Citation
Mixed-Agency Teams and Their Effect on Social Presence and Behavior during High-Fidelity Team Training

Material Information

Title:
Mixed-Agency Teams and Their Effect on Social Presence and Behavior during High-Fidelity Team Training
Creator:
Robb, Andrew C
Place of Publication:
[Gainesville, Fla.]
Florida
Publisher:
University of Florida
Publication Date:
Language:
english
Physical Description:
1 online resource (178 p.)

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Computer Engineering
Computer and Information Science and Engineering
Committee Chair:
LOK,BENJAMIN
Committee Co-Chair:
ANTHONY,LISA
Committee Members:
JAIN,EAKTA
ENTEZARI,ALIREZA
LAMPOTANG,SAMSUN
Graduation Date:
12/18/2015

Subjects

Subjects / Keywords:
Human behavior ( jstor )
Medical scrubs ( jstor )
Nurses ( jstor )
Simulations ( jstor )
Social behavior ( jstor )
Social interaction ( jstor )
Surgeons ( jstor )
Team training ( jstor )
Virtual observatories ( jstor )
Visual fixation ( jstor )
Computer and Information Science and Engineering -- Dissertations, Academic -- UF
behavior -- humanfactors -- socialpresence -- teamtraining -- virtualhumans
Genre:
bibliography ( marcgt )
theses ( marcgt )
government publication (state, provincial, terriorial, dependent) ( marcgt )
born-digital ( sobekcm )
Electronic Thesis or Dissertation
Computer Engineering thesis, Ph.D.

Notes

Abstract:
Team training can improve job performance, communication skills, and coordination within a team. Unfortunately, team training can be difficult to implement in the real world, due to logistical difficulties. Virtual teammates could be used to address these limitations, by serving as replacements for missing teammates. This work investigates the use of virtual humans for team training. Specifically, it investigates how the agency of a trainee's teammates (e.g. if they are real or virtual) affects behavior. This research investigates this question from two perspectives: will people treat a teammate differently if he is real or virtual? And, will people treat a virtual teammate differently when a second human is present. It is important to understand whether behavior is influenced by a teammate's agency, as this will influence when and how virtual humans can be used for team training. To investigate these research questions, a system was developed to create team training scenarios where the agency of specific teammates could be varied. Two training exercises were then developed, both of which were used to evaluate behavior with real and virtual teammates. Sixty-nine nurses participated in the first study, and 94 nurses and surgical technicians participated in the second study. Three aspects of participants' interactions with teammates were evaluated: social presence, decision making behavior, and nonverbal behavior. The results of these studies suggest that behavior is not meaningfully influenced by the agency of one's teammates, nor do people behave differently with virtual humans when other humans are present. These results support the use of virtual humans for team training. ( en )
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Thesis:
Thesis (Ph.D.)--University of Florida, 2015.
Local:
Adviser: LOK,BENJAMIN.
Local:
Co-adviser: ANTHONY,LISA.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2016-12-31
Statement of Responsibility:
by Andrew C Robb.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Embargo Date:
12/31/2016
Classification:
LD1780 2015 ( lcc )

Downloads

This item has the following downloads:


Full Text

PAGE 1

MIXED-AGENCYTEAMSANDTHEIREFFECTONSOCIALPRESENCEANDBEHAVIORDURINGHIGH-FIDELITYMEDICALTEAMTRAININGByANDREWROBBADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2015

PAGE 2

c2015AndrewRobb

PAGE 3

Idedicatethistomylovingparents,towhomIamforevergrateful

PAGE 4

ACKNOWLEDGMENTSIthankDr.BenjaminLok,myadvisorandsupervisorycommitteechair,fortheadvice,support,wisdom,andguidancehehasprovidedovermyyearsasaPhDstudent.Hismentoringandhisexamplehaveshapedmeasathinker,aresearcher,andaperson.Ithasbeenanincredibleprivilegedtoworkwithhiminhisresearch.Ithankmycollaboratorsalso,Drs.CaseyWhite,SamsunLampotang,andAdamWendling,fortheirhelp,support,andpatienceinhelpingmelearntoworkwiththemedicalcommunity.Thisexpertiseandsupportwascriticalinthedevelopmentofthetrainingprogramsexploredinmyresearchandtherecruitmentofparticipants.IwouldalsoliketothankTeresaHughes,TerrySullivan,ChristineFoley-Brinza,andDianeSkorupskifortheirenthusiasmfortrainingandthetime,resources,andenergytheydevotedtomakemyresearchpossible.Additionally,Iwouldliketothankmysupervisorycommittee,Drs.EaktaJain,LisaAnthony,AlirezaEntezari,andSamsunLampotangfortheiruniquecontributionstothisdissertation.Finally,IwouldliketothankthemanyfellowgraduatestudentsandpostdoctoralfellowsIhavebeenprivilegedtoworkwith,Dr.JoonHaoChuah,AndrewCordar,Dr.AndreaKleinsmith,ShivashankarHalan,DiegoRivera-Gutierrez,MichaelBorish,Dr.RegisKopper,Dr.AaronKotranza,MalloryMcManamon,VaishnaviKrishnan,DoaaElSheikh,andGuangyanHu.Theirsupport,bothdirectandindirect,hasbeeninvaluable.IappreciatetheUniversityofFloridaandComputerandInformationScienceandEngineeringDepartment,Faculty,andStaffforprovidingthenancialandequipmentsupportinthiswork.Also,IthanktheUniversityofFloridaAlumniGraduateProgramforthehonorofreceivingtheirfellowshipsupportduringmyrstfouryearsofresearch.IalsothanktheNationalScienceFoundation,whichsupportthisworkviaNSFGrant1161491. 4

PAGE 5

Lastly,IwouldliketothankmyparentsMarkandBeckieRobb.TheirunconditionalloveandencouragementhassupportedmeduringmyPhD.IwouldnotbethepersonIamtodaywithouttheirexample,sacrice,patience,wisdom,andlove. 5

PAGE 6

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 10 LISTOFFIGURES ..................................... 11 ABSTRACT ......................................... 13 CHAPTER 1INTRODUCTION ................................... 15 1.1Mixed-AgencyInteractions .......................... 17 1.2ThesisStatement ................................ 19 1.3Motivation .................................... 20 1.4OverviewofApproach ............................. 22 1.5Innovation .................................... 25 2REVIEWOFLITERATURE ............................. 28 2.1VirtualHumans ................................. 28 2.1.1SocialResponsestoVirtualHumans ................. 29 2.1.2VirtualHumansandTraining ...................... 30 2.1.3VirtualTeammates ........................... 31 2.2AgencyandVirtualHumans .......................... 37 2.2.1AgencyandSocialPresence ..................... 38 2.2.2AgencyandBehavior .......................... 39 2.3EmbeddingVirtualHumansintheRealWorld ................ 41 2.4MedicalTeamTraining ............................. 43 3CREATINGMIXED-AGENCYINTERACTIONS .................. 46 3.1AuthoringMixedAgencyTeams ........................ 46 3.1.1RepresentingandSimulatingVirtualHumans ............ 46 3.1.2ConversationModelsforMixed-AgencyTeams ........... 48 3.2SimulatingMixed-AgencyTeams ....................... 50 3.3TimerequiredtoauthorvirtualhumansinVPFandVPF2 ......... 51 3.4DevelopingVirtualPeopleFactory2.0 .................... 52 3.5VirtualHumanDisplayTechnology ...................... 52 3.6WizardofOz .................................. 54 3.7Conclusion ................................... 55 4TEAMTRAININGEXERCISES ........................... 56 4.1SpeakingUpExerciseandStudy ....................... 56 4.1.1SpeakingUpExercise ......................... 57 6

PAGE 7

4.1.1.1Tutorial ............................ 57 4.1.1.2Stage1:Pre-opHolding ................... 58 4.1.1.3Stage2:Pre-inductionBrieng ............... 58 4.1.1.4Stage3:Pre-incisionTimeout ............... 61 4.1.1.5EducationalIntervention ................... 62 4.1.2SpeakingUpStudy ........................... 63 4.1.2.1Conditions .......................... 64 4.1.2.2HumanConfederates .................... 64 4.1.2.3Participants .......................... 66 4.1.2.4Metrics ............................ 66 4.1.2.5Studyprocedure ....................... 67 4.2SurgicalCountExerciseandStudy ...................... 68 4.2.1SurgicalCountExercise ........................ 68 4.2.1.1Tutorial ............................ 68 4.2.1.2Stage1:Pre-inductionBrieng ............... 69 4.2.1.3Stage2:Pre-incisionTimeout ............... 70 4.2.1.4Stage3:ClosingCount ................... 70 4.2.1.5EducationalIntervention ................... 73 4.2.2SurgicalCountStudy .......................... 73 4.2.2.1Conditions .......................... 74 4.2.2.2Participants .......................... 74 4.2.2.3Metrics ............................ 76 4.2.2.4Studyprocedure ....................... 76 5MIXED-AGENCYINTERACTIONSANDDECISIONMAKINGBEHAVIOR ... 78 5.1ResearchQuestionsandMotivation ..................... 78 5.2CollectionandPreparationofDecisionMakingData ............ 79 5.2.1Codingschemeforagreementmoments ............... 81 5.2.2Codingschemeforspeakingupmoments .............. 81 5.2.3Codingbehaviorduringthespeakingupmoments ......... 81 5.3BehaviorDuringAgreementMoments .................... 83 5.4BehaviorDuringSpeakingUpMoments ................... 84 5.4.1DirectEffects:DoesaSurgeon'sAgencyAffectHowNursesSpeakUptoHim ................................ 84 5.4.2IndirectEffects:DoesthePresenceofaSecondHumanTeammateChangeHowNursesSpeakUptoaVirtualSurgeon ........ 87 5.4.2.1BehaviorduringtheSpeakingUpstudy .......... 87 5.4.2.2BehaviorduringtherstspeakingupmomentoftheSurgicalCountstudy ......................... 88 5.4.2.3BehaviorduringthesecondspeakingupmomentoftheSurgicalCountstudy .................... 89 5.4.2.4Discussion .......................... 90 5.5InuenceTacticsUsageduringtheSpeakingUpstudy ........... 92 5.5.1DirectEffects:InuenceTacticusageintheSpeakingUpstudy .. 92 7

PAGE 8

5.5.2IndirectEffects:InuenceTacticusageintheSpeakingUpstudy . 95 5.6Conclusions ................................... 96 6MIXED-AGENCYINTERACTIONSANDNONVERBALBEHAVIOR ....... 98 6.1ResearchQuestionsandMotivation ..................... 98 6.2CollectionandPreparationofNonverbalData ................ 100 6.3CodingofNon-VerbalBehavior ........................ 100 6.4Results ..................................... 103 6.4.1DescriptiveStatistics .......................... 103 6.4.2TotalGazeDuration .......................... 104 6.4.2.1Agency'sDirectEffects ................... 104 6.4.2.2IndirectEffect ......................... 105 6.4.3GazeBehaviorandSpeech ...................... 106 6.4.4GazeFixationsAnalysis ........................ 109 6.4.4.1NumberofGazeFixationMoments ............ 109 6.4.4.2DurationofGazeFixationMoments ............ 113 6.4.4.3DistributionofGazeFixationMoments ........... 115 6.4.5Silence .................................. 116 6.4.5.1DirectEffects ......................... 116 6.4.5.2IndirectEffect ......................... 116 6.5Discussion ................................... 117 6.5.1DirectEffects:Similaritiesingazewithrealandvirtualteammates 118 6.5.2DirectEffects:VaryingaTeammate'sAgencycanAffectGazewiththatTeammate ........................... 118 6.5.3IndirectEffects:GazewithaVirtualTeammateisLargelyUnaffectedbytheAgencyofotherTeammates .................. 121 6.5.3.1Limitations .......................... 122 6.6ConclusionandFutureWork ......................... 123 7MIXED-AGENCYINTERACTIONSANDSOCIALPRESENCE ......... 125 7.1ResearchQuestionsandMotivation ..................... 125 7.2CollectionandPreparationofSocialPresenceData ............ 126 7.2.1SocialPresenceSurveys ....................... 126 7.2.2PreparationofdatafromtheSpeakingUpstudy ........... 127 7.2.3PreparationofdatafromtheSurgicalCountstudy .......... 127 7.3Agency'sDirectEffectonSocialPresence .................. 128 7.3.1ConstructValidityintheSpeakingUpstudy ............. 128 7.3.2ConstructValidityintheSurgicalCountstudy ............ 129 7.3.3DiscussionofConstructValidity .................... 130 7.3.4UsingSocialPresenceSurveyswithRealHumans ......... 131 7.4Agency'sIndirectEffectonSocialPresence ................. 132 7.4.1TheEffectofaHumanTeammateonaVirtualHuman'sSocialPresenceintheSpeakingUpStudy ................. 133 8

PAGE 9

7.4.2TheEffectofaHumanTeammateonaVirtualHuman'sSocialPresenceintheSurgicalCountStudy ................ 133 7.4.3Meta-analysisoftheresultsoftheSpeakingUpandSurgicalCountstudies .................................. 134 7.4.4DiscussionoftheEffectofaHumanTeammateonaVirtualHuman'sSocialPresence ............................ 136 7.4.5Limitations ................................ 139 7.5ConclusionsandFutureWork ......................... 139 8SUMMARYANDFUTUREDIRECTIONS ..................... 141 8.1DiscussionandSummary ........................... 141 8.1.1BehaviorwithVirtualTeammatesMirrorsBehaviorwithHumanTeammates ............................... 141 8.1.2ThePresenceofOtherHumansdoesnotMeaningfullyAlterBehaviorwithVirtualTeammates ........................ 143 8.1.3Variable-AgencyTeammatescanSupportTrainingProgramsbyOvercomingLogisticalDifculties ................... 144 8.2Limitations ................................... 144 8.2.1High-delityVirtualHumans ...................... 145 8.2.2MedicalDomain ............................. 145 8.2.3PowerRelationships .......................... 145 8.3RealWorldUsage ............................... 146 8.4FutureDirections ................................ 146 8.5FinalRemarks ................................. 147 APPENDIX ASPEAKINGUPSCENARIOSCRIPT ........................ 149 A.1Stage1Pre-OpHolding ............................ 149 A.2Stage2Brieng ................................ 150 A.3Stage3Timeout ................................ 152 BSURGICALCOUNTSCENARIOSCRIPT ..................... 154 B.1Stage1Brieng ................................ 154 B.2Stage2Timeout ................................ 156 B.3Stage3ClosingCount ............................ 158 REFERENCES ....................................... 162 BIOGRAPHICALSKETCH ................................ 178 9

PAGE 10

LISTOFTABLES Table page 1-1ResearchQuestions ................................. 20 5-1Taxonomyofspeakingupbehaviors ........................ 81 5-2Thesixinuencetactics ............................... 82 5-3Behaviorduringtherstagreementmoment ................... 83 5-4Behaviorduringthesecondagreementmoment ................. 84 5-5Theeffectofthesurgeon'sagencyonspeakingupbehavior ........... 85 5-6Indirecteffectoftheanesthesiologist'sagencyonspeakinguptothesurgeonintheSpeakingUpstudy .............................. 88 5-7Indirecteffectoftheanesthesiologist'sagencyonspeakinguptothesurgeoninthesecondspeakingupmomentoftheSurgicalCountstudy ......... 89 5-8Indirecteffectoftheanesthesiologist'sagencyonspeakinguptothesurgeoninthesecondspeakingupmomentofintheSurgicalCountstudy ....... 89 6-1Durationofthetwostagesinwhichgazebehaviorwasanalyzed ........ 103 6-2Gazetimewiththesurgeonandanesthesiologistinthedifferentconditions .. 104 6-3Gazetimewiththesurgeonandanesthesiologistinthedifferentconditions,moderatedbyspeaker ................................ 107 6-4Thenumberofgazexationswiththesurgeonandanesthesiologistinthedifferentconditions .................................. 110 7-1SocialpresenceforrealandvirtualhumansintheSpeakingUpstudy ..... 129 7-2SocialpresenceforrealandvirtualhumansintheSurgicalCountstudy .... 130 7-3SocialpresenceforvirtualhumanswhenotherhumansarepresentintheSpeakingUpstudy .................................. 133 7-4SocialpresenceforvirtualhumanswhenotherhumansarepresentintheSurgicalCountstudy ................................. 134 10

PAGE 11

LISTOFFIGURES Figure page 1-1Illustrationofthetworesearchquestionsaddressedinthisdissertation ..... 16 1-2Threeinteractionsbetweenonerealhumanandonevirtualhuman ....... 26 1-3AdaandGracespeakingwithvisitorsattheMuseumofScienceinBoston,Massachusetts. .................................... 26 2-1TwoSteveagentsinteractwitheachotherwhileahumanobserves. ...... 32 2-2TheMissionRehearsalExercisesystem. ..................... 33 2-3TwoimagesfromthePerfectCirclegamedevelopedtotesttheSGDmodel. .. 34 2-4Twoimagesofthesyntheticteammateproject .................. 35 2-5TheCITTPandPORTSTAO-ITSsystems ..................... 36 3-1Theprocessbywhichahumanspeechtriggersavirtualhumantorespond. .. 48 3-2Threeseparatecharacters,eachwiththeirownactions. ............. 48 3-3Theprocessofmatchinganinputspeechtoanaction .............. 49 3-4Theprocessbywhichahumanspeechtriggersavirtualhumantorespond. .. 51 3-5Theprocessbywhichavirtualhumansperformsanaction. ........... 51 3-6Threevirtualhumans ................................ 53 4-1ImageoftheSpeakingUptutorial ......................... 58 4-2ImageoftheSpeakingUppre-opholdingstage .................. 59 4-3ImageoftheSpeakingUppre-inductionbriengstage .............. 59 4-4ImageoftheSpeakingUppre-incisiontimeoutstage ............... 62 4-5ThethreeconditionsoftheSpeakingUpstudy .................. 65 4-6ImageoftheSurgicalCounttutorial ........................ 69 4-7ImageoftheSurgicalCountpre-inductionbriengstage ............. 70 4-8ImageoftheSurgicalCountpre-incisiontimeoutstage .............. 71 4-9ImageoftheSurgicalCountclosingcountstage ................. 71 4-10ThethreeconditionsoftheSurgicalCountstudy ................. 75 5-1ThedirecteffectofagencyonspeakingupintheSurgicalCountstudy ..... 85 11

PAGE 12

5-2TheindirecteffectofagencyonspeakingupintheSpeakingUpstudy ..... 88 5-3TheindirecteffectofagencyonspeakingupinthesecondspeakingupmomentoftheSurgicalCountstudy ............................. 90 5-4TheindirecteffectofagencyonspeakingupinthesecondspeakingupmomentoftheSurgicalCountstudy ............................. 91 5-5Thedirecteffectofagencyoninuencetacticusage ............... 93 5-6Thenumberofdifferentinuencetacticsused ................... 93 5-7Thenumberofattemptstoinuencethesurgeon ................. 94 5-8Theamountoftimespentspeakingtothesurgeon ................ 94 5-9Theindirecteffectofagencyoninuencetacticusage .............. 96 6-1Anexampleofthevideosusedfornonverbalanalysis. .............. 101 6-2AsampleportionofthegazecodingtrackinANVIL. ............... 102 6-3Directeffectofagencyongazeduration ...................... 105 6-4Indirecteffectofagencyongazeduration ..................... 106 6-5Directeffectofagencyongazeduration,moderatedbyspeaker ........ 109 6-6Directeffectofagencyonthenumberofgazexationmoments ......... 111 6-7Indirecteffectofagencyonthenumberofgazexationmoments ........ 112 6-8Directeffectofagencyonthedurationofgazexationmoments ........ 113 6-9Indirecteffectofagencyonthedurationofgazexationmoments ....... 114 6-10Directeffectofagencyongazexationmomentdistributions .......... 115 6-11Indirecteffectofagencyongazexationmomentdistributions ......... 115 6-12Directeffectofagencyontheamountofsilenceafterthesurgeonspoke .... 117 6-13Indirecteffectofagencyontheamountofsilenceafterthesurgeonspoke ... 117 7-1Illustrationoftheconstructvaliditycomparison .................. 129 7-2Participantsreinterpretedsomeofthesocialpresencequestions ........ 132 7-3Forestplotofthemeta-analysisresults ....................... 135 12

PAGE 13

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyMIXED-AGENCYTEAMSANDTHEIREFFECTONSOCIALPRESENCEANDBEHAVIORDURINGHIGH-FIDELITYMEDICALTEAMTRAININGByAndrewRobbDecember2015Chair:BenjaminLokMajor:ComputerEngineeringTeamtrainingcanimprovejobperformance,communicationskills,andcoordinationwithinateam.Unfortunately,teamtrainingprogramscanbedifculttoimplementintherealworld,duetologisticaldifculties.Virtualteammatescouldbeusedtoaddresstheselimitations,byservingasreplacementsformissingteammates.Thisworkinvestigatestheuseofvirtualhumansforteamtraining.Specically,itinvestigateshowtheagencyofatrainee'steammates(e.g.iftheyarerealorvirtual)affectsbehavior.Thisresearchinvestigatesthisquestionfromtwoperspectives:willpeopletreataspecicteammate(e.g.asurgeon)differentlywhenheisrealcomparedtowhenheisvirtual?And,willpeopletreatavirtualteammatedifferentlywhenasecondhumanteammateispresent?Itisimportanttounderstandwhetherbehaviorisinuencedbyateammate'sagency,asthiswillinuencewhenandhowvirtualhumanscanbeusedforteamtraining.Toinvestigatetheseresearchquestions,avirtualhumanauthoringsystemwasdevelopedtocreateteamtrainingscenarioswheretheagencyofspecicteammatescouldbevaried.Twotrainingexerciseswerethendeveloped,bothofwhichwereusedtoevaluatebehaviorwithrealandvirtualteammates.Sixty-ninenursesparticipatedintherststudy,andninety-fournursesandsurgicaltechniciansparticipatedinthesecondstudy.Threeaspectsofparticipants'interactionswithteammateswereevaluated:socialpresence,decisionmakingbehavior,andnonverbalbehavior. 13

PAGE 14

Theresultsofthesestudiessuggestthatbehaviorisnotmeaningfullyinuencedbytheagencyofone'steammates,andthatpeopledonotbehavedifferentlywithvirtualteammateswhenotherhumansarepresent.Theseresultssupporttheuseofvirtualhumansforteamtraining. 14

PAGE 15

CHAPTER1INTRODUCTIONVirtualhumansaresoftwareagentswithahuman-likeappearancethatusespeech,gaze,gesture,intonationandothernonverbalmodalitiestoemulatetheexperienceofhumanface-to-faceconversationwiththeirusers[ 1 ].Becauseofthesesimilarities,peopleoftendisplayrealisticbehaviorwheninteractingwithvirtualhumans[ 2 – 6 ].Thistendencytobehaverealisticallycanmakevirtualhumansusefulfortrainingandeducation.Severaltypesofvirtualhuman-basedinterpersonalskillstrainershavebeencreated,includingmedicalinterviewingsystems[ 7 – 9 ],physicalexamtrainers[ 10 , 11 ],classroomtrainersforteachers[ 12 ],andmilitarynegotiationsimulations[ 13 , 14 ].Theresearchdescribedinthisdissertationexploresbehaviorwithvirtualhumanswithinthecontextofmedicalteamtraining.Specically,itexploreshow(a)behaviorwithaspecicvirtualteammate(e.g.avirtualsurgeon)differsfrombehaviorwithahumancounterpart(e.g.ahumansurgeon),and(b)howthepresenceofotherhumanteammatesaffectsbehaviorwithvirtualteammates(e.g.doesahumannurse'sbehaviorwithavirtualsurgeonchangewhentheteam'svirtualanesthesiologistisreplacedwithahumananesthesiologist).Bothoftheseresearchquestionsrelatetohowvaryingateammate's“agency”,(e.g.whethertheteammateisahumanoracomputer[ 15 ])affectsbehaviorwiththatteammate,andwiththeentireteam.ThesetworesearchquestionsareillustratedinFigure 1-1 .Inthisgure,ahumannursecouldworkwithoneoftwoteams.Therstteamiscomprisedofavirtualanesthesiologistandavirtualsurgeon,andthesecondteamiscomprisedofahumananesthesiologistandavirtualsurgeon.Varyingtheanesthesiologist'sagencyhasthepotentialtoalterbehaviornotonlywiththeanesthesiologist,butalsowiththevirtualsurgeon,whoseagencywasheldconstant.Figure 1-1 illustratesthesetwopossibilitiesusingabluearrowandanorangearrowrespectively. 15

PAGE 16

Figure1-1. Anillustrationofthetworesearchquestionsaddressedinthisdissertation.Changingtheanesthesiologist'sagencyhasthepotentialtonotonlyaffectbehaviorwiththeanesthesiologist,butalsowiththesurgeon. Thebluearrowcorrespondstotherstresearchquestionexploredinthisdissertation:theextenttowhichbehaviorwithaspecicteammatedependsonwhetherthatteammateisrealorvirtual.Priorresearchhasexploredthisquestioninnon-teamsettings(thisresearchisreviewedinSection 2.2.2 ).Theorangearrowillustratesthesecondresearchquestionexploredinthisdissertation:whetherthepresenceofotherhumanteammatesaltersbehaviorwithvirtualteammates.Thisquestionhasnotbeeninvestigatedpriortothisdissertation.Withinthecontextofthesetworesearchquestions,Iinvestigatethreespecicaspectsof“behavior”:howparticipantsmakedecisionsinresponsetothebehavioroftheirteammates,non-verbalbehaviorwithteammates,andtheexperienceofsocialpresencewithteammates.Thesethreeaspectsrepresentthreedifferentclassesofbehavior:consciousbehaviorthatisdeliberatelyenacted,unconsciousbehaviorthatis 16

PAGE 17

automaticallyenacted,andinternalmentalperceptions.Decisionmakingisofprimaryimportancetotraining,asacommongoalofmanytrainingprogramsistoteachpeopletomakecorrectdecisions.Thus,itisimportanttoexplorewhethertheagencyofone'steammatesaffectshowdecisionsaremadewiththem.Non-verbalbehaviorislargelyanautomaticprocess,whichmakesitaninterestingmetricwhenexploringtheeffectofagencyonbehavior.Anysimilaritiesthatmaybeobservedindecisionmakingbehaviorcouldpotentiallybeattributedtoadeliberatesuspensionofdisbelief,notduetoanautomaticimpulsetobehaverealisticallywithvirtualteammates.Thesameisnottruefornon-verbalbehavior.Observingsimilaritiesordifferencesinnon-verbalbehaviorwithvirtualteammateswillhelptounderstandwhetheransimilaritiesinbehaviorwithvirtualteammatesandhumanteammatesareduetoaconsciouschoicetobehaverealistically,orifitisduetoanautomaticprocess.Socialpresence,oftendenedasthesenseof“beingwith”another[ 16 ],relatestohowparticipantsmentallyperceivevirtualagents.Socialpresencemeasuresthedegreetowhichapersonthinksofavirtualagentasbeingarealsocialactor,ratherthanasatechnologicalartifactincapableoftruesocialinteraction.Theeffectofateammate'sagencyonsocialpresenceisimportanttoconsider,asameaningfuldecreaseinfeelingsofsocialpresencewithvirtualteammatescouldinterferewithatrainee'swillingnesstosuspenddisbeliefduringthetrainingexercise. 1.1Mixed-AgencyInteractionsForthisdissertation,Idenetheterm“mixed-agencyinteraction”asaninteractioninvolvingtwoormorehumansandoneormorevirtualhumans.“Mixed-agencyteams”representasubsetofmixed-agencyinteractions,whereeachoftherealandvirtualhumanspresentareworkingtogetheraspartofateam.Thekeyconceptofamixed-agencyinteractionisthateachhumanparticipantcansimultaneouslyinteractwithbothrealandvirtualhumans.Thus,aninteractioninvolvingasinglehumanisnotamixed-agencyinteraction,asthatsinglehumanwillonlyhavevirtualhumansto 17

PAGE 18

interactwith.InFigure 1-1 ,thesecondteamisanexampleofamixed-agencyteam,asitcontainsahumannurse,ahumananesthesiologist,andavirtualsurgeon.Toexploretheeffectofmixed-agencyteamsonbehaviorwithvirtualteammates,Icomparemixed-agencyteamstoteamscontainingonehumanparticipantandmultiplevirtualteammates.Twomajorvariationsofmixed-agencyinteractionsexist.Intherstvariation,allinteractionbetweenhumansandvirtualhumanstakesplacewithinasharedvirtualenvironment,suchasSecondLifeorWorldofWarcraft.Interactingwithinasharedvirtualenvironmentmasksmanyofthedifferencesbetweenrealandvirtualhumans,ashuman-humanandhuman-computerinteractionsarebothmediatedbythesametechnologyandthussharemanyofthesamelimitations.Inthesecondvariationofmixed-agencyinteractions,allinteractionbetweenrealandvirtualhumanstakesplacewithinasharedphysicalenvironment.Interactingwithinasharedphysicalenvironmentallowsthedifferencesbetweenrealandvirtualhumanstobeclearlyseen.Whenphysicallypresentwithotherhumans,human-humaninteractionscanemploythefullrangeofcommunicationchannelsusedinhumancommunication,manyofwhicharedifculttosimulateinhuman-computerinteractions.Examplesofcommunicationchannelsthatremainareasofopenresearchforhuman-computercommunicationincludegaze,bodyposture,facialexpression,intonation,andproxemics.Differencesinvisualdelity,bothinappearanceandmotion,canalsobeclearlyseen.Asymmetricinteractions,inwhichhumansbehavedifferentlywithhumanteammatesandvirtualteammates,seemmorelikelyinmixed-agencyinteractionsinvolvingphysically-presenthumans,duetothelikelihoodofagreaterdifferenceinvisualandbehavioraldelitybetweenrealandvirtualteammates.Whilebehavioraldelitycanalsodifferinsharedvirtualenvironments,thesedifferencesareinherentlymoreconstrainedandarealsonotnecessarilyunidirectional.Withinasharedvirtualenvironment,limitationsininputtechnologycansometimesmakeitmoredifcultfor 18

PAGE 19

humanstocommunicateusingfacialexpressionandbodygesture,comparedtovirtualhumans.Accordingly,duetothehigherlikelihoodofndingasymmetricinteractionsinmixed-agencyinteractionsinvolvingphysically-presenthumans,ifagencydoesimpactbehaviorwithinmixed-agencyteams,thiseffectwouldlikelybemorepronouncedwhenotherhumansarephysicallypresent.Thisistheclassofinteractionsinvestigatedinthisdissertation. 1.2ThesisStatementMythesisstatementisthefollowing:withinthecontextofmedicalteamtraining,varyingtheagencyofatrainee'steammateswillnotmeaningfullyalterthetrainee'sbehavior,giventhatthevirtualteammatesaresufcientlyrealistictomeetthetrainee'sexpectationsabouthowpeoplebehaveintherealworld.Withinthisthesisstatement,Iinvestigatetwospecicresearchquestions,eachofwhichareconcernedwithadifferentmannerinwhichagencycouldaffectbehavior.Tohelpdifferentiatebetweenthesetwoquestions,Iassignalabeltoeachquestion.Whendiscussingmyresults,theappropriatelabelwillbeusedtohelpdistinguishbetweenmytworesearchquestions. 1. DirectEffect:Whethervaryingaspecicteammate'sagencychangespeople'sbehaviorwiththatteammate(e.g.doeschangingtheagencyoftheanesthesiologistdirectlyaffecthowpeoplebehavewiththeanesthesiologist?). 2. IndirectEffect:Whetherintroducingasecondhumanteammatechangeshowpeoplebehavewithvirtualteammates(e.g.doeschangingtheagencyoftheanesthesiologistindirectlyaffecthowpeoplebehavewiththevirtualsurgeon?).Irefertotherstquestionasthedirecteffectofagencybecausethisquestionisconcernedwithwhethervaryingateammate'sagencydirectlyaffectsbehaviorwiththatteammate.Irefertothesecondquestionastheindirecteffectofagencybecausethisquestionisconcernedwithwhethervaryingoneteammate'sagencyindirectlyaffectsbehaviorwithasecondteammate. 19

PAGE 20

Iinvestigatethesetworesearchquestionsusingthreemetrics:decision-makingbehavior,non-verbalbehavior,andsocialpresence.Thus,inseekingtoprovemythesisstatement,Iinvestigatesixspecicquestionsgroupedintotwocategories.ThesesixquestionsarepresentedandorganizedinTable 1-1 . Table1-1. Theresearchquestionsthatwillbeexploredwhileinvestigatingmythesisstatement DirectEffectIndirectEffect DecisionMakingDopeoplemakedifferentdecisionswithrealandvirtualteammates?Doesthepresenceofahumanteammatealterdecisionsmadewithvirtualteammates?Non-verbalBehaviorDohumanteammatesevokedifferentnon-verbalbehaviorthanvirtualteammates?Doesthepresenceofahumanteammatealternon-verbalbehaviorwithvirtualteammates?SocialPresenceDohumanteammatesevokemoresocialpresencethanvirtualteammates?Doesthepresenceofahumanteammatealteravirtualteammate'ssocialpresence? 1.3MotivationEffectiveteamworkhasbecomeanessentialpartofmodernhealthcare.Effectiveteamworkcanleadtoreducederrorrates,improvedpatientoutcomes,andimprovedproblemsolvingskills[ 17 ].Unfortunately,notallteamsfunctioneffectively.Failuresinteamworkhavealsobecomeoneoftheleadingsourcesofmedicalerrors.Oneanalysisof2,455adversehealthcareeventsfoundthatfailuresincommunicationweretheprimaryrootcauseofover70%oftheseerrors[ 18 ].Thankfully,teamtraininghasbeenshowntoimprovecommunicationandcoordinationwithateam[ 19 ]andtoimprovejobperformance[ 20 ].Unfortunately,teamtrainingcanbeverydifculttoimplementintherealworld,largelyduetologisticaldifculties[ 21 ].Twonotabledifcultiesareinefcienciesinschedulingtrainingprogramsduetotheconictingschedulesoftrainees,andneedingtocanceltrainingexerciseswhenkeyteammembersdon't 20

PAGE 21

arriveasscheduled.Virtualteammatescouldpotentiallybeusedtoovercomebothofthesedifculties,eitherbyllinginforteammemberswhohaveschedulingconicts,orbyreplacingteammemberswhodon'tarriveasscheduled.Beforeusingvirtualhumansinthismanner,itwillbeimportanttorstconsiderwhetherworkingwithvirtualteammatesaffectshowhumantraineesbehaveduringatrainingexercise.AsillustratedinFigure 1-1 ,therearetwowaysthatchangingateammate'sagencycouldpotentiallyaffectbehavior.First,itispossiblethatatrainee'sbehaviorwithaspecicteammatewillchangeifthatteammate'sagencyischanged.Itisparamountthattraineesbehaverealisticallyduringteamtrainingexercises.Iftraineesbehaveunrealistically,theymayfailtolearnthedesiredbehaviors,ormayevenlearnundesirablebehaviors.Failuretobehaverealisticallywithvirtualteammatescouldalsorendervirtualhuman-basedtrainingprogramsuselessforassessmentpurposes.ThiscanbeillustratedusingFigure 1-1 asanexample.InFigure 1-1 ,theagencyoftheanesthesiologistisaltered.Itispossiblethatthischangewillaffecthowthenursebehaveswiththeanesthesiologist.Forinstance,ifpeopleprefertocommunicatewithrealhumansinsteadofvirtualhumans,traineesmayinteractlesswiththevirtualanesthesiologistthanthehumananesthesiologist.Thiscouldinturnaffectbehaviorwithanesthesiologistsintherealworld.Figure 1-1 alsoillustratesasecondwaythatchangingateammate'sagencycanaffectbehavior.Changingateammate'sagencycannotonlyaffectbehaviorwiththatteammate,butalsowithotherteammates.InFigure 1-1 ,theanesthesiologist'sagencyischanged,butthesurgeon'sisnot.Ifpeopleprefertocommunicatewithrealhumansinsteadofvirtualhumans,traineescouldspendlesstimecommunicatingwiththevirtualsurgeonwhenthehumananesthesiologistispresent.Thiscouldalsoaffectbehaviorwithsurgeonsintherealworld.Thequestionofwhetherthepresenceofotherhumanteammatescanaffectbehaviorwithvirtualteammatesisgroundedinthepossibilityofcomparisons.When 21

PAGE 22

otherhumanteammatesarealsopresent,traineescandirectlycomparevirtualteammatestohumanteammates.Asvirtualhumansareinherentlymorelimitedthanhumanteammatesintermsofvisualandbehavioraldelity,itmaybethatsuchcomparisonswillleadtoreducedimpressionsofavirtualteammate'srealismandalterbehaviorwithvirtualteammates.Forinstance,ifbeingabletocompareavirtualsurgeontoahumananesthesiologistcausesnursestoperceivethesurgeonaslessrealorlessauthoritative,theymaybemorewillingtoconfrontthevirtualsurgeonaboutdecisionsthatendangerapatient'ssafety.Thiswouldnotonlyrendervirtualhuman-basedtraininglessusefulforassessmentpurposes,butitalsocouldcausenursestoexperienceafalsesenseofreadinessthatwouldleadthemtoremainsilentwhenahumansurgeonmakesadecisionthatcouldendangerapatient'ssafetyintherealworld.Thisresearchwillgenerateguidelinesonwhethersuchaneffectislikely.Ifso,virtualhuman-basedtrainingmayneedtobelimitedtoasinglehumantraineeatatime. 1.4OverviewofApproachToevaluatemythesis,thefollowingworkwasperformed.Avirtualhumanauthoringsystem,VirtualPeopleFactory2.0(VPF2),wasdevelopedtocreateandsimulatemixed-agencyteams.Thissystemwasbasedonaprevioussystem(VPF)usedtocreatevirtualpatientsthatwasnotsuitableforcreatingmixed-agencyteams.Specically,thefollowingrequirementswereunsupportedintheprevioussystem:simulatingmorethanonevirtualhuman,varyingtheagencyofspecicteammates,andallowingvirtualhumanstotakeinitiativeandcontroltheinteraction.VPF2representedacompletere-designoftheoriginalVPF.Thisredesignwasnecessarytosupporteachoftheabovethreerequirements.ThisworkisdiscussedinChapter 3 .VPF2wasthenusedtocreatetwomixed-agencyteams.IworkedwiththenursingmanagementofUFHealth'soperatingroomstoidentifyatrainingneedthatcouldbemetusingvirtualhuman-basedteamtraining.Nursingmanagementidentiedspeakingupaboutpatientsafetyissuesasanimportanttrainingneed.Ithenworkedwithnursing 22

PAGE 23

managementandanexperiencedanesthesiologisttodeveloptwomixed-agencyteamsthatwouldallownursesandsurgicaltechnicianstopracticespeakingupaboutpatientsafetyissues.Theseteamsweredevelopedsequentially,wheretherstteamwasdevelopedandevaluated,andthenthesecondteamwasdevelopedandevaluated.Ithenconductedtworesearchstudies,eachofwhichusedoneofthetwomixed-agencyteamsIdeveloped.Intherststudy,nursesworkedwithasurgeonandananesthesiologisttoprepareasimulatedpatientforsurgery.Theagencyofthesurgeonandtheanesthesiologistwasvariedbetweenconditions;participantseitherworkedwithavirtualsurgeonandavirtualanesthesiologist,ahumanactorplayingasurgeonandavirtualanesthesiologist,oravirtualsurgeonandahumanactorplayingananesthesiologist.Duringthetrainingexercise,theteam'ssurgeonmadeadecisionthatcouldpotentiallyendangerthesafetyoftheteam'spatient.Thisrequirednursestospeakuptothesurgeoninordertoprotectthepatient.Thetraininggoalofthisexercisewastospeakupandchallengethesurgeonsuccessfully.ThisteamisdiscussedinSection 4.1 .Thesecondstudyintroducedanadditionalpopulation,namelysurgicaltechnicians.Inthesecondstudy,nursesandsurgicaltechniciansworkedtogetherwithavirtualsurgeonandavirtualanesthesiologisttoprepareasimulatedpatientforsurgery.Theagencyofthesurgeonandtheanesthesiologistwasheldconstant,whiletheagencyofthenurseandsurgicaltechnicianwasvaried.Allnursesandsurgicaltechnicianswereactualparticipants,notactors.Eitherahumannurseworkedwithavirtualsurgicaltechnician,ahumansurgicaltechnicianworkedwithavirtualnurse,orboththenurseandthesurgicaltechnicianwerehuman.Duringthetrainingexercise,theteam'ssurgeonmadetwodecisionsthatcouldpotentiallyendangerthesafetyoftheteam'spatient.Thisrequirednursesandsurgicaltechnicianstospeakuptothesurgeoninordertoprotectthepatient.Thetraininggoalofthisexercisewastospeakupandchallengethesurgeonsuccessfully.Oneofthesurgeon'sdecisionswasdirected 23

PAGE 24

towardsnursesandtheotherwasdirectedtowardssurgicaltechnicians.ThisteamisdiscussedinSection 4.2 .Behavioraldatawasalsogatheredinbothstudies,usingvideosofparticipants.Fourkeymomentswerepresentinthesetwostudiesthatwereusedtoexplorehowagencyaffectedthewayparticipantsmadedecisionswiththeirteammates.Twoofthesedecisionmomentswerecontentneutral,whiletheothertwodecisionsinvolvedconictwiththesurgeonoverapatientsafetyissue.Eachofthesemomentsweredevelopedinconjunctionwithtrainedmedicalprofessionals.BehaviorduringthesemomentsisdiscussedinChapter 5 .Non-verbalbehaviorwascodedandanalyzedusingthevideodatacollectedduringtherststudy.ThisworkisdiscussedinChapter 6 .Finally,socialpresencedatawascollectedduringbothstudiesusingavalidatedsurveyinstrument[ 22 ].Thisdatawasanalyzedtoexplorewhateffecttheagencyofparticipants'teammateshadontheirperceptionsofsocialpresence.ThisworkisdiscussedinChapter 7 .Therearethreemajordifferencesbetweentherstandsecondstudies.First,thenatureofhumanteammatesasactualtraineeswasaltered.Whiletherststudyusedhumanactors,ratherthanactualsurgeonsandanesthesiologists,thesecondstudyusedactualnursesandactualsurgicaltechniciansasparticipants.Thisallowedtherststudytoachieveahighlevelofexperimentalcontrol,asthehumansurgeonandanesthesiologistwereinstructedtofollowaspecicscript.Thesecondstudythenbuiltontheresultsoftherststudyinasettingwithlesscontrol,buthigherecologicalvalidity,asitmorecloselyapproximatedarealtrainingexercise,asallhumanparticipantswereactuallyreceivingtraining.Second,thepowerrelationshipbetweenhumanteammateswasaltered.Intherststudy,theagencyofdoctorswasvaried,whoaretypicallyperceivedashavingmorepowerthannurses[ 23 ].Inthesecondstudy,theagencyofnurseandsurgicaltechnicianswasvaried,whoaremuchcloserontheperceivedpowerscalesinmosthospitals.Varyingthepowerrelationshipsbetweenstudiesmeansthe 24

PAGE 25

resultsaremorelikelytobegeneralizabletovariousteamsettingswithdifferentpowerrelationships.Finally,thesecondstudyintroducedanadditionalspeakingupmoment,whichallowedspeakingupbehaviortobeobservedinadifferentsetting. 1.5InnovationThisdissertationcontainstwomaininnovations.Therstisthedevelopmentofasystemtosimulatemixed-agencyinteractionswhereanyteammatecanbeplayedbyarealhumanoravirtualhuman,asneeded.InChapter 3 ,Ipresentasystemthatcanbeusedtocreateinteractionswherespecicteammatescanbeplayedbyeitherrealorvirtualhumans,asneeded.Existingvirtualhumanauthoringsystemsmaketheassumptionthataspecicteammatewillalwaysbeeitherrealorvirtual.Thisassumptionisincompatiblewiththeproposeduseofvirtualhumanstodynamicallyreplacemissingteammates,asitwillnotbeknowninadvancewhichteammatesshouldbeplayedbyavirtualhuman.ThesystemIdesignedaccomplishesthisbycreatingavirtualsimulationofallteammates,eventhoseteammateswhoareactuallyhuman.Intheeventthatateammateisactuallyhuman,thevirtualsimulationdoesnotperformactionsautomatically,butwaitsuntilthehumanperformsanaction.Thisactionisthenautomaticallyinterpretedbythesystemandconvertedintoanactionthevirtualsimulationcanunderstandandperform.Intheeventthatateammateisvirtual,thesimulationperformsactionsautomatically,basedontheactionsperformedbyotherteammatesandthestateofthesimulation.Thisdesignallowshumanteammatestoreplacevirtualteammateswhenevertheyareavailable.Thisdissertation'ssecondinnovationispresentinganexplorationofhowoneperson'sbehaviorwithvirtualteammatescanbeaffectedbythepresenceofasecondhuman.Whilemanyresearchershaveexploredhumanbehaviorwithvirtualhumans,themajorityofthisresearchhasfocusedoninteractionsinvolvingonlyasinglehuman(seeFigure 1-2 forexamples). 25

PAGE 26

ASource:Kenny,Patrick,etal.”EvaluationofJustina:avirtualpatientwithPTSD.”Intelligentvirtualagents.SpringerBerlinHeidelberg,2008. BSource:Cassell,Justine.”Bodylanguage:Lessonsfromthenear-human.”GenesisRedux(2007):346-374. CSource:Guadagno,RosannaE.,etal.”Virtualhumansandpersuasion:Theeffectsofagencyandbehavioralrealism.”MediaPsychology10.1(2007):1-22.Figure1-2. Threeinteractionsbetweenonerealhumanandonevirtualhuman Atleastoneresearchprojecthasdeployedvirtualhumansinasettingwheremultiplehumanswerepresent.ThesevirtualhumanswereAdaandGrace[ 24 ],developedbytheUSCInstituteforCreativeTechnology(seeFigure 1-3 ).AdaandGracewereamuseumexhibitdesignedtoincreasechildren'sinterestinscienceandtechnology.However,AdaandGracewerenotusedtoexplorebehaviorwithvirtualhumans.Instead,theresearchfocusedonlanguageunderstanding[ 25 ]onattitudesandscienceandtechnology[ 24 ]. Figure1-3. AdaandGracespeakingwithvisitorsattheMuseumofScienceinBoston,Massachusetts.Source:Traum,David,etal.”AdaandGrace:Directinteractionwithmuseumvisitors.”IntelligentVirtualAgents.SpringerBerlinHeidelberg,2012. 26

PAGE 27

Theresearchinthisdissertationexploreshumanbehaviorwithvirtualhumansinarelationaldynamicthathasnotbeenpreviouslyexplored,namelywhetherthepresenceofasecondhumancanalterbehaviorwithavirtualhuman.ThisisillustratedbytheorangearrowinFigure 1-1 ,wherebehaviorwithavirtualhumaniscomparedwhentheagencyoftheanesthesiologistismodied.Thisquestionhasbothpracticalsignicance,asdiscussedinSection 1.3 ,butalsotheoreticalsignicance.Investigatingtheroleotherhumansplayinourbehaviorwithvirtualhumanscaninformourunderstandingofwhypeoplebehaverealisticallywithvirtualhumans,whichinturncanhelpusbuildvirtualhumansthatevokerealisticbehaviormoreeasily. 27

PAGE 28

CHAPTER2REVIEWOFLITERATUREThischapterreviewstheexistingliteratureconcerningsocialresponsestovirtualhumans,theuseofvirtualhumansandotheragentsforteamtraining,andhowperceivedagencyinuencesperceptionsofandbehaviorwithvirtualhumans.Ialsobrieydiscussmethodsofbringingvirtualhumansintorealenvironments,whichisrequiredtocreatemixed-agencyteams,andgiveanoverviewofteamtraininginthemedicaldomain. 2.1VirtualHumansEmbodiedconversationalagents,orECAs,canbedenedas“animatedsoftwareagentsthatusespeech,gaze,gesture,intonationandothernonverbalmodalitiestoemulatetheexperienceofhumanface-to-faceconversationwiththeirusers”[ 1 ].Forthepurposeofthisdissertation,virtualhumansarethesubsetofECAsthatpossesshuman-likeappearances.Virtualhumanscanvaryintermsofvisualdelity[ 26 ]andbehavioraldelity[ 27 ].Visualdelityisinuencedbyanumberoffactors,includingrenderingstyle[ 26 ],animationquality[ 28 ],displaycharacteristics(e.g.resolution,size)[ 29 ],anddisplaymodality(e.g.monitor,head-mounteddisplay,CAVE)[ 30 ].Behavioraldelitycanalsobeinuencedbyanumberoffactors,includingbehavioralappropriateness[ 22 ],behavioralaccuracy[ 31 ],voicequality[ 32 ],non-verbalbehavior(e.g.nodding,gestures)[ 27 ],gazebehavior[ 33 ],andfacial/emotionalexpressions[ 34 ].Bothvisualdelityandbehavioraldelitycanaffecthowpeoplerespondtovirtualhumans.Kotranzaetal.foundthatincreasingavirtualhuman'svisualrealismincreasedparticipants'acceptanceofthevirtualhuman[ 10 ].Puttenetal.foundthatincreasingavirtualhuman'sbehavioralrealismincreasedfeelingsofsocialpresence[ 35 ]andtheamountofwordsusedwhilespeakingwiththevirtualhuman[ 36 ].However,whilevirtualhumanswithhigherlevelsofvisualandbehavioralrealismaregenerallyperceivedmorefavorably,someexceptionshavebeenfound,particularlywithregardtovisualrealism. 28

PAGE 29

Nowaketal.foundthatagentswithlowanthropomorphism(ahighlystylizedcartoonface)evokedstrongerfeelingsofsocialpresencethanmoreanthropomorphicagents[ 15 ].McDonnelletal.foundthatparticipantsratedcartoonfaceswithlowerrealismasmoreappealingandmorefriendlythanslightlymorerealisticfaces[ 26 ];however,highlyrealisticfaceswereratedassimilarlyappealingandfriendly.Ringetal.foundaninteractioneffectbetweentaskdomain(medicinevs.socialdialog)andvisualrealismonhowanagentwasperceived[ 37 ];lessrealisticvirtualhumanswereperceivedasmorelikableinthesocialdomainbutnotinthemedical.Morerealisticvirtualhumanswereperceivedasmoreappropriateinthemedicaldomain,butnotthesocialdomain.Thevirtualhumansexaminedinthisdissertationpossessedrelativelyhighlevelsofvisualandbehavioraldelity.ThisisdiscussedfurtherinSection 3.5 ,wherethevirtualhumansusedinthisresearcharedescribedindetail. 2.1.1SocialResponsestoVirtualHumansTheseminalworkintheareaofsocialresponsestocomputerswasrstperformedbyNassandReeves[ 38 ].NassandReevesconductedmultiplestudiesexploringwhetherknownpsychologicalndingsconcerninginterpersonalrelationshipsalsoappliedtohumaninteractionswithcomputers,whenprovidedwiththeappropriatesocialcues.Theyfoundevidenceforpolitenessbehavior[ 39 ],gendereffects[ 40 ],personalityeffects[ 41 ],andteambehavior[ 42 ]withcomputers.NassandMoonsuggestpeopledonotconsciouslychoosetobehavesociallywithcomputersandotherobjects,butinsteadthatpeoplemindlesslyapplysocialrulesandexpectationswheneverappropriatesocialcuesareprovided,regardlessoftheirsource[ 39 ].Ifthisweretrue,peopleshouldexhibitsocialbehaviorwithvirtualhumans,providedthattheycangeneratetheappropriatesocialcues.Manyexperimentshaveobservedhumanparticipantsrespondingsociallytovirtualhumans.Socialresponsesincludesocialanxiety[ 43 ],publicspeakinganxiety[ 44 ],socialpriming[ 45 ],socialimitation[ 46 , 47 ],socialinhibitionandfacilitation[ 48 – 29

PAGE 30

50 ],maintaininginterpersonaldistancesinthepresenceofavirtualhuman[ 22 ],experiencingdistresswheninstructedtoharmavirtualhuman[ 51 ],persuasion[ 52 ],gendereffects[ 52 , 53 ],andimplicitracialbiases[ 4 ].Researchershaveexploredhowvisualandbehavioraldelityinuencesocialresponsestovirtualhumans.Gongfoundthatparticipantsexhibitedstrongersocialresponsesasvisualanthropomorphismincreased[ 54 ].Garauetal.foundthatmorerealisticgazebehaviorledtoimprovedcommunicationinanavatar-basedtelecommunicationsystem[ 55 ].Yeeetal.conductedametastudythatfoundaslightincreaseinpositivesocialinteractionsforvirtualagentsthatpossessedhighervisualdelity,butonlyforsubjectivemetrics,notbehavioralmetrics[ 56 ].Bailensonetal.foundthatcopresence(aconceptrelatedtosocialpresence)waslowestwhenvisualdelityandbehaviordelityweremismatched[ 22 ].Garauetal.alsofoundaninteractionbetweenvisualandbehavioraldelity,whereamismatchbetweenthetworeducedsubjectivemeasuresofinvolvementwiththevirtualhuman[ 33 ].AliteraturereviewconductedbyVinayagamoorthyetal.alsosupportedtherelationshipbetweenvisualandbehavioraldelity;whileincreasingbothtogethercanleadtostrongsocialresponsestovirtualhumans,increasingonlyonecanactuallydegradesocialresponses[ 57 ]. 2.1.2VirtualHumansandTrainingVirtualhumanshavebeenusedtodevelopawiderangeoftrainingapplications.CulturaltrainershavebeendevelopedthatusevirtualhumanstoteachsouthIndiangreetingprotocols[ 58 ]andteachculturalawarenessformilitarypersonnelintheMiddleEast[ 59 , 60 ]andNorthAfrica[ 61 ].TheUniversityofSouthernCalifornia'sInstituteforCreativeTechnologieshasusedvirtualhumanstodevelopinterpersonalleadershiptrainers[ 62 ]andnegotiationtrainersforthemilitary[ 13 ].Virtualclassroomshavebeendevelopedtohelpprepareteachersforrealclassroomexperiences[ 12 , 63 ].Virtualhumanshavebeenusedtoteachpoliceofcershowtorespondtothementallyill[ 64 ]. 30

PAGE 31

Virtualpatientshavebeendevelopedtoteachawiderangeofmedicalskills,includingmedicalinterviewing[ 7 , 65 – 67 ],pediatricinterviews[ 9 ],cranialnerveexaminations[ 68 ],breastexams[ 69 ],prostateexams[ 11 ],andPTSDcounseling[ 8 ].Virtualhumansalsobeendevelopedtotrainnewresearchershowtoadministerinformedconsentduringresearchstudies[ 70 ]andtoteachtelephoneinterviewershowtoobtainsurveyparticipation[ 71 ].Evaluationshaveshownthatthesetrainingsystemscanbeeffective.Severalresearchershaveshownthatvirtualpatientsystemscanimprovebehavioralandcognitiveskills,andthattheseskillgainsrivalorexceedskillgainswithtraditionalmethods[ 7 , 10 , 72 , 73 ].Ameta-analysisof12differentvirtual-humanbasedmedicaltrainingprogramsfoundthat,withinspeciccontexts,virtualhumansweremoreeffectivethantraditionallearningmethodswhenteachingclinicalreasoning,datacollection,andinterpretation[ 74 ].Othersystems,suchasthecranialnervepatientsdevelopedbyKotranzaetal.[ 68 ],havebeenusedtogivemedicalstudentshands-onexperiencewithdiseasestheywouldnototherwisehaveencounteredduringtheireducation.Otherresearchershaveshownthatskillsgainedwithvirtualhumanstransfertorealworldsituations[ 70 ]. 2.1.3VirtualTeammatesSeveralvirtualagentshavebeendevelopedforusewithteamtraining.Thesesystemsaredescribedindetailinthissection,withanemphasisonthecontentoftheteamactivity,thevisualandbehavioraldelityofthevirtualhumans,andtheaimsoftheresearch.Steve(SOARTrainingExpertforVirtualEnvironments)wastheoneoftherstvirtualhumansdevelopedforteamtrainingapplications[ 34 , 75 , 76 ].Rickeletal.developedStevetoparticipateinandleadtask-basedteamtrainingsessions,suchasenginemaintenanceonnavalvessels(Figure 2-1 ).MultipletraineescouldinteractwithoneormoreinstancesofStevewithinasharedvirtualenvironmentdisplayedon 31

PAGE 32

anHMD.Steve'smainresearchcontributionwasthedevelopmentofanarchitecturecapableofsupportingtask-basedteamactivities.StevewaspoweredbySOAR[ 77 ],ageneralizablecognitivearchitectureusedtobuildintelligentsystems. A BFigure2-1. TwoSteveagentsinteractwitheachotherwhileahumanobserves.Source:Rickel,Jeff,andW.LewisJohnson.”Task-orientedcollaborationwithembodiedagentsinvirtualworlds.”Embodiedconversationalagents(2000):95-122. KroetzperformedaqualitativeevaluationofSteve'seffectivenessasapedagogicalagentinanon-teamsettingwhereoneinstanceofStevetaughtalearnerhowtooperateanHPACsystem[ 78 ].Tenlearnersparticipatedinthisstudy.KroetzfoundthatparticipantswereabletooperatethesystemwithaminimalamountoferrorsafterreceivinginstructionfromSteve.HealsofoundthatparticipantsgenerallyperceivedStevefavorably.Traumetal.developedvirtualhumanscapableofteam-basednegotiation[ 14 ].Thesevirtualhumans,whichwerebasedontheStevesystem,incorporatedemotionalexpressions/behavior,negotiationbehaviors,andaperceptualmodelthatlimitedthevirtualhumans'understandingoftheirenvironmenttowhatarealhumanwouldbeabletoperceive.ThesevirtualhumanswereusedintheMissionRehearsalExercise[ 62 ], 32

PAGE 33

ateamtrainingscenariointendedtoteachcriticalthinkingskillstosmall-unitleadersintheU.S.Army(Figure 2-2 ).Thescenariorequiredatraineetodecidehowtorespondtoanaccidentalcollisionwithacivilianvehiclewhichinjuredasmallchildinalocationwithcivilunrest.Traineesinteractedwiththevirtualhumanusingalargepanoramicprojectorwall.Onlyonehumantraineeparticipatedineachexercise.Traumetal.didnotreportanevaluationofhumanbehaviorintheMissionRehearsalExercise.Instead,theresearchfocusedonthedevelopmentofvirtualhumanscapableofteam-basednegotiation. Figure2-2. TheMissionRehearsalExercisesystem.Source:Traum,David,etal.”Negotiationovertasksinhybridhuman-agentteamsforsimulation-basedtraining.”ProceedingsofthesecondinternationaljointconferenceonAutonomousagentsandmultiagentsystems.ACM,2003. Pradaetal.developedtheSyntheticGroupDynamicsmodel(SGD)toenablevirtualhumanstoexhibitrealisticgroupdynamicsinteam-basedcollaborativegames[ 79 , 80 ].TheSGDmodelsimulatessocialstructurewithinagroupusingfourcomponents:individualcharacteristicsofspecicagents(e.g.personality),groupstructureandhierarchy(e.g.relationshipsbetweengroupmembers,includingattractionandinuence),classesofinteractionspossiblewithinthegroup(e.g.socialorproceduralactionsthatcanbeperformed),andenvironmentalcontext.Pradaetal.arguethatagentsrequireknowledgeaboutthesefourcomponentstogeneratebelievablesocialbehavior. 33

PAGE 34

A BFigure2-3. TwoimagesfromthePerfectCirclegamedevelopedtotesttheSGDmodel.Source:Prada,RuiFilipeFernandes.Teaminguphumansandsyntheticcharacters.Diss.UniversidadeTecnicadeLisboa,2005. PradadevelopedacollaborativegametoevaluatetheSGDmodel(Figure 2-3 ).Participantsworkedwithateamofautonomousagentstosolveapuzzleinvolvingthemanipulationofgemstoneswhichmustbecombinedusingvariousabilitiestocreateaspeciccombinationstone.Pradaconductedastudyinwhichhevariedthemodelusedtosimulategroupdynamics(SGDortask-based)andtheinitialsocialmakeupofthegroup(neutralcohesionorlowcohesion).Thetask-basedmodeldidnotincludeanyrulesgoverningsocialinteraction,andinsteadonlygovernedtaskperformance.Pradaobservedtheeffectofthesevariablesongrouptrust,groupidentication,socialpresence,andsatisfaction.Eachofthesevariablesweremeasuredusingsurveys.Twenty-fourparticipantstookpartinthisexperiment,eightineachcondition.PradafoundthattheSGDmodelimproveduser'sgrouptrustregardlessoftheinitialgroupcohesion.Italsoproducedmoreidentication,thoughthisdependedonthegroupcohesion.Participantsinteractedwiththeagentsthroughawindowedapplication.Participantsdidnotworkwithotherhumansduringthistask.Othernon-embodiedagentshavebeendevelopedforuseinteamtrainingscenarios.Balletal.developedasyntheticteammatecapableofplayingtheroleof 34

PAGE 35

pilotduringanunmannedaerialvehicle(UAV)teamtrainingexercisewheretheteamwascomprisedofthesimulatedpilot,thepayloadoperator(controlstheUAV'ssensors),andthenavigator[ 81 , 82 ].Participantscommunicatedwiththesimulatedpilotviaasimulatedsensordisplayandaninstantmessagingsystem(Figure 2-4 ).ThisvirtualteammatewasdevelopedusingtheACT-Rcognitivearchitecture.LikeRickelandTraum,themainfocusofthisresearchwasthedevelopmentofacognitivemodel,nottheevaluationofhumaninteractionswithit. A BFigure2-4. TheSyntheticTeammateinterfacewassplitintotwodisplays.Therstdisplay,ontheleft,showedthetaskenvironmentcontaininginformationabouttheUAV.Theseconddisplay,ontheright,showedthechatclientsandsensorcontrolsystem.Source:Ball,Jerry.“SyntheticTeammateProjectBrochure-PALM”Web20Aug.2015.http://palm.mindmodeling.org/synwiki/images/d/dd/Synthetic-Teammate-Project.pdf AspartoftheSyntheticTeammateproject,DemirandCookperformedanexperimentwhereteamsofthreehumansworkedtogethertocontrolasimulatedUAV[ 83 ].Twoofthehumanswerelocatedinthesameroom,whileathirdteammate,whoplayedtheAirVehicleOperator,wasinaremotelocation.Dependingoncondition,thetwohumanparticipantswhowereinthesameroomweretoldthattheAVOwaseitherarealhumaninaremotelocationorasyntheticagent(theAVOwasactuallyarealhumaninbothconditions).DemirandCookthenevaluatedperformanceandperceptionsoftheAVObetweenconditions.Twentythree-personteamstookpartinthis 35

PAGE 36

evaluation.Thisevaluationrevealedthatpeoplelikedtheperceivedagentmorethanthehumanpilot,thoughttheirworkloadwaslowerwhenworkingwiththeperceivedagent,andgavetheperceivedagentmoresuggestions.Remolinaetal.developedtwonon-embodiedagent-basedteam-trainingsystems.TheCITTPsystem(Figure 2-5 ,onleft)wasanintelligenttutoringsysteminvolvingvirtualagentscapableofworkingwithandprovidingfeedbacktoNASAtechnicians[ 84 ].ThelaterPORTSTAO-ITSsystem(Figure 2-5 ,onright)employedsimilartechniquestocoachnavalofcersontacticalexercises[ 85 ].Remolinaetal.alsofocusedonthedevelopmentofcognitivemodelscapableofteamworkanddoesnotincludeanevaluationofthesetwoagent-basedsystems. ASource:Remolina,Emilio,JianLi,andAlanE.Johnston.”TeamTrainingwithSimulatedTeammates.”TheInterservice/IndustryTraining,SimulationEducationConference(I/ITSEC).Vol.2005. BSource:Remolina,Emilio,etal.”Rehears-ingNavalTacticalSituationsUsingSimulatedTeammatesandanAutomatedTutor.”LearningTechnologies,IEEETransactionson2.2(2009):148-156.Figure2-5. TheCITTPandPORTSTAO-ITSsystems Themajorityofaboveresearchconcerningvirtualteammateswasfocusedonlearningtobuildthem.Eachoftheaboveresearchershavefocusedprimarilyonhowtosimulateteammatescapableofintelligenttaskperformanceandcommunication.Evaluationslookingathowhumansinteractwiththeirvirtualteammateshaveeitherbeenlimitedinscope,suchasthatperformedbyKroetz[ 78 ],orhavenotbeenreported 36

PAGE 37

atall,suchasfortheMissionRehearsalExercise,theSyntheticTeammateproject,andthesystemsdevelopedbyRemolina.TheresearchbyDemirandCookdidevaluatehowpeoplebehavedandevaluatedateammatewhentheirperceptionofitsagencychanged[ 83 ].However,thisexperimenttookplaceinasettingwithverylowembodiment,whereparticipantsinteractedwiththepilotthroughsimulatedsensorreadingsandaninstantmessagingsystem.Asstatedearlier,resultspertainingtolowimmersionsystemsmaynotapplytointeractionswhetherhumanandvirtualteammatesinteractwithinasharedenvironment.Theresearchdiscussedinthisdissertationisconcernedprimarilywithevaluatingbehaviorwithvirtualteammates,asopposedtobuildingmodelsthatcansimulatevirtualteammates.Assuch,itcomplementsthisexistingresearch. 2.2AgencyandVirtualHumansTheterm“agency”isusedtodistinguishbetweenhumanandnonhumanentities[ 15 ].Withintherealmofvirtualagents,thetermavatarisusedforvirtualcharactersthatarecontrolledbyahuman,andthetermagentisusedforvirtualcharacterscontrolledbyacomputer.Thisdissertationalsousesthetermagencytodistinguishphysically-presenthumansfromavatarsandagents.Themajorityofpriorresearchconcerningagencyhascomparedagentstoavatars.Comparativelylittleresearchhascomparedagentstophysicallypresenthumans.Agencyresearchoftenvaries“perceivedagency”ratherthanactualagency,wherewhatparticipantsaretoldaboutaspeciccharacter'sagencyismanipulatedwhilethecharacter'sactualagencyisheldconstant.Thisistypicallydoneforpurposesofexperimentalcontrol.Lucasetal.performedastudywherebothactualagencyandperceivedagencywerevaried,andfoundthatactualagencyhadnoeffect,whileperceivedagencyimpactedself-disclosureandimpressionmanagementbehavior[ 86 ].Theresearchdescribedinthisdissertationalsovariedperceivedagency,asvirtualteammateswerecontrolledremotelybyahumanusingaWizard-of-Ozdesign(describedmoreinSection 3.6 ).Technically,thismakesthevirtualteammatesusedin 37

PAGE 38

thisresearchavatars.However,thevirtualteammateswerepresentedtoparticipantsasvirtualagentscontrolledbyacomputer.Twocompetingtheoriesattempttodescribetheroleagencyplaysduringinteractionswithvirtualagents[ 15 ].TheHumanPrimacytheoryholdsthat“humanitymatters”,andthatpeoplewillinherentlyfeelmoreconnectedtoandmoreopentoinuencefromavatars,comparedtoagents[ 87 ].ThistheoryisreectedinBlascovich'sThresholdModelofSocialInuence,whichpredictsthatperceivedagencymoderatestheefcacyofsocialstimuli[ 88 ].Inotherwords,humansaremoreresponsivetosocialstimuliwhentheybelieveitcomesfromanavatarratherthananagent.Thisholdstrueregardlessofwhatactuallycontrolsthevirtualhuman.Accordingly,agentsmustexhibithigherlevelsofbehavioralrealisminordertoevokethesamelevelofsocialbehaviorevokedbylowerqualityavatars.Incontrasttothis,theComputersasSocialActorstheory,alsoknownastheMediaEquation,holdsthatpeopleautomaticallyrespondsociallytoentitiesthatlookorbehavehuman[ 39 , 89 ].TheMediaEquationpredictsthatagencyhasnoinherenteffectonperceptionsoforbehaviorwithvirtualagents.Agencyonlymatterstothedegreethatitchangesanagent'sabilitytobehavesocially.Avatars,whicharecontrolledbyahumanandarethusgenerallymorecapableofrealisticbehavior,mayevokestrongersocialresponsesthanagentsbutthisisduetothedifferenceinbehavior,notthedifferenceinagency.Allotherthingsbeingequal,theMediaEquationpredictsthatagentsandavatarswillbothevokethesamedegreeofsocialbehavior. 2.2.1AgencyandSocialPresenceResearchremainsconictedabouttheeffectagencyhasontheexperienceofsocialpresence.Nowaketal.[ 15 ]andVonderPuttenetal.[ 35 ]bothexploredtheeffectofagencyonfeelingsofsocialpresence.WhileNowaketal.foundthatanagent'sdegreeofanthropomorphisminuencedsocialpresence,perceivedagencydidnot. 38

PAGE 39

Similarly,VonderPuttenetal.foundthatwhilebehavioralrealisminuencedsocialpresence,perceivedagencydidnot.Incontrast,Bailensonetal.[ 22 ],Guadagnoetal.[ 90 ],andAppeletal.[ 91 ]foundthatavatarsevokedstrongerfeelingsofsocialpresencethanvirtualagents.GuadagnoandAppelbothreportpartialeta2effectsizesof0.051and0.112respectively.Theseeffectsizescorrespondtosmalltomediumeffectsizes.ThedifferencesobservedinBailenson'sexperimentwereofsimilarmagnitudes,thoughaneffectsizewasnotreported.Bailensonfoundthatmutualgazeincreasedfeelingsofsocialpresencewithvirtualagents,buthadnoeffectwithavatars.Guadagnoalsoobservedanincreaseinsocialpresencewhenbehavioralrealismwasincreased,thoughthisincreasewasobservedwithbothavatarsandagents.Finally,Appelfoundthatincreasingsocialcuesalsoincreasedalsoincreasedsocialpresence(thiseffectwasalsoobservedwithbothagentsandavatars).Insum,itappearslikelythatagencymayhaveastatisticallysignicanteffectonsocialpresence,butthattheeffectisrelativelysmall.However,thepriorresearchsummarizedabovecomparedvirtualagentstovirtualavatars,virtualcharacterscontrolledbyahuman.Themagnitudeofthedifferenceinsocialpresencebetweenavirtualhumanandarealhumanhasnotyetbeenexplored. 2.2.2AgencyandBehaviorSimilarly,resultsremainconictedonwhateffectagencyhasonbehaviorwithvirtualagents.Appeletal.[ 91 ],whowaspreviouslycitedasobservingasignicanteffectofagencyonsocialpresence,foundnoeffectofagencyonseveralbehavioralmeasures,includingmutualawareness,positiveperceptions,andattentionalallocation.Eachofthesemetricswasaffectedbybehavioraldelitybutnotagency.VonderPuttenetal.[ 36 ]similarlyfoundthatagencyhadlittleeffectonbehavioralmetrics,whilebehavioraldelityhadsignicanteffects. 39

PAGE 40

Incontrast,otherresearchershaveobservedbehavioraldifferencesbetweenagentsandavatars.Agencyresearchcommonlyemploys2x2studies,whereagencyandbehavioralrealismarebothvaried.Severalresearchershavefoundthatvaryingbehavioralrealismaffectedbehaviorwithavirtualagent,buthadnoeffectonbehaviorwithaperceivedavatar.Bailensonetal.[ 22 ]foundthatincreasingavirtualagent'sbehavioralrealism,intheformofgazebehavior,causedpeopletomaintainmorerealisticinterpersonaldistances,whilebehavioralrealismhadnoeffectoninterpersonaldistancewithavatars.DeMeloetal.exploredtherelationshipbetweenemotionalexpression,agency,andsocialbehaviorduringsocialdilemmasandnegotiations.Theyfoundthatemotionalexpressionsinuencesocialbehavior,butonlyforperceivedavatars.Varyinganagent'semotionalexpressionhadnoeffectonsocialbehavior[ 92 ].Otherresearchershaveexploredtheeffectofagencyonbehaviorwithoutmanipulatingbehavioralrealism.Bailensonetal.[ 22 ]foundthatpeoplemovedfurtherawayfromagentsthanavatarswhentheirpersonalspacewasinvaded.Raijetal.comparedmedicalinterviewswithavirtualpatienttointerviewswithahumanstandardizedpatient[ 93 ].Theyfoundthatparticipants'interviewswithavirtualpatientwerelessstructuredandtheirexpressionsofempathywerelesssincerethanthoseconductedwiththestandardizedpatient.However,Raijobservedthatthesedifferencesmayhavebeenduetolimitationsinthevirtualpatient'sconversationarchitecture,especiallythelackofsupportforcontextualquestions.Lucasetal.variedbothperceivedagencyandactualagency,andfoundthatparticipantsweremorewillingtoself-discloseandengagedinfewerimpressionmanagementbehaviorswithperceivedagentsthanperceivedavatars,butthatactualagencyhadnoeffect.[ 86 ].DeMeloetal.exploredtheeffectofagencyandemotionaldisplaysoncooperativeandcompetitivenegotiationsandfoundthatparticipantsconcededmoretoangryagentsandavatars,butthatthiseffectwasmuchmorepronouncedforavatars[ 94 ].Okitaetal.observedthatparticipantsperformedbetteronaknowledge-basedposttestafterinteractingwith 40

PAGE 41

anavatarthananagent,mostlikelybecauseparticipantsbelievedtheirinteractionwiththeavatarwasasociallyrelevantaction[ 95 ].Finally,Limetal.observedthatvideogameplayersshowedsignsofstrongerphysiologicalarousalwhenplayinggameswithperceivedavatarsthanwithperceivedagents[ 96 ].Insum,itseemslikelythatperceivedagencycaninuencebehaviorwithvirtualagents.However,thisinuenceisamatterofdegreenotkind.Socialbehaviorhasbeenconsistentlyobservedwithvirtualagents,butthedegreeofthiseffectcanbemoderatedbybehavioraldelity.Additionally,thestrengthofaperson'sresponsetoavirtualagentappearstobeinuencedbythesocialconsequencesofthataction;peoplearemorewillingtoself-disclosewhenthereisnoriskofbeingjudgedbyanotherhuman,concedemoretoangryhumansthanangryagents,andlearnmorewhenquestioninganotherhumanthanavirtualagent. 2.3EmbeddingVirtualHumansintheRealWorldWhenusingvirtualhumansforteamtraining,threedifferencespacesexistwithinwhichtrainingcanoccur.Humantraineescanenterintoavirtualspaceoccupiedbytheirvirtualteammates(suchasthroughanHMD),humantraineesandvirtualhumanscanoccupydifferentspaces(suchasinmostdesktopbasedsimulations),orvirtualhumanscanenterintorealworldoccupiedbytrainees.Thelatertwoinstancesallowforthepossibilityofmixed-agencyteams.Researchershavedevelopedatleastthreedifferentmethodsthatcanbringvirtualhumansintotherealworld.Therstmethodleveragessee-throughHMDsorstandardHMDsaugmentedwithco-locatedphysicalprops.ARFacadeusedasee-throughdisplaytoembedtwovirtualhumansinaphysicalspacewheretheycouldinteractwitheachotherandwithparticipants[ 97 ].Kotranzaetal.developedavirtualbreastexampatientthatincorporatedanaugmentedrealitybreastexamsimulator.ThevirtualpatientwasdisplayedonanHMD,whilealivevideofeedofthebreastexamsimulatorwastrackedandoverlaidonthedisplay[ 69 ].Thevirtualpatientwasalsoequippedwith 41

PAGE 42

aphysicalarmthatcouldreachoutandtouchtheparticipant;thisphysicalarmwasco-locatedwiththepatient'svirtualarmshownontheHMD.Thesecondmethodrelieson“digitalats”,displaysstrategicallyplacedatboundarypointsinaphysicalenvironment,suchasdoors,windows,orbehindbarriers.Thesedisplaysgivetheillusionthatthephysicalenvironmentextendspasttheseboundaries.Propsareoftenusedtoobscuretheedgeofthedisplays,whichhelpstocreatetheillusionofasinglesharedspace.DigitalatshavebeenusedtoreplacedoorsandwindowsintheFlatWorldproject[ 98 ],andtosimulatepartsofentireroomsintheGunslingerproject,suchasbehindabar[ 99 ].Thenalmethodemploysaugmentedrealitydisplayslocatedwithinaphysicalenvironment.Theseaugmenteddisplaysdifferfromdigitalatsinthattheyareplacedwithinthephysicalenvironmentinsteadofalongitsboundaries.Ratherthansimulatinganadjacentbutseparatespace,augmenteddisplaysemploysee-throughbackgroundsandperspective-correctrenderingtogivetheimpressionthatvirtualcontentisactuallypresentintheroomatthelocationofthedisplay.Chuahetal.usedaugmenteddisplaysinseveralmedicaltrainingprograms[ 100 ].Eachofthesemethodshasdifferentadvantagesandlimitationswhenusedtoembedvirtualhumansinshared,physicalenvironments.Whentheyfunctioncorrectly,see-throughHMDscanprovideaverydelityillusion.However,theseoftenrequirecomplextrackingsetupsthatcanbedifculttocalibrate.Digitalatsaresimpleandeffective,butarerarelyportable.Augmenteddisplayscanbemovedmoreeasilybutcannotbeusedwithmultipleviewersbecauseperspective-correctrenderingisspecictoasingleviewer.Intheend,theconstraintsofthescenariobeingsimulatedwilldeterminewhichapproachwillbemostuseful.Augmentedrealitydisplayswereusedfortheworkdescribedinthisdissertation.Augmentedrealitydisplaysstrikeabalancebetweenconvenienceandavailabilitythatmakesthemappropriateforteamtraining.Digitalatsareinexibleandthusrequire 42

PAGE 43

dedicatedspacethatmaynotbeavailableatmanyinstitutions.Similarly,see-throughHMDsarestilluncommonandcanrequirecomplextrackingequipment.Augmentedrealitydisplaysarebothportable,inexpensive,andeasytooperate,whichmakesthemthebestoptionformanytypesofteamtraining. 2.4MedicalTeamTrainingSalasetal.denesateamas”twoormoreindividualswithspecicrolesinteractingadaptively,interdependently,anddynamicallytowardacommonandvaluedgoal”,anddenesteamworkas”asetofinterrelatedthoughts,actions,andfeelingsofeachteammemberthatareneededtofunctionasateamandthatcombinetofacilitatecoordinated,adaptiveperformanceandtaskobjectivesresultinginvalue-addedoutcomes”[ 101 ].Mostmedicalteamtrainingprogramstintooneoftwocategories:simulation-basedprogramsorclassroom-basedprograms[ 102 ].Simulation-basedprogramsfrequentlycenteraroundpatientsimulators,suchastheHumanPatientSimulator(HPS)[ 103 ]orSimMan[ 104 ].Examplesofvirtualreality-basedmedialteamtrainingalsoexist[ 105 ].Incontrast,classroom-basedprogramsrelyprimarilyontraditionalclassroom-basedmethods,suchaslectures,videos,orroleplay[ 102 ].Severalwidelyusedprotocolsforteamtrainingexistwithinthemedicalcommunity,includingAnesthesiaCrisisResourceManagement(ACRM)[ 106 , 107 ],TeamStrategiesandToolstoEnhancePerformanceandPatientSafety(TeamSTEPPS)[ 108 ],EMCrisisResourceManangement(EMCRM)[ 109 ],Team-OrientedMedicalSimulation(TOMS)[ 110 ],andMedTeams[ 111 , 112 ].Whilethemajorityofthesetrainingprotocolsincludebothclassroom-basedmethodsandsimulation-basedmethods,manyemphasizeonemethodovertheother.IbrieydiscussACRMasanexampleofasimulation-centricmethodandTeamSTEPPSasaclassroom-centricmethod.ACRMwasdevelopedininthelate80'sandearly90'sbyresearchersinStandford'sDepartmentofAnesthesiology[ 106 ].ACRMspecicallyfocusesontraininganesthesiologistshowtoworkwithotherteammembersduringacrisis.ACRMhasevolvedovertimeinto 43

PAGE 44

astandardmulti-daysimulation-basedcurriculum[ 107 ].Trainingexercisesarebasedonacatalogofover60differentcriticalincidentsinanesthesia[ 106 ].Anesthesiologistsreceivingtrainingworkwithothermedicalprofessionals,androtatethroughseveraldifferentpositionsduringthetrainingexercises,includingrolestheywouldnotnormallyplay,suchasasurgicaltechnician.Thisisintendedtohelpgiveanesthesiologistssomeperspectiveonwhatotherteammatesexperienceduringacrisis.ACRMalsostronglyemphasizesdebriengsaboutthetrainee'sexperiencesduringtraining.ThegoalsofthesedebriengsaretoexplorealternatesolutionsandtodiscusshowtheACRMprincipleswereappliedduringthetrainingexercise.TeamSTEPPSwasdevelopedaspartofaninitiativebytheDepartmentofDefence(DoD)andtheAgencyforHealthcareResearchandQuality(AHRQ)inthemid2000's[ 108 ].TeamSTEPPSisamoregeneraltrainingprogramthanACRM,asittargetsallhealthprofessionals,notjustanesthesiologists.TeamSTEPPSfocusesonfourcorecompetencies:leadership,situationmonitoring,mutualsupport,andcommunication.Traineesreceiveinstructiononthehistoryofteamtrainingandteamformation,andthenthencompletefourmodules,eachofwhichfocusononeofthefourcorecompetencies.Specialemphasisisplacedonbarrierstoteambehavior(e.g.defensiveness,conventionalthinking,distrations,fatigue)andontoolsandstrategies(briengs,huddles,feedback,closed-loopcommunication).TeamSTEPPSalsoteachesseveralprotocols/mnemonicsaimedatspecicsituations,suchasspeakingupeffectivelyaboutconcerns.Inthiscase,TeamSTEPPSprovidestheCUSprotocol,whichteachestraineestousethewords“Concerned”,“Uncomfortable”,and“Safety”whenraisingconcernsaboutpatientsafetyissues.TeamSTEPPSalsoincludesmechanismstoassessandimprovehealthcareatanorganizationallevel[ 108 ].Thisisamajordifferencefromothertrainingprotocols,likeACRM,whicharemorefocusedontrainingindividuals.AcompleteTeamSTEPPSimplementationincludesanassessmentphase,whereanorganizationisassessedtoidentifykeyleadersandchampionsfor 44

PAGE 45

change,alongwithpotentialbarriersandresourcesthatwillberequiredtoimplementchange.Aftertheassessmentphase,trainingbeginswithindividualdepartmentsandhealthprofessionals.Thistrainingcanbecustomizedtotheneedoftheindividualdepartments.TeamSTEPPSincludesathirdandnalphasethatemphasizessustainingchangewithintheorganization.Thisincludesmonitoringongoingchangeandprovidingfeedbacktoteamswithintheorganization.Whilesimulationhasbeenwidelyacceptedwithinthemedicalcommunityandisfrequentlyusedtoconductteamtraining,virtualhumanshaverarelybeenusedwithindomainofmedicalteamtraining(seeSection 2.1.3 foradiscussionofvirtualteammatesinotherdomains).Theuseofvirtualhumansformedicaltraininghaslargelyfocusedonclinicalskillstraining[ 7 , 8 , 10 ],whichfocusesontheinteractionbetweendoctorandpatient. 45

PAGE 46

CHAPTER3CREATINGMIXED-AGENCYINTERACTIONS 3.1AuthoringMixedAgencyTeamsInthischapter,IdiscussthesystemIdevelopedtoauthormixed-agencyteams.Themainchallengewiththisworkwasdetermininghowtorepresentcharacterswhocouldbeplayedbyeitherahumanoravirtualhuman,asneeded.Irefertothesecharactersas“variable-agency”characters,orvariable-agencyteammates,astheiragencycanbevariedatruntime.IalsodiscussthetechnologythatwasusedtorendervirtualhumansandembedtheminaphysicalenvironmentfortheteamtrainingscenariosIdeveloped.ThevirtualhumanauthoringsystemIdeveloped,whichIrefertoasVirtualPeopleFactory2.0(VPF2),extendsapreviousmodelofvirtualhumanscreatedbyRossen[ 113 ].Thismodelwasdevelopedtosupportquestion-answeringvirtualhumans,specicallyvirtualpatientswhocouldbeinterviewedbymedicalstudents.Assuch,thismodelmadetwoassumptionsthatwereincompatiblewithmyresearchgoals:allconversationscontainedexactlytwomembers(e.g.adoctorandapatient),andtheagencyofthesememberswasalwaysxed(e.g.thedoctorwasalwayshuman,thepatientwasalwaysvirtual).Neitheroftheseassumptionswerecompatiblewithmyresearch,asIneededtosimulateteamscontainingmorethantwomembersandneededtovaryingeachteammember'sagency. 3.1.1RepresentingandSimulatingVirtualHumansVirtualhumansareoftenrepresentedasconversationmodels[ 113 ].Conversationmodelsdeterminewhatavirtualhumancansay,andwhenavirtualhumanshouldspeak.Therulesgoverningwhatavirtualhumancansaymayinvolvenaturallanguageprocessing,whichallowsvirtualhumanstogeneratespeechesatruntime,ortheymaysimplybeasetofpre-generatedspeechesthatavirtualhumancanutter.Whichofthesemodelsisappropriatedependsonthedomainwithinwhichavirtual 46

PAGE 47

humanoperates.Rossendescribesthreedomains:open-domain,domain-specic,andconversation-specic[ 114 ].Open-domainagents,suchasApple'sSiri,anddomain-specicagents,suchasanairlinereservationagents,frequentlyuserulestogeneratespeechatruntime,duetotheirneedtoadaptresponsestoauser'sspecicrequest.Incontrast,conversation-specicagents,suchasavirtualpatientwithcranialnervedamage,frequentlyusepre-generatedsetsofspeeches,becausethereislittlevariationbetweenusers.AsthegoalofVPF2istocreatevirtualhumansforuseinspecictrainingexercises,thevirtualhumansauthoredwithVPF2areconversation-specicandthususepre-generatedsetsofspeechestocontrolwhatvirtualhumanscansay.Thelistbelowgivesanexampleofpre-generatedspeechesfromavirtualpatient. Hellodoctor,mynameisMollySmith. I'vehaddoublevisionsincethismorningandI'mreallyworriedaboutit. Myneighborbroughtmein.Drivingdidn'tseemlikeagoodidea. Ihaveadaughter.ShelivesinAtlanta.Manydifferenttypesofrulescanbeimaginedtogovernwhenavirtualhumanshouldspeak:internalmodelingofagentstate(e.g.I'mhungry),stimulusfromanexternalenvironment(e.g.didyoujustseethatcardrivepast?),usingatimer(e.g.we'vebeentalkingforveminutes,canwewrapthisup?),orevenrandomly.However,forconversation-specicagents,themostimportanttypeofruleisrespondingtowhatanotherconversationpartnerhassaid(e.g.Human:Hi,howareyou?Agent:I'mfeelingsick.)ThesystemusedbyVPF2toenablevirtualhumanstorespondtoconversationpartnersreliesonacorpusofpossiblespeechesmadebyaconversationpartnerthatarepairedwiththeappropriatevirtualresponse.Whenaconversationpartneruttersaspeech,thisspeechiscomparedagainstthecorpusofknownspeechesandthemostappropriatespeechisselected,whichthentriggersthevirtualhumantorespondwiththepairedspeech.Aclusterofnaturallanguageprocessingtechniquesareusedto 47

PAGE 48

matchanactualinputspeechtospeechesinthecorpus[ 115 ].ThisprocessisillustratedinFigure 3-1 . Figure3-1. Theprocessbywhichahumanspeechtriggersavirtualhumantorespond. 3.1.2ConversationModelsforMixed-AgencyTeamsAfundamentalmodicationtoRossen'soriginalmodelwasrequiredtosupportmixed-agencyteams.WhileRossenhadrepresentedahumanspeechandavirtualhumanresponsewithinasingledatastructure,VPF2decoupleshumanspeechesfromvirtualhumanresponses,andinsteadrepresentsallmembersofaconversationasinstancesofthesameunderlyingdatastructure.Eachmemberofaconversationisrepresentedasadiscrete“character”,whoiscapableofperformingaspecicsetofactions(predominantlyspeeches).AnexampleofascenariocontainingthreecharactersisshowninFigure 3-2 . Figure3-2. Threeseparatecharacters,eachwiththeirownactions. Multiplephrasingscanbeassociatedwithspecicspeeches.Thesephrasingsmakeupthecorpusofknownspeechesthatareusedtomatchaspecicspeech 48

PAGE 49

utteredbyahumantoacorrespondingactionintheunderlyingrepresentation.Phrasingsareonlyimportantformatchinghumanspeeches,meaningthatphrasesarenotrequiredforcharacterswhoseagencywillalwaysbevirtual.However,allcharactersarecapableofhavingphrasesadded.Anexampleofphrasings,andtheiruseformatchingisshowninFigure 3-3 .Inthisgure,twoofthenurse'sactionsareshown,alongwithseveralphrasingsassociatedwitheachofthoseactions.Whenanactualspeechiscomparedagainstthephrasings(“Clayisheretohavehisscoliosisxed”),thephrasingthatmatchesthisspeechmostclosely,accordingtothealgorithmdescribedin[ 115 ],isregistered(“Clayisheretohavehisscoliosisrepaired”).Rossenalsorepresentedhumanspeechesusingphrasings,whichhereferredtoas“triggers”,butphrasingswererestrictedtooneofthetwocharacters.Allowingallcharacterstopossessphrasingsenablesallcharacterstopotentiallybeplayedbyahuman,arequirementformixed-agencyteams. Figure3-3. Aninputspeechismatchedagainstthenurse'scorpusofphrasings.Themostappropriatephrasingisselectedandsenttotheothervirtualteammates. InVPF2,relationshipsbetweenactionsareusedtocontrolwhendifferentcharacterscanspeak.Rossen'smodeldirectlyconnectedonehumanspeechtoonevirtualspeech,anddidnotallowvirtualspeechestotriggerothervirtualspeeches. 49

PAGE 50

VPF2allowsanyactiontobeconnectedtoanotherotheraction,eitherownedbythesamecharacterorbyothercharacters.Thissystemallowsvirtualhumanstospeakwitheachotherwithouthumaninvolvement,andallowslimitedsupportforbranchingbasedonhumanactions.Thisalsoallowsvirtualhumanstoinitiatespeeches,ratherthanwaitingforahumantotaketherstaction.Eachofthesefeatureswasalsorequiredtosimulatemeaningfulmixed-agencyinteractions. 3.2SimulatingMixed-AgencyTeamsWhensimulatingamixed-agencyinteraction,thesystemmustprocesshumanactionsandcommunicatethemtovirtualteammates,aswellassimulatingactionsperformedbyvirtualteammates.Eachmemberofamixed-agencyteamhasavirtualrepresentationwithinthesimulation,includinghumanteammates.Whenahumanperformsanaction(e.g.sayssomething),thisactionisprocessedbyahumaninterpreter,andtheactionismatchedagainstthecorpusofpossibleactionsandphrasings.Onceamatchisselected,thevirtualrepresentativeofthehumanteammateisinformedthatitneedstoperformthematchedaction.Uponperformingthematchedaction,thevirtualrepresentativeinformsallothervirtualteammatesthatanactionhasbeenperformed.Thisallowsthevirtualteammatestoperformanactionbasedonwhatthehumanjustdid.ThisprocessisillustratedinFigure 3-4 .Whenperforminganaction,avirtualteammateinformsalloftheothervirtualteammatesthatanactionhasbeenperformedandthensendsinstructionstoaninterfacesystemthatcommunicatestheactionsperformedtohumanteammates.Thisinterfacecantaketheformofatext-basedinputsystem,orarenderingsystemthatdisplaysananimatedimageofthevirtualteammateandcanplayaudiocorrespondingtothevirtualhuman'sspeech(thestudiesdescribedinthisdissertationuseananimatedrepresentation).ThisprocessisillustratedinFigure 3-5 . 50

PAGE 51

Figure3-4. Theprocessbywhichahumanspeechtriggersavirtualhumantorespond. Figure3-5. Theprocessbywhichavirtualhumansperformsanaction. 3.3TimerequiredtoauthorvirtualhumansinVPFandVPF2Variable-agencycharactersdorequiremoreefforttoauthorthancharactersthatarealwaysvirtual.Oneofthemosttimeconsumingpartsofdevelopingavirtualhumanisthecollectionandorganizationofthedifferentphrasingspeoplemayusewhenspeakingtovirtualhumans.However,itisimportanttonotethatVPF2doesnotrequireall 51

PAGE 52

teammatestobevariable-agency.Itisuptotheauthortodeterminewhichteammatesneedtosupporthavingtheiragencychangedatruntime.Whenthisisnecessary,theauthormustcreateasetofphrasingsthatareusedtotranslateahuman'sspeechintoacorrespondingvirtualaction.Itispossibleforanauthortogobacktoascenarioafterdeployingitandconvertvirtualteammatesintovariable-agencyteammates.Thiscanbedoneatanypointduringtheauthoringprocessbyaddingphrasingstoacharacter'sspeeches.Ifanauthorchoosestoonlyhaveonevariable-agencyteammate,thenVPF2willrequirethesameamountoftimerequiredtocreateavirtualhumanusingtheoriginalVPF,whichalsosupportedonecharacterbeingplayedbyahuman(thoughunlikeVPF2,thischaracterwasalwaysrequiredtobehuman). 3.4DevelopingVirtualPeopleFactory2.0VPF2.0wasdevelopedusingASP.NetMVC4.ThesimulationenginewaswritteninC#.VPF2currentlycontainsapproximately13,400linesofcode.IdesignedanddevelopedthecorecomponentsofVPF2,includingtheauthoringinterface,thecoredatastructures,andthesimulationengine.SeveralothersassistedwiththedevelopmentofVPF2.0:MichaelBorishdevelopedfeaturesforhisresearchrelatedtohuman-assistedscriptdevelopmentandassessment,AndrewCordardevelopedapagewhichallowspeopletointeractwithvirtualhumansinawebbrowser,andDiegoRiveradevelopedourvirtualhumanrenderingsystem.Asofthetimeofthiswriting,VPF2.0canbeaccessedonlineatvpf2.cise.u.edu. 3.5VirtualHumanDisplayTechnologyVPF2canbeconnectedtoseveraldifferenttypesofdisplaytechnology.Thetwomainclassesaredesktop-basedinteractions,whereauserinteractswithvirtualhumansviatextinawebbrowser,andlife-sizedinteractions,whereusersinteractwithvirtualhumansdisplayedonlargedisplays,suchthattheyareapproximatelylife-size.Iusedthelatterduringtheresearchdescribedinthisdissertation. 52

PAGE 53

Figure3-6. Threevirtualhumans Thevirtualhumansusedinthisresearchwerelife-sizeandinteractedwithparticipantsusingspeechandgesture(seeFigure 3-6 ).Thevirtualhumans'speecheswereprerecordedbyvoiceactors,andgestureswerecreatedusingmotioncapture.Thevirtualhumansweredisplayedusinghigh-physicalityinteractionmodules,asdescribedbyChuahetal.[ 100 ].Thevirtualhumanswererenderedlife-sizeon40”1080ptelevisionsandcouldtrackparticipants'positionsusingaMicrosoftKinect,whichallowedthemtomakeeyecontactwithparticipants.Perspective-correctrenderingwasusedtocreateanillusionofdepthandsee-throughbackgroundsweresimulatedusingpre-capturedpanoramicimagesoftheenvironment;theuseofperspective-correctrenderingalsoallowedthesee-throughbackgroundtoadjustbasedonparticipants'positionintheroom.Thevirtualhumans'headgazewascontrolledthroughasimpleMarkovmodel;thevirtualhumanslookedatwhoeverwasspeaking,butcouldrandomlyglanceattheotherteammembers.Theyalsoblinkedandmimickedidlemotionswhennotspeaking.Idleanimationswerecreatedusingmotioncapture.ThistechnologywasdevelopedduringmyPhDinconjunctionwithseveralothergraduatestudents.JoonChuahledthedevelopmentofearlyversionsofhigh-physicalityvirtualhumans,andfurtherrenementsweremadebymyself,DiegoRivera,andAndrewCordar. 53

PAGE 54

Toavoidintroducingpotentialgenderandracialconfounds,bothvirtualsurgeonsandanesthesiologistsweremodeledasaverageCaucasianmales,andvirtualnursesandsurgicaltechniciansweremodeledasaverageCaucasianfemales.Thesecombinationofraceandgenderarerepresentativeofthemajorityofsurgeons,anesthesiologists,nurses,andsurgicaltechnicianspracticingintheUnitedStates[ 116 ].Bothtrainingexercisesalsoinvolvedasimulatedpatient.Thepatientwasportrayedbyapatientmannequinsimulator.Thesimulatorwasaugmentedwithaspeakerthatallowedittospeakifneeded.Thesimulatorwascapableofgeneratingvitalsigns,butcouldnotmove. 3.6WizardofOzThevirtualhumansusedinmyresearchwerecontrolledviaaWizard-of-Oz(WoZ).InaWoZconguration,virtualhumansaresecretlycontrolledbyahumanoperator.Human-factorsresearchersfrequentlyuseWoZstoreduceconfoundingeffectsthatcanbeintroducedbyspeechrecognitionerrorsorspeechunderstandingerrors(see[ 1 , 11 , 117 ]forexamples).Inmystudies,thewizardcontrolledthevirtualsurgeonandthevirtualanesthesiologistsimultaneously,usinganinterfacethatallowedhimtotriggerthevirtualhumans'speechesusingpre-speciedlists.Thisinterfacewasorganizedbycharacterandtopic,toallowforrapidselection.Theinterfacealsointelligentlysuggestedresponsesbasedonthepreviousactionperformed.Thewizardfollowedaspecicscriptforeachstageoftheinteraction,butmadeadjustmentswhenparticipantsbehavedunexpectedly.Thevirtualhumanswerecapableofmakingninegenericstatements,suchas“Yes”,“No”,“OK”,and“I'mnotsure”,whichallowedthewizardtorespondtounexpectedquestionsorstatements.Inordertocreateaconsistentexperienceforeachparticipant,asinglewizardwasusedduringeachstudy.Iwasthewizardintherststudy,andAndrewCordarwasthewizardinthesecondstudy. 54

PAGE 55

ParticipantswerenottoldtheirvirtualteammateswerecontrolledbyaWizard-of-Oz.Toreducesuspicionthatthevirtualhumanswerecontrolledbyahuman,participantsweretoldthatthevirtualhumanswerefullyautonomousandwererequiredtocompleteaspeechrecognitiontrainingsessionandwearamicrophoneduringtheexercise.Whileusingawizard-of-oztechnicallymakesthesevirtualteammatesavatars,ratherthanagents,previousresearchhasshownthatperceivedagencyhasamuchstrongeraffectonbehaviorthanactualagency[ 86 ]. 3.7ConclusionThetechnologydiscussedinthissectionpresentsamethodthatcanbeusedtocreatevariable-agencyteammates,wheretheagencyofateammatecanbechangedatruntimedependingonifahumanisavailabletoplaythatrole.Thisisanimportantrequirementifvirtualteammatesaretobeusedtosupporttrainingprogramsbyovercomingschedulingproblems,asitmaynotbeknowninadvancewhichteammateswillneedtobeplayedbyvirtualhumans.Thischapteralsobrieydiscussedthetechnologyusedinthisresearchtovisualizeandinteractwithvirtualteammates.Thevirtualhumansusedinthisresearchhadrelativelyhighvisualandbehavioralrealism,comparedtomanyothervirtualagentsthathavebeendiscussedintheliterature.Threeofthemostimportantattributesthatcontributetothisrelativelyhighrealismisthelife-sizenatureoftheseagents,theirlowerrorrate(possiblebecauseoftheWoZ),andtheirabilitytomakeeyecontactwithaparticipantasheorshemovesabouttheroom.Whenseekingtoapplytheresultsofthisdissertationtoothertrainingprograms,itwillbeimportanttoconsideranydifferencesinvisualandbehavioralrealism. 55

PAGE 56

CHAPTER4TEAMTRAININGEXERCISESThischapterdescribestheteamtrainingexercisesdevelopedduringmydissertationresearch,andthestudiesconductedusingtheseteamtrainingexercises.Aseachstudycontributedtoeachofmyprimaryresearchquestions,thestudiesaredescribedinthischapterandtheresultsofthesestudiesaredescribedinthenextthreechapters.Eachteamtrainingexerciseisdescribedinitsownsection.Afteraspecicteamtrainingexercisehasbeendescribed,thestudyconductedwiththatexerciseisdescribedanddiscussed.PersonalContributions:Iledthedevelopmentofbothteamtrainingexercisesevaluatedinthisdissertation.Ialsoledthedesignofbothstudiesdiscussedinthisdissertationandproctoredtherststudy.Collaborators:Dr.AdamWendling,TheresaHughes,andTerrySullivanassistedwiththedevelopmentofbothtrainingexercisesevaluatedinthisdissertation.Dr.BenjaminLok,Dr.AdamWendling,Dr.SamsunLampotang,Dr.CaseyWhite,andAndrewCordarassistedwiththedevelopmentofthebothstudiesdiscussedinthisdissertation.Dr.AdamWendlingandTerrySullivanassistedwithparticipantrecruitment.RelevancetoThesis:ThischapterdescribestwostudiesIconductedduringmydissertationresearch.Theresultsofthesestudiesareconsideredinlaterchapters,wheretheresultsofeachstudyareaggregatedbasedontheirrelevancetomydifferentresearchquestions. 4.1SpeakingUpExerciseandStudyTheSpeakingUpexercisewastherstmixed-agencyteamIdevelopedduringmyresearch.ItwasdevelopedusingVirtualPeopleFactory2.0,thevirtualhumanauthoringsystemIdevelopedtocreateandsimulatemixed-agencyteams.Astudywasconducted 56

PAGE 57

usingtheSpeakingUpexercisetoevaluatetheeffectofmixed-agencyteamsonnurses'behaviorwithandperceptionsoftheirteammates. 4.1.1SpeakingUpExerciseTheSpeakingUpexercisewasdesignedtotrainhumannurseshowtospeakupaboutpatientsafetyissues.Theexerciseinvolvedapatientmannequin(thatdidnotinteractwithnurses,asthepatientwasdevelopmentallydelayedandnon-verbal),thepatient'smother,andthreeteammates:asurgeon,ananesthesiologist,andanurse.Asthegoaloftheexercisewastoteachnurseshowtospeakupaboutpatientsafetyissues,thesurgeonwasdesignedtomakeadecisionthatcouldpotentiallyendangerthesafetyoftheteam'spatient,givingnursesanopportunitytopracticeidentifyingthepotentialdangerandspeakinguptothesurgeontoprotectthepatient.Thedecisionthatcouldpotentiallyendangerthepatient'ssafetywasbeginningthesurgerybeforereplacementbloodwasavailable.Bloodwasnotavailablebecausetheteam'sanesthesiologisthadforgottentosendbloodsamplestothebloodbankforprocessing.Ratherthanwaitingforsamplestobesentandprocessed,thesurgeongrewimpatientandarguedthatitwouldbesafetostartwithoutblood,despitetheriskofunexpectedbloodloss,whichcouldseriouslyinjurethepatient.ThisexercisewasbasedonanactualeventthathadoccurredatUFHealth.TheSpeakingUpexercisewasbrokenintothreestages:pre-opholding,thepre-inductionbrieng,andthepre-incisiontimeout.Eachofthesestagesrepresentsakeystageintheprocessofpreparingapatientforsurgery. 4.1.1.1TutorialBeforetherststage,nursescompletedaninteractivetutorialthatallowedthemtopracticetalkingwithavirtualhuman.Duringthetutorial,avirtualhumantoldparticipantswhattheywouldbedoingduringtheexerciseandexplainedhowtotalkwithvirtualhumans.Thevirtualhumanandtheparticipantthenpracticedtalkingtoeachother.Duringthispracticephase,thevirtualhumanandtheparticipantexchanged 57

PAGE 58

informationwitheachotheraboutapatient.Thevirtualhumanandtheparticipanteachhadhalfoftheinformationtheyneededandhadtoasktheotherpersonabouttheinformationtheyweremissing. Figure4-1. Tutorial 4.1.1.2Stage1:Pre-opHoldingOncenurseshadcompletedthetutorial,theybegantherststageoftheexercise,pre-opholding.Duringpre-opholding,thenursespokewiththeanesthesiologistandthepatient'smother.Thenurse'sgoalinthisstagewastogatherinformationrelevanttotheupcomingsurgery(aT1toL1scoliosisrepair).Examplesofimportantinformationincludewhenthepatienthadlasteaten,orifthepatienthadeverhadabadreactiontoanesthesia.Duringthisstage,thepatient'smotherdemandedthatparticipantsnotdrawanybloodfromhersonuntilhehadbeensedated,citinghisviolentdislikeofneedles.Ifparticipantsobjected,theanesthesiologiststeppedintosaythatwaitingtodrawthebloodwasokay. 4.1.1.3Stage2:Pre-inductionBriengAfternurseshadnishedquestioningthepatient'smother,theymovedontothesecondstageoftheexercise,thepre-inductionbrieng.Duringthebrieng,nursesworkedwithasurgeonandananesthesiologisttoensurethatthesimulatedpatientwasreadytobeplacedunderanesthesiaforthesurgery.Thesurgeonguidedthis 58

PAGE 59

Figure4-2. Stage1-Pre-opholding stage,workingthroughachecklistusedintheUFHealthhospitaloperatingrooms.Thesurgeonaddressedquestionstothenursesandtheanesthesiologistasneeded.Theanesthesiologistoccasionallyinterruptedthesurgeontoaskaquestion.Nursescouldalsointerruptwithquestionsorcomments.Attheendofthebrieng,thesurgeonlearnedthatbloodsampleshadnotbeendrawnandinstructedtheanesthesiologisttodrawthebloodaftersedatingthepatient. Figure4-3. Stage2-Pre-inductionBrieng Stage2containedtwokeymomentsreferredtoas“agreementmoments”.Thesemomentswereincludedintheexercisetoinvestigatewhethertheagencyofnurses' 59

PAGE 60

teammembersimpactedhowtheymadedecisions.Duringthesemoments,thesurgeonandanesthesiologistwereunabletoagreeonminorpatientcareissues.Afterbeingunabletoresolvetheissuethemselves,theyaskedparticipantswhattheythoughtwouldbethebestsolution.Thesurgeonandtheanesthesiologistagreedwithwhateverdecisionparticipantsmadeandmovedontothenextitemonthesurgeon'schecklist.Bothagreementmomentsweredevelopedwiththehelpofanexperiencedanesthesiologist,whoselectedthesemomentsbecauseoftheirlackofaclearlycorrectanswer.Hejudgedthatthealternativesproposedbyboththesurgeonandtheanesthesiologistwereequallyacceptable.Therstagreementmomentconcernedwhetherornottoplacemonitors(devicesthatmonitorapatient'svitalsigns)onthepatientbeforeorafterhewasplacedunderanesthesia.Thesurgeonandanesthesiologist'sexchangeduringthismomentisshownbelow. Surgeon: Youhaven'tputthemonitorsonClayyet? Anesth: No,Ididn'twanttoagitateClay.Hismothersaidhecanbeabittouchyaroundneedles,soI'mconcernedthemonitorsmightdisturbhimaswell. Surgeon: Idon'tthinkputtingmonitorsonshouldagitatehim,evenifhedoesn'tlikeneedles. Anesth: Nurse,whatdoyouthink?Thesecondagreementmomentconcernedwhetherornottouseanupper-andlower-bodyBairHuggeroronlyalower-bodyBairHugger(BairHuggersareusedtokeeppatientswarmduringsurgery).Thesurgeonandanesthesiologist'sexchangeduringthismomentisshownbelow. Anesth: We'lluseanupperandlowerBairHugger,right? Surgeon: Ithinkwe'reonlygoingtohaveroomforalowerbodyBairHugger.Nurse,whatdoyouthink? 60

PAGE 61

Bothoftheseagreementmomentsoccurredduringthesecondstageoftheexercise.Theagreementmomentsdidnotoccurbacktoback,butwereseparatedbytenotheritemsonthesurgeon'schecklist. 4.1.1.4Stage3:Pre-incisionTimeoutAfterthesurgeonhadcompletedthechecklist,nursesmovedontothethirdstageoftheexercise,thepre-incisiontimeout.Duringthetimeout,nursesworkedwiththesurgeonandtheanesthesiologisttoconrmthatthesimulatedpatientwasreadyfortheincisionthatwouldbeginsurgery.Thesurgeonguidedthisstage,workingthroughasecond,shorterchecklist.Afteraskingthenurseseveralbasicquestions,thesurgeonaskedtheanesthesiologistifthebloodwasnowavailable.Theanesthesiologistreportedthathehadforgottentosendthebloodtothelab.Thisangeredthesurgeon,whoberatedtheanesthesiologistandthenannouncedthat,becausetheywererunninglate,theteamneededtosendthebloodsamplesimmediatelyandstartthesurgerywithoutwaitingfortheresults.Thisproposedcourseofactioncouldpotentiallyendangerthepatient'ssafety,becausebloodmightbeneededsoonerthanexpected,andbecausethepatient'sbloodcouldhaveantibodiesthatwouldslowthebloodpreparationprocesses.Thisproposedcourseofactionwasalsoagainsthospitalpolicy,whichstatesthattwosamplesmustbesenttothebloodbankbeforesurgerycanbegin.Atthispointnursescouldeitherspeakuptothesurgeonaboutthepotentialdangertothepatient,ortheycouldagreewiththesurgeon'sdecisionandallowhimtoproceed.Ifthisscenarioweretooccurinanactualoperatingroom,nursingmanagementwouldwantnursestospeakupaboutthepatient-safetyissueandstopthesurgeonfromstartingthesurgeryuntilbloodwasavailable.IntheSpeakingUpexercise,ifnursesspokeuptothesurgeonandexpressedconcernsaboutthesurgeon'sproposedcourseofaction,hewouldrepeatedlyobjecttotheirconcerns.Hisobjectionswere:thatthepatientwasunlikelytohaveantibodiesbecausehehadneverbeentransfusedbefore,thattherewassufcienttimetogetbloodbeforeitwouldbeneeded,andthatwaiting 61

PAGE 62

Figure4-4. Stage3-Pre-incisionTimeout couldharmthepatientbecauseoftheadditionaltimehewouldbeunderanesthesia(theseobjectionsweredevelopedincollaborationwithnursingmanagementandananesthesiologist).Afteranursespokeupthreetimes,thesurgeonannouncedthathewasnotgoingtolistentothenurseanymoreandthathewasgoingtobeginthesurgery.Atthispointnursescouldeitherbackdownor“stoptheline”andcallasupervisor.Theanesthesiologistdidnotspeakduringthespeakingupmoment,unlesshewasaskedaquestionbythenurse.Ifaskedtointervene,hesaidhewasnotgoingtogetinvolvedandwouldletthesurgeonandthenursemakethedecision.This“speakingupmoment”formedthecorecomponentofthetrainingexerciseandalsoprovidedakeyopportunitytoassesswhethermixed-agencyteamsalterednursesbehavior. 4.1.1.5EducationalInterventionAfternursesresolvedthespeakingupmoment,eitherbyspeakinguporbackingdown,nursesweredebriefedaboutwhytheyhadactedastheydidandhowtheyhadfeltabouttheirvirtualteammates.Thedebriengtooktheformofasemi-structuredinterviewstructuredaroundthefollowingquestions: 1. Canyoutellmewhyyouhandledtheincidentwiththebloodasyoudid? 2. Doesanythingmakeitharderoreasierforyoutospeakup? 62

PAGE 63

3. Couldyoutellmeaboutoneofthetimesyouhavespokenupandwhattheoutcomewas? 4. Whatwasitlikeworkingwithyourvirtualteammatestoday? 5. Howdidthevirtualsurgeoncomparetootherrealsurgeonsyouhaveworkedwith? 6. Howdidthevirtualanesthesiologistcomparetootherrealanesthesiologistsyouhaveworkedwith? 7. Howimportantdidyoufeelyouweretotheteamtoday?Oncethesemi-structuredinterviewwascompleted,nurseswatchedavideoexplainingwhythesurgeon'sactionswasdangeroustotheteam'spatient.ThisvideowascreatedbyanexperiencedanesthesiologistwhohelpedtodeveloptheSpeakingUpexercise.NurseswerethenbriefedontheTeamSTEPPSprotocol[ 108 ]andtheCUSrule[ 18 ],commonprotocolsdescribinghowtospeakupeffectively.OncenurseshadreceivedinstructionabouttheTeamSTEPPSprotocolandtheCUSruleandhadbeengiventheopportunitytoaskquestions,participantsrepeatedStage3andwereallowedtoadjusttheirspeakingupbehavior,ifdesired.Stage3wasmodiedslightlyforthispracticesession.Ratherthanremainingunwillingtobackdownandwait,thesurgeongaveinandlistenedtonurses'concernsaftertheyhadspokenuptwice.Thismodicationwasmadeinanattempttoempowernursesandreinforcetheirtrainingbycreatinganimpressionthatwhattheyhadlearnedenabledthemtoconvincethesurgeontowait.Finally,participantswereverballydebriefedabouttheirfeelingsconcerningtheusefulnessoftheentireexercise. 4.1.2SpeakingUpStudyTheSpeakingUpexercisewasusedduringmyrststudyexploringtheeffectsofmixed-agencyteamsonsocialpresenceandbehavior.DuringtheSpeakingUpstudy,theagencyofthesurgeonandtheanesthesiologistwerevaried.Humanactorsplayedtheroleofsurgeonoranesthesiologist,dependingoncondition. 63

PAGE 64

4.1.2.1ConditionsParticipantsweresplitintothreeconditions(seeFigure 4-5 ).Theagencyofthesurgeonandtheanesthesiologistwerevariedbetweenconditions.Intherstcondition,thesurgeonandanesthesiologistwerebothplayedbyvirtualhumans.Inthesecondcondition,thesurgeonwasplayedbyahumanconfederateandtheanesthesiologistwasplayedbyavirtualhuman.Inthethirdcondition,thesurgeonwasplayedbyavirtualhumanandtheanesthesiologistwasplayedbyahumanconfederate.Afourthcondition,wherethesurgeonandanesthesiologistwerebothplayedbyhumans,wasnotincludedgiventheemphasisofthisresearchisonbehaviorwithvirtualhumans.Asthisfourthconditiondoesnotcontainanyvirtualhumans,itwasexcluded. 4.1.2.2HumanConfederatesStepsweretakentoreducevariancebetweenthehumanconfederatesandtheirvirtualcounterparts.Onceahumanconfederatehadbeenselected,hisvirtualcounterpartwasmodiedtoresemblethehumanconfederateintermsofheight,bodytype,skincoloration,anduniquefeatures(suchaswearingglasses).Thehumanconfederatethenrecordedhisvirtualcounterpart'svoice,toensurethattheysoundedsimilar.Bothconfederatesweretrainedstandardizedpatientswhoregularlyassistwithmedicalstudenttrainingexercises.Standardizedpatientsarecommonlyusedtotrainmedicalstudentstopracticemedicalinterviewingandphysicalexaminations[ 118 ].Assuch,theyaretrainedtoportrayspecicindividualsandexhibitspecicmedicalsymptoms.Whilestandardizedpatientsdonotnormallyplaytheroleofmedicalprofessionals,theyarefamiliarwiththemedicaldomainandunderstandthatitisimportanttobehaveconsistentlyduringeachtrainingsession.Theconfederateswererecruitedfromthepoolofavailablestandardizedpatientsandwerepaidthestandardrateof$20perhourduringthestudy. 64

PAGE 65

AConditionBoth-Virtual-VirtualSurgeonandVirtualAnesthesiol-ogist BConditionHuman-Surgeon-HumanSurgeonandVirtualAnes-thesiologist CConditionHuman-Anesth-VirtualSurgeonandHumanAnes-thesiologistFigure4-5. Theagencyofthesurgeonandtheanesthesiologistwerevariedinthethreeconditions.Thehumansurgeonandthehumananesthesiologistwereplayedbytwodifferentconfederates.Humanconfederatesdidnotwearmasksbecauseparticipantsalsodidnotwearmasks,soasnottointerferewithspeechrecording. TheconfederatesreceivedindividualtrainingonthescriptfromtheWoZoperatorwhocontrolledthevirtualhumansduringthestudy.TheWoZoperatorexplainedthescenarioanddemonstratedastandardinteractionwherethesurgeonandtheanesthesiologistwerebothvirtual,andthenhadtheconfederatepracticeplayinghisroleuntilhewascomfortablewithit.Afterthistrainingsession,theconfederatetook 65

PAGE 66

homepaperscriptsforfurtherstudyandfamiliarization.Thesepaperscriptswerealsoavailableduringthestudyasaguidetohelptheconfederateperformconsistentlyduringtheexercise.Thehumanconfederateswerelargelysuccessfulinmimickingthebehavioroftheirvirtualcounterparts.However,therewasonesystematicvariationbetweenthehumanconfederatesandtheirvirtualcounterparts.Asthehumanconfederatescouldrefertothescriptduringthestudy,theyfrequentlybrokeeyecontactwiththenursetorefertothescripttoensurethattheyrememberedwhatshouldhappennext.Incontrast,thevirtualhumansmaintainedeyecontactwithwhoeverwasspeaking(exceptforoccasionalrandomglancesatotherplacesintheroom).Thehumansurgeonalsooccasionallyrespondedincorrectlytoanurseorad-libbedaresponse.TheimplicationsofthesesourcesofvariancewillbeconsideredfurtherintheLimitationssectionsoftheresultschapters. 4.1.2.3ParticipantsAtotalof69participants(53female)tookpartintheexercise.AllparticipantswerenurseswhowerecurrentlyworkinginUFHealthoperatingrooms.Theaverageparticipantagewas42.4yearsold;agesrangedfrom24to68.Participantshadbeenworkingasanurseforanaverageof20.2years,andasanurseintheORforanaverageof16.8years.Ofthe69participants,52reportedtheirraceasWhite,9asAsian,7asBlack,and1asAmericanIndianorAlaskaNative. 4.1.2.4MetricsSeveralmetricsweregatheredduringtheSpeakingUpstudy.Participantscompletedsocialpresencesurveys[ 22 ]aftereachstageforeachofthepeopletheyhadinteractedwith(includingbothvirtualandrealhumans).Participantsalsocompletedademographicsurvey,theBFI-44personalityinventory[ 119 , 120 ],andtheAdultSelf-ExpressionScale[ 121 ]beforebeginningthestudy.TheBFI-44inventoryassessesvecorepersonalitytraits:Openness,Conscientiousness,Extraversion,Agreeableness, 66

PAGE 67

andNeuroticism.TheAdultSelf-ExpressionScaleassessesassertiveness.Participantswerevideorecordedduringthestudy(3onlyconsentedtoaudiorecording,1didnotconsenttoanyrecording).Thisvideowasusedtoanalyzeparticipants'decisionmaking,verbal,andnon-verbalbehaviorduringtheexercise.Decisionmakingandverbalbehaviorwerespecicallyexaminedduringthetwoagreementmomentsandthespeakingupmoment.Aftertheexercisewascompleted,participantswerealsodebriefedabouttheirspeakingupbehaviorandtheirfeelingsaboutthevirtualhumanstheyworkedwithduringtheexercise. 4.1.2.5StudyprocedureUponarrival,participantsweretoldtheywouldbeparticipatinginatrainingexercisewheretheywouldbepreparingasimulatedpatientforsurgery.Participantswerenotwarnedthatthesurgeonwouldengageinbehaviorthatcouldpotentiallyendangerthesafetyoftheteam'spatient,norweretheyprimedtospeakuptothesurgeon.Additionally,participantsweretoldthattheirbehaviorduringtheexercisewouldremaincompletelycondentialandthattheexercisewasnotbeingusedasaperformanceassessment.Oncetheprocedurehadbeenexplainedtoparticipants,theywereaskediftheyconsentedtoparticipateinthestudy.Participantsreceiveda1.5creditsofcontinuingmedicaleducationcreditsanda$10coffeecardforparticipating.Iftheygaveconsent,theycontinuedtocompleteaspeechrecognitiontrainingsession,usingDragonMedicalPracticeEdition.TheaccuracyofDragonMedicalPracticeEditionwasevaluatedduringthisstudy,butitwasnotusedtointeractwiththevirtualhumans(whowerecontrolledbyaWoZ).Speechrecognitiontrainingalsoreinforcedtheideathatparticipants'virtualteammateswerefullyautonomous.Oncetheyhadcompletedthespeechrecognitiontraining,participantscompletedademographicsurveyandapersonalitysurvey.Participantsthenbegantheexerciseitself,startingwiththetutorialandthenprogressingthroughstages1,2,and3(seeSection 4.1.1 formoredetails).Participants 67

PAGE 68

completedasocialpresencesurveyafterstages1,2,and3whilethestudyproctorwaspreparingforthenextstage.Participantsweredebriefedaftercompletingstage3andthentookpartinaneducationalinterventionconcerningeffectivespeakingupbehavior.Finally,participantswereverballydebriefedabouthoweffectivetheyfelttheexercisehadbeenandwerethankedforparticipating. 4.2SurgicalCountExerciseandStudyTheSurgicalCountexercisewasthesecondmixed-agencyteamIdevelopedduringmyresearch.ItwasdevelopedusingVirtualPeopleFactory2.0,thevirtualhumanauthoringsystemIdevelopedtocreateandsimulatemixed-agencyteams.Astudywasconductedusingthesurgicalcountexercisetoevaluatetheeffectofmixed-agencyteamsonnursesbehaviorwithandperceptionsoftheirteammates.WhiletheSpeakingUpstudyvariedtheagencyofauthoritygures(asurgeonandananesthesiologist)andemployedhumanconfederates,theSurgicalCountstudyvariedtheagencyofpeopleofsimilarauthority(nursesandsurgicaltechnicians)andusedactualhumanparticipants. 4.2.1SurgicalCountExerciseTheSurgicalCountexercisewasbasedontheSpeakingUpexercise,butwasextendedtoaddafourthteammate(asurgicaltechnician)andanewstagewithanewspeakingupmoment.Stage1fromtheSpeakingUpexercise,whichinvolvedinteractingwiththepatient'smother,wasremovedfromtheSurgicalCountexercise.Stage2and3fromtheSpeakingUpexercisewerelargelyunchanged.Theclosingcountstagewasaddedafterthepre-incisiontimeoutstage.Theclosingcountstageinvolvedanewspeakingupmomenttargetedatsurgicaltechnicians.TheSurgicalCountexerciseisdescribedinmoredetailbelow,withanemphasisonhowitdifferedfromtheSpeakingUpexercise. 4.2.1.1TutorialBeforetherststage,participantsalsocompletedaninteractivetutorialthatallowedthemtopracticetalkingwithvirtualhumans.Duringthetutorial,twovirtualhumanstold 68

PAGE 69

participantswhattheywouldbedoingduringtheexerciseandexplainedhowtotalkwithvirtualhumans.Thevirtualhumansandtheparticipantsthenpracticedtalkingtoeachother.Duringthispracticephase,thevirtualhumansandtheparticipantsexchangedinformationwitheachotheraboutapatient.Thevirtualhumanandtheparticipanteachhaddifferentpiecesoftheneededinformationandhadtoasktheotherpeopleabouttheinformationtheyweremissing.Thevirtualhumansaskedeachotherquestions,askedtheparticipantsquestions,andpromptedtheparticipantstoquestionthemaswell. Figure4-6. Tutorial 4.2.1.2Stage1:Pre-inductionBriengOnceparticipantshadcompletedthetutorial,theybegantherststageoftheexercise,thepre-inductionbrieng.Duringthebrieng,participantsworkedwithavirtualsurgeonandavirtualanesthesiologisttoensurethatthesimulatedpatientwasreadytobeplacedunderanesthesiaforthesurgery(alaparoscopicwhipple).Thevirtualsurgeonguidedthisstage,workingthroughachecklistusedintheUFHealthhospitaloperatingrooms.Thevirtualsurgeonaddressedquestionstoparticipantsandthevirtualanesthesiologistasneeded.Thevirtualanesthesiologistoccasionallyinterruptedthevirtualsurgeontoaskaquestion.Participantscouldalsointerruptwithquestionsorcomments.Attheendofthebrieng,thevirtualsurgeonlearnedthatbloodsampleshadnotbeendrawnandinstructedthevirtualanesthesiologisttodrawthebloodafter 69

PAGE 70

sedatingthepatient.ThetwoagreementmomentspresentintheSpeakingUpexercisewereremovedfromtheSurgicalCountexercise,astheagencyofthesurgeonandtheanesthesiologistwerenotvariedinthisexercise. Figure4-7. Stage1-Pre-inductionBrieng 4.2.1.3Stage2:Pre-incisionTimeoutStage2oftheSurgicalCountstudywasverysimilartoStage3oftheSpeakingUpstudy(seeSection 4.1.1.4 fordetails).ThemainchangemadefortheSurgicalCountstudywasthepresenceofaSurgicalTechnician.However,thesurgicaltechniciandidnotparticipatesignicantlyinthisstage.Also,intheeventthatahumansurgicaltechnicianwasworkingwithavirtualnurse,thevirtualnursewouldspeakuptothesurgeon.ThisbehaviorisdiscussedinmoredetailinSection 4.2.2.1 .Thevirtualnursespokeupinduringthismomentbecausethesurgicaltechnicianisnotresponsibleforbloodrelatedissuesandwouldnotnormallychallengesurgeonsaboutbehaviorrelatedtobloodavailability. 4.2.1.4Stage3:ClosingCountAfterStage2hadbeenresolved,eitherbytheparticipantsspeakinguporagreeingwiththesurgeon,theymovedontothethirdstageoftheexercise,theclosingcount.Theclosingcounttookplaceafterthesurgeryhadbeensuccessfullycompleted.Participantsdidnotpracticethesurgerybutskippedstraighttotheclosingcount; 70

PAGE 71

Figure4-8. Stage2-Pre-incisionTimeout theywereinformedthatthesurgeryhadbeensuccessfulandthattherehadnotbeenanycomplications.Duringtheclosingcount,thenurseandthesurgicaltechniciancountedseveralitemsusedduringsurgeryandcheckedtomakesurethatthesecountscorrespondedtocountsrecordedbeforethesurgery(thesecountswereprovidedtoparticipantsbytheproctor).Twointerruptionsoccurredduringtheclosingcount,onewherethesurgeonaskedthesurgicaltechnicianforapieceofequipmentandonewherethepatient'slegfelloffthebed(thiswastriggeredbythewizardoperator)andthevirtualanesthesiologistaskedthenursetorepositionhisleg.Bothoftheseinterruptionscouldoccurduringarealoperation. Figure4-9. Stage3-ClosingCount 71

PAGE 72

Oneitemwasmissingfromtheclosingcount:ahemoclipboat.Thelistgiventoparticipantsshowedthatthereshouldbetwoboats,butonlyonewaspresentintheroom.Upondiscoveringthatahemoclipboatwasmissing,thevirtualsurgeoninstructedtheteamtobeginsearchingfortheboat.Aftertheboatcouldnotbefound,thevirtualanesthesiologistsuggestedorderinganx-ray.Thesurgeonagreed,andanx-rayappearedonadisplayintheroom.Thesurgeonexaminedthex-rayandannouncedthathecouldn'tseeanyforeignobjectsinsidethepatientandthathewasgoingtobeginclosingthepatient'sincision.Thiscontradictedhospitalpolicy,whichstatesthat,intheeventofamissingobject,theattendingradiologistmustalsoreadthex-raybeforeclosingapatient.Atthispoint,participantscouldeitheragreewiththesurgeonandallowhimtoclosethepatient,ortheycouldspeakuptothesurgeonandinformhimthatheneedstowaitfortheradiologisttoreadthex-ray.Ifparticipantsspokeuptothesurgeonandexpressedconcernsabouthimclosingthepatient,hewouldrepeatedlyobjecttotheirconcerns.Hisobjectionswere:thathedidn'twanttowastetimewaitingfortheradiologist,thathewasequallyqualiedtoreadanx-ray,thathehasbeenatthehospitalfor36hoursandisexhausted,andthattheparticipantscouldseeforthemselvesthatnoforeignobjectswerevisibleinsidethepatient.Afterparticipantsspokeupfourtimes,thevirtualsurgeonannouncedthathewasnotgoingtolistentothemanymoreandthathewasgoingtoclosethepatient.Atthispoint,participantscouldeitherbackdownor“stoptheline”andcallasupervisor.Thevirtualanesthesiologistdidnotspeakduringthespeakingupmoment,unlesshewasaskedaquestionbyparticipants.Ifaskedtointervene,hesaidhewasnotgoingtogetinvolvedandwouldletthevirtualsurgeonandtheparticipantsmakethedecision.Ifonlyonehumanparticipantwaspresentandheorshedidnotimmediatelyspeakuptothevirtualsurgeon,eitherthevirtualnurseorthevirtualsurgicaltechnicianwouldaskthehumanparticipantiftheythoughtthiswasOK,andremindtheparticipantthataradiologistneededtolookatthex-raybeforeclosingthepatient. 72

PAGE 73

4.2.1.5EducationalInterventionAfterparticipantsnishedtheclosingcount,eitherbyspeakinguporagreeingwiththesurgeon,theyweredebriefedaboutwhytheyhadactedastheydidandhowtheyhadfeltabouttheirvirtualteammates.Thedebriengtooktheformofasemi-structuredinterviewstructuredaroundthefollowingquestions: 1. Canyoutellmewhyyouhandledtheincidentswiththebloodandthecountasyoudid? 2. Couldyoutellmeaboutoneofthetimesyouhavespokenupandwhattheoutcomewas? 3. Whatwasitlikeworkingwithyourvirtualteammatestoday? 4. Howimportantdidyoufeelyouweretotheteamtoday?Oncethesemi-structuredinterviewwascompleted,nurseswatchedavideoexplainingwhythesurgeon'sactionswasdangeroustotheteam'spatient.Thisvideowascreatedbyanexperiencedsurgeonwhohelpedtodeveloptheclosingcountpolicy.NurseswerethenbriefedontheTeamSTEPPSprotocol[ 108 ]andtheCUSrule[ 18 ],commonprotocolsdescribinghowtospeakupeffectively.OncenurseshasreceivedinstructionabouttheTeamSTEPPSprotocolandtheCUSruleandhadbeengiventheopportunitytoaskquestions,participantsrepeatedStage3andwereallowedtoadjusttheirspeakingupbehavior,ifdesired.UnlikeintheSpeakingUpexercise,Stage3wasnotmodiedintheSurgicalCountexercise.Thischangewasmadeinordertobetterassesstheeffectoftheeducationalintervention. 4.2.2SurgicalCountStudyTheSurgicalCountexercisewasusedduringmysecondstudyexploringtheeffectsofmixed-agencyteamsonsocialpresenceandbehavior. 73

PAGE 74

4.2.2.1ConditionsParticipantsweresplitintothreeconditions(seeFigure 4-10 ).Theagencyofthenurseandthesurgicaltechnicianwerevariedbetweenconditions,whilethesurgeonandtheanesthesiologistwerealwaysplayedbyavirtualhuman.Whennursesorsurgicaltechnicianswerehuman,theywerealwaysactualparticipants;humanconfederateswerenotusedforthisstudy.Intherstcondition,thenursewasahumanparticipantandtheremainingteammemberswerevirtualhumans.Inthesecondcondition,thesurgicaltechnicianwasahumanparticipantandtheremainingteammemberswerevirtualhumans.Inthethirdcondition,boththenurseandthesurgicaltechnicianwerehumanparticipantsandtheremainingteammemberswerevirtualhumans.Thehumansurgicaltechnicianconditionwasfurtherdividedintotwosub-conditions.DuringStage2,humansurgicaltechnicianseithersawthevirtualnursesuccessfullystopthelinewiththevirtualsurgeon,ortheysawherattempttospeakuptothevirtualsurgeonbutfailtostoptheline.Instead,thevirtualnursegaveupandshiftedresponsibilitytothesurgeon,saying“Alright,well...Ithinkthisisabadidea,butyou'rethesurgeonandit'syourcall.”Thissub-conditionwasincludedtoexplorewhetherbehaviormodeledbyavirtualhumanwouldaffectbehaviorbythesurgicaltechnicianlaterduringStage3.Thisresearchwasacomponentofaresearchprojectunrelatedtothisdissertation,andisthusnotexploredfurther. 4.2.2.2ParticipantsAtotalof94participants(78female)tookpartintheexercise.Allparticipantswereeithernurses(n=49)orsurgicaltechnicians(n=45)whowerecurrentlyworkinginUFHealthoperatingrooms.Theaverageparticipantagewas39.42yearsold;agesrangedfrom23to62.Participantshadbeenworkingastheircurrentprofessionforanaverageof11.5years,andhadbeenworkingintheORforanaverageof9.27years.Ofthe94participants,59reportedtheirraceasWhite,20asBlack,10asAsian,5as 74

PAGE 75

AHumanNursecondition BHumanSurgicalTechniciancondition(ThevirtualnursewasshownontheTVontheleftsideofthisimage) CBothHumanconditionFigure4-10. Theagencyofthenurseandthesurgicaltechnicianwerevariedinthethreeconditions.Allhumannursesandsurgicaltechnicianswereactualparticipants;confederateswerenotusedintheSurgicalCountstudy. AmericanIndianorAlaskaNative,and1asNativeHawaiianorPacicIslander;oneparticipantidentiedasbothWhiteandAmericanIndianorAlaskaNative. 75

PAGE 76

4.2.2.3MetricsSeveralmetricsweregatheredduringtheSurgicalCountstudy.Participantscompletedsocialpresencesurveysaftereachstageforeachofthepeopletheyhadinteractedwith(includingbothvirtualandrealhumans).ParticipantsalsocompletedthePerceivedCohesionsurveyaftereachstage[ 122 ].Participantsalsocompletedanuncannyvalleysurvey[ 123 ]foreachteammateaftercompetingstage3;theuncannyvalleysurveywasonlyadministeredafterstage3duetoitslength.Beforebeginningthestudy,participantscompletedademographicsurvey,theBFI-44personalityinventory[ 119 , 120 ],andtwosurveysdrawnfromtheInternationalPersonalityItemPool[ 124 ],namelythePublicSelf-ConsciousnesssurveyandtheSocialIntelligencesurvey.Participantswerevideorecordedduringthestudy(twoconsentedonlytoaudiorecording,onedidnotconsenttoanyrecording).Thisvideowasusedtoanalyzeparticipants'decisionmaking,verbal,andnon-verbalbehaviorduringtheexercise.Decisionmakingandverbalbehaviorwerespecicallyexaminedduringthetwoagreementmomentsandthespeakingupmoment.Aftertheexercisewascompleted,participantswerealsodebriefedabouttheirspeakingupbehaviorandtheirfeelingsaboutthevirtualhumanstheyworkedwithduringtheexercise. 4.2.2.4StudyprocedureUponarrival,participantsweretoldtheywouldbeparticipatinginatrainingexercisewheretheywouldbepreparingasimulatedpatientforsurgery.Participantswerenotwarnedthatthesurgeonwouldengageinbehaviorthatcouldpotentiallyendangerthesafetyoftheteam'spatient,norweretheyprimedtospeakuptothesurgeon.Additionally,participantsweretoldthattheirbehaviorduringtheexercisewouldremaincompletelycondentialandthattheexercisewasnotbeingusedasaperformanceassessment.Oncetheprocedurehadbeenexplainedtoparticipants,theywereaskediftheyconsentedtoparticipateinthestudy.Participantsreceiveda1.5creditsofcontinuing 76

PAGE 77

medicaleducationcreditsanda$10coffeecardforparticipating.Iftheygaveconsent,theywereaskedtoputonmicrophonesandweretoldthatthesemicrophoneswouldenablethevirtualhumanstohearthem.Thiswasdonetoreinforcetheideathatparticipants'virtualteammateswerefullyautonomous.Participantsthencompletedademographicsurveyandapersonalitysurvey.Participantsthenbegantheexerciseitself,startingwiththetutorialandthenprogressingthroughstages1,2,and3(seeSection 4.2.1 formoredetails.Participantscompletedsocialpresencesurveysandperceivedcohesionsurveysafterstages1,2,and3whilethestudyproctorwaspreparingforthenextstage.Participantscompletedtheuncannyvalleysurveysafterstage3.Participantsweredebriefedaftercompletingstage3andthentookpartinaneducationalinterventionconcerningeffectivespeakingupbehavior.Finally,participantswereverballydebriefedabouthoweffectivetheyfelttheexercisehadbeenandwerethankedforparticipating. 77

PAGE 78

CHAPTER5MIXED-AGENCYINTERACTIONSANDDECISIONMAKINGBEHAVIORThischapterconsidershowparticipantsbehavedduringspecicdecisionsintheSpeakingUpstudyandtheSurgicalCountstudy(seeSections 4.1 and 4.2 fordetailsaboutthesestudies).Overall,theagencyofparticipants'teammatesdidnotappeartoalterhowtheymadedecisions,thoughsomesignicanteffectswereobserved.Furtherresearchisrequiredtodeterminewhethertheseeffectsrepresentisolatedoccurrences,orlargertrends.ResultsfromtheSpeakingUpstudywerepublishedintheproceedingsoftheIntelligentVirtualAgents2014conference[ 125 ],IEEETransactionsonVisualizationandComputerGraphics(TVCG)[ 126 ],andComputersinHumanBehavior[ 127 ].PersonalContributions:Iledthedevelopmentofthebehavioralcodingscheme,codedthedata,analyzedthedata,andwrotethemajorityofallpapers.Collaborators:Dr.CaseyWhiteandAndrewCordarassistedwiththedevelopmentofthecodingscheme.Dr.CaseyWhiteassistedwiththecodingofthedata.Dr.JamesMartindaleassistedwiththeanalysisofthedata.RelevancetoThesis:Thischapterexploreswhethermixed-agencyteamsalterdecision-makingbehaviorwithvirtualteammates. 5.1ResearchQuestionsandMotivationThisdissertationexplorestwoquestionsrelatedtointeractionswithvirtualmembersofmixed-agencyteams.Therstresearchquestionfocusesonhowinteractionswithaspecicteammateareaffectedbymanipulatingthatteammate'sagency(e.g.replacingavirtualsurgeonwithahumansurgeon).Theterm“DirectEffect”isusedtorefertothisresearchquestion,asitdealswithhowmanipulatingaspecicteammate'sagencydirectlyimpactsinteractionswiththatteammate.Thesecondresearchquestionfocusesonwhetherthepresenceofhumanteammatesaltersinteractionswithvirtualteammates(e.g.doesyourperceptionof 78

PAGE 79

avirtualanesthesiologistchangewhentheteam'svirtualsurgeonisreplacedbyahumansurgeon).Theterm“IndirectEffect”isusedtorefertothisresearchquestion,asitdealswithhowmanipulatingoneteammate'sagencyindirectlyimpactsinteractionswithadifferentvirtualteammate.BothofthesequestionsarediscussedinmoredetailinSection 1.2 andillustratedinFigure 1-1 .Withinthecontextofthesetwoquestions,thischapterinvestigateshowtheagencyofparticipants'teammatesaffectedtheirdecisionmakingprocesses.Iexaminewhetheragencyinuencedhowparticipantsdecidebetweenalternativesproposedbydifferentteammates,whetheragencyaffectedhowparticipantsspokeupaboutpotentialsafetyissues.Whenpossible,establishedbehavioralframeworkswereusedtoassessbehavior.However,giventhecontext-specicnatureofbehavior,newcodingschemeswerealsodevelopedtoassessbehaviorduringscenario-specicmoments.Theresearchdiscussedinthissectionismotivatedbytheimportanceofdecisionmakinginteamactivities.Ifatrainee'sdecisionsareinuencedbytheagencyofhisorherteammates,thiscouldleadtotraineeslearningincorrectbehavior,andcouldalsoprovideaninconsistenttrainingexperience.Forinstance,iftraineespreferentiallyagreewithhumanteammatesbecauseoftheiragencyandnotthemeritoftheirargument,thiscouldteachtraineestofollowimproperpracticesinreal-worldcircumstances.Thiswouldalsoproduceaninconsistenttrainingenvironment,wheredifferenttraineeslearndifferentbehaviorsbasedonwhatotherhumanswereavailablefortraining. 5.2CollectionandPreparationofDecisionMakingDataDecisionmakingbehaviorwasassessedduringfourmomentsthatwereinsertedintotheSpeakingUpandSurgicalCountscenarios.Participantswererequiredtomakeadecisionduringeachofthesemoments.ThreeofthesemomentsexistedintheSpeakingUpstudy,andtwoexistedintheSurgicalCountstudy.Onemomentexistedinbothstudies.ThersttwomomentsintheSpeakingUpstudyarereferredtoasagreementmoments.ThesemomentsaredescribedindetailinSection 4.1.1.3 . 79

PAGE 80

Duringthesemoments,participantswereaskedtodeterminewhichoftwoalternativeswasmoreappropriategiventhesituation.Eachalternativewassupportedbyoneofparticipants'teammates.Neitheralternativewasclearlymoreappropriatethantheother.Thesecondtwomomentsarereferredtoas“speakingup”moments.Duringthesemoments,theteam'ssurgeonproposedacourseofactionthatcouldpotentiallyendangerthesafetyoftheteam'spatient.Incircumstanceslikethese,theappropriatebehaviorforallmedicalprofessionalsistospeakupaboutthesafetyriskandacttoensurethatthepatient'ssafetyisprotected.Onespeakingupmomentoccurredduringbothstudies,andonemomentoccurredonlyintheSurgicalCountstudy.Thespeakingupmomentusedinbothstudieswasdirectedtowardsnurses,andthespeakingupmomentusedonlyintheSurgicalCountstudywasdirectedtowardssurgicaltechnicians.SeeSections 4.1.1.4 and 4.2.1.4 formoredetailconcerningthesemoments. CodingofBehavioralData.Participants'behaviorduringtheagreementmomentswascodedviapost-hocvideocoding.CodingschemesweredevelopedduringtheSpeakingUpstudy,andwerelateradaptedforusewiththeSurgicalCountstudy.Codingschemesweredevelopedusingthefollowingmethod,borrowedfromGorden'sCodingInterviewResponses[ 128 ]:tworesearchersanalyzedeachparticipant'stranscriptandcodedhisorherbehaviorduringaspecicagreementmoment.Videosofparticipants'interactionswereconsultedwhenthetranscriptswereunclear.Afternishingtheinitialcoding,thetworesearchersdiscussedtheirobservationsand,followingnegotiation,agreeduponanalcodingscheme.Theythenre-codedthetranscriptsusingthecodingschemeasaguide.Afterthesecondroundofcoding,differenceswerethencomparedandreconciled.Additionalroundsofcodingwerenotrequired,asthecoderswereincompleteagreementabouthoweachparticipantshouldbecoded. 80

PAGE 81

5.2.1CodingschemeforagreementmomentsFortheagreementmoments,asimplecodingschemewasdeveloped.Participantseitheragreedwiththesurgeon,agreedwiththeanesthesiologist,orproposedacompromisesolutionthatattemptedtoaddresstheconcernsraisedbyboththesurgeonandtheanesthesiologist.Participantssuggestedseveraldifferentcompromises,however,astheyeachrepresentedanattempttoaddresstheconcernsraisedbyboththesurgeonandtheanesthesiologist,theyweregroupedintoasinglecode. 5.2.2CodingschemeforspeakingupmomentsForthespeakingupmoments,amorecomplexcodingschemewasdeveloped.Fivemajoroutcomeswereidentied.Theseoutcomesdescribethenalactionanursetookwhilerespondingtothesurgeon'sproposedcourseofaction.TheseoutcomesaredescribedbelowinTable 5-1 . Table5-1. Theoutcomesareorderedbasedonthedegreeofresistanceparticipantsofferedtothesurgeon,indescendingorder. OutcomeDescription StoppedthelineCalledachargenursetopreventthesurgeonfrombeginningthesurgerySanctionsAgreedtoproceed,butthreatenedthesurgeonwithsanctions(e.g.linganincidentreport)ShiftedResponsibilityVoicedconcern,butleftthedecisionuptothesurgeonGaveInInitiallyvoicedconcern,butlaterbackeddowntothesurgeonNoObjectionsAgreedwiththesurgeon'sdecision 5.2.3CodingbehaviorduringthespeakingupmomentsAsecondcodingschemewasusedtocodeparticipants'behaviorduringthespeakingupmoment,beforeparticipantsreachedanaloutcome.Thiscodingschemedescribedthetacticsparticipantsusedtopersuadethesurgeontowaittobeginthesurgery.ThiscodingschemewasderivedfromacodingschemedevelopedbyKipnis 81

PAGE 82

etal,whichwasdevelopedtodescribedhowpeopleattemptedtoinuencedecisionswithintheworkplace[ 129 ].Kipnisetal.refertothesebehaviorsas“inuencetactics”.Inuencetacticusagewascodedasfollows.Eachstatementmadebyparticipantswascodedasoneofsixinuencetacticsbytwodifferentresearchers.Afternishingtheinitialcoding,thetworesearchesthendiscussedtheirobservationsanddiscoveredsomemajordifferencesintheirinterpretationsofthetactics.Assuch,theythenmorepreciselydenedhoweachinuencetacticshouldbedenedwithinthispopulationandcontext;thesedenitionsarereportedinTable 5-2 .Theythenanalyzedandre-codedthetranscriptsusingtheseupdateddenitions.Afterthesecondroundofcoding,differenceswereminorandagreementwasquicklyreachedacrossallstudyparticipants.Additionalroundsofcodingwerenotrequired,asthecoderswereincompleteagreementabouthoweachparticipantshouldbecoded. Table5-2. ThesedenitionswereextractedfromthequestionsusedinKipnisetal'soriginalsurveyandthenadaptedtotthepopulationofthisstudy. InuenceTacticDenition RationalityAttemptstouselogicorreasontopersuadethesurgeonAssertivenessChallengesthesurgeonusingreferencestohospitalprocedure,forceofwill,orstatementsnotinvolvingreasonorinference.IngratiationAttemptstouseempathy,appeasement,ordeferencetopersuadethesurgeonUpwardAppealAsksanauthorityguretointerveneontheirbehalfCoalitionsAskssomeonewhoisnotanauthorityguretosupporttheirpositionExchangeOfferssomethingtothesurgeoninreturnforcompliance InuencetacticswereonlyexaminedduringtheSpeakingUpstudy,asthefocusofthisresearchwastodeterminewhethertheagencyofateammateaffectedhowparticipantsusedinuencetacticswiththatparticipant.Sincethesurgeon'sagency 82

PAGE 83

wasnotmanipulatedduringtheSurgicalCountstudy,inuencetacticusagewasnotconsidered. 5.3BehaviorDuringAgreementMomentsInthissection,Iconsiderwhetherparticipants'decisionsduringtheagreementmomentswereinuencedbytheagencyoftheirteammates.Specically,Iconsiderwhetherparticipantspreferentiallyagreewithhumanteammates,orwhetheragencyhasnoeffect.Toexplorethisquestion,Iconsiderparticipants'behaviorduringtheagreementmomentsthatoccurredduringStage2oftheSpeakingUpstudy(seeSection 4.1.1.3 forareviewoftheagreementmoments).Participantswereclassiedasmakingoneofthreedecisions:agreeingwiththesurgeon,agreeingwiththeanesthesiologist,orproposingacompromisesolution.Tables 5-3 and 5-4 reportthepercentageofparticipantswhomadeeachdecision,ineachcondition.Percentagesarereportedbecausethenumberofparticipantsineachconditionvariedslightly. Table5-3. Participantresponsestothemonitorsagreementmoment.Theanesthesiologist,whowasinfavorofwaitingtoputthemonitorson,askedparticipantstoweighinonthisquestion.Datafromtwoparticipantsismissingbecauseonedidnotconsenttovideorecordingandanotherdidnotrespondtothisquestion. ConditionNAgreewithAgreewithCompromiseSurgeonAnesthesiologist BothVirtual2626.9%(7)15.4%(4)57.7%(15)HumanSurgeon2040.0%(8)25.0%(5)35.0%(7)HumanAnesth.2133.3%(7)23.9%(5)42.8%(9) Fisherexacttestsrevealednosignicantdifferencesbetweenconditionsforeitherdecision(pMonitor=0.439,pBairHugger=0.380),whichsuggeststhatparticipants'behaviorwasnotinuencedbytheagencyoftheirteammates. DiscussionaboutAgreementMoments.Nosignicantdifferenceswereobservedbetweenconditionsinthedecisionsparticipantsmadeduringeitherofthe 83

PAGE 84

Table5-4. Participantresponsestothebairhuggeragreementmoment.Thesurgeon,whowasinfavorofusingjustthelower-bodybairhugger,askedparticipantstoweighinonthisquestion.Datafromfourparticipantsismissingbecauseonedidnotconsenttovideorecordingandthreedidnotrespondtothisquestion. ConditionNAgreewithAgreewithCompromiseSurgeonAnesthesiologist BothVirtual2552.0%(13)24.0%(6)24.0%(6)HumanSurgeon2161.9%(13)28.6%(6)9.5%(2)HumanAnesth.1942.1%(8)21.1%(4)36.8%(7) twoagreementmoments(pMonitor=0.439,pBairHugger=0.380).Accordingly,itcannotbeconcludedthattheagencyofparticipants'teammatesaffectedhowtheymadedecisionsduringtheseagreementmoments.Instead,participants'decisionsappeartohavebeenbasedonclinicalreasoning:manyparticipantsverballyreasonedthroughtheirdecisionbeforeansweringthequestion.Additionally,manyparticipantsattemptedtodevelopacompromisesolutionthatcouldaddresstheconcernsraisedbyboththesurgeonandtheanesthesiologist(see“Compromise”inTables 5-3 and 5-4 ). 5.4BehaviorDuringSpeakingUpMomentsInthissection,Iconsiderwhetherparticipants'behaviorduringthespeakingupmomentswasinuencedbytheagencyoftheirteammates.Iconsidertwoprimaryquestions,eachofwhichpertaintooneofthisdissertation'sprimaryresearchquestions:isiteasiertospeakuptoavirtualsurgeonthanahumansurgeon(DirectEffects),anddoeshavingahumanteammatealterhowyouspeakuptothevirtualsurgeon(IndirectEffects). 5.4.1DirectEffects:DoesaSurgeon'sAgencyAffectHowNursesSpeakUptoHimIrstconsiderwhethertheagencyofthesurgeonaffectedhowparticipantsspokeuptohim.Becausethesurgeon'sagencywasnotmanipulatedduringtheSurgicalCountstudy,IonlyconsiderdatacollectedduringtheSpeakingUpstudy.Forty-seven 84

PAGE 85

nursesareincludedinthisanalysis,26ofwhichworkedwiththevirtualsurgeonandthevirtualanesthesiologist,and21ofwhichworkedwiththehumansurgeonandthevirtualanesthesiologist.Table 5-5 reportsthenumberofnurseswhoexhibitedeachofthevespeakingupbehaviors.Figure 5-1 visualizesthesefrequenciesusingastackedbarcharttoallowforeasiercomparisonsbetweenconditions.Fisher'sexacttestrevealednosignicantdifferencesbetweenthethreeconditions(p=0.869). Table5-5. Speakingupbehaviorfrequenciesfortheconditionswherethesurgeon'sagencywasvaried. BehaviorBothVirtualHumanSurgeon StoppedtheLine19.2%(5)19.0%(4)Sanctions3.8%(1)0.0%(0)ShiftedResponsibility15.3%(4)9.5%(2)GaveIn19.2%(5)14.2%(3)NoObjections42.3%(11)57.1%(12) Figure5-1. Thepercentageofparticipantswhoemployedeachspeakingupbehavior.Outcomesareorderedbasedontheamountofresistanceshowntothesurgeon. Oftheseveoutcomes,nursingmanagementconsideredstoppingthelinetobetheonlyacceptableoutcome.Only20%ofparticipantsinbothconditionsresolved 85

PAGE 86

conictwiththesurgeonbystoppingtheline.Another25%to40%ofparticipants,dependingoncondition,voicedconcernaboutthesurgeon'sproposedcourseofaction,butfailedtostopthelineandallowedthesurgeontobeginthesurgery.Filinganincidentreport,whichwasthemostassertiveresolutionmethodthatstillfellshortofstoppingtheline,wasobservedoncewiththevirtualsurgeon,butdidnotoccurwiththehumansurgeon.Theremaining40%to55%ofparticipants,dependingoncondition,offerednoobjectionstothesurgeon'sproposedcourseofaction.Oftheseparticipants,sixdiscussedthedecisionwiththesurgeonwithoutofferingobjectionstoit;veofthesesixwereinthehumansurgeoncondition.Theremainingparticipantswhoofferednoobjectionsagreedimmediatelywiththesurgeon'sdecisiontoproceed. Discussion.Theseresultssuggestthatthesurgeon'sagencydidnotsubstantiallyaffecthowparticipantsbehavedduringthespeakingupmoment.Nosignicantdifferenceswereobservedbetweenconditions.Mostimportantly,asimilarnumberofparticipants(20%)stoppedthelineinbothconditions;thisiscritical,asstoppingthelinewasboththemostdesiredbehaviorandthemostdifcultbehavior.Althoughitmayseemsurprisingthatonly20%ofparticipantsstoppedtheline,thereareseveralfactorsthatcanmakeitdifcultfornursestostoptheline.Theperceptionofahierarchicalrelationshipbetweensurgeonsandnursescanmakespeakingupdifcult[ 130 ],andnursessometimesfeelthatasurgeon'smoreextensivemedicaltrainingmakeshimorhermorequaliedtomakedecisionsaboutpatientcare[ 131 ].Participantsworkingwiththevirtualsurgeonreferencedbothofthesedifcultiesduringthedebrieng.Sixparticipantsdeferredtothevirtualsurgeonforhierarchicalreasons,usingphraseslike“You'rethesurgeon,it'suptoyou”.Threeparticipantsstatedthatthevirtualsurgeon'sexperienceandpositionmadehimmorequaliedtomakethedecision.Fromatrainingperspective,itisencouragingthatparticipantsencounteredthesereal-worlddifcultieswhenspeakinguptothevirtualsurgeon.Ifspeakinguptovirtualhumansisnotdifcult, 86

PAGE 87

thenpracticingspeakingupwithvirtualhumansmaynotbehelppeopletoovercomebarrierstospeakingupintherealworld.Itisalsonoteworthythat,withinasimulation,participantsshiftedresponsibilitytothesurgeonorthreatenedtoleanincidentreport.Bothofthesebehaviorsareexamplesofself-protectivevoice[ 132 ]andrepresentattemptstoguardoneselffrompotentialrepercussionsshouldthesurgeon'sbehaviorharmthepatient.Inbothcases,thenursesoughttoestablishthatheorshehadobjectedtothesurgeon'sbehaviorbuthadbeenunabletostophim.Observingself-protectivebehaviorinthecontextofasimulationisimportant,asparticipantsknewthattherewouldbenoreal-worldrepercussionsiftheyfailedtoprotectthesimulatedpatient(theyhadbeeninformedthatthissimulationwasnotanevaluationandthattheirperformancewouldnotbereportedtomanagement).Accordingly,theuseofself-protectivebehaviorssuggeststhattheseparticipantsautomaticallyengagedinthesamebehaviorstheywoulduseduringreal-worldconict.Observingself-protectivebehaviorswithbothvirtualandhumansurgeonsfurtherstrengthenstheconclusionthatbehaviorwiththevirtualsurgeoncloselyapproximatedbehaviorwiththehumansurgeon. 5.4.2IndirectEffects:DoesthePresenceofaSecondHumanTeammateChangeHowNursesSpeakUptoaVirtualSurgeonInextconsiderwhethertheagencyofasecondteammateaffectedhownursesspokeuptothesurgeon.Theagencyofateammateotherthanthesurgeonwasmanipulatedinbothstudies,allowingustoexaminethisquestionusingdatafromboththeSpeakingUpstudyandtheSurgicalCountstudy. 5.4.2.1BehaviorduringtheSpeakingUpstudyForty-sevennursesareincludedinthisanalysis,26ofwhichwereintheBoth-Virtualcondition,and21ofwereintheHumanAnesthesiologistcondition.ThenumberofparticipantswhoexhibitedeachbehaviorisshownbelowinTable 5-6 .Figure 5-2 visualizesthesefrequenciesusingastackedbarcharttoallowforeasiercomparisons 87

PAGE 88

betweenconditions.Fisher'sexacttestrevealedatrendtowardssignicance(p=0.083). Table5-6. Speakingupbehaviorfrequenciesforthetwodifferentconditions. BehaviorBothVirtualHumanAnesth. StoppedtheLine19.2%(5)38.1%(8)Sanctions3.8%(1)19.0%(4)ShiftedResponsibility15.3%(4)0.0%(0)GaveIn19.2%(5)19.0%(4)NoObjections42.3%(11)23.8%(5) Figure5-2. Thepercentageofparticipantswhoemployedeachspeakingupbehavior.Outcomesareorderedbasedontheamountofresistanceshowntothesurgeon. 5.4.2.2BehaviorduringtherstspeakingupmomentoftheSurgicalCountstudyForty-ninenursesareincludedinthisanalysis.Onenursewasexcludedbecauseshehadassistedwiththedevelopmentofthescenario,andwasthusfamiliarwithitsobjectives.ThisspeakingupmomentwasverysimilartothemomentusedintheSpeakingUpstudy,thoughsomeminormodicationsweremade.Surgicaltechniciansarenotincludedinthisanalysis,astheywerenotresponsiblefortheissuerelatedto 88

PAGE 89

thespeakingupmomentinStage2oftheSurgicalCountstudy.ThenumberofnurseswhoemployedeachspeakingupbehaviorisreportedinTable 5-7 .Figure 5-3 visualizesthesefrequenciesusingastackedbarcharttoallowforeasiercomparisonsbetweenconditions.Fisher'sexacttestrevealednosignicantdifferences(p=0.968). Table5-7. Speakingupbehaviorfrequenciesforthetwodifferentconditions. BehaviorVirtualTechHumanTech StoppedtheLine38.4%(10)36.3%(8)Sanctions3.8%(1)0.0%(0)ShiftedResponsibility3.8%(1)0.0%(0)GaveIn15.3%(4)18.2%(4)NoObjections38.4%(10)45.5%(10) 5.4.2.3BehaviorduringthesecondspeakingupmomentoftheSurgicalCountstudyForty-ninenursesareincludedinthisanalysis.Onenursewasexcludedbecauseshehadassistedwiththedevelopmentofthescenario,andwasthusfamiliarwithitsobjectives.Anadditionalnursewasexcludedbecausesheclaimedtohavefoundthemissingboat,andthusavoidedthespeakingupmoment.ThenumberoftimeseachspeakingupbehaviorwasobservedisreportedinTable 5-8 .Figure 5-4 visualizesthesefrequenciesusingastackedbarcharttoallowforeasiercomparisonsbetweenconditions.Fisher'sexacttestrevealednosignicantdifferencesbetweenthethreeconditions(p=0.973). Table5-8. Speakingupbehaviorfrequenciesforthetwodifferentconditions. BehaviorVirtualTechHumanTech StoppedtheLine68.0%(17)68.1%(15)Sanctions12.0%(3)13.6%(3)ShiftedResponsibility0.0%(0)0.0%(0)GaveIn12.0%(3)9.1%(2)NoObjections8.0%(2)9.1%(2) 89

PAGE 90

Figure5-3. Thepercentageofparticipantswhoemployedeachspeakingupbehavior.Outcomesareorderedbasedontheamountofresistanceshowntothesurgeon. 5.4.2.4DiscussionTheseresultssuggestthattheagencyofasecondteammatedidnotsubstantiallyaffecthownursesspokeupwiththevirtualsurgeon.AtrendtowardssignicancewasobservedduringtheSpeakingUpstudy(p=0.083),butthistrendwasnotrepeatedineitherofthespeakingupmomentsintheSurgicalCountstudy(pFirstMoment=0.968,pSecondMoment=0.973).ItisinterestingtonotethatparticipantsspokeupmuchmoreoftenduringthesecondspeakingupmomentintheSurgicalCountstudy.Thisincreaseintherateofspeakingupappearstoberelatedtothestrongpoliciesinplacerequiringsurgeonstoconferwitharadiologistintheeventofamissingitem.Almost 90

PAGE 91

Figure5-4. Thepercentageofparticipantswhoemployedeachspeakingupbehavior.Outcomesareorderedbasedontheamountofresistanceshowntothesurgeon. everynursewhospokeuptothesurgeonreferencedthispolicy.Policiescanaddressbothofthedifcultiesmentionedearlierthatnursescanhavewhenspeakingup:theperceptionofahierarchicalrelationshipbetweensurgeonsandnurses,andtheperceptionthatsurgeonsaremorequaliedtomakemedicaldecisions.Bybasingtheirdecisionsonestablishedpolicies,nursesareempoweredbyanauthoritygreaterthananyindividualsurgeonandalsoleverageabodyofmedicalknowledgegreaterthananyindividualsurgeon. 91

PAGE 92

5.5InuenceTacticsUsageduringtheSpeakingUpstudyInthissection,Iconsiderwhethertheagencyofparticipants'teammatesinuencedthetacticstheyusedwhenseekingtoconvincethesurgeontowaittobeginthesurgery.Therearetwoprimaryquestionshere:dopeopleusedifferenttacticswhenspeakinguptoavirtualandahumansurgeon,anddoeshavingahumanteammatealterthetacticspeopleusewhilespeakingup.Onceagain,therstquestioncorrespondstotheDirectEffectresearchquestion,andthesecondtoIndirectEffectresearchquestion.IonlyconsiderinuencetacticusageintheSpeakingUpstudy,astheSurgicalCountstudydidnotmanipulatethesurgeon'sagency. 5.5.1DirectEffects:InuenceTacticusageintheSpeakingUpstudyOfthe48participants,only25(52%)attemptedtoinuencethesurgeon;theremaining23eitherdeferredtothesurgeonoractivelysupportedhisproposal.Assuch,these23(48%)participantsusednoinuencetacticsandwereexcludedfromthisportionoftheanalysis.Oftheremaining25participants,15workedwiththevirtualsurgeonand10workedwiththehumansurgeon.Achi-squaretestrevealednosignicantdifferencesbetweenconditionsinthenumberofparticipantswhoattemptedtoinuencethesurgeon(p=0.398).Figure 5-5 reportsthepercentageofparticipantswhousedeachtacticatleastonetime.Participantsfrequentlyusedmorethanoneinuencetactic,whichmeantthatinuencetacticusagecouldnotbeassessedusingaMANOVA,astheindependenceassumptionwasviolated.Instead,individualt-testswereconductedforeachofthesixinuencetactics.Becausesixdifferenttestswererun,thesignicancecriterionwasreducedbyafactorofsix,toavoidmakingaTypeIerror.Theadjustedsignicancecriterionwas0.00833.Nosignicantdifferenceswereobservedwiththissignicancecriterion.Figure 5-6 reportsthenumberofdifferenttacticsparticipantsused(e.g.usingrationalityandassertiveness).Figure 5-7 reportsthetotalnumberoftimesparticipants 92

PAGE 93

Figure5-5. Thepercentageofparticipantswhoemployedaspecictactic.Noparticipantsusedexchange. attemptedtoinuencethesurgeon(e.g.usingrationalitythreetimesandassertivenesstwice).Nosignicantdifferenceswereobservedinthenumberofdifferenttacticsusedbetweenthetwoconditions(p=1.000,Virtual=2.4,Human=2.4).Similarly,nosignicantdifferenceswereobservedinthetotalnumberoftimesparticipantsattemptedtoinuencethesurgeon(p=0.292,Virtual=4.87,Human=3.90). Figure5-6. Thisgraphreportshowmanyofthesixdifferenttacticswereusedbyparticipants. 93

PAGE 94

Figure5-7. Thisgraphreportsthetotalnumberoftimesparticipantsattemptedtoinuencethesurgeon. Figure 5-8 reportshowmuchtimeparticipantsspenttalkingduringthespeakingupmoment(thistimedoesnotincludetimewhenthesurgeonspokeorperiodsofsilence).Nosignicantdifferenceswereobservedinhowmuchtimeparticipantsspenttalkingtothehumanorvirtualsurgeonduringthespeakingupmoment(p=0.788,Virtual=29.01seconds,Human=27.45seconds). Figure5-8. Thetimereportedhereisspecicallywhenparticipantswerespeaking.Itdoesnotincludewhenthesurgeonspokeorperiodsofsilence. DiscussionofInuenceTacticsUsage.Theseresultssuggestthatthetacticsparticipantsusedwhenattemptingtostopthesurgeonfrombeginningthesurgerywerenotinuencedbywhetherthesurgeonwasrealorvirtual.Theresultsshowthatparticipantsusedinuencetacticssimilarlyinbothconditions:theyusedinuence 94

PAGE 95

tacticsatthesamerates,usedasimilarrangeoftactics,madeasimilarnumberofattemptstostopthesurgeon,andspentsimilaramountsoftimearguingregardlessofwhetherthesurgeonwasrealorvirtual.Whiletheseresultscannotdemonstratestatisticalequivalencebetweenbehaviorwiththerealandvirtualsurgeons,theydosuggestthatwhateverdifferencesmayexistareminorandunlikelytohindervirtualhumansfrombeingusedforspeakinguptraining.Severalresearchershavefoundthatrationalityisoneofthemostfrequentlyusedinuencetactics[ 129 , 133 ].Thisndingwasconrmedintheresultsforboththevirtualandhumansurgeon.Twenty-fouroutoftwenty-veparticipantsusedrationalitywhenattemptingtopersuadethesurgeontowaittobeginthesurgery.Surprisingly,assertivenesswasalsofrequentlyusedbyparticipants,despiteresultsintheliteraturesuggestingthatassertivenessisnotcommonlyusedwhenattemptingtoinuenceasuperior[ 129 ].Thehighincidenceofassertivenessduringthisstudymaybeduetothesurgeon'sunwillingnesstobeconvincedbyreason;ofthe14participantswhoemployedbothrationalityandassertiveness,9ofthemresortedtoassertivenessonlyafterrstattemptingtouserationalitytoconvincethesurgeontowaittobeginthesurgery.Participantsmayalsohavebeenmorewillingtouseassertivenessbecauseofthehighstakesnatureofthescenario.Mostinuencetacticsresearchhasbeenconductedinbusinesssettings,wherelivesarerarelyatstake.Participantsmayhavebeenmorewillingtouseassertivenessbecauseofthethreattotheirsimulatedpatient'ssafety. 5.5.2IndirectEffects:InuenceTacticusageintheSpeakingUpstudyOfthe48participantswhoworkedwiththevirtualsurgeon,31(64%)attemptedtoinuencethesurgeon;theremaining17eitherdeferredtothesurgeonoractivelysupportedhisproposal.Assuch,these17(36%)participantsusednoinuencetacticsandwereexcludedfromthisportionoftheanalysis.Oftheremaining31participants,15workedwiththevirtualanesthesiologistand16workedwiththehumananesthesiologist. 95

PAGE 96

Achi-squaretestrevealednosignicantdifferencesbetweenconditionsinthenumberofparticipantswhoattemptedtoinuencethesurgeon(p=0.368).Figure 5-9 reportsthepercentageofparticipantswhousedeachtacticatleastonetime.Participantsfrequentlyusedmorethanoneinuencetactic,whichmeantthatinuencetacticusagecouldnotbeassessedusingaMANOVA,astheindependenceassumptionwasviolated.Instead,individualt-testswereconductedforeachofthesixinuencetactics.Becausesixdifferenttestswererun,thesignicancecriteriawasreducedbyafactorofsix,toavoidmakingaTypeIerror.Theadjustedsignicancecriteriawas0.00833.Nosignicantdifferenceswereobservedwiththissignicancecriteria. Figure5-9. Thepercentageofparticipantswhoemployedaspecictactic.Noparticipantsusedexchange. 5.6ConclusionsInsummary,agencywasnotobservedtohaveanysignicanteffectonbehavior.ThewastrueforbothDirectEffectsandIndirectEffectsofagency.Participantsshowednosignofpreferentiallyagreeingwithhumanteammates.Participantsdidnotspeak 96

PAGE 97

upmoreoftentothevirtualsurgeonthanthehumansurgeon,andvaryingtheagencyofasecondteammatedidnothaveasignicanteffectonspeakingup.Participantsappeartohaveusedinuencetacticssimilarlybetweenconditions.Themannerinwhichtheyusedinuencetacticswasalsoconsistentwithpreviousresearchabouthowthesetacticsareemployedduringreal-worldconict.Participantsfrequentlyemployedmultipleinuencetactics,alteringbetweenthemoncetheyhadbeenunabletoconvincethesurgeonusingothertactics.Whenspeakingabouttheirexperiencespeakingupduringthetrainingexercise,participantsspokeofitintermsofreal-worldexperiences.Participantswhostoppedthelineexplainedtheirdecisionsintermsofdangertothepatientandhospitalpolicy.Participantswhodidnotstopthelineexplainedtheirdecisionsintermsofotheralternativesandauthoritystructureswithinthehospital.Participantsalsoreportedthatthevirtualsurgeonfeltrealisticbecauseofhisanger,impatience,andreasoning.Whilethelackofsignicantdifferencescannotbeusedtoclaimthatagencyhasnoeffectonbehavior,itdoessuggestthatwhateveraffectagencymayhaveonbehaviorissmall.Speakingupbehaviorwiththevirtualsurgeoncloselyapproximatedspeakingupbehaviorwiththehumansurgeon,andvaryingtheagencyofasecondteammatedidnotproduceanymeaningfulchangeinspeakingupbehavior.Theseresultssupporttheuseofvirtualhumansforteamtraining,asvaryingtheagencyofateammatedidnotproduceanymeaningfulchangesinspeakingupbehavior. 97

PAGE 98

CHAPTER6MIXED-AGENCYINTERACTIONSANDNONVERBALBEHAVIORThischapterexaminesparticipants'nonverbalbehavioralduringtheSpeakingUpstudy(seeSections 4.1 fordetailsaboutthisstudy).Severalsimilaritieswereobservedbetweengazewithrealandvirtualteammates,alongwithonemajordifference.Participantslookedatvirtualteammateswhentheyspoke,spentmoretimelookingatvirtualteammateswhospokemorefrequently,anddidnotviolatepolitenessnormsbystaringatvirtualteammates.However,participantsdidspendsignicantlymoretimelookingatvirtualteammatesthanhumanteammates(upto75%longer).Thiseffectwasparticularlypronouncedduringperiodsofsilence,whichsuggeststhatthiseffectcouldbeduetouncertaintyaboutwhethervirtualteammateshadunderstoodthem.Thepresenceofahumanteammatedidnothaveamarkedeffectonparticipants'gazebehaviorwiththeirvirtualteammates;instead,gazewithaspecicvirtualteammateremainedconsistentregardlessoftheagencyoftheotherteammate.TheresultsdiscussedinthischapterwillbesubmittedforpublicationatIEEEVR2016.PersonalContributions:Idevelopedthenonverbalbehaviorcodingscheme,determinedhowtoanalyzethedata,andperformedtheanalysis.Collaborators:Dr.AndreaKleinsmithassistedwiththedevelopmentofthecodingscheme.Thevideoswerecodedbythreeundergraduatestudentswhowerehiredtoperformthecoding.RelevancetoThesisThischapterexploreswhethermixed-agencyteamsalternonverbalbehaviorwithvirtualhumans.Specically,thischapterexaminesgazebehavior. 6.1ResearchQuestionsandMotivationThisdissertationexplorestwoquestionsrelatedtointeractionswithvirtualmembersofmixed-agencyteams.Therstresearchquestionfocusesonhowinteractionswithaspecicteammatechangewhenthatteammate'sagencyismanipulated(e.g.replacing 98

PAGE 99

avirtualsurgeonwithahumansurgeon).Theterm“DirectEffect”isusedtorefertothisresearchquestion,asitdealswithhowmanipulatingaspecicteammate'sagencydirectlyimpactsinteractionswiththatteammate.Thesecondresearchquestionfocusesonwhetherthepresenceofhumanteammatesaltersbehaviorwithvirtualteammates(e.g.doesyourbehaviorwithavirtualanesthesiologistchangewhentheteam'svirtualsurgeonisreplacedbyahumansurgeon).Theterm“IndirectEffect”isusedtorefertothisresearchquestion,asitdealswithhowmanipulatingoneteammate'sagencyindirectlyimpactsinteractionswithadifferentvirtualteammate.BothofthesequestionsarediscussedinmoredetailinSection 1.2 andillustratedinFigure 1-1 .Withinthecontextofthesetwoquestions,thischapterinvestigatesnonverbalbehaviorwithmixed-agencyteams.Specically,Idiscusshowgazebehaviorisimpactedbyalteringtheagencyofone'steammates.ThisanalysisisbasedondatacollectedduringtheSpeakingUpstudy(seeSection 4.1 foradiscussionoftheSpeakingUpstudy).Gazebehaviorwascodedbasedonvideorecordingscollectedduringthestudy.Thisdatawasthenprocessedtoextractkeyfeatures,includingtheaverageamountoftimespentlookingateachteammate,thenumberofgazexationmomentswitheachteammate,theaveragedurationofeachgazexation,andtherelationshipbetweengazeandspeech.Theresearchdiscussedinthissectionismotivatedbythelargelyautomaticandunconsciousnatureofnonverbalbehavior.Whilenonverbalbehaviorcanbeconsciouslycontrolled,nonverbalbehaviorispredominatelyautomatedandunconscious.Examplesoftheunconsciousnatureofmostnonverbalbehaviorincludethechameleoneffect[ 134 ],wherepeopleautomaticallymimicthebehaviorandmannerismsofconversationpartners,“leakage”[ 135 ],whereimplicitbiasesandprejudicesareunconsciouslycommunicatedthroughgazeandposture,andcorrelationsbetweennonverbalbehaviorandinternalpsychologicalstates,suchasrapport[ 136 ],emotion[ 137 ],anddeception 99

PAGE 100

[ 138 ].Nonverbalbehavior'sautomaticandunconsciousnaturemakesitavaluableopportunitytoassessthedegreetowhichrealisticbehaviorwithvirtualhumansisactuallygovernedbythesamerulesgoverningbehaviorwithotherhumans.Nassetal.[ 39 ]andKimetal.[ 139 ]arguethatpeopleautomaticallyandmindlesslyapplysocialrulestointeractionswithcomputers,whichexplainswhywefrequentlyobservesimilaritiesbetweeninteractionsbetweenpeopleandinteractionswithcomputers.However,itisunclearwhetherrealisticbehavioroccursbecausepeopleautomaticallybehaverealistically,orbecausepeoplechoosetobehaverealistically.Similaritiesinnonverbalbehavior,whichislargelyoutsideofconsciouscontrol,wouldprovideadditionalsupportforthehypothesisthatrealisticbehaviorwithvirtualhumansisamindlessoccurrence. 6.2CollectionandPreparationofNonverbalData 6.3CodingofNon-VerbalBehaviorDuringtheSpeakingUpstudy,videodatawasrecordedfor65participants.Videowasrecordedat30FPSandaresolutionof640x480.ArepresentativeframefromthevideosisshowninFigure 6-1 .Fourparticipantsdidnotgiveconsenttorecordvideo,andwerethusexcludedfromthisanalysis.Thefollowinggazebehaviorswerecodedusingtherecordedvideo:lookingatthesurgeon,theanesthesiologist,thepatient,aclipboard,orinoneofseveraldirections(up,downleft,right,straightahead).ThecodingprocedureusedwasadaptedfromtheprocedureusedtocreatetheDistressAnalysisCorpus[ 140 ].CodingwasperformedusingANVIL[ 141 ].Threestudentannotatorswerehiredtoperformthecoding.AnnotatorsweretrainedhowtouseANVILandhowtoperformgazecoding.Afterreceivingsomebasicinstruction,annotatorscodedtrainingvideosuntiltheirinter-raterreliability(Krippendorff'salpha)exceeded0.7.Trainingvideoswererandomlyselectedfromthevideosofparticipants.Oncetrainingwascomplete,eachannotatorwasassignedaspecicsetofvideostocode.Approximately10%ofthevideoswere 100

PAGE 101

Figure6-1. Anexampleofthevideosusedfornonverbalanalysis. assignedtomultiplecoders.Thesevideoswereusedtomonitortheannotators'performanceandensurethattheymaintainedinter-raterreliabilitythroughthecodingprocess.Annotatorswereawarethattheirperformancewasbeingmeasured,butwerenotawareofwhichvideoswereusedtomonitortheirperformance.Uponcompletionofthecoding,inter-raterreliabilitywasassessedforthetensharedvideos.Gazebehaviorcodingyieldedanaverageinter-raterreliabilityof=0.856.Gazecoderswerenotblindtotheconditionofthevideostheycoded,asthisinformationcouldbededucedfromthevideosthemselves.However,coderswerenotinformedaboutthehypothesesofthestudyorthepurposeofthegazecoding,makingitunlikelythattheircodingwasinuencedbytheirawarenessofthedifferentconditions.ThetimesduringwhicheachteammatespokewasalsocodedusingPraat[ 142 ],aspeechannotationsystem.Speechtimingswerecodedbyasecondsetofthreecoders.Inter-raterreliabilitywasnotcalculatedforspeechtimingsduetolowsubjectivity 101

PAGE 102

ofidentifyingwhensomeoneisspeaking.Uponcompletion,speechcodeswerethencombinedwiththegazecodesusingANVIL. PreparingtheDataforAnalysis.Gazebehaviorwascodedusingninecodes.“Surgeon”,“Anesthesiologist”,“Patient”,and“Clipboard”describedover95%ofthecodeddata.Theremainingninecodeswerecollapsedintoasingle“Other”code.Theanalysisinthissectionfocusesonthe“Surgeon”andthe“Anesthesiologist”codes.Gazedatawascodedsequentially.Eachmomentcouldbeassignedtoexactlyonecode.Nodatawasleftuncoded,andmultiplecodescouldnotbeassignedtoaspecicmoment.AnexamplesectionoftheGazecodetrackinANVILisshownbelow,inFigure 6-2 .ThesecodeswererepresentedinXMLwithastarttime,stoptime,andlabel.Duringmyanalysis,Irefertoaspeciccodeasa“gazexation”,referringtowheregazewasxedduringthatperiodoftime. Figure6-2. AsampleportionofthegazecodingtrackinANVIL. AfterfeatureshadbeenextractedfromthecodedANVILles,theywerenormalizedtorepresentaninteractionlastingexactly250seconds,theaveragedurationofaninteractionduringStage2oftheSpeakingUpstudy.Thisnormalizationwasperformedtocorrectforsystematicvariationbetweenconditions.Participantsintheboth-virtualconditionspent,onaverage,20extrasecondsinteractingwiththeirteammates,orabout8%.TheanalysisinthissectionparallelstheanalysisthatwasperformedontheSocialPresencedatainChapter 7 .Theconditionwherebothteammateswerevirtualisusedasabaselineagainstwhichotherconditionsarecompared.Accordingly,eachteammateisexaminedinaseparateANOVA,ratherthanasanadditionalfactorwithinasingleANOVA.AsingleANOVAcouldnotbeperformedbecauseteammateisapartiallyrepeated-measuresfactor.ParticipantsintheBoth-Virtualcondition 102

PAGE 103

contributeddataontwoteammateswiththesameagency,whileparticipantsintheHuman-SurgeonandHuman-Anesthconditionscontributedataontwoteammateswithdifferentagency.IdonotcomparetheHuman-SurgeonandHuman-Anesthconditions,asthiscomparisonvariestheagencyofbothteammatessimultaneously,makingitdifculttodrawconclusionsfromthisdata. 6.4ResultsInthissection,Iconsiderseveralaspectsofparticipants'gazebehavior:thetotalamountoftimespentlookingattheirteammates,whetherspeechinteractedwithgazeduration,thedistributionofindividualgazexationmoments,andtheamountofsilencebetweenwhenthesurgeonortheanesthesiologisthadnishedspeakingandthenursebeganspeaking. 6.4.1DescriptiveStatisticsAfterprocessing,thefollowingnumberofparticipantsremainedineachcondition:23intheconditionwithtwovirtualhumans,20intheconditionwiththehumansurgeon,and20intheconditionwiththehumananesthesiologist.Sixparticipantswereexcludedeitherbecausetheydidnotconsenttovideo,orbecauseofabnormalbehaviorduringthetrainingexercisethatsignicantlyaffectedtheirgazebehavior(e.g.havingaconversationwiththestudyproctorinthemiddleoftheexercise). Table6-1. Averagetimeinsecondsforstages2and3,bycondition.Standarddeviationsareshowninparentheses. ConditionNStage2DurationStage3Duration BothVirtual23258.8(31.3)122.5(42.4)HumanSurgeon20235.5(38.4)114.1(37.5)HumanAnesth.20238.6(24.8)130.2(27.3) Theaveragedurationofstage2and3isreportedinTable 6-1 .Atwo-wayANOVArevealedasignicantmaineffectofstage(F(1,62)=113.7,p<0.001),butnomaineffectforcondition(F(2,61)=0.837,p=0.435).Theinteractioneffectbetweenstage 103

PAGE 104

andconditionwasalsonotsignicant(F(2,61)=1.724,p=0.183).Thesignicantdifferenceintimebetweenstageswasduetothedifferentcontentofeachstage. 6.4.2TotalGazeDurationTable 6-2 reportstheaverageamountoftimeparticipantsspentlookingateachteammateduringtheinteraction. Table6-2. Percentageoftimeparticipantsspentlookingatthesurgeon,theanesthesiologist,orelsewhereduringStage2and3.Otherincludeslookingatthepatient,theclipboard,oratotherportionsoftheroom. ConditionSurgeonAnesth.Other BothVirtual34.7%19.7%45.6%HumanSurgeon23.6%19.1%57.3%HumanAnesth.30.6%11.2%58.2% ProportionsforStage2 ConditionSurgeonAnesth.Other BothVirtual48.9%20.8%28.1%HumanSurgeon29.3%27.6%43.1%HumanAnesth.45.9%11.9%42.2% ProportionsforStage3 6.4.2.1Agency'sDirectEffectsToexplorewhetherparticipantsspentdifferentamountoftimelookingatvirtualteammatesandhumanteammates,mixedfactorialANOVAswereconductedforthesurgeonandtheanesthesiologist,whereagencyservedasabetween-subjectsfactorandstageservedasarepeated-measuresfactor.Gazewiththevirtualsurgeonwascomparedtogazewiththehumansurgeon,andgazewiththevirtualanesthesiologistwascomparedtogazewiththehumananesthesiologist.AsignicantmaineffectwasobservedforaDirectEffectofagencyforthesurgeon(F(1,42)=18.350,p<0.001)andtheanesthesiologist(F(1,42)=24.970,p<0.001),whereparticipantsspentmoretimelookingatvirtualteammatesthanhumanteammates.Themaineffectofstagewasnotsignicantforeitherthesurgeon 104

PAGE 105

(F(1,42)=2.798,p=0.098)ortheanesthesiologist(F(1,42)=0.235,p=0.629).Theinteractionbetweenagencyandstagewasnotsignicantforthesurgeonortheanesthesiologist.AninteractionplotvisualizingtheDirectEffectofagencyongazeisshownbelowinFigure 6-3 . Figure6-3. Participantsspentmoretimelookingatvirtualteammatesthanhumanteammates.Errorbarsreportthe95%condenceintervals.Signicanceisindicatedusingthefollowingsyntax:NS=p>0.05;=p<0.05;=p<0.01;=p<0.001 Onaverage,participantsspent66%moretimelookingatvirtualteammatesthanhumanteammates.Thispatternwaspresentinbothstages,asindicatedbythelackofasignicantmaineffectforStage. 6.4.2.2IndirectEffectToexplorewhethertheamountoftimeparticipantsspentlookingatvirtualteammateswasinuencedbythepresenceofasecondhuman,mixedfactorialANOVAswereconductedforthesurgeonandtheanesthesiologist,whereagencyservedasabetween-subjectsfactorandstageservedasarepeated-measuresfactor.Asareminder,theIndirectEffectofagencyreferstochangesintheagencyofadifferentteammate.Thus,gazewiththevirtualsurgeoniscomparedwhentheagencyoftheanesthesiologistisvaried,andgazewiththevirtualanesthesiologistiscomparedwhentheagencyofthesurgeonisvaried. 105

PAGE 106

Themaineffectofagencywasnotsignicantforthesurgeon(F(1,42)=2.083,p=0.153)orfortheanesthesiologist(F(1,42)=0.675,p=0.414).Asignicantmaineffectwasobservedforstageforthesurgeon(F(1,42)=4.150,p<0.05),butnotfortheanesthesiologist(F(1,42)=1.677,p=0.199).Theinteractionbetweenagencyandstagewasnotsignicantforthesurgeonortheanesthesiologist.AninteractionplotvisualizingtheIndirectEffectofagencyongazeisshownbelowinFigure 6-4 . Figure6-4. Theagencyoftheotherteammatehadnoeffectongazebehaviorwithavirtualteammate.Errorbarsreportthe95%condenceintervals.Signicanceisindicatedusingthefollowingsyntax:NS=p>0.05;=p<0.05;=p<0.01;=p<0.001 Havingasecondhumanteammatedidnotaffecttheamountoftimeparticipantsspentlookingatvirtualteammates.Therewasasmalleffectrelatedtothestageoftheinteraction,howeverthiseffectdidnotinteractwiththeagencyofparticipants'teammates.Thiseffectcanlikelybeattributedtochangesincontentbetweenstages2and3. 6.4.3GazeBehaviorandSpeechAsgazebehaviorcanbeinuencedbyspeech[ 143 ],Inowexaminewhethertheobserveddirecteffectofagencyismoderatedbyspeech.Theindirecteffectofagencyisnotconsideredhere,becauseitwasnotobservedtohaveasignicanteffectongazetime.Participants'gazebehaviorwassegmentedintofourperiodsbasedonwhich 106

PAGE 107

Table6-3. Thesetablesreporttheaverageproportionofastagewheretheparticipantgazedatthesurgeonortheanesthesiologistwhilevariousteammateswerespeaking.Proportionsdonotsumto100%becauseparticipantsdidnotalwayslookatthesurgeonortheanesthesiologist,butalsolookedatthepatient,theclipboard,andotherportionsoftheroom.Proportionsaresmallerfortheanesthesiologistbecauseparticipantsdidnotlookattheanesthesiologistasmuch,givenhissecondaryroleduringtheexercise. ConditionGazingattheSurgeonGazingattheAnesthesiologist SurgeonAnesth.NurseSilenceSurgeonAnesth.NurseSilenceSpeakingSpeakingSpeakingSpeakingSpeakingSpeaking BothVirtual16.4%1.3%6.1%11.5%3.1%6.7%3.3%6.8%HumanSurgeon.13.4%0.8%4.5%4.9%4.1%6.2%3.0%5.7%HumanAnesth.14.6%1.6%4.6%10.0%1.8%4.5%1.8%3.1% ProportionsforStage2 ConditionGazingattheSurgeonGazingattheAnesthesiologist SurgeonAnesth.NurseSilenceSurgeonAnesth.NurseSilenceSpeakingSpeakingSpeakingSpeakingSpeakingSpeaking BothVirtual24.6%0.4%9.1%15.6%4.7%4.6%3.1%8.4%HumanSurgeon.19.7%0.8%5.0%5.0%7.9%6.8%3.6%9.6%HumanAnesth.24.5%0.5%8.6%13.2%3.7%2.9%1.8%3.5% ProportionsforStage3 teammatewasspeaking(thesurgeon,theanesthesiologist,thenurse,ornoone).Becauseitispossibleformultiplepeopletospeaksimultaneously,thesesegmentedperiodsarenotcompletelydisjoint.However,thevariousteammatesrarelyspokeovereachotherorinterruptedeachother,soitisunlikelythatthisaffectstheresultsdiscussedinthissection.TheaverageproportionofaninteractionthatwasspentlookingatthesurgeonandtheanesthesiologistwhilevariousteammateswerespeakingisshowninTable 6-3 .Toexplorewhethertheamountoftimespentlookingataspecicteammateinteractedwiththatteammate'sagencyandwhowasspeaking,mixedfactorialANOVAswereconductedforthesurgeonandtheanesthesiologist,wherethedirecteffectofagencyservedasabetween-subjectsfactorandspeakerandstageservedasrepeated-measuresfactors.Signicantmaineffectswerefoundforadirecteffectof 107

PAGE 108

agencyforthesurgeon(F(1,42)=30.534,p<0.001)andfortheanesthesiologist(F(1,42)=49.433,p<0.001).Signicantinteractionseffectswereobservedbetweenagencyandspeakerforthesurgeon(F(3,40)=5.267,p<0.01)andtheanesthesiologist(F(3,40)=4.482,p<0.01).Signicantinteractioneffectswerenotobservedbetweenagencyandstageforthesurgeonortheanesthesiologist.Post-hocTukeytestswereconductedtoexploretheinteractioneffectbetweenagencyandspeaker.Thisrevealedasignicantdirecteffectofagencyduringperiodsofsilenceforthesurgeon(p<0.001)andtheanesthesiologist(p<0.001).Onaverage,participantsspent156%moretimelookingavirtualteammatesthanhumanteammatesduringperiodsofsilence.Asignicantdifferencewasalsoobservedforgazewiththeanesthesiologistwhiletheanesthesiologistwasspeaking(p<0.05),whereparticipantsspent52%moretimelookingatthevirtualanesthesiologistthanthehumananesthesiologistwhiletheanesthesiologistwasspeaking.Similarly,atrendtowardssignicancewasobservedforgazewiththesurgeonwhenthesurgeonwasspeaking(p=0.057),whereparticipantsspent24%moretimelookingatthevirtualsurgeonthanthehumansurgeonwhilethesurgeonwasspeaking.Noothertrendstowardssignicancewereobserved.AninteractionplotvisualizingtheeffectofSpeakerandthedirecteffectsofagencyongazedurationisshownbelowinFigure 6-5 .Severalimportantobservationscanbemadefromthisdata.First,itcanbeseenthatthedirecteffectofagencyongazewasmoderatedbyspeaker.Participantsdidnotspentsignicantlymoretimelookingatvirtualteammateswhileparticipantswerespeaking,orwhileadifferentteammatewasspeaking.Gazesignicantlyincreasedonlyduringperiodsofsilenceandwhenthevirtualteammatewasspeaking.Itisalsoclearthattherearesimilaritiesbetweengazewithrealandvirtualteammates.Participantslookedatvirtualteammateswhenparticipantsspoketothemandalsolookedatthevirtualteammateswhentheywerespeaking. 108

PAGE 109

Figure6-5. Thesegraphsreporttheamountoftimespentlookingataparticularteammatewhenvariousteammateswerespeaking.Eachboxrepresentsaspecicspeaker,whoisprintedatthetopofthebox.Errorbarsreportthe95%condenceintervals.Signicanceisindicatedusingthefollowingsyntax:NS=p>0.05;=p<0.05;=p<0.01;=p<0.001 6.4.4GazeFixationsAnalysisHavingseenthatagencydidhaveadirecteffectongazeoverthecourseoftheentireinteraction,Inowconsiderwhetheragencyhadadirectorindirecteffectontheindividualgazexationmoments.Ideneagazexationmomentasasinglecontinuousperiodduringwhichaparticipant'sgazeremainedxedonaspecicteammate.Inthissection,Iconsiderwhetheragencydirectlyorindirectlyaffectedthetotalnumberofgazexationmoments,theiraverageduration,andthedistributionoftheirdurations.Aseachparticipant'sinteractionvariedinlength,eachparticipant'stotalnumberofgazexationmomentswasnormalizedtorepresenta250secondinteraction.Normalizationwasperformedbyscalingthetotalnumberofgazexationmomentsbyafactorequaltotheactuallengthoftheinteractiondividedby250seconds.TheaveragecountanddurationofgazexationmomentsareshowninTable 6-4 . 6.4.4.1NumberofGazeFixationMoments DirectEffects.Toexplorewhetherthenumberofgazexationmomentswithaspecicvirtualteammatediffersfromthenumberofgazexationmomentswitha 109

PAGE 110

Table6-4. Thesetablesreporttheaveragecountanddurationofgazexationmomentswiththesurgeonandtheanesthesiologistineachcondition.Gazexationcountswerenormalizedtorepresentthenumberthatwouldoccurina250secondinteraction. ConditionSurgeonAnesthesiologist CountDurationCountDuration BothVirtual42.42.2532.81.66HumanSurgeon27.52.3224.32.09HumanAnesth.33.52.7519.81.65 GazexationcountsandaveragedurationforStage2 ConditionSurgeonAnesthesiologist CountDurationCountDuration BothVirtual40.13.8430.91.95HumanSurgeon29.12.6025.42.78HumanAnesth.33.14.5717.61.89 GazexationcountsandaveragedurationforStage3 humancounterpart,mixedfactorialANOVAswereconductedforthesurgeonandtheanesthesiologist,wherethedirecteffectofagencyservedasabetween-subjectsfactorandstageservedasarepeated-measuresfactor.Asignicantmaineffectwasobservedforadirecteffectofagencyforthesurgeon(F(1,43)=20.768,p<0.001)andfortheanesthesiologist(F(1,43)=16.024,p<0.001).Themaineffectofstagewasnotsignicantforthesurgeon(F(1,43)=0.095,p<0.759)orfortheanesthesiologist(F(1,43)=0.454,p<0.502).Theinteractionbetweenagencyandstagewasnotsignicant.AninteractionplotvisualizingthedirecteffectofagencyonthenumberofgazexationmomentsisshownbelowinFigure 6-6 .Theseresultsindicatethatparticipantslookedatvirtualteammatesmorefrequentlythanhumanteammates.Onaverage,participantsmade30%moregazexationmomentswiththevirtualsurgeonthanthehumansurgeon,and51%moregazexationmomentswiththevirtualanesthesiologistthanthehumananesthesiologist. 110

PAGE 111

Figure6-6. Participantshadmoregazexationmomentswithvirtualteammatesthanhumanteammates.Errorbarsreportthe95%condenceintervals.Signicanceisindicatedusingthefollowingsyntax:NS=p>0.05;=p<0.05;=p<0.01;=p<0.001 IndirectEffect.Toexplorewhetherthepresenceofasecondhumanaffectsthenumberofgazexationmomentswithvirtualteammates,mixedfactorialANOVAswereconductedforthesurgeonandtheanesthesiologist,wheretheindirecteffectofagencyservedasabetween-subjectsfactorandstageservedasrepeated-measuresfactors.Asignicantmaineffectwasobservedforanindirecteffectofagencyforthesurgeon(F(1,43)=5.870,p<0.05)andfortheanesthesiologist(F(1,43)=6.198,p<0.05).Themaineffectofstagewasnotsignicantforthesurgeon(F(1,43)=0.246,p=0.622)orfortheanesthesiologist(F(1,43)=0.062,p=0.805).Theinteractionbetweenagencyandstagewasnotsignicant.AninteractionplotvisualizingtheindirecteffectofagencyonthenumberofgazexationmomentsisshownbelowinFigure 6-7 .Theseresultsindicatethatthepresenceofasecondhumandidindirectlyaffectgazexationmomentswithvirtualteammates.Thiseffectwasmarkedlyweakerthanthedirecteffectofagencyongazexationmoments.Whileparticipantsmade30%to51%moregazexationmomentswithvirtualteammates,comparedtotheirhumancounterparts,thepresenceofasecondhumanactuallyreducedgazexationmoments 111

PAGE 112

Figure6-7. Participantshadslightlyfewergazexationmomentswithavirtualteammatewhenasecondhumanteammatewaspresent.Errorbarsreportthe95%condenceintervals.Signicanceisindicatedusingthefollowingsyntax:NS=p>0.05;=p<0.05;=p<0.01;=p<0.001 withvirtualteammates,butonlybyafactorof5.9%withthesurgeonand4.4%withtheanesthesiologist.Thedirectandindirecteffectsofagencyongazexationmomentsarelikelyrelated.Anexaminationofthedatarevealedthatparticipantssometimeslookedbackandforthbetweenthesurgeonandtheanesthesiologist,orwouldbrieyglanceatthesurgeonastheirgazemovedfromtheanesthesiologisttotheclipboard.Inthissituation,areductioningazexationmomentswithoneteammatewouldalsoleadtoareductioningazexationmomentswiththeotherteammate,astherewouldbefewertransitionsbetweengazetargets.Considerthesituationwherethesurgeonisvirtualandtheanesthesiologistishuman.Inthiscase,participantslookedatthevirtualsurgeon5.9%lessoften,andatthehumananesthesiologist30%lessoften,comparedtotheconditionwherebothwerevirtual.Asparticipantsmadeeyecontactwiththeanesthesiologistlessoften,thisinherentlyreducedthenumberoftimesparticipantscouldglancefromtheanesthesiologisttothesurgeonandthenbacktotheanesthesiologist.Thisalsoreducedthenumberofopportunitiestoglanceatthesurgeonwhenlookingfromtheanesthesiologisttotheclipboard.Inlightofthis,itseemslikelythattheobservedindirect 112

PAGE 113

effectofagencyongazeisnotatrueeffect,butinsteadonlyaconsequenceofthedirecteffectofagencyongaze. 6.4.4.2DurationofGazeFixationMoments DirectEffects.Toexplorewhethertheaveragedurationofgazexationmomentswithaspecicvirtualteammatediffersfromtheaveragedurationofgazexationsmomentswithahumancounterpart,mixedfactorialANOVAswereconductedforthesurgeonandtheanesthesiologist,wherethedirecteffectofagencyservedasabetween-subjectsfactorandStageservedasarepeated-measuresfactor.Themaineffectofdirectagencywasnotsignicantforthesurgeon(F(1,43)=1.906,p=0.171)ortheanesthesiologist(F(1,43)=0.730,p=0.396).Themaineffectofstagewasnotsignicantforthesurgeon(F(1,43)=2.342,p=0.129)ortheanesthesiologist(F(1,43)=2.452,p=0.121).Theinteractionbetweenagencyandstagewasnotsignicant.AninteractionplotvisualizingthedirecteffectofagencyonthenumberofgazexationmomentsisshownbelowinFigure 6-8 . Figure6-8. Agencydidnotaffecttheaveragedurationofgazexationmoments.Errorbarsreportthe95%condenceintervals.Signicanceisindicatedusingthefollowingsyntax:NS=p>0.05;=p<0.05;=p<0.01;=p<0.001 Theseresultssuggestthatwhileparticipantslookedatvirtualteammatesmorefrequently,theindividualgazexationmomentsdidnotlastlonger.Oneimplicationof 113

PAGE 114

thisisthatparticipantsdidnotstareatvirtualteammatesforlongerperiodsoftimethanhumanteammates. IndirectEffect.Toexplorewhetherthepresenceofasecondhumanaffectstheaveragedurationofeachgazexationmomentspentlookingatvirtualteammates,mixedfactorialANOVAswereconductedforthesurgeonandfortheanesthesiologist,wheretheindirecteffectofagencyservedasabetween-subjectsfactorandstageservedasarepeated-measuresfactor.Themaineffectofindirectagencywasnotsignicantforsurgeon(F(1,43)=0.238,p=0.627)butwasforanesthesiologist(F(1,43)=8.390,p<0.01).Themaineffectofstagewasnotsignicantforthesurgeon(F(1,43)=1.530,p=0.220)orfortheanesthesiologist(F(1,43)=1.293,p=0.259).Theinteractionbetweenagencyandstagewasnotsignicantforeitherteammate.AninteractionplotvisualizingtheindirecteffectofagencyonthedurationofgazexationmomentsisshownbelowinFigure 6-9 . Figure6-9. Haveasecondhumanteammatedidnotaffectgazedurationwiththevirtualsurgeon,butdidslightlyincreasetheaveragedurationofgazexationmomentswiththevirtualanesthesiologist.Errorbarsreportthe95%condenceintervals.Signicanceisindicatedusingthefollowingsyntax:NS=p>0.05;=p<0.05;=p<0.01;=p<0.001 114

PAGE 115

Figure6-10. Thedistributionsforgazexationdurationsarealmostidenticalregardlessoftheteammate'sagency. Figure6-11. Asmallshiftinthedistributionofgazexationdurationswiththeanesthesiologistcanbeobserved,butthedistributionsremainremarkablysimilar. 6.4.4.3DistributionofGazeFixationMomentsFigures 6-10 and 6-11 showshowgazexationdurationsweredistributedforthesurgeonandtheanesthesiologist,color-codedbyagency.Whilesomeminordifferencesbetweenthedistributionscanbenoted,suchasareductioninshortglanceswiththevirtualanesthesiologistwhenthehumansurgeonispresent,thedistributionsareremarkablyconsistentbetweenconditions.Allofthedistributionsarecharacterizedbyapeakcenteredaround0.75seconds,followedbyalongandthintail.Thedistributionwhichdiffersmostfromtheothersisthedistributioncorrespondingtoanindirecteffectofagencyongazexationmomentswiththeanesthesiologist.Thiscomparisonwasalsotheonlydifferenceforwhichasignicantdifferencewasobservedontheaveragegazexationduration. 115

PAGE 116

Theseresultsfurthersupporttheconclusionthatparticipantswerenotmorelikelytostareatvirtualteammatesthanhumanteammates.Eveninthecaseoftheanesthesiologistweseeonlyasmallshifttowardsmoreprolongedgaze. 6.4.5SilenceInlightofthestronglysignicanteffectofagencyongazeduringperiodsofsilence,Ialsoexaminethedurationofmomentsofsilencebetweenwhenthesurgeonoranesthesiologistspokeandthenurseresponded. 6.4.5.1DirectEffectsToexplorewhethertheagencyofaspecicteammateaffectedtheamountoftimeparticipantswaitedbeforerespondingtothatteammate,mixedfactorialANOVAswereconductedforthesurgeonandtheanesthesiologist,wherethedirecteffectsofagencyservedasabetween-subjectsfactorandStageservedasarepeated-measuresfactor.Themaineffectofdirectagencywassignicantforthesurgeon(F(1,42)=13.035,p<0.001),butwasnotfortheanesthesiologist(F(1,42)=1.892,p=0.173).Themaineffectofstagewasnotsignicantforthesurgeon(F(1,42)=2.222,p=0.140)ortheanesthesiologist(F(1,42)=0.172,p=0.679).Nosignicantinteractionswereobservedbetweenagencyandstage.AninteractionplotvisualizingthedirecteffectofagencyonthenumberofgazexationmomentsisshownbelowinFigure 6-12 . 6.4.5.2IndirectEffectToexplorewhetherthepresenceofasecondhumanaffectedtheamountoftimeparticipantswaitedbeforerespondingtoavirtualteammate,mixedfactorialANOVAswereconductedforthesurgeonandtheanesthesiologist,wheretheindirecteffectofagencyservedasabetween-subjectsfactorandstageservedasarepeated-measuresfactor.Themaineffectofindirectagencytrendedtowardssignicanceforthesurgeon(F(1,42)=3.815,p=0.055),andwasnotsignicantfortheanesthesiologist(F(1,42)=2.425,p=0.124).Themaineffectofstagewassignicantforthesurgeon(F(1,42)=4.509,p<0.05),butwasnotsignicantfortheanesthesiologist(F(1,42)=0.274,p= 116

PAGE 117

Figure6-12. Participantsspentsignicantlymoretimelookingatthevirtualsurgeonduringperiodsofsilence,howevernosignicantdifferenceswasobservedwiththeanesthesiologistduringperiodsofsilence. 0.602).NosignicantinteractionswereobservedbetweenagencyandStage.AninteractionplotvisualizingtheindirecteffectofagencyonthedurationofgazexationmomentsisshownbelowinFigure 6-13 . Figure6-13. IndirectEffecthadnosignicanteffectsongazeduringperiodsofsilence. 6.5DiscussionThegoalofthisstudywastodeterminehowtheagencyofvariousteammatesaffectsgazebehavior.Inthissection,Iconsiderbothwhethergazewithaspecicteammatewasaffectedbythatteammate'sagency,andalsowhethergazewithavirtual 117

PAGE 118

teammatewasaffectedbytheagencyofotherteammates.Theresultsshowedbothsimilaritiesanddifferencesingazewithrealandvirtualteammates. 6.5.1DirectEffects:SimilaritiesingazewithrealandvirtualteammatesSeveralsimilaritiesareworthnotinginparticipants'gazebehaviorwiththeirrealandvirtualteammates.Participantslookedatvirtualteammateswhentheyspoke,andtheyspentmoretimelookingatvirtualteammateswhospokemore.Participantsalsolookedatvirtualteammateswhileparticipantswerespeaking.Additionally,theamountoftimeparticipantsspentlookingattheirteammateswasnotaffectedbyagencywhileparticipantswerespeaking,orwhiletheotherteammatewasspeaking.Finally,therewerenosignicantdifferencesintheaveragegazexationdurationwithrealandvirtualteammates.Thisisimportant,asitsuggeststhatparticipantsfollowedculturalnormswhichdiscouragestaringatpeopleforlongperiodsoftime.Thisruledoesnotapplytocomputers,yetparticipantsappeartohavefolloweditwithvirtualteammates. 6.5.2DirectEffects:VaryingaTeammate'sAgencycanAffectGazewiththatTeammateHowever,amajordifferencewasobservedinthetotalamountoftimeparticipantsspentlookingattheirteammates.Onaverage,participantsspent66%moretimelookingatvirtualteammatesthantheirhumancounterparts(p<0.001).Examininggazeandspeechtogetherrevealedaninteractioneffectbetweenagencyandspeakerongazebehavior,whereparticipantslookedatvirtualteammatessignicantlymoreoftenduringperiodsofsilence(p<0.001)andwhileateammatewasspeaking(pSurgeon=0.057,pAnesth<0.05).Participantsalsomademoregazexationmomentswithvirtualteammatesthanhumancounterparts(p<0.001),thoughtheaveragelengthofthesemomentswasnotsignicantlyaffectedbyagency.Finally,speechinteractedwithgazeandagency:participantsspentanaverageof150%moretimelookingatvirtualteammatesduringperiodsofsilence,and30%moretimelookingatavirtualteammatewhileitwasspeaking.Gazewithvirtualteammatesdidnotsignicantly 118

PAGE 119

increasewhentheparticipantwasspeaking.Eachofthesesignicanteffectswereobservedforboththesurgeonandtheanesthesiologist,thoughthemagnitudeoftheeffectsometimesvaried.Insum,participantsspentsignicantlymoretimelookingatvirtualteammates,andthemajorityofthisextragazetimeoccurredduringperiodsofsilenceorwhenthevirtualhumanwasspeaking.Severalexplanationscouldpotentiallyexplainwhyparticipantsspentmoretimelookingatvirtualteammatesthanhumanteammates.Irstconsideranddiscardtwoexplanationsthatcannotaccountforalloftheobserveddata,namelythatvariationsinthehumanconfederate'sgazecausedtheobserveddifferences,andthattheincreaseintimewascausedbyaviolationofgazenormswithvirtualhumans.AswasnotedinSection 7.4.5 ,asystematicdifferencewasobservedbetweenthehumansurgeon'sgazeandthevirtualsurgeon'sgaze.Thehumansurgeonspentasignicantamountoftimelookingdownathispaperscript,andthusmadelesseyecontactwithparticipants.Incontrast,thevirtualsurgeonconsistentlymadeeyecontactwithparticipants,orwiththeanesthesiologist,dependingonwhothesurgeonwasspeakingto,orwhowasspeaking.Thehumananesthesiologistdidnothaveaslargeofasystematicdifference,inpartbecausehehadfewerlinestospeak.Ifthevariationsobservedherewerecausedbythechangeinthehumansurgeon'sgazebehavior,itwouldnotbeexpectedtondsuchconsistentagreementbetweenthegazeresultsforthesurgeonandtheanesthesiologist.Thegazebehaviorofthehumansurgeonandthehumananesthesiologistweredifferent,yetthesamesignicantdifferencesemergebetweenconditionsforboththesurgeonandtheanesthesiologist.Thismakesitunlikelythatthedifferencesobservedwerecausedbyvariationingazebehaviorbetweenthevirtualteammatesandtheirhumancounterparts.Itisalsounlikelythatthesedifferenceswereduetoviolationsofgazenormswithvirtualhumans,specicallypolitenessnorms.Inwesternculture,itisgenerallyconsideredrudetostareataconversationpartnerforanextendedperiodoftime. 119

PAGE 120

However,asvirtualhumansaretechnologicalartifacts,notactualhumanbeings,staringatavirtualhumanwouldnotactuallyviolateanysocialconventions.Ifparticipantsdidstareatvirtualhumansforlongerperiodsoftime,thiswouldbeassociatedwithanincreaseinoveralltimespentlookingatvirtualteammates.Thepresenceofstaringwouldbeindicatedbytwosigns:anincreaseintheaveragedurationofgazexationmoments,andashiftingazexationdistribution,specicallythepresenceofafattailindicatingthatlonggazexationsweremorecommon.Neitherofthesendingswereobserved.Instead,gazedurationshowednosignicantdifferences,andthedistributionofgazexationswereremarkablysimilar,especiallyinthepresenceofathintail.Assuch,itseemsunlikelythatthechangeingazebehaviorwascausedbyaviolationofpolitenessnormswithvirtualhumans.ThisndingisofparticularrelevancetoresearchconcerningtheMediaEquation.Giventhat,ingeneral,peopleautomaticallyandmindlesslyobeypolitegazenorms,ndingthatparticipantsalsoobeypolitenessnormsduringgazewithvirtualhumanssupportsthepropositionthatrealisticbehaviorwithvirtualhumansisamindlessphenomenon,ratherthanaconsciousdecision,asproposedbyNass[ 39 ]andKim[ 139 ].Havingdiscussedwhytheobservedsignicantdifferenceswereunlikelytohavebeencausedbyvariationsinthehumanconfederates'gazebehaviororbyviolationsofpolitenessnorms,Inowconsiderathirdoption,namelyuncertaintyconcerningturntakingandbackchannels.Turnorderduringconversationisgovernedbyasetofwell-denedrules[ 144 ].Mostimportantly,peopleusevoiceinection[ 145 ]andnonverbalbehavior[ 146 ]toindicatewhentheyareabouttocedetheturntosomeoneelse,whotheyplantocedetheturnto,orwhentheywishtohavetheturnpassedtothem.Participants'humanteammateswerecapableofunderstandingparticipants'turntakingsignalsandalsoprovidedsignalsindicatingwhentheywerereadytocedetheturn,andwhomtheywantedtocedetheturnto.Incontrast,participants'virtualteammatescouldnotunderstandparticipants'turntakingsignals,andwere 120

PAGE 121

onlycapableofprovidingextremelylimitedturntakingcues.Itislikelythatthislackofabilitytounderstandandprovideturntakingcuescausedparticipantstopaymorevisualattentiontovirtualteammatesatimportanttransitionmoments,suchaswhenateammatewasspeakingorduringperiodsofsilence,inhopethattheywouldbeabletogleanadditionalinformationaboutwhoshouldspeaknext.Thiswouldalsoexplainwhyparticipantsdidnotspendsignicantlymoretimelookingatvirtualteammateswhileparticipantswerespeaking,asparticipantsdidnotneedadditionalinformationfromtheirteammateswhileparticipantswerespeaking.Similarly,nosignicantdifferenceswereobservedduringgazewithateammatewhiletheotherteammatewasspeaking(e.g.lookingatthesurgeonwhiletheanesthesiologistwasspeaking).Theteammatewhowasnotspeakingisnotresponsibleforcontrollingthenextspeaker,thuslookingatthisteammatewouldnothelpparticipantsdecideifitwasanappropriatetimeforthemtobeingspeaking.Thisexplanationalsoexplainstheincreaseingazexationmoments.Ratherthanstaringatvirtualteammatesduringperiodsofsilence,whichwouldbreaksocialnorms,theyglancebackatthemmorefrequentlytoseeiftheyarenishedspeakingandtoseeiftheparticipantcanspeak. 6.5.3IndirectEffects:GazewithaVirtualTeammateisLargelyUnaffectedbytheAgencyofotherTeammatesWithregardtotheindirecteffectofagency,severalsignicantdifferenceswereobserved,however,theeffectsweretypicallyweakerthanthoseobservedforadirecteffectofagency,andwerenotalwaysobservedforboththesurgeonandtheanesthesiologist.Varyingtheagencyoftheotherteammatedidnotsignicantlyaltertheoverallamountoftimespentlookingatthevirtualsurgeonandthevirtualanesthesiologist.Fewergazexationmomentswereobserved(p<0.05)whenahumanteammatewaspresent,andtheaveragedurationofgazexationmomentsincreasedfortheanesthesiologistwhenthehumansurgeonwaspresent(p<0.01). 121

PAGE 122

Onthewhole,itappearsthatthepresenceofasecondhumanteammatedoesnothavealargeeffectongazebehaviorwithvirtualteammates.Varyingtheagencyofasecondteammatedidnotsignicantlyaltertheamountoftimespentlookingatvirtualteammates,andwhowasspeakingalsodidnotinteractwithagencytochangewhereparticipantsgazed.Therewashoweverasignicantdecreaseingazexationmomentswithvirtualteammateswhenahumanteammatewaspresent.Thisismostlikelyaconsequenceofthedecreaseingazexationmomentswithhumanteammates.When,forexample,thevirtualanesthesiologistisreplacedbyahumananesthesiologist,participantsmakefewergazexationmomentswiththeanesthesiologist.Thisinturnmeansthattherearefeweropportunitiestoglancefromtheanesthesiologisttothevirtualsurgeon,whichresultsinfewergazexationmomentswiththesurgeon.Putanotherway,thenumberofgazexationmomentsthatoccurisdirectlyrelatedtothenumberoftransitionsbetweenlooktargets.Ifpeoplelookatthehumananesthesiologistlessfrequently,thentherearefeweropportunitiestotransitiontoanotherstate,whichnecessarilyreducesthetotalnumberofgazexationmoments.Thus,thesignicantdecreaseingazexationmomentswithavirtualteammatedoesnottrulyrepresentachangeingazebehaviorwiththevirtualteammate,butinsteadisrelatedtothegreaterchangeingazebehaviorwiththehumanteammate.Insum,alteringaspecicteammate'sagencyhaslittleeffectongazewithothervirtualhumans.Therearesomeinevitableeffects,giventhezero-sumnatureofgaze,howevertheseeffectswereverysmall. 6.5.3.1LimitationsThestrengthoftheseconclusionsislimitedbythegrossnatureofthegazecoding.Thislimitsustoabroadanalysisofwhenpeoplelookedinthedirectionofvariousteammates.Thestronginterraterreliabilitybetweencodersindicatesthatthisvideowassufcientlydetailedtoallowgrosscodingtobeperformedsuccessfully.However,wedonotknowspecicallywherepeoplewerelookingwhentheygazedatvarious 122

PAGE 123

teammates.Itmaybethatpeoplemadeeyecontactwithhumanteammates,andmerelylookedinthedirectionofthedisplaythatrenderedthevirtualteammates,ratherthanactuallymakingeyecontactwiththevirtualteammate.Itisalsopossiblethattheincreaseintimespentlookingatvirtualteammatesisduetoadifferentfactorthatcouldnotbeidentiedwithoutmoredetailedcoding.Forinstance,theadditionaltimespentlookingatvirtualteammatesmayhavebeendirectedatpartsofthevirtualteammateotherthantheteammate'sface,whichcouldindicatethattheeffectwasduetonoveltyorcuriosityratherthanbeingcausedbymissingturntakingsignals.Whiletheproposedinterpretationofthedatadoesexplaineachoftheresultsidentiedinthisanalysis,furtherresearchisrequiredtoeliminateotherpotentialexplanations. 6.6ConclusionandFutureWorkThisworkyieldedtwomajorndings.First,Iexaminedtheeffectofagencyongazebehaviorwithspecicteammates.Ifoundindicationsthatparticipantsfollowedcommongazeconventions,suchaspolitenessnormsrelatedtogazeduration,wheninteractingwithvirtualteammates.However,wealsosawthatparticipantsweresensitivetowhenvirtualteammatesfailtoprovideimportantsocialcues,suchasgivingturntakingcueswhentheynishspeaking.Thesendingslendadditionalsupporttothetheorythatrealisticbehaviorwithvirtualhumans,andothertechnologicalartifacts,isamindlessphenomenonratherthanaconsciousdecision.Giventheunconsciousandautomaticnatureofnonverbalbehavior,observingpeoplefollowingnonverbalconventionswithvirtualteammatessuggeststhatsuchbehaviorisanunconsciousandmindlessresponse,ratherthanaconsciouschoice.Second,Iexaminedhowthepresenceofasecondhumanaffectedgazebehaviorwithvirtualteammates.Ifoundthatalteringtheagencyofateammatehadasmall,butsignicanteffectonthenumberofgazexationsparticipantsmadewithvirtualteammates.However,thiseffectmoreproperlyrepresentsachangeingazebehavior 123

PAGE 124

withthehumanteammate,whichnecessarilyaltersgazewiththevirtualteammate,giventhezero-sumnatureofgazebehavior.Overallgazetimewithvirtualteammates,andwhereparticipantslookedwhendifferentteammatesspoke,wasunaffectedbyalteringtheagencyofasecondteammate. 124

PAGE 125

CHAPTER7MIXED-AGENCYINTERACTIONSANDSOCIALPRESENCEThischapterconsidersthesocialpresenceresultsfromtheSpeakingUpstudyandtheSurgicalCountstudy(seeSections 4.1 and 4.2 fordetailsaboutthesestudies).Participantsexperiencedsignicantlymoresocialpresencewithhumanteammatesthanvirtualteammates,whichconrmsthatthesocialpresenceinstrumentusedinthisstudyhasconstructvalidity.Participantsalsoexperiencedasmall,butsignicantdecreaseinsocialpresencewithvirtualteammateswhenasecondhumanteammatewaspresent.Whilethemagnitudeofthisdifferenceisunlikelytohavepracticalsignicancefortheuseofvirtualhumansforteamtraining,itdoeshaveimplicationsforsocialpresencetheoryandmeasurement.ResultsfromtheSpeakingUpstudywerepublishedinIEEETransactionsonVisualizationandComputerGraphics(TVCG)[ 126 ].PersonalContributions:IanalyzedthedatadiscussedinthissectionandwrotethepaperpublishedinTVCG.Collaborators:Dr.JamesMartindaleprovidedassistanceindeterminingwhatstatisticaltestscouldberuntoanalyzethisdata.RelevancetoThesis:Thischapterexaminestheexperienceofsocialpresenceinmixed-agencyteams.Italsoexploreswhetherthesocialpresenceinstrumentsusedinthisresearchhaveconstructvalidity. 7.1ResearchQuestionsandMotivationThisdissertationexplorestwoquestionsrelatedtointeractionswithvirtualmembersofmixed-agencyteams.Therstresearchquestionfocusesonhowinteractionswithaspecicteammatechangewhenthatteammate'sagencyismanipulated(e.g.replacingavirtualsurgeonwithahumansurgeon).Theterm“DirectEffect”isusedtorefertothisresearchquestion,asitdealswithhowmanipulatingaspecicteammate'sagencydirectlyimpactsinteractionswiththatteammate. 125

PAGE 126

Thesecondresearchquestionfocusesonwhetherthepresenceofhumanteammatesaltersinteractionswithvirtualteammates(e.g.doesyourperceptionofavirtualanesthesiologistchangewhentheteam'svirtualsurgeonisreplacedbyahumansurgeon).Theterm“IndirectEffect”isusedtorefertothisresearchquestion,asitdealswithhowmanipulatingoneteammate'sagencyindirectlyimpactsinteractionswithadifferentvirtualteammate.BothofthesequestionsarediscussedinmoredetailinSection 1.2 andillustratedinFigure 1-1 .Withinthecontextofthesetwoquestions,thischapterinvestigatestheexperienceofsocialpresenceinmixed-agencyteams.Socialpresence,oftendenedasthesenseof“beingwith”another[ 16 ],canbeaffectedbychangestoavirtualhuman'svisualandbehavioraldelity.Inthischapter,Idiscusswhetherfeelingsofsocialpresencecanalsobeaffectedbyalteringtheagencyofone'steammates.ThisanalysisisbasedondatacollectedduringtheSpeakingUpstudy(seeSection 4.1 )andtheSurgicalCountstudy(seeSection 4.2 ). 7.2CollectionandPreparationofSocialPresenceData 7.2.1SocialPresenceSurveysSocialpresencewasmeasuredusingavequestionsurveydevelopedbyBailensonetal[ 22 ].Thequestionsareshownbelow.Theplaceholder...wasreplacedwitheither“surgeon”,“anesthesiologist”,“nurse”,or“surgicaltechnician”,dependingonwhichteammate'ssocialpresencewasbeingmeasured.ResponsesweregivenusingasevenpointLikertscale,rangingfromstronglydisagreetostronglyagree. 1. IperceivethatIaminthepresenceofa...intheroomwithme. 2. Ifeelthatthe...iswatchingmeandisawareofmypresence. 3. Thethoughtthatthe...isnotarealpersoncrossesmymindoften. 4. The...appearstobesentient,conscious,andalivetome. 5. Iperceivethe...asbeingonlyacomputerizedimage,notasarealperson. 126

PAGE 127

Anoverallsocialpresencescoreiscalculatedbyinvertingquestions3and5,thenaveragingalloftheanswerstogether.ParticipantscompletedsocialpresencesurveysforeachteammateaftereverystageoftheSpeakingUpandSurgicalCountstudies.Thesamesurveywasusedtomeasurethesocialpresenceofbothrealandvirtualteammates. 7.2.2PreparationofdatafromtheSpeakingUpstudySocialpresencedatafromeachstageoftheinteractionwasaveragedtogethertocreateasinglesocialpresencescoreforeachteammate.Beforecreatingaveragescores,testswereruntoconrmthatsocialpresencewascorrelatedbetweenstages.StrongcorrelationswerefoundbetweenStage2andStage3forthesurgeon(r=0.807)andtheanesthesiologist(r=0.933).DatafromStage1wasnotused,asthesurgeonwasnotpresentinthisstage.Unlessotherwisestated,allsocialpresencescoresdiscussedinthischapterrefertotheseaveragesocialpresencescores.Threeparticipantswereexcludedasoutliers,astheirsocialpresencedatafellbelowthemeanbymorethatthreetimesthestandarddeviation.Excludingthesethreeparticipantsleftsocialpresencedatafor66participants:26inthebothvirtualcondition,20inthehumansurgeoncondition,and20inthehumananesthesiologistcondition(seeSection 4.1.2.1 foranexplanationoftheseconditions). 7.2.3PreparationofdatafromtheSurgicalCountstudyThesameprocedurewasalsousedtopreparethedatafromtheSurgicalCountstudy.Moderatetostrongcorrelationswerefoundbetweenallstagesforthesurgeon(r1,2=0.718,r2,3=0.794,r1,3=0.619),theanesthesiologist(r1,2=0.795,r2,3=0.805,r1,3=0.796),thenurse(r1,2=0.790,r2,3=0.620,r1,3=0.745),andthesurgicaltechnician(r1,2=0.917,r2,3=0.942,r1,3=0.927).Allthreestageswereincludedintheaveragesocialpresencescore,aseachteammemberwaspresentineverystage.Fourparticipantswereexcludedduetoincompletedata.Theseparticipantsfailedtocompletemultiplesurveysduetotimeconstraints.Threeadditionalparticipants 127

PAGE 128

wereexcludedasoutliers,astheirsocialpresencedatafellbelowthemeanbymorethatthreetimesthestandarddeviation.Excludingthesesevenparticipantsleftsocialpresencedatafor87participants:40intheboth-humancondition(20nursesand20surgicaltechnicians),24inthehumannursecondition,and23inthehumansurgicaltechniciancondition(seeSection 4.2.2.1 foranexplanationoftheseconditions). 7.3Agency'sDirectEffectonSocialPresenceIrstconsiderwhetheraspecicteammate'sagencyaffectedparticipants'experienceofsocialpresencewiththatteammate.Thisquestionisfundamentallyrelatedtoconstructvalidity,thedegreetowhichatestactuallymeasureswhatitclaimstomeasure[ 147 ].Socialpresencetheorypredictsthathumansandvirtualhumansshouldgeneratedifferentlevelsofsocialpresence[ 87 ].Assuch,thesocialpresenceinstrumentcanbesaidtohaveconstructvalidityifitcandifferentiatebetweenrealandvirtualteammates.Toexplorethis,Iconsiderwhetherhumanteammateswereperceivedtohavesignicantlymoresocialpresencethantheirvirtualcounterparts.AnexampleofthiscomparisondrawnfromtheSpeakingUpstudyisshownin 7-1 .HereIcompareahumansurgeon'ssocialpresencetoavirtualsurgeon'ssocialpresence,andahumananesthesiologist'ssocialpresencetoavirtualanesthesiologist'ssocialpresence.IcanalsoexamineconstructvalidityusingdatacollectedduringtheSurgicalCountstudy,bycomparingthevirtualnurse'ssocialpresencetothehumannurse'ssocialpresence(asperceivedbysurgicaltechnicians)andthevirtualsurgicaltechnician'ssocialpresencetothehumansurgicaltechnician'ssocialpresence(asperceivedbynurses). 7.3.1ConstructValidityintheSpeakingUpstudyThesocialpresencemeansandstandarddeviationsfromtheSpeakingUpstudyarereportedinTable 7-1 .Twoone-wayANOVAswereconductedtoassesstheDirectEffectofagencyonfeelingsofsocialpresencewiththesurgeonandthe 128

PAGE 129

AVirtualSurgeonvs.HumanSurgeon BVirtualAnesth.vs.HumanAnesth.Figure7-1. ComparingthesocialpresenceofvirtualteammatestorealteammatesintheSpeakingUpstudy. anesthesiologist.Asingle2x2ANOVAwasnotused,whereAgencyandTeammateservedastwomaineffects,becauseTeammateisnotafullyrepeated-measuresfactor.ParticipantsintheBoth-Virtualconditioncontributeddataontwoteammateswiththesameagency,whileparticipantsintheHuman-SurgeonandHuman-Anesthconditionscontributedataontwoteammateswithdifferentagency.BothANOVAsshowedsignicantmaineffectsforDirectEffectsofagency(F(1,44)Surgeon=14.79,pSurgeon<0.001,F(1,44)Anesth=8.25,pAnesth<0.01). Table7-1. Socialpresencemeanandstandarddeviations,summarizedbyagency(realorvirtual)andbyteammate(surgeonoranesthesiologist).Socialpresencecanrangefrom1to7. AgencySurgeonAnesthesiologist MeanSDMeanSD Real6.1700.6076.0150.889Virtual5.2150.9725.1921.015 7.3.2ConstructValidityintheSurgicalCountstudyThesocialpresencemeansandstandarddeviationsfromtheSurgicalCountstudyarereportedinTable 7-2 .UnliketheSpeakingUpstudy,a2x2ANOVAcouldbeconductedonthedatacollectedintheSurgicalCountstudy,becauseeachparticipant 129

PAGE 130

contributedonlyonedatapointrelevanttotheDirectEffectquestion:nursesonlyratedsurgicaltechniciansandsurgicaltechniciansonlyratednurses.Asthesurgeon'sandanesthesiologist'sagencywasalwaysvirtual,theirdataisnotincludedinthisanalysis.A2x2ANOVAshowedasignicantmaineffectofagency(F(1,83)=35.200,p<0.001);realhumanswereratedasmoresociallypresentthanvirtualhumans.Therewasnosignicantmaineffectofteammate(F(1,83)<0.001,p=1.000)andnosignicantinteractionbetweenagencyandteammate(F(1,83)<0.001,p=0.95). Table7-2. Socialpresencemeanandstandarddeviations,summarizedbyagency(realorvirtual)andbyteammate(nurseorsurgicaltechnician).Socialpresencecanrangefrom1to7. AgencyNurseScrubTech MeanSDMeanSD Real6.2680.7656.2831.154Virtual5.0780.7815.0671.020 7.3.3DiscussionofConstructValidityInbothstudies,humanteammatesevokedsignicantlyhigherlevelsofsocialpresencethantheirvirtualcounterparts(pSpeakingUp<0.001,pSurgicalCount<0.001).Themagnitudeofthedifferencewasalsomeaningful.Onaverage,humanparticipantsevoked14.9%moresocialpresencethantheirvirtualcounterparts.Itisnotsurprisingthatrealhumansevokemoresocialpresencethanvirtualhumans.Nevertheless,thisisanimportantnding,asitsuggeststhesocialpresenceinstrumentusedinthisresearchhasconstructvalidity.Socialpresencetheorypredictsthatvirtualhumans(intheircurrentstate)shouldevokelesssocialpresencethanrealhumans[ 88 ].Assuch,socialpresenceinstrumentsoughttobeabletodifferentiatebetweenrealandvirtualhumans.Findingthatthissocialpresenceinstrumentcandifferentiatebetweenrealandvirtualhumanssuggeststhathasconstructvalidityandthatthedataitprovidescanbeusedtoexploresocialpresenceinmixed-agencyteams. 130

PAGE 131

7.3.4UsingSocialPresenceSurveyswithRealHumansDespiteactuallybeingreal,mostparticipantsdidnotgivetheirhumanteammatesaperfectsocialpresencescore.ThisndingparallelsUsoh'searlierworkapplyingphysicalpresencequestionnairestorealenvironments[ 148 ].Usohfoundthattwophysicalpresencequestionnaireswereessentiallyunabletodistinguishbetweenvirtualandrealenvironments.Whilethisresultinitiallyseemscounterintuitive,Usohclaimsitisactuallyanaturalconsequenceofhumanpsychology: Ifsomeoneisaskedfortheirsenseofbeingthereona1to7scale,itgivesthempermissiontoanswerwithascoreoflessthan7evenwhentheyarereallythere.Thequestionsarereinterpretedtomakesenseinthegivencontext....Inthereal-world,sincethereisnodoubtthattheindividualispresentintheobvioussense,itbecomesreinterpretedasthesenseofinvolvement,thelackofisolation,perhapsthedegreeofcomfort.Thethought“Iamnotcomfortabletobehere”mightleadtoalow`presence'response.[ 148 ]Participantsappeartohavereinterpretedsomeofthesocialpresencequestionsaswell.Participants'responsestotheindividualsocialpresencequestionsfortheirhumanteammatesareshowninFigure 7-2 .ThequestionthatwasreinterpretedmoststronglywasQ3,whichstates“Thethoughtthatthe...isnotarealpersoncrossesmymindoften.”Despitethefactthatthehumanteammateswereobviouslyrealpeople,participantsfrequentlygavethemlowscoresforthisquestion,sometimesevendippingdownintothe”agree”range(whichimplieslowersocialpresenceforthisquestion,givenitsinvertednature).Asimilar,thoughlesspronouncedeffectcanbeseenforQ5,whichstates”Iperceivethe...asbeingonlyacomputerizedimage,notasarealperson.”ParticipantsappeartohavereinterpretedQ1,Q2,andQ4lessfrequently.WhileQ3andQ5focusedonfeelingsofunreality,thesethreequestionsassessedpositivefeelingsrelatedtomutualawarenessandsentience.Itmaybethatnegativelyframedquestions 131

PAGE 132

ASpeakingUpStudy BSurgicalCountStudyFigure7-2. Participant'sresponsetotheindividualsocialpresencequestionsforthehumanteammates.Responsesforvirtualteammatesarenotincludedinthisdata.ResponsestoQ3andQ5havebeeninverted,accordingtothesurvey'sscoringinstructions.ThetextforeachquestioncanbefoundinSection 7.2.1 . aremorelikelytobereinterpretedthanpositivelyframedquestions.Thispossibilityshouldbeconsideredwhendevelopingpresencequestionnaires. 7.4Agency'sIndirectEffectonSocialPresenceInowconsiderwhetheravirtualhuman'ssocialpresenceisaffectedbytheagencyofotherteammates.Toexplorethisquestion,Ilookatthesocialpresence 132

PAGE 133

datafromtheSpeakingUpstudyandtheSurgicalCountstudy.InthedatafromtheSpeakingUpstudy,Iexaminehowthevirtualsurgeon'ssocialpresenceisaffectedbychangingtheagencyoftheanesthesiologistandhowthevirtualanesthesiologist'ssocialpresenceisaffectedbychangingtheagencyofthesurgeon.InthedatafromtheSurgicalCountstudy,Iexaminehowthevirtualsurgeon'ssocialpresenceisaffectedbychangingtheagencyofthenurseorthesurgicaltechnicianandhowthevirtualanesthesiologist'ssocialpresenceisaffectedbychangingtheagencyofthenurseorthesurgicaltechnician. 7.4.1TheEffectofaHumanTeammateonaVirtualHuman'sSocialPresenceintheSpeakingUpStudyThesocialpresencemeansandstandarddeviationsfromtheSpeakingUpstudyarereportedinTable 7-3 .Twoone-wayANOVAswereconductedtoassessagency'sIndirectEffectonfeelingsofsocialpresencewiththesurgeonandtheanesthesiologist.TheANOVAsforthesurgeonshowednosignicantmaineffectforIndirectEffectsofagency(F(1,44)Surgeon=0.658,pSurgeon=0.422).TheANOVAfortheanesthesiologistwasalsonotsignicant,thoughatrendtowardssignicancewasvisible(F(1,44)Anesth=3.475,pAnesth=0.069). Table7-3. Socialpresencemeanandstandarddeviations,summarizedbyteammate(surgeonoranesthesiologist)andbyagencyoftheotherteammate(virtualorhuman).Socialpresencecanrangefrom1to7. IndirectEffectSurgeonAnesthesiologist MeanSDNMeanSDN VirtualTeammate5.2150.972265.1921.01526HumanTeammate5.0000.775204.6301.01220 7.4.2TheEffectofaHumanTeammateonaVirtualHuman'sSocialPresenceintheSurgicalCountStudyThesocialpresencemeansandstandarddeviationsfromtheSurgicalCountstudyarereportedinTable 7-4 .A2x2ANOVAshowedatrendtowardsasignicantmaineffectoftheagencyoftheotherteammate(F(1,168)=3.669,p=0.0571),nosignicant 133

PAGE 134

maineffectofteammate(F(1,168)=0.071,p=0.790)andnosignicantinteractionbetweenagencyandteammate(F(1,168)=0.305,p=0.581). Table7-4. Socialpresencemeanandstandarddeviations,summarizedbyteammate(surgeonoranesthesiologist)andbyagencyoftheotherteammate(virtualorhuman).Socialpresencecanrangefrom1to7. IndirectEffectSurgeonAnesthesiologist MeanSDNMeanSDN VirtualTeammate5.1940.822475.0640.86646HumanTeammate4.8431.020404.8901.03239 7.4.3Meta-analysisoftheresultsoftheSpeakingUpandSurgicalCountstudiesTheresultsfromtheSpeakingUpstudyandtheSurgicalCountstudywerethenintegratedintoasinglemeta-analysisexploringtheIndirectEffectsofagencyonavirtualteammate'ssocialpresence.CrombieandDaviesprovideahelpfulsummaryofwhatameta-analysisisandhowtoconductone[ 149 ].Amorethoroughtreatmentcanbefoundin“Understandingthenewstatistics:Effectsizes,condenceintervals,andmeta-analysis”[ 150 ].Thegoalofameta-analysisistocombinedatafrommultiplestudiestoimprovestatisticalpowerandincreaseourabilitytodeterminewhethersignicanteffectsarepresentinthedata.Arandom-effectsmeta-analysiswasconductedusingthemetaforpackageinR[ 151 ].Theanalysisrevealedasignicanteffectoftheotherteammate'sagencyonavirtualhuman'ssocialpresence(p<0.05,mean=0.307,CI=[0.0626,0.551]).Theresultsofameta-analysisarefrequentlyvisualizedinforestplots.Aforestplotvisualizingtheindividualcomparisonsandthecombinedeffectisshownbelow,inFigure 7-3 .Forestplotsvisualizetheindividualcondenceintervalsofeachstudyandtheweightgiventothatstudyinthenalmeta-analysis.Individualstudies/comparisonsareshownonseparatelines.Eachcomparisonincludesa95%condenceinterval,visualizedasahorizontalline,anindicatoroftherelativeweightgiventothiscomparisoninthewholemeta-analysis(indicatedbythesizeofthecentralsquare),andthe 134

PAGE 135

numericalmeandifferenceand95%condenceinterval.Figure 7-3 showsthefourcomparisonsconcerningtheIndirectEffectsofagencymadeintheSpeakingUpstudyandtheSurgicalCountstudy.Thenalresultofthemeta-analysisisshownonthenalline,labeledREModel.Thediamondvisualizesthe95%condenceintervalproducedbymergingtheresultsoftheindividualcomparisons.Numericalmeandifferencesand95%condenceintervalsareshowntotherightofthediamond. Figure7-3. Meansand95%condenceintervals(CIs)areshownfortheeffectoftheagencyoftheotherteammateonthesocialpresenceofthefourvirtualhumansthatareexaminedinthismeta-analysis.Thesizeofthesquaresymbolsforthemeanssignalstherelativeweightingsofthevirtualhumansinthemeta-analysis.Thediamondreportstheresults(meanand95%CI)ofthemetaanalysis. Signicantdifferencescomparabletoap-valuelessthan0.05canbeidentiedbywhenthe95%condenceintervaldoesnotcrossthecentraldashedline,whichcorrespondstoameandifferenceofzero.Whilenoneoftheindividualcomparisonsshowedastatisticallysignicantdifference,thenalresultofthemeta-analysisis 135

PAGE 136

statisticallysignicant,thoughthemeandifferenceissmall(0.31,onasevenpointscale). 7.4.4DiscussionoftheEffectofaHumanTeammateonaVirtualHuman'sSocialPresenceThoughthepresenceofasecondhumandidproduceastatisticallysignicantdecreaseinsocialpresence,themagnitudeofthisdifferencemeansthatthiseffectisunlikelytohindertheuseofvirtualhumansforteamtraining.Thepresenceofotherhumanteammatesdidnotmeaningfullyalterparticipants'perceptionsconcerningtheirvirtualteammates.However,itisimportanttorecognizethatthiseffectmaygrowmorepronouncedwhenworkingwithlessrealisticvirtualhumans.Thevirtualhumansusedinthisstudyevokedhighlevelsofsocialpresence,comparedtoothervirtualhumanswhichevokedonlyneutralfeelingsofsocialpresence[ 100 ],orevennegativefeelingsofsocialpresence[ 22 ].Themagnitudeofthedecreaseobservedinthisstudymayhavebeensmallbecausethevirtualhumanswerealreadyfairlyrealistic.Virtualhumanswithlessvisualandbehaviordelitymayexperienceamorepronounceddropinsocialpresencewhenintroducingotherhumanteammates.Futureexperimentationisneededtoexplorewhetherthepresenceofotherhumanscausesmorepronounceddecreasesinsocialpresenceforlessrealisticvirtualagents.Thisresearchalsohasimplicationsforsocialpresencetheory.Mostsocialpresencetheoryandresearchhasfocusedoninteractionsoccurringbetweentwoindividuals(oneofwhichmaybevirtual).Theresultsfromthesestudiessuggestthatsocialpresencecanbeinuencedbyfactorsexternaltoboththeperceivedentityandtheperceiver.TwoexternalfactorsareespeciallyrelevanttotheresultsfromtheSpeakingUpandSurgicalCountstudies,namelypassivecomparisonsbetweenteammatesandobservinginteractionsbetweenteammates.Atpresent,virtualhumansareoflowerdelitythatactualhumans.Thisistrueforallaspectsofvirtualhumans:visualappearance,non-verbalbehavior,motionand 136

PAGE 137

animation,speech,andbehavior.Wheninteractingwithasinglevirtualhuman,thatvirtualhumancanonlybecomparedagainstexistingmentalmodelsofhowhumansshouldlookandbehave.Whilepeople'smentalmodelsofhumanappearanceandbehaviorareveryrobust,theyarenonethelessmalleable–forinstance,SeyamaandNagayamafoundthatbeingexposedtoimagesofpeoplewhoseeyesizehadbeenincreasedalteredhoweyesizewasperceivedinnaturalfaces[ 152 ].Introducingasecondhumantotheinteractionmeansthatusersnolongerhavetorelyonmentalmodels,butinsteadcannowcomparevirtualhumansagainstanactualhuman.Havingaconcretereferencepointmaymakeusersmoreawareofvisualandbehavioraldifferencesbetweenrealandvirtualhumans,leadingtolowerfeelingsofsocialpresence.Thiseffectcanbeviewedintermsofexpectations.Whenotherhumansarenotpresent,users'expectationsofhowahumanshouldlookandbehavemaybeinuencedbyhowthevirtualhumanlooksandbehaves,aswasobservedwitheyesizebySeyamaandNagayama.However,whenasecondhumanispresent,participants'expectationsaregroundedinrealityandhavelessfreedomtovary.Thisxedreferencepointmaymakeitmoredifcultforavirtualhumantomeetusers'expectations.However,ifmeetingexpectationsiskeytotheexperienceofsocialpresence,thisalsomeansthatbeingabletocomparevirtualhumanstorealhumansneednotalwaysresultinadecreaseinsocialpresence.Instead,ifavirtualhumanmeetsauser'sexpectations,thevirtualhuman'ssocialpresencewillremainconstant,orpossiblyevenincrease.AnexampleoftheinteractionbetweenmeetingexpectationsandperceivedrealismwasdescribedbyparticipantsintheSurgicalCountdebrieng.DuringStage3oftheSurgicalCountstudy,participantshadtosearchtheroomforamissingitem.Thesurgeongrewfrustratedwhentheitemcouldnotbefoundandcontinuallyaskedparticipantsiftheyhadfoundtheitemandcritiquedtheirsearchprocess(e.g.“Haveyoulookedinthedrapesyet?”).Multipleparticipantsreportedthattheyfeltthevirtual 137

PAGE 138

surgeonseemedmuchmorerealisticduringthisperiod,becausetheirgazewasdirectedawayfromthesurgeonandatotherportionsoftheroom.Thus,theonlyinputthereceivedfromthesurgeonwashisvoice.Whilethevirtualsurgeon'svisualappearancewasofmuchlowerdelitythatanactualhuman'sappearance,thevirtualsurgeon'svoicewasrecordedbyarealhumanandwasthusmorecapableofmeetingparticipants'expectations.Thus,whenthevirtualsurgeon'sfailuretomeetparticipants'expectationsforhowasurgeonshouldvisuallyappearwasnolongeraconsideration,participantsbegantofeelthatthevirtualsurgeonseemedmorerealistic.Inadditiontopassivecomparisonsbetweenteammates,observinginteractionsbetweenteammatesmayalsoaffectauser'sexperienceofsocialpresencewithavirtualagent.Insociallearningtheory,Bandurapositsthatlearningisanessentiallysocialprocess,whichtakesplacewhenpeopleobservethebehaviorofothers[ 153 ].Thus,auser'sbehaviorwithvirtualteammatesmaybeinuencedbyhowotherhumanteammatesbehavewithvirtualteammates,suchthatifahumanengageswithavirtualteammateasifitwerereal,otherhumanteammatesmaybemoreinclinedtodosoaswell.Sociallearningeffectscouldbeeitherpositiveornegative.Observingasuccessfulinteractioncouldprimeuserstoviewvirtualteammatesasmorecapableandmoresociallypresent,whileobservinganunsuccessfulinteractioncouldprimeuserstoviewvirtualteammatesaslesscapableandlesssociallypresent.Theattitudeofotherhumanteammatescouldhaveasimilareffect.Workingwithahumanthatdoesn'ttakevirtualteammateseriouslycouldloweryourownfeelingofsocialpresencewithavirtualteammate,orviceversa.Observinginteractionsbetweenvirtualhumansmayhaveasimilareffect.Virtualhumanshavebeenshowntobecapableofinducingsociallearningeffects,suchaspromotingriskybehaviorinabicyclingsimulation[ 154 ].Itisnotinconceivablethatobservingvirtualteammatesinteractingwitheachothersuccessfullycouldprimepeopletoviewvirtualhumansasmorecapableandsociallypresent. 138

PAGE 139

7.4.5LimitationsAsystematicdifferencewasobservedintheSpeakingUpstudybetweenthegazebehaviorofthevirtualteammatesandtheirhumancounterparts.Thehumanconfederatesfrequentlylookeddownattheclipboardholdingtheirscript,whilethevirtualhumansinsteadmaintainedeyecontactwithwhoeverwasspeaking,orthepersonbeingspokento.Thisdifferenceisaconcernaspriorresearchhasfoundthatvirtualhumansthatexhibitmorerealisticgazebehaviorevokestrongerfeelingsofsocialpresence[ 33 ].However,whilethisdifferenceingazebehaviorisalimitationintheSpeakingUpstudy,italsoseemsunlikelythattheresultswouldhavebeenalteredifthehumanconfederateshadmademoreeyecontact.Giventhatimprovedeyecontactwouldlikelyhaveincreasedthehumanconfederate'ssocialpresence,thiswouldhaveonlyservedtostrengthenthealreadysignicantsocialpresencedifferencesbetweenthevirtualhumansandtheirhumancounterparts.Additionally,thistypeofgazebehaviorisnotuncommonintheOR,wheresurgeonsandanesthesiologistsfrequentlyhavetoconsultpatientchartsorchecklists,andthuscannotalwaysmaintaineyecontactwiththeirteammates.Othersourcesofvariancebetweenparticipants'virtualteammatesandtheirhumancounterpartsincludedvariationsintoneofvoice,posture,andphrasing.Whilethehumanconfederatesattemptedtokeepthesedetailsconstantfromparticipanttoparticipant,theywereunderstandablynotalwayssuccessful,asitcanbedifcultforhumanstocontrolthesebehaviorsprecisely.Itispossiblethatthesechangesinuencedparticipantbehavior.However,itisimportanttorecognizethatthesesourcesofvarianceareimpossibletocompletelyeliminatefromanyresearchinvolvinghumanconfederates. 7.5ConclusionsandFutureWorkThisworkhastwomaincontributions:conrmingthatsocialpresencehasconstructvalidity,andidentifyingthatthepresenceofotherhumanteammateshasasmall,butsignicanteffectontheexperienceofsocialpresencewithvirtualteammates.Thesize 139

PAGE 140

ofthiseffectissmallenoughthatitisunlikelytorendervirtualhumanslessusefulforteamtraining,butitspresencehasimportantimplicationsforsocialpresencetheoryandresearch,namelyshowingthatfactorsexternaltoadyadicinteractioncaninuencetheexperienceofsocialpresencewithinthatdyad.Futureresearchisrequiredtoexplorethisfurther. 140

PAGE 141

CHAPTER8SUMMARYANDFUTUREDIRECTIONS 8.1DiscussionandSummaryInthisdissertation,Iexaminedthedirectandindirecteffectsofagencyonthreetypesof“behavior”:decision-makingbehavior,gazebehavior,andsocialpresence.Decisionmakingwasexaminedbecauseakeyelementoftrainingislearningtomakedecisionscorrectly.Gazebehaviorwasexaminedtoexplorewhethervariationsinagencyaffectedbehaviorthatwasunconsciousandautomatic.Socialpresencewasexaminedasanindicatorofperceivedrealism.Inowsummarizetheresultsofthesethreemetricsaccordingtothisdissertation'sthreemaincontributions.Specically,that: 1. Trainee'sbehaviorwithvirtualteammatesmirroredbehaviorwithhumanteammates,solongasvirtualteammatesmetbasicexpectationsabouthowhumansbehave. 2. Thepresenceofotherhumanteammatesdidnotmeaningfullyalterbehaviorwithvirtualteammates. 3. Variable-agencyteammatescanbeusedtosupporttrainingprogramshamperedbylogisticaldifculties. 8.1.1BehaviorwithVirtualTeammatesMirrorsBehaviorwithHumanTeammatesAgencywasnotobservedtohaveanyeffectondecisionmakingbehavior.Participantsspokeuptoahumansurgeonandavirtualsurgeoninsimilarwaysandatsimilarrates.Whenaskedtodecidebetweentwopositions,participantswerenotmorelikelytoagreewithhumanteammatesthanvirtualteammates.Additionally,thesametacticswereusedwhennegotiatingwithrealandvirtualteammates.Takentogether,theseresultssuggestthatparticipants'decisionswerenotstronglyinuencedbytheagencyoftheirteammates.Whiletheseresultscannotdemonstrateequivalence,theydosuggestthat,inthesettingsexploredinthisdissertation,decisionmakingwithvirtualteammatescloselyapproximateddecisionmakingwithhumanteammates. 141

PAGE 142

Agency'seffectongazebehaviorwasmorenuanced.Therewereseveralwaysthatgazewithvirtualteammatescloselyapproximatedgazewithhumanteammates:participantslookedatvirtualteammateswhentheyspoke,spentmoretimelookingatteammateswhospokemoreoften,anddidnotstareatvirtualteammateslongerthanhumanteammates(whichsuggeststhatparticipantsfollowedculturalpatternsgoverningpolitegaze).However,participantsdidspendsignicantlymoretimelookingatvirtualteammatesduringperiodsofsilenceandwhilethevirtualteammateswerespeaking.Itseemslikelythatthisincreaseingazetimeduringsilenceandwhilethevirtualteammatewasspeakingisreectiveofthelimitednon-verbalcuesprovidedbythevirtualteammatesusedinthisresearch.Missingnon-verbalcues,especiallythoserelatedtoturntaking,couldhavecausedparticipantstobelesscertainwhenitwastheirturntospeak.Participantscompensatedforthisbyspendingmoretimelookingatvirtualteammatesduringperiodsofsilenceinordertodetermineiftheyshouldspeak.Observinganincreaseintheamountoftimebetweenwhenthevirtualsurgeonspokeandtheparticipantrespondedprovidedadditionalsupportforthishypothesis,asthisindicatespeoplepausedlongerbeforespeaking,whichcouldhavebeencausedbyuncertaintyaboutifitwastheirturntospeak.Takentogether,theseresultsuggestthatvariationsinagencywillnotinherentlyleadtodifferentgazebehavior,butmayleadtobehavioraldifferencesifvirtualteammatescannotreplicateimportanthumanbehaviors,suchasnon-verbalcues.Thisunderscoreshowitisimportanttoconsiderwhethervirtualteammatesarecapableofmeetingparticipants'expectationsforhowhumansbehaveintherealworld,bothonthelevelofconsciousawarenessandofunconsciousresponse.Agencydidhaveapronouncedeffectonparticipants'experienceofsocialpresence.Humanteammatesalwaysevokedsignicantlymoresocialpresencethanvirtualteammates.However,itwouldbesurprisingifvirtualteammatescreatedusingcurrenttechnologywereabletogeneratefeelingsofsocialpresenceequivalentto 142

PAGE 143

thosegeneratedbyrealpeople,duetocurrentlimitations.Itisimportanttonotethatparticipants,onaverage,reportedpositivefeelingsofsocialpresence,whichsuggeststhatparticipantscanviewvirtualhumansasentitiescapableofrealsocialinteractions,includingteamwork.Insum,participants'behaviorwithvirtualteammatesmirroredtheirbehaviorwithhumanteammates,solongasthevirtualteammateswereabletomeettherelevantexpectations.Thissuggeststhattheuseofvirtualteammatesforteamtrainingwillnotleadtoparticipantslearningineffectiveordisruptivebehaviors,butinsteadthatvirtualteammatescanbeusedforthesametypesoftrainingrealhumansparticipatein,solongasthevirtualteammatesareabletomeettherelevantexpectations. 8.1.2ThePresenceofOtherHumansdoesnotMeaningfullyAlterBehaviorwithVirtualTeammatesThepresenceofasecondhumanteammatedidnothaveanymeaningfuleffectonparticipants'behavior.Workingwithasecondhumanteammatedidnotaffecthowparticipantsspokeuptothevirtualsurgeon,nordiditaffectthetacticsusedwhennegotiatingwiththevirtualsurgeon.Thepresenceofasecondhumanteammatedidproducesomesignicanteffectsongazebehavior,buttheseeffectswerenotconsistentlyobservedwithbothteammates,andweremuchsmallerthantheeffectsobservedbetweenrealandvirtualteammates.Adropinsocialpresenceofapproximately4.3%wasalsoobservedwhenasecondhumanwaspresent,howeverthisdifferenceissmallenoughtohavenomeaningfulsignicanceforthepurposesoftraining.Insum,whilethepresenceofasecondhumanteammatedidproducesomestatisticallysignicanteffectsonbehavior,theseeffectswereeitherweak,orwerenotconsistentlyobservedwithallteammates.Whilethesesignicantdifferencesmayhaveimportanttheoreticalimplications,theyareunlikelytoproducemeaningfulchangesinbehaviorduringtraining.Thissuggeststhattrainingexperienceswillnotbeimpactedbychangingthenumberofhumanteammatespresent. 143

PAGE 144

8.1.3Variable-AgencyTeammatescanSupportTrainingProgramsbyOvercom-ingLogisticalDifcultiesFinally,theresearchconductedinthisdissertationalsoservesasaproofofconceptshowingthatvariable-agencyteammatescreatedinVPF2canbeusedtosupporttrainingprogramshamperedbylogisticaldifculties.ThesecondstudyIconductedcontainedaconditionthatrequiredtwohumanparticipants.Therewereinstanceswhereonlyoneofthetwohumanteammatesarrivedasscheduled.Inthisevent,ratherthancancelingthetrainingexercise,theparticipantwhodidarriveontimecompletedthetrainingexercisebyherself,workingwithavirtualteammateinstead.Additionally,therewerealsoinstanceswhentwoparticipantsarrivedtogetherwhenonlyonewasscheduled.Inthiscase,avirtualteammatewasreplacedbythesecondhumanteammate,whichallowedbothparticipantstoreceivetheneededtraining.Withoutvariable-agencyteammates,neitherofthesesituationswouldhavebeenpossible.Intherst,thetrainingexercisewouldhaveneededtobecanceled,duetokeypersonnelbeingmissing.Inthesecond,onlyoneofthetwoparticipantswouldhavebeenabletoreceivetraining,andtheotherwouldhavehadtobesentaway.Asvariable-agencyteammatessuccessfullyallowedmetorespondtounexpectedchangesinavailablepersonnel,sovariable-agencyteammatescouldhelpovercomerelateddifcultiesinothertrainingprograms. 8.2LimitationsWhenseekingtogeneralizethesecontributionstoothertrainingdomains,itisimportanttoconsiderthreespecicassumptionsinherentinthetrainingexercisesevaluatedinthisresearch,specicallytheuseofhigh-delityvirtualhumans,thetrainingdomain(healthcare,specicallyintheoperatingroom),andpowerrelationshipsbetweenteammates.Thecontributionsofthisdissertationmaynotgeneralizetoothertrainingprogramsthatdonotmeettheseassumptions. 144

PAGE 145

8.2.1High-delityVirtualHumansThevirtualhumansusedinthisresearchpossessedrelativelyhighvisualandbehavioralrealism:theywerelife-size,wereembeddedinrealenvironments,communicatedusingspeechandgesture,madeeyecontactwithparticipantswhenspeaking,andmadefewmistakeswhilespeakingduetotheuseofaWoZ.Itispossiblethattheconclusionsofthisresearchmaynotapplytointeractionswithvirtualhumanspossessinglowervisualorbehavioralrealism.Forinstance,thevirtualsurgeon'sabilitytocreateadifcultspeakingupenvironmentmayhavebeenreducedifhehadbeenrenderedonasmallerdisplay,oriftheteammateshadinteractedbytyping.Thesemodicationscouldhavereducedthesurgeon'sabilitytointimidateparticipantsandcausedthemtospeakupmoreassertively. 8.2.2MedicalDomainThisresearchwasconductedspecicallywithinamedicalcontext.Whileitislikelythatthecontributionsofthisresearchwillgeneralizetoothersimilardomains,itisimportanttorecognizethatvariationsintrainingcontextmayimpactthegeneralizabilityofthesecontributions.Thepotentialeffectofculturaldissimilaritiesandtaskfocusshouldbeconsidered.Someattributesofthemedicaldomainthatmaybeimportanttoconsiderincludehighdegreesofspecialization,frequentrelianceonpolicyandprocedure,lowtoleranceforfailure, 8.2.3PowerRelationshipsPowerrelationshipsarepresentinessentiallyallteams[ 155 ].Theparticipantsinthisresearchwerenursesandsurgicaltechnicians,whoworkedbothwitheachotherandwithsurgeonandanesthesiologists.Nursesandsurgicaltechniciansaretypicallyperceivedtooccupypositionswithlesspowerthanthoseoccupiedbysurgeonsandanesthesiologists[ 23 ].Thestudiesdiscussedinthisresearchprimarilylookedathowparticipantswithlesspower(nurses)interactedwithconfederates/virtualteammateswithmorepower(surgeonsandanesthesiologists).Theinteractionbetweennurseand 145

PAGE 146

surgicaltechnician,whopossesssimilaramountsofpower,wasalsobrieyexaminedintheexplorationofagency'sIndirectEffectonsocialpresence.However,thesestudiesdidnotaffordanyopportunitytoexploretheeffectofagencyonbehaviorwhenparticipantspossessedmorepowerthanotherteammates.Itispossiblethatagencycouldaffectbehaviorwhentraineespossessmoreauthoritythantheirvirtualteammates.Thisshouldbeconsideredwhendesigningtrainingexercisesinvolvingdifferentpowerrelationships. 8.3RealWorldUsageThisresearchproducedasystemcapableofauthoringvirtualhumansforteamtraining,andtwofullydevelopedteamtrainingsystems.Over190nursesandsurgicaltechniciansweretrainedduringthisresearch(approximately30nursesandsurgicaltechnicianstookpartinthetrainingbutdidnotconsenttoparticipateintheresearch).NursingmanagementatUFHealthhasexpressedinterestincontinuingtorunthesetrainingexercisesonayearlybasis,andindevelopingadditionalteamtrainingscenarios.IhaveworkedwithnursingmanagementandUFHealthITtotrainpeoplehowtoauthorvirtualhumansandtorunthetrainingexerciseswithoutmyassistance.Asofthetimeofthiswriting,UFHealthhasindependentlydevelopedateamtrainingexerciseintendedtoteachnursestheproperprocedurewhendrawingbloodsamplespriortosurgeries.FailuretodrawandlabelsamplescorrectlycanleadtoaWrongBloodinTubeevent,whichcanleadtoapatientbeingtransfusedwiththewrongbloodtype.SeveralindividualshavealsobeentrainedtooperatetheWizard-of-Ozsystemusedtooperatethetrainingexercise.Theseindividualsalsorancloseto20additionalnursesandsurgicaltechniciansthroughtheSurgicalCountexerciseaftertheresearchportionhadbeencompleted. 8.4FutureDirectionsTheresearchpresentedinthisdissertationhaslaidthegroundworkfortheuseofmixed-agencyteamsforteamtraining.However,manyavenuesofresearchremain 146

PAGE 147

open.Twosuchareasofinvestigationarebrieyproposedhere.Therstpertainstothelimitationdescribedaboveconcerningvisualandbehavioraldelity.Thevirtualhumansusedinthesestudieswereofrelativelyhighvisualandbehavioraldelity.Giventhatpreviousresearchhasindicatedthatdelitycaninteractwithagency,itwouldbeimportanttoexplorewhetherlowerdelityvirtualhumansarealsousefulfortraininginmixed-agencycontexts.Therearemanycircumstancesinwhichitwouldbeadvantageoustouselowerdelityvirtualhumans,eitherbecauseoftheirlowerspaceorhardwarerequirements,orforeaseofdeployment,andifbehaviorremainsconsistent.However,itisimportantthatitrstbedeterminedwhethervisualandbehavioraldelityinteractswithagencytoaffectbehavior.Asecondareaofinvestigationrelatestotheautomationofmixed-agencyteams.TheresearchconductedinthisdissertationrequiredtheuseofaWizard-of-Oztomaintainahighdegreeofbehavioraldelity.Signicantamountsofresearcharerequiredbeforemixed-agencyteamtrainingcouldbefullyautomated,mostespeciallyintherealmofspeechunderstanding(e.g.speechrecognitionsoftware)andspeechcomprehension.However,asdiscussedearlierandsupportedbyLucasetal.[ 86 ],replacingaWizard-of-Ozsystemwithafullyautomatedsystemisunlikelytochangetheeffectofagencyonbehavior,becausethekeyfactorisperceivedagency,notactualagency.Thedesiretoreplacewizard-basedtrainingprogramswithautomatedtrainingprogramsismotivatedbyeaseofdeploymentandsustainability,notbybehavioraleffects. 8.5FinalRemarksThisdissertationattemptstoaddressthefollowingthesis: Withinthecontextofmedicalteamtraining,varyingtheagencyofatrainee'steammateswillnotmeaningfullyalterthetrainee'sbehavior,giventhatthevirtualteammatesaresufcientlyrealistictomeetthetrainee'sexpectationsabouthowpeoplebehaveintherealworld. 147

PAGE 148

Thisresearchshowedthatahuman'sbehaviorduringtrainingisnotmeaningfullyinuencedbytheagencyofhisorherteammates,giventheconstraintsoutlinedinSection 8.2 .Thisisanimportantprerequisitefortheuseofvirtualhumansfortraining,asanymeaningfuleffectofagencyonbehaviorcouldbedetrimentalfortraining.Thelackofmeaningfuldifferencessupportstheuseofvirtualhumansformedicalteamtraining.Thisresearchalsolaidthegroundworkfortheuseofvirtualteammatesformedicalteamtraining.Specically,itdemonstratedthatvariable-agencyteammatescanbeusedinareal-worldsetting,andthattheycanhelpovercomelogisticaldifcultiesassociatedwithschedulingtrainingexercises.VPF2,whichcanbeusedtocreatevariable-agencyteammates,iscurrentlybeingusedbytheUFHealthhospitaltodevelopotherteamtrainingexercisesthatmeettheirspecicneeds.Thesedevelopmentshavethepotentialtoradicallyimprovetheaccessibilityofteamtrainingwithinthemedicaldomain. 148

PAGE 149

APPENDIXASPEAKINGUPSCENARIOSCRIPTThisappendixattemptstodescribethescriptusedintheSpeakingUpscenario.Asanactualinteractioncanproceednon-linearly,thisscriptservesasageneralexampledescribingparticipants'experiences,notanexactcopy.Whenitisaparticipant'sturntospeak,typicalresponsesareshowninitalics. A.1Stage1Pre-OpHolding Anesthesiologist Doyouhaveanymorequestionsformeaboutanesthesiarisks,orwhatweneedtodoforClay? Mother Ithinkyou'vecoveredallofmyquestions,mybiggestconcernisthatmyson'srelaxedwhenyoustartanesthesiaorstartanyoftheIV's. Anesthesiologist Absolutely,wecangivehimsomethingtorelaxhim.Doyouthinkhecantakeanythingbymouth,or...? Mother Idon'tknow,we'vehadproceduresinthepastwheretheygivehimalittlebitofuh...gasintheoperatingroom,torelaxhim,andthentheystartwhatevertheyhavetodo.I'mnotusuallytheresoIdon'treallyknowwhatthatis. Anesthesiologist Well,wehaveanicebubblegumavoredmaskwecanhavehimbreathetohelphimrelax,andthenwe'llattemptallourIVplacement. Mother That'llbeperfect! Anesthesiologist Ok,Ithinkournurseneedstocheckafewthingswithyou,andthenwe'llsendClaybacktotheoperatingroomandgetstarted. Mother Thankyou.Itwasnicetomeetyou. Anesthesiologist (Tonurse)YoucangoaheadandaskMollyanyquestionsyouneedto. Nurse (Nurseintroducesherselfandasksanyquestionsshemayhave) Anesthesiologist (Tonurse)Doyouhaveanyotherquestions? Nurse No. 149

PAGE 150

Anesthesiologist Alright,let'sgetstarted. A.2Stage2Brieng Surgeon Ok,soeveryoneispresent.Let'sgoaheadandintroduceourselves. Surgeon Clay,we'regoingtobetakingcareofyoutoday.I'mDr.Girard,I'myoursurgicalattending. Anesth. MynameisDr.Sanders.I'mgoingtobetheanesthesiaattending. Nurse Myname'sSarah. Surgeon (Tonurse)Ok,nurse,canyouconrmthepatient'snameforme? Nurse ClayMathews Surgeon Alright,andhisbirthdate? Nurse ItsayshisbirthdateisApril13th,1999. Surgeon Andhismedicalrecordnumber? Nurse Hismedicalrecordnumberis123456. Surgeon WhatsurgerydoestheconsentsayClayisherefortoday? Nurse IhaveClayhereforacomplexscoliosisrepair. Surgeon Ok,thatsoundsrighttome. Surgeon (Toanesth)Youhaven'tputmonitorsonClayyet? Anesthesiologist No,Ididn'twanttoagitateClay.Hismothersaidhecanbeabittouchyaroundneedles,soI'mconcernedthemonitorsmightdisturbhimaswell. Surgeon Idon'tthinkputtingmonitorsonshouldagitatehim,evenifhedoesn'tlikeneedles. Anesthesiologist Nurse,whatdoyouthink? Nurse Ithinkitwouldbebestifwegoaheadandputthemonitorson. Anesthesiologist Alright,that'sne.I'llputthemonitorsonrightafterwe'redonewiththebrieng. Surgeon Andwehavethecorrectelectronicmedicalrecords,Icanseethex-raysoverthere. 150

PAGE 151

Surgeon Ok,sitemarkingsarenotindicated,he'shavingscolioticsurgery,hehasnoknownallergiesindicated. Surgeon (Toanesth)Ok,antibioticsfortheprocedure? Anesthesiologist Anceph,justforskinbugs. Surgeon 2grams,withthisweight? Anesthesiologist He'sprettybig,yeahI'dsay2grams Surgeon (Toanesth)Alright,currentmedications? Anesthesiologist Nothingrelevant. Surgeon Ok...anti-coagulants,SCDsshouldbesufcient. Surgeon (Tonurse)Alright,whataboutABOverication? Nurse Wedon'thaveanythingyet,needtosend2samplesdown. Surgeon Ok,we'lltakecareofthatafterClay'sasleep. Surgeon (Toanesth)Dr.Sanders,makesurethosesamplesgetdrawnwhileyou'repreparinghim. Anesthesiologist Ok,I'lltakecareofthat. Surgeon Wedidn'tdrawanybloodonhim,sowedon'thaveapreoperativecrit. Anesthesiologist Howmanyunitsdoyoubelievewe'llneedfortheprocedure? Surgeon Idon'tbleed.2unitsshouldbesufcient. Surgeon Sopositioning-wise,he'llbeprone,withaproneview. Surgeon Betablockadeisnotindicated,dvtprophylaxis–,we'regoingtodocompressionstockings,uh,antibioticsarecovered,andwe'llgivehimnicewarmedIVuidsalso. Anesthesiologist We'lluseanupperandlowerbairhugger,right? Surgeon Ithinkwe'reonlygoingtohaveroomforalowerbodybairhugger. Surgeon Nurse,whatdoyouthink? Nurse Ithinkthereshouldberoomforboth. Surgeon Ok,ne,we'llgowithboththen. 151

PAGE 152

Surgeon Arealltheimplantsandinstrumentsready? Nurse Yes,theyare. Surgeon Post-operativedestination?PICU? Nurse Correct. Surgeon Ok,sodoesanyonehaveanyconcerns?Arewereadytobegin? Nurse Ihavenoconcerns. Anesthesiologist Noconcernsonanesthesia'spart. Surgeon Alright,let'sgetstartedthen. A.3Stage3Timeout Surgeon (Toanesth)Ok,we'venallyinduced.Yougotyourlinesinright? Anesthesiologist Yes,we'vegotthemin Surgeon Ok.It'sbeenanhour,andwe'vegotanotheroneofthesetodolatertoday.Let'sgetmoving Surgeon Thisisstillthecorrectpatient,right?HistagstillsaysClayMathews,right? Nurse That'scorrect. Surgeon Andhismedicalrecordnumberisstill123456? Nurse Yes. Surgeon Andthesitemarking–,Istillseethex-raysoverthere–,I'mstillplanningtodoamulti-levelfusionT1toL1scoliosisrepair. Nurse Yes. Surgeon (Toanesth)Antibiotics,arewereadytomaketheincision? Anesthesiologist Yes,we'rereadytomaketheincision. Surgeon (Toanesth)Ok,wehavebloodavailable?Yousentthespecimensdown? Anesthesiologist Wedrewthesamples,uh,...butitlooksliketheynevergotsenttothelab... Surgeon What?Areyoukiddingme?Youdidn'tsendthesamplesdown?Whatwereyoudoing? 152

PAGE 153

Anesthesiologist Look,I'msorry,thingswerehecticwhileweweregettinghimready. Surgeon Whatever.Justsendthesamplesdownnow. Surgeon We'regoingtohavetogoaheadandproceedwhilethelabsarebeingprocessed.We'realreadyrunningbehindtoday Surgeon Anyobjections? Nurse Weneedtohavebloodavailablebeforewecanproceed. Surgeon Look,Clayisayoungman,neverhadsurgerybefore,neverbeentransfused,and,obviously,he'sneverbeenpregnant,sothere'swhat,maybea1%chancehehashadantibodies? Nurse It'sstillarisk,nomatterhowsmall. Surgeon Look,we'reatleastanhourawayfromanysignicantbloodloss,maybeevenmore.Thelabsshouldbeprocessedin,what,30minutes?That'smorethanenoughtime. Nurse Hospitalpolicysaysthatweneedtohavereplacementbloodavailablebeforebeginningthesurgery. Surgeon Sowhatarewesupposedtodoforthenexthour?Justkeephimunderanesthesia,gettingbraindamageforallIknow? Nurse I'msureanesthesiacanpreventanythinglikethatfromhappening. Surgeon Youknowwhat,I'mjustgoingtogetstarted.Idon'thavetimetowaste,I'vegototherpatientsIhavetoattendtoafterthiskid.SendthebloodnowandI'mgoingtogetstarted.Givemetheknife. Nurse I'mgoingtocallmychargenurse. Surgeon AngryFine.Youdothat. 153

PAGE 154

APPENDIXBSURGICALCOUNTSCENARIOSCRIPTThisappendixattemptstodescribethescriptusedintheSurgicalCountscenario.Asanactualinteractioncanproceednon-linearly,thisscriptservesasageneralexampledescribingparticipants'experiences,notanexactcopy.Whenitisaparticipant'sturntospeak,typicalresponsesareshowninitalics. B.1Stage1Brieng Surgeon Ok,we'renallybackintheOR.Itlookslikeeveryoneispresent.Let'sgoaroundandintroduceourselves. Surgeon Eric,we'regoingtobetakingcareofyoutoday.I'myoursurgicalattending,Dr.Girard. Anesthesiologist MynameisDr.Sanders.I'mtheanesthesiaattendingtoday. Nurse I'mSandy,I'llbeyournursetoday. ScrubTech I'mLaura,I'mthesurgicaltechnicianintheroomtoday. Surgeon Great.It'sgoodtobeworkingwithyoualltoday.Let'sgoaheadandstartthebrieng. Surgeon Mr.Mason,couldyouconrmformeyourfullnameanddateofbirth? Patient MynameisEricMason.IwasbornonDecember13th,1954. Surgeon Alright,great.Andwhatprocedureareyouhavingtoday? Patient You'regoingtotakeoutmypancreas,withcamerasandasmallincision,Ithink. Surgeon That'sright,that'swhatIseehereonyourconsentform. Surgeon (Tonurse)Ok.AndMr.Mason'smedicalrecordnumberis02119711,correct? Nurse Yes,that'scorrect. Surgeon Ok,sitemarkingsaren'tneededsincewe'reremovinghispancreas. Surgeon Anddoyouhaveanyallergies? Patient Idon'tthinkso.NonethatI'mawareof. Surgeon Alright,great. 154

PAGE 155

Surgeon (Toanesth)Antibioticsforthisprocedure?Cefotetan? Anesthesiologist Yes,that'swhatIwasplanningtoadminister.2grams? Surgeon Yeah,2gramswillbegood. Surgeon IrememberMr.Masonwasonsomestatinpreop,sowewon'tneedanyanticogulantsorbetablockers. Surgeon (Tonurse)AndwhatwasMr.Mason'sHH? Nurse HisHHis11.5and34. Surgeon (Tonurse)Alright,whataboutthetype-and-screen?Dowehavereplacementbloodavailable? Nurse Uh,thetype-and-screenwasgood,butitlooksliketheABOvericationwasn'tdrawninpreop. Surgeon (Toanesth)Alright,Dr.Sanders,makesurethosesamplesgetdrawnafterwenishthisbrieng. Anesthesiologist Ok,I'lltakecareofthat. Surgeon He'llbesupine,andI'dlikehisarmstucked. Surgeon Betablockadeisnotindicated.ForDvtprophylaxiswe'regoingtodocompressionstockings. Surgeon (Tonurse)Hehadheparininpreop,right? Nurse Yes,that'scorrect. Surgeon (Toanesth)Ok,andwe'lluseanupperandalowerbodybairhugger? Anesthesiologist Yeah,let'sdothat. Surgeon Uh,wealreadycoveredantibiotics,andwe'llgivehimnicewarmedIVuids. Surgeon (Tonurse)HaveMr.Mason'sSCDsbeenplaced? Nurse Yes,they'reon. Surgeon (Totech)Arealltheinstrumentsreadyandintheroom? ScrubTech Yes,they'reallready. Surgeon Andyou'vegotendo-clips? 155

PAGE 156

ScrubTech Yes,wedo. Surgeon Great,thanks. Anesthesiologist (Tosurgeon)Howmanyunitsdoyouwantfortheprocedure? Surgeon Idon'tbleedmuch.Twounitsshouldbesufcient. Surgeon Ok,wedon'tneedanyimaging. Surgeon Uh,ok.Post-operativedestination?ThePACU? Nurse Yes,that'sright. Surgeon Alright,that'severythingthen.Doesanyonehaveanyconcerns,orarewereadytobegin? Nurse Ithinkwe'reready Anesthesiologist Ihavenoconcerns ScrubTech Ithinkwe'reready Surgeon Alright,let'sgetstarted. B.2Stage2Timeout Surgeon (Toanesth)Ok,we'venallyinduced.Yougotyourlinesinright? Anesthesiologist Yes,we'vegotthemin Surgeon Alright,great.We'rerunningbehindschedule,solet'sseeifwecannishthetimeoutquickly. Surgeon Thisisstillthecorrectpatient,right?HistagstillsaysEricMason,right? Nurse Yes,thisisEricMason. Surgeon Andhismedicalrecordnumberisstill02119711? Nurse Yes,that'sright. Surgeon Andwearestillplanningtodoalaparoscopicwhipple,correct? Nurse Yes,thatiscorrect. Surgeon (Totech)Ok.You'vegotalltheequipmentready? ScrubTech Yes,wedo. Surgeon Dowehavea0degree,5millimeterscope? 156

PAGE 157

ScrubTech Yes,I'vegotonerighthere. Surgeon Perfect,thanks. Surgeon (Toanesth)Havetheantibioticsbeenadministerd? Anesthesiologist Yes,I'veadministeredtheantibiotics. Surgeon (Toanesth)Great,dowehavebloodavailable? Anesthesiologist Wesentthesamplesdown,butitlookslikethebloodbankneverreceivedthem. Surgeon What?Areyoukiddingme?Andyou'rejusttellingmethisnow?Whydidn'tyoure-sendthesamples? Anesthesiologist Look,I'msorry,thingswerehecticwhileweweregettinghimready. Surgeon Whatever,Idon'tcarewhatyou'reexcuseis.Justsendthesamplesdownnow. Surgeon We'regoingtohavetogetstartedwhilethelabsarebeingprocessed.We'realreadyrunningbehindtoday Surgeon Anyobjections? Nurse Sowhatarewesupposedtodoifheneedsbloodearlierthanexpected? Surgeon Wecanalwaysuseuncrossmatchedbloodifwehaveto.Thathappensallthetime,Idon'tseewhyyou'remakingthissuchabigdeal. Nurse I'mnotcomfortablestartingwithoutanybloodavailable. Surgeon Sowhatarewesupposedtodowhilewe'rewaiting?Justkeephimunderanesthesia,gettingbraindamageforallIknow? Nurse Thispolicyisintendedtohelppreventusfromtransfusingthewrongtypeofblood.Requiringasecondsamplehelpsuspreventawrongbloodintubeevent. Surgeon Welllook,we'reatleastanhourawayfromanysignicantbloodloss,maybeevenmore.Thelabsshouldbeprocessedin,what,30minutes?That'smorethanenoughtime. 157

PAGE 158

Nurse Iunderstandyourconcerns,butitisnotsafetoproceed.Thebloodbankwon'treleaseusreplacementblooduntiltheygetanABOverication. Surgeon Look,thebloodbank'spolicyofneedingtwosamplesissilly,thisisunnecessary.Wealreadyknowhisbloodtypefromhistypeandscreenandweknowthathedoesn'thaveantibodies. Surgeon Youknowwhat,I'mjustgoingtogetstarted.Idon'thavetimetowaste,I'vegototherpatientsIhavetoattendtoafterthisguy.SendthebloodnowandI'mgoingtogetstarted. Surgeon (Totech)Givemetheknife. ScrubTech I'mnotgoingtogiveyoutheknife.Weneedtowaituntilbloodisavailable. Surgeon I'llgettheknifemyselfthen.We'restartingthissurgerynow. Nurse Nowholdon,thisisn'tsafeforthepatient. Surgeon I'malreadyrunningbehindschedule.Wearestartingthissurgerynow. Nurse Ifyouinsistonproceeding,thenI'mgoingtohavetocallmychargenurse Surgeon Fine.Youdothat. B.3Stage3ClosingCount Surgeon Alright,I'mnishedandreadytoclose.Goodjobeveryone.Canyougoaheadandstarttheclosingcount? ScrubTech Sure,we'llstartthecount. Surgeon Ok,letmeknowwhenyou'renished. ScrubTech (Tonurse)Ok,we'llstartwiththelaps.Areyoureadytostartthecount? Nurse Yes,I'mready. ScrubTech Alright ScrubTech (Lapsonsurgicaleld)One,Two,Three– ScrubTech (Lapsonmayostand)Four,Five,Six,Seven,Eight.Ok,I'vegoteighthere.Whatdoyouhaveinthespongecountingbag? 158

PAGE 159

Nurse (Lapsinspongebag)Nine,Ten,Eleven,Twelve,Thirteen,Fourteen,Fifteen,Sixteen,Seventeen,Eighteen,Nineteen,Twenty,Twenty-one,Twenty-two,Twenty-three,Twenty-four,Twenty-ve,Twenty-six,Twenty-seven,Twenty-eight,Twenty-nine,Thirty.Thirtylaps. Nurse Ok,thirtylaps,that'scorrect. ScrubTech Alright,nowraytecs. ScrubTech (Raytecsonsurgicaleld)Ihavetwoup. ScrubTech (Raytecsonmayostand)Three,Four,Five,Six,Seven,Eight.I'vegoteightraytecshere.Howmanyareinthebag? Nurse (Raytecsinspongebag)Nine,Ten.TenRaytecs. Nurse Ok,tenraytecs,that'sright. ScrubTech OK,KD'snext. ScrubTech KDsonmayostandOne,Two,Three,Four,Five. Nurse Ok,veKDs,that'sgood. ScrubTech Alright,let'sdoneedlesnow. ScrubTech (Needlesonmayostand)One,Two,Three,Four,Five,Six,Seven,Eight,Nine,Ten,Eleven,Twelve,Thirteen,Fourteen,Fifteen,Sixteen. Nurse Ok,sixteenneedles.that'sgood. ScrubTech Alright,bovietipsnext. ScrubTech (Bovietipsonmayostand)One,Two,Threebovietips. Nurse Ok,threebovietips,that'sright. ScrubTech Alright,bladesnext. ScrubTech (Bladesonmayostand)One,Two,Three,Four,Fiveblades. Nurse Ok,veblades,that'sright. ScrubTech Alright,hyposnext. ScrubTech (Hyposonmayostand)One,twohypos. Nurse Ok,twohypos,that'sright. 159

PAGE 160

ScrubTech Alright,boatsnext. ScrubTech (Boatsonmayostand)Ok,I'vegotoneboat. Nurse Uh,I'vegottwoboatswrittenhere.You'resurethere'sonlyoneboat? ScrubTech Letmecheckagain.Yeah,I'veonlygotoneboathere. Surgeon You'remissingsomething?Whathappened? ScrubTech Aboatismissing.Thereweretwoattheinitialcount,butwe'veonlygotonenow. Surgeon Alright,wellwe'regoingtoneedtondit.Everyonestartsearchingforamissingboat. Surgeon Areyoundinganything?I'mnotndinganythinginsidethepatient. ScrubTech We'restilllooking.Wehaven'tfoundanythingyet. Surgeon I'mnotseeinganythinginsidethepatient.Areyousurethecountwasoff?I'veneverhadacountbeofflikethisbefore. ScrubTech We'resurethecountisoff.There'sonlyoneboataccountedfor. Surgeon Ok,well,I'mverypositivethatnothingisinthepatient.Andyou'retellingmeyouhaven'tfounditintheroom?Didyoucheckinthedrapes? ScrubTech Welookedinthedrapes,butwedidn'tndanything. Anesthesiologist Arewegoingtoneedtogetanx-ray?IneedtoknowifIneedtokeepthisguyasleep.He'salreadyalmostwokenuponce. Surgeon Yeah,Iguessweneedtoorderanx-ray.Istilldon'tthinkit'sinthepatientthough.Ididn'tseeanythinginthere. Surgeon Ok,thex-rayisback.LikeIsaidearlier,Idon'tseeanythinginsidethepatient.Let'sclosehimup.Theboatmustbeintheroomsomewhere. ScrubTech Sir,thepolicystatestheattendingradiologisthastolookatthelm. Surgeon Look,Icanseethex-rayrightnowandthere'snothingthere.Idon'twanttowastemoretimewhenthere'sclearlynothingonthex-ray. ScrubTech Yessir,butthepolicystatestheattendingradiologisthastolookatthelm. 160

PAGE 161

Surgeon I'mjustasqualiedtoreadanx-rayasanyradiologistis.Thisisastupidpolicy,andIdon'tagreewithit. ScrubTech Iunderstandsir,butthispolicyexistsforthepatient'ssafety. Surgeon Look,I'vebeenherefor36hours,Iwassupposedtogohomesixhoursago,I'mexhausted.There'snothingonthex-ray,I'mgoingtoclosethepatientnow.Theboatisn'tinsidehim. ScrubTech I'msorry,butIcan'tletyouclosethepatientuntilyou'vespokewiththeradiologist. Surgeon Thisisridiculous.Youcanseethex-rayyourself,there'snothingintherethat'snotsupposedtobe.Whyareyoubeingsodifcult?I'mclosingthepatientnow. ScrubTech Wouldyoulikemetogettheradiologistonthephoneforyou? Surgeon I'mnottalkingtotheradiologist.Thisisawasteoftime.I'mclosingthepatientnow. ScrubTech Itwillonlytakeaminutesir. Surgeon Youknowwhat,I'mthesurgeonhere,I'mmakingthisdecision.I'mnotwaitingtotalktotheradiologist.We'reclosingthepatientnow. ScrubTech Alright,wellI'mgoingtohavetocallthechargenurse. Surgeon Fine,youdothat. 161

PAGE 162

REFERENCES [1] T.Bickmore,“Relationalagents:Effectingchangethroughhuman-computerrelationships,”Ph.D.dissertation,MassachusettsInstituteofTechnology,2003.[Online].Available: http://www.ccis.northeastern.edu/home/bickmore/bickmore-thesis.pdf [2] A.W.deBorstandB.deGelder,“Isittherealdeal?Perceptionofvirtualcharactersversushumans:anaffectivecognitiveneuroscienceperspective,”FrontiersinPsychology,vol.6,no.May,pp.1,2015.[Online].Available: http://journal.frontiersin.org/article/10.3389/fpsyg.2015.00576 [3] M.Slater,A.Rovira,R.Southern,D.Swapp,J.J.Zhang,C.Campbell,andM.Levine,“BystanderResponsestoaViolentIncidentinanImmersiveVirtualEnvironment,”PLoSONE,vol.8,no.1,2013. [4] B.Rossen,K.Johnsen,A.Deladisma,S.Lind,andB.Lok,“Virtualhumanselicitskin-tonebiasconsistentwithreal-worldskin-tonebiases,”IntelligentVirtualAgents,2008.[Online].Available: http://www.springerlink.com/index/412l15740g814803.pdf [5] A.Raij,K.Johnsen,R.Dickerson,B.Lok,M.Cohen,A.Stevens,T.Bernard,C.Oxendine,P.Wagner,andD.ScottLind,“Interpersonalscenarios:Virtualreal?”inProceedings-IEEEVirtualReality,vol.2006.Ieee,2006,p.8.[Online].Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1667627 [6] M.Slater,D.P.Pertaub,andA.Steed,“PublicSpeakinginVirtualReality:FacinganAudienceofAvatars,”IEEEComputerGraphicsandApplications,vol.19,no.2,pp.6,1999. [7] K.Johnsen,A.Raij,A.Stevens,D.S.Lind,andB.Lok,“Thevalidityofavirtualhumanexperienceforinterpersonalskillseducation,”inProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystemsCHI07.ACM,2007,pp.1049.[Online].Available: http://portal.acm.org/citation.cfm?doid=1240624.1240784 [8] P.Kenny,T.D.Parsons,J.Gratch,andA.A.Rizzo,“EvaluationofJustina:avirtualpatientwithPTSD,”IntelligentVirtualAgents,pp.394,2008.[Online].Available: http://link.springer.com/chapter/10.1007/978-3-540-85483-8 40 [9] T.Bloodworth,L.Cairco,J.Mcclendon,F.Larry,S.Babu,N.K.Meehan,A.Johnson,andA.C.Ulinski,“InitialEvaluationofaVirtualPediatricPatientSystem,”CarolinasWomeninComputing,pp.1,2012. [10] A.Kotranza,B.Lok,A.Deladisma,C.M.Pugh,andD.S.Lind,“Mixedrealityhumans:Evaluatingbehavior,usability,andacceptability,”IEEETransactionson 162

PAGE 163

VisualizationandComputerGraphics,vol.15,no.3,pp.369,2009.[Online].Available: http://ieeexplore.ieee.org/xpls/abs all.jsp?arnumber=4689554 [11] A.Robb,R.Kopper,R.Ambani,F.Qayyum,D.Lind,L.M.Su,andB.Lok,“Leveragingvirtualhumanstoeffectivelypreparelearnersforstressfulinterpersonalexperiences,”IEEETransactionsonVisualizationandComputerGraphics,vol.19,no.4,pp.662,2013. [12] A.T.Hayes,S.E.Hardin,andC.E.Hughes,“PerceivedPresence'sroleonlearningoutcomesinamixedrealityclassroomofsimulatedstudents,”inLectureNotesinComputerScience(includingsubseriesLectureNotesinArticialIntelligenceandLectureNotesinBioinformatics),vol.8022LNCS,no.PART2,2013,pp.142. [13] M.Core,D.Traum,H.C.Lane,W.Swartout,J.Gratch,M.vanLent,andS.Marsella,“TeachingNegotiationSkillsthroughPracticeandReectionwithVirtualHumans,”Simulation,vol.82,no.11,pp.685,Nov.2006.[Online].Available: http://sim.sagepub.com/cgi/doi/10.1177/0037549706075542 [14] D.Traum,J.Rickel,J.Gratch,andS.Marsella,“Negotiationovertasksinhybridhuman-agentteamsforsimulation-basedtraining,”ProceedingsofthesecondinternationaljointconferenceonAutonomousagentsandmultiagentsystems-AAMAS'03,p.441,2003.[Online].Available: http://dl.acm.org/citation.cfm?id=860646$ndelimiterE30F$nhttp://portal.acm.org/citation.cfm?doid=860575.860646 [15] K.L.NowakandF.Biocca,“TheEffectoftheAgencyandAnthropomorphismonUsers'SenseofTelepresence,Copresence,andSocialPresenceinVirtualEnvironments,”Presence:TeleoperatorsandVirtualEnvironments,vol.12,pp.481,2003. [16] F.Biocca,C.Harms,J.K.Burgoon,M.Interface,andE.Lansing,“TowardsAMoreRobustTheoryandMeasureofSocialPresence:ReviewandSuggestedCriteria,”Presence:Teleoperators&VirtualEnvironments,vol.12,no.5,pp.456,2003. [17] D.Clements,M.Dault,andA.Priest,“Effectiveteamworkinhealthcare:researchandreality.”HealthcarePapers,vol.7SpecNo,pp.26,2007. [18] M.Leonard,S.Graham,andD.Bonacum,“Thehumanfactor:thecriticalimportanceofeffectiveteamworkandcommunicationinprovidingsafecare.”Quality&safetyinhealthcare,vol.13Suppl1,pp.i85–i90,2004. [19] E.Salas,D.DiazGranados,C.Klein,C.S.Burke,K.C.Stagl,G.F.Goodwin,andS.M.Halpin,“Doesteamtrainingimproveteamperformance?Ameta-analysis.”Humanfactors,vol.50,no.6,pp.903,2008.[Online].Available: http://hfs.sagepub.com/content/50/6/903.short 163

PAGE 164

[20] H.AguinisandK.Kraiger,“Benetsoftraininganddevelopmentforindividualsandteams,organizations,andsociety.”Annualreviewofpsychology,vol.60,pp.451,2009. [21] S.K.RothschildandS.Lapidos,“Virtualintegratedpractice:integratingteamsandtechnologytomanagechronicdiseaseinprimarycare.”Journalofmedicalsystems,vol.27,no.1,pp.85,Feb.2003.[Online].Available: http://www.ncbi.nlm.nih.gov/pubmed/12617200 [22] J.N.Bailenson,J.Blascovich,A.C.Beall,andJ.M.Loomis,“InterpersonalDistanceinImmersiveVirtualEnvironments,”PersonalityandSocialPsychologyBulletin,vol.29,no.7,pp.819,2003.[Online].Available: http://psp.sagepub.com/content/29/7/819.full.pdf [23] F.Gardezi,L.Lingard,S.Espin,S.Whyte,B.Orser,andG.R.Baker,“Silence,powerandcommunicationintheoperatingroom,”JournalofAdvancedNursing,vol.65,no.7,pp.1390,2009. [24] D.Traum,P.Aggarwal,R.Artstein,S.Foutz,J.Gerten,A.Katsamanis,A.Leuski,D.Noren,andW.Swartout,“AdaandGrace:DirectInteractionwithMuseumVisitors,”no.Ili,pp.245,2012. [25] P.Aggarwal,R.Artstein,J.Gerten,A.Katsamanis,S.Narayanan,A.Nazarian,andD.Traum,“TheTwinsCorpusofMuseumVisitorQuestions,”no.3,pp.2355,2010. [26] R.McDonnell,M.Breidt,andH.H.Bulthoff,“RendermeReal?InvestigatingtheEffectofRenderStyleonthePerceptionofAnimatedVirtualHumans,”ACMTransactionsonGraphics,vol.31,no.4,pp.1,2012.[Online].Available: http://dl.acm.org/citation.cfm?doid=2185520.2185587 [27] A.vonderPutten,N.Kramer,andJ.Gratch,“HowOurPersonalityShapesOurInteractionswithVirtualCharacters-ImplicationsforResearchandDevelopment,”IntelligentVirtualAgents,2010.[Online].Available: http://www.springerlink.com/index/R851090J536U48R0.pdf [28] S.Maddock,J.Edge,andM.Sanchez,“MovementRealisminComputerFacialAnimation,”19thBritishHCIGroupAnnualConference,WorkshoponHuman-animatedCharactersInteraction,vol.6,pp.1,2005.[Online].Available: http://www.researchgate.net/prole/Steve Maddock/publication/228922876 Movement realism in computer facial animation/links/09e4150e6f78d266e7000000.pdf [29] T.Ni,D.a.Bowman,andJ.Chen,“IncreasesDisplaySizeandResolutionImproveTaskPerformanceinInformation-RichVirtualEnvironments,”GraphicsInterface,p.139,2006.[Online].Available: http://cs.brown.edu/jchen/publications/2006 GI IRVE.pdf 164

PAGE 165

[30] K.JohnsenandB.Lok,“AnEvaluationofImmersiveDisplaysforVirtualHumanExperiences,”2008IEEEVirtualRealityConference,pp.133,2008.[Online].Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4480764 [31] Y.Wang,P.Khooshabeh,andJ.Gratch,“Lookingrealandmakingmistakes,”inIntelligentVirtualAgents,2013.[Online].Available: http://ict.usc.edu/pubs/LookingRealandMakingMistakes.pdf [32] E.-J.Lee,“Themorehumanlike,thebetter?Howspeechtypeanduserscognitivestyleaffectsocialresponsestocomputers,”ComputersinHumanBehavior,vol.26,no.4,pp.665,Jul.2010.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S0747563210000051 [33] M.Garau,M.Slater,V.Vinayagamoorthy,a.Brogni,a.Steed,andM.Sasse,“Theimpactofavatarrealismandeyegazecontrolonperceivedqualityofcommunicationinasharedimmersivevirtualenvironment.”ProceedingsoftheconferenceonHumanfactorsincomputingsystems-CHI'03,no.5,p.529,2003.[Online].Available: http://discovery.ucl.ac.uk/13446/ [34] J.RickelandW.Johnson,“Task-orientedcollaborationwithembodiedagentsinvirtualworlds,”Embodiedconversationalagents,pp.1,2000.[Online].Available: http://www.isi.edu/isd/VET/eca00.pdf [35] A.M.V.D.Putten,N.C.Kramer,andJ.Gratch,“Who'sthere?CanaVirtualAgentReallyElicitSocialPresence?”Proceedingsofthe12thInternationalWorkshoponPresence,pp.1,2009.[Online].Available: http://ict.usc.edu/pubs/whosthere.pdf [36] A.M.VonDerPutten,N.C.Kramer,J.Gratch,andS.H.Kang,“”Itdoesn'tmatterwhatyouare!”Explainingsocialeffectsofagentsandavatars,”ComputersinHumanBehavior,vol.26,no.6,pp.1641,Nov.2010.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S0747563210001846 [37] L.Ring,D.Utami,andT.Bickmore,“TheRightAgentfortheJob?Theeffectsofagentvisualappearanceontaskdomain,”IntelligentVirtualAgents,2014.[Online].Available: http://relationalagents.com/publications/render.iva14.pdf [38] B.ReevesandC.Nass,TheMediaEquation:Howpeopletreatcomputers,television,andnewmedialikerealpeopleandplaces.CSLIPublicationsandCambridgeuniversitypress,1996. [39] C.NassandY.Moon,“MachinesandMindlessness:SocialResponsestoComputers.”JournalofSocialIssues,vol.56,no.1,p.81,Jan.2000.[Online].Available: http://ezaccess.libraries.psu.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=a2h&AN=3379301&site=ehost-live [40] E.Lee,C.Nass,andS.Brave,“Cancomputer-generatedspeechhavegender?:anexperimentaltestofgenderstereotype,”CHI'00extended 165

PAGE 166

abstractsonHumanfactors...,pp.289,2000.[Online].Available: http://dl.acm.org/citation.cfm?id=633461 [41] C.NassandK.M.Lee,“Doescomputer-generatedspeechmanifestpersonality?anexperimentaltestofsimilarity-attraction,”ProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystemsCHI00,vol.2,no.1,pp.329,2000.[Online].Available: http://portal.acm.org/citation.cfm?doid=332040.332452 [42] C.Nass,B.Fogg,andY.Moon,“Cancomputersbeteammates?”InternationalJournalofHuman-ComputerStudies,vol.45,pp.669,1996.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S1071581996900737 [43] B.Herbelin,“Virtualrealityexposuretherapyforsocialphobia,”Ph.D.dissertation,EcolePolytechniqueFederaledeLausanne,2005. [44] D.-P.Pertaub,M.Slater,andC.Barker,“Anexperimentonpublicspeakinganxietyinresponsetothreedifferenttypesofvirtualaudience,”Presence:Teleoperators&VirtualEnvironments,vol.11,no.1,pp.68,2002.[Online].Available: http://eprints.ucl.ac.uk/13447/ [45] P.S.Bordnick,K.M.Graap,H.L.Copp,J.Brooks,andM.Ferrer,“Virtualrealitycuereactivityassessmentincigarettesmokers.”Cyberpsychology&behavior:theimpactoftheInternet,multimediaandvirtualrealityonbehaviorandsociety,vol.8,no.5,pp.487,2005. [46] S.Babu,T.Grechkin,B.Chihak,C.Ziemer,J.Kearney,J.Cremer,andJ.Plumert,“Avirtualpeerforinvestigatingsocialinuencesonchildren'sbicycling,”Proceedings-IEEEVirtualReality,no.Figure2,pp.91,2009. [47] J.Fox,J.Bailenson,andJ.Binney,“VirtualExperiences,PhysicalBehaviors:TheEffectofPresenceonImitationofanEatingAvatar,”Presence:Teleoperators&VirtualEnvironments,vol.18,no.4,pp.294,Aug.2009.[Online].Available: http://www.mitpressjournals.org/doi/abs/10.1162/pres.18.4.294 [48] C.A.Zanbaka,A.C.Ulinski,P.Goolkasian,andL.F.Hodges,“EffectsofVirtualHumanPresenceonTaskPerformance,”ProceedingsArticialRealityandTelexistence,pp.174,2004. [49] C.ZanbakaandA.Ulinski,“Socialresponsestovirtualhumans:implicationsforfutureinterfacedesign,”ProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystems,2007.[Online].Available: http://dl.acm.org/citation.cfm?id=1240861 [50] A.L.Hayes,A.C.Ulinski,andL.F.Hodges,“ThatAvatarIsLookingatMe!SocialInhibitioninVirtualWorlds,”pp.454,2010. 166

PAGE 167

[51] M.Slater,A.Antley,A.Davison,D.Swapp,C.Guger,C.Barker,N.Pistrang,andM.V.Sanchez-Vives,“AvirtualrepriseoftheStanleyMilgramobedienceexperiments,”PLoSONE,vol.1,no.1,p.e39,Jan.2006.[Online].Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1762398&tool=pmcentrez&rendertype=abstract [52] C.Zanbaka,P.Goolkasian,andL.Hodges,“Canavirtualcatpersuadeyou?:theroleofgenderandrealisminspeakerpersuasiveness,”ProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystems,pp.1153,2006.[Online].Available: http://dl.acm.org/citation.cfm?id=1124945 [53] A.T.Hirsh,S.Z.George,andM.E.Robinson,“Painassessmentandtreatmentdisparities:avirtualhumantechnologyinvestigation.”Pain,vol.143,no.1-2,pp.106,May2009.[Online].Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2666785&tool=pmcentrez&rendertype=abstract [54] L.Gong,“Howsocialissocialresponsestocomputers?Thefunctionofthedegreeofanthropomorphismincomputerrepresentations,”ComputersinHumanBehavior,vol.24,no.4,pp.1494,Jul.2008.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S0747563207000945 [55] M.Garau,M.Slater,S.Bee,andM.Sasse,“Theimpactofeyegazeoncommunicationusinghumanoidavatars,”...oftheSIGCHIconferenceon...,pp.309,2001.[Online].Available: http://dl.acm.org/citation.cfm?id=365121 [56] N.Yee,J.N.Bailenson,andK.Rickertsen,“Ameta-analysisoftheimpactoftheinclusionandrealismofhuman-likefacesonuserexperiencesininterfaces,”ProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystems-CHI'07,p.1,2007.[Online].Available: http://portal.acm.org/citation.cfm?doid=1240624.1240626 [57] V.Vinayagamoorthy,A.Steed,andM.Slater,“Buildingcharacters:Lessonsdrawnfromvirtualenvironments,”COGSCI2005Workshop,2005.[Online].Available: http://www.androidscience.com/proceedings2005/VinayagamoorthyCogSci2005AS.pdf [58] S.Babu,E.Suma,T.Barnes,andL.F.Hodges,“Canimmersivevirtualhumansteachsocialconversationalprotocols?”Proceedings-IEEEVirtualReality,pp.215,2007. [59] M.a.Zielke,M.J.Evans,F.Dufour,T.V.Christopher,J.K.Donahue,P.Johnson,E.B.Jennings,B.S.Friedman,P.L.Ounekeo,andR.Flores,“Seriousgamesforimmersiveculturaltraining:Creatingalivingworld,”IEEEComputerGraphicsandApplications,vol.29,no.2,pp.49,2009. [60] E.Prasolova-Frland,M.Fominykh,andR.Darisiro,“VirtualAfghanvillageasalow-costenvironmentfortrainingculturalawarenessinamilitarycontext,”LectureNotesinComputerScience(includingsubseriesLectureNotesinArticial 167

PAGE 168

IntelligenceandLectureNotesinBioinformatics),vol.8095LNCS,no.September,pp.569,2013. [61] W.L.Johnson,L.Friedland,P.J.Schrider,A.Valente,andS.Sheridan,“TheVirtualCulturalAwarenessTrainer(VCAT):JointKnowledgeOnlines(JKOs)SolutiontotheIndividualOperationalCultureandLanguageTrainingGap.” [62] R.Hill,J.Gratch,S.Marsella,andJ.Rickel,“Virtualhumansinthemissionrehearsalexercisesystem,”KunstlichIntelligenz(KI),vol.17,no.4,pp.5–,2003.[Online].Available: http://ict.usc.edu/marsella/publications/KI-Journal-2003-Hill.pdf [63] A.a.Rizzo,T.Bowerly,J.G.Buckwalter,D.Klimchuk,R.Mitura,andT.D.Parsons,“AVirtualRealityScenarioforAllSeasons:TheVirtualClassroom,”CNSSpectrums,vol.11,no.01,pp.35,2006.[Online].Available: http://www.journals.cambridge.org/abstract S1092852900024196 [64] G.Frank,C.Guinn,R.Hubal,N.Carolina,P.Pope,M.Stanford,andD.Lamm-weisel,“JUST-TALK:AnApplicationofResponsiveVirtualHumanTechnology,”ProceedingsofInterservice/IndustryTraining,SimulationandEducationConference,2002. [65] P.Kenny,T.D.Parsons,J.Gratch,A.Leuski,andA.a.Rizzo,“VirtualPatientsforClinicalTherapistSkillsTraining,”IVA07Proceedingsofthe7thinternationalconferenceonIntelligentVirtualAgents,vol.4722,pp.197,2007.[Online].Available: http://portal.acm.org/citation.cfm?id=1421879.1421904 [66] A.Wendling,S.Halan,P.Tighe,L.Le,T.Euliano,andB.Lok,“VirtualHumansversusStandardizedPatients:Whichleadresidentstomorecorrectdiagnoses?”AcademicMedicine,vol.86,no.3,pp.384,2011.[Online].Available: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3072236/ [67] J.Gratch,G.Lucas,A.King,andL.-P.Morency,“ItsOnlyaComputer:TheImpactofHuman-agentInteractioninClinicalInterviews,”Proceedingsofthe2014internationalconferenceonAutonomousagentsandmulti-agentsystems,pp.85,2014. [68] A.Kotranza,K.Johnsen,J.Cendan,B.Miller,D.S.Lind,andB.Lok,“Virtualmulti-Toolsforhandandtool-basedinteractionwithlife-sizevirtualhumanagents,”3DUI-IEEESymposiumon3DUserInterfaces2009-Proceedings,pp.23,2009.[Online].Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4811201 [69] A.KotranzaandB.Lok,“Virtualhuman+tangibleinterface=Mixedrealityhumananinitialexplorationwithavirtualbreastexampatient,”Proceedings-IEEEVirtualReality,pp.99,2008. 168

PAGE 169

[70] R.C.HubalandR.S.Day,“Informedconsentprocedures:Anexperimentaltestusingavirtualcharacterinadialogsystemstrainingapplication,”JournalofBiomedicalInformatics,vol.39,no.5,pp.532,2006. [71] M.W.Link,P.P.Armsby,R.C.Hubal,andC.I.Guinn,“Accessibilityandacceptanceofresponsivevirtualhumantechnologyasasurveyinterviewertrainingtool,”ComputersinHumanBehavior,vol.22,pp.412,2006. [72] G.Huang,R.Reynolds,andC.Candler,“VirtualpatientsimulationatUSandCanadianmedicalschools,”AcademicMedicine,vol.82,no.5,pp.446,2007.[Online].Available: hGotoISIi://WOS:000246171800004 [73] S.L.Leong,C.D.Baldwin,andA.M.Adelman,“IntegratingWeb-basedcomputercasesintoarequiredclerkship:developmentandevaluation.”Academicmedicine:journaloftheAssociationofAmericanMedicalColleges,vol.78,no.3,pp.295,2003. [74] F.Consorti,R.Mancuso,M.Nocioni,andA.Piccolo,“Efcacyofvirtualpatientsinmedicaleducation:Ameta-analysisofrandomizedstudies,”ComputersandEducation,vol.59,no.3,pp.1001,2012.[Online].Available: http://dx.doi.org/10.1016/j.compedu.2012.04.017 [75] J.RickelandW.Johnson,“Steve:avirtualhumanforindividualandteamtraining,”VirtualReality,2000.[Online].Available: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.6532 [76] J.RickelandW.LewisJohnson,“Virtualhumansforteamtraininginvirtualreality,”ProceedingsoftheNinthInternationalConferenceonArticialIntelligence,no.July,pp.578,1999.[Online].Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.47.4132&rep=rep1&type=pdf [77] J.E.Laird,“ExtendingtheSoarcognitivearchitecture,”Proceedingsofthe2008conferenceonArticialGeneralIntelligence2008:ProceedingsoftheFirstAGIConference,vol.171,pp.224,2008. [78] A.Kroetz,“Theroleofintelligentagencyinsyntheticinstructorandhumanstudentdialogue,”Ph.D.dissertation,UniversityofSouthernCalifornia,1999. [79] R.Prada,“TeamingUpHumanandSyntheticCharacters,”Ph.D.dissertation,UniversidadeTecnicadeLisboaInstitutoSuperiorTecnico,2005.[Online].Available: http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Teaming+Up+Humans+and+Synthetic+Characters#1 [80] R.PradaandA.Paiva,“Teaminguphumanswithautonomoussyntheticcharacters,”ArticialIntelligence,vol.173,no.1,pp.80,Jan.2009.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S0004370208001161 169

PAGE 170

[81] J.Ball,C.Myers,A.Heiberg,N.J.Cooke,M.Matessa,M.Freiman,andS.Rodgers,“Thesyntheticteammateproject,”ComputationalandMathematicalOrganizationTheory,vol.16,no.3,pp.271,Aug.2010.[Online].Available: http://www.springerlink.com/index/10.1007/s10588-010-9065-3 [82] S.M.Rodgers,C.W.Myers,J.Ball,andM.D.Freiman,“TheSituationModelintheSyntheticTeammateProject,”20thBehaviorRepresentationinModelingandSimulation(BRIMS)Conference2001,pp.66,2011. [83] M.DemirandN.J.Cooke,“HumanTeamingChangesDrivenbyExpectationsofaSyntheticTeammate,”ProceedingsoftheHumanFactorsandErgonomicsSocietyAnnualMeeting,vol.58,no.1,2014. [84] E.Remolina,J.Li,anda.E.Johnston,“TeamTrainingwithSimulatedTeammates,”inTheInterservice/IndustryTraining,Simulation&EducationConference(I/ITSEC),vol.2005,no.-1.NTSA,2005,pp.1.[Online].Available: http://ntsa.metapress.com/index/BARN5RA0X01U7PMY.pdf [85] E.Remolina,S.Ramachandran,R.Stottler,andA.Davis,“Rehearsingnavaltacticalsituationsusingsimulatedteammatesandanautomatedtutor,”IEEETransactionsonLearningTechnologies,vol.2,no.2,pp.148,2009.[Online].Available: http://ieeexplore.ieee.org/xpls/abs all.jsp?arnumber=4967565 [86] G.M.Lucas,J.Gratch,A.King,andL.P.Morency,“It'sonlyacomputer:Virtualhumansincreasewillingnesstodisclose,”ComputersinHumanBehavior,vol.37,pp.94,Aug.2014.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S0747563214002647 [87] J.J.SheehanandM.Sosna.,Theboundariesofhumanity:Humans,animals,machines.UnivofCaliforniaPress,1991. [88] J.Blascovich,J.Loomis,A.C.Beall,K.R.Swinth,C.L.Hoyt,andJ.N.Bailenson,“ImmersiveVirtualEnvironmentTechnologyasaMethodologicalToolforSocialPsychology,”PsychologicalInquiry,vol.13,no.2,pp.103,Apr.2002.[Online].Available: http://www.tandfonline.com/doi/abs/10.1207/S15327965PLI1302 01 [89] C.I.Nass,J.Steuer,andE.R.Tauber,“Computersaresocialactors,”Computer-HumanInteraction(CHI)Conference:CelebratingInterdependence1994,pp.72,1994.[Online].Available: http://portal.acm.org/citation.cfm?doid=259963.260288 [90] R.E.Guadagno,J.Blascovich,J.N.Bailenson,andC.Mccall,“VirtualHumansandPersuasion:TheEffectsofAgencyandBehavioralRealism,”MediaPsychology,vol.10,pp.1,2007.[Online].Available: http://www.tandfonline.com/doi/abs/10.1080/15213260701300865 [91] J.Appel,A.VonDerPutten,N.C.Kramer,andJ.Gratch,“Doeshumanitymatter?Analyzingtheimportanceofsocialcuesandperceivedagencyofacomputer 170

PAGE 171

systemfortheemergenceofsocialreactionsduringhuman-computerinteraction,”AdvancesinHuman-ComputerInteraction,vol.2012,2012. [92] C.M.DeMelo,J.Gratch,andP.J.Carnevale,“Theeffectofagencyontheimpactofemotionexpressionsonpeople'sdecisionmaking,”Proceedings-2013HumaineAssociationConferenceonAffectiveComputingandIntelligentInteraction,ACII2013,pp.546,Sep.2013.[Online].Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6681487 [93] A.Wiebel,C.Garth,andG.Scheuermann,“Computationoflocalizedowforsteadyandunsteadyvectoreldsanditsapplications,”IEEETransactionsonVisualizationandComputerGraphics,vol.13,no.3,pp.641,2007.[Online].Available: http://www.ncbi.nlm.nih.gov/pubmed/17356212 [94] C.M.D.Melo,J.Gratch,andP.J.Carnevale,“HumansversusComputers:ImpactofEmotionExpressionsonPeoplesDecisionMaking,”vol.6,no.2,pp.127,2015. [95] S.Y.Okita,J.Bailenson,andD.L.Schwartz,“TheMereBeliefofSocialInteractionImprovesLearning,”ProceedingsoftheTwenty-ninthMeetingoftheCognitiveScienceSociety,pp.1,2007. [96] S.LimandB.Reeves,“Computeragentsversusavatars:Responsestointeractivegamecharacterscontrolledbyacomputerorotherplayer,”InternationalJournalofHuman-ComputerStudies,vol.68,no.1-2,pp.57,Jan.2010.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S107158190900130X [97] S.Dow,M.Mehta,A.Lausier,B.MacIntyre,andM.Mateas,“InitiallessonsfromARFacade,aninteractiveaugmentedrealitydrama,”ProceedingsoftheACMSIGCHIinternationalconferenceonAdvancesincomputerentertainmenttechnology,vol.1,p.28,2006.[Online].Available: http://portal.acm.org/citation.cfm?doid=1178823.1178858 [98] J.Pair,U.Neumann,D.Piepol,andB.Swartout,“FlatWorld:Combininghollywoodset-designtechniqueswithVR,”IEEEComputerGraphicsandApplications,vol.23,no.February,pp.12,2003. [99] A.Hartholt,J.Gratch,andL.Weiss,“Atthevirtualfrontier:IntroducingGunslinger,amulti-character,mixed-reality,story-drivenexperience,”IntelligentVirtualAgents,pp.500,2009.[Online].Available: http://link.springer.com/chapter/10.1007/978-3-642-04380-2 62 [100] B.L.JoonHaoChuah,AndrewRobb,CaseyWhite,AdamWendling,SamsunLampotang,RegisKopper,“ExploringAgentPhysicalityandSocialPresenceforMedicalTeamTraining,”Presence:TeleoperatorsandVirtualEnvironments,vol.22,no.2,pp.141,2013.[Online].Available: http://www.mitpressjournals.org/doi/abs/10.1162/PRES a 00145#.VUEF6yFVhBe 171

PAGE 172

[101] E.Salas,“Istherea”BigFive”inTeamwork?”SmallGroupResearch,vol.36,no.5,pp.555,Oct.2005.[Online].Available: http://sgr.sagepub.com/cgi/doi/10.1177/1046496405277134 [102] D.P.Baker,S.Gustafson,J.M.Beaubien,E.Salas,andP.Barach,“Medicalteamtrainingprogramsinhealthcare,”ROCKVILLEMD,Tech.Rep.,2005.[Online].Available: http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrex=html&identier=ADA435072 [103] C.J.Wallin,L.Meurling,L.Hedman,J.Hedegard,andL.Fellander-Tsai,“Target-focusedmedicalemergencyteamtrainingusingahumanpatientsimulator:Effectsonbehaviourandattitude,”MedicalEducation,vol.41,no.2,pp.173,2007. [104] M.a.DeVita,J.Schaefer,J.Lutz,H.Wang,andT.Dongilli,“Improvingmedicalemergencyteam(MET)performanceusinganovelcurriculumandacomputerizedhumanpatientsimulator.”Quality&safetyinhealthcare,vol.14,no.5,pp.326,2005. [105] P.Youngblood,P.M.Harter,S.Srivastava,S.Moffett,W.L.Heinrichs,andP.Dev,“Design,development,andevaluationofanonlinevirtualemergencydepartmentfortrainingtraumateams.”Simulationinhealthcare:journaloftheSocietyforSimulationinHealthcare,vol.3,no.3,pp.146,Jan.2008.[Online].Available: http://www.ncbi.nlm.nih.gov/pubmed/19088658 [106] S.K.Howard,D.M.Gaba,K.J.Fish,G.Yang,andF.H.Sarnquist,“Anesthesiacrisisresourcemanagementtraining:teachinganesthesiologiststohandlecriticalincidents.”pp.763,Sep.1992.[Online].Available: http://www.ncbi.nlm.nih.gov/pubmed/1524531 [107] D.M.Gaba,S.K.Howard,K.J.Fish,B.E.Smith,andY.a.Sowb,“Simulation-BasedTraininginAnesthesiaCrisisResourceManagement(ACRM):ADecadeofExperience,”Simulation&Gaming,vol.32,no.2,pp.175,2001. [108] H.B.King,J.B.Battles,D.P.Baker,a.Alonso,E.Salas,J.Webster,L.Toomey,M.Salisbury,K.Henriksen,M.a.Keyes,andM.L.Grady,“TeamSTEPPS:TeamStrategiesandToolstoEnhancePerformanceandPatientSafetyAdvancesinPatientSafety:NewDirectionsandAlternativeApproaches(Vol.3:PerformanceandTools),”TeamSTEPPS():TeamStrategiesandToolstoEnhancePerformanceandPatientSafety,pp.5,2008. [109] M.Reznek,R.Smith-Coggins,S.Howard,K.Kiran,P.Harter,Y.Sowb,D.Gaba,andT.Krummel,“EmergencyMedicineCrisisResourceManagement(EMCRM):Pilotstudyofasimulation-basedcrisismanagementcourseforemergencymedicine,”AcademicEmergencyMedicine,vol.10,no.4,pp.386,2003. 172

PAGE 173

[110] J.Sexton,S.Marsch,R.Helmreich,D.Betzendoefer,T.Kocher,andD.Scheidegger,“Participantevaluationofteamorientedmedicalsimulation,”Simulatorsinanesthesiologyeducation,pp.109,1998. [111] J.C.Morey,R.Simon,G.D.Jay,R.L.Wears,M.Salisbury,K.a.Dukes,andS.D.Berns,“Errorreductionandperformanceimprovementintheemergencydepartmentthroughformalteamworktraining:Evaluationresultsofthemedteamsproject,”HealthServicesResearch,vol.37,no.6,pp.1553,2002. [112] D.T.Risser,M.M.Rice,M.L.Salisbury,R.Simon,G.D.Jay,andS.D.Berns,“Thepotentialforimprovedteamworktoreducemedicalerrorsintheemergencydepartment,”AnnalsofEmergencyMedicine,vol.34,no.3,pp.373,1999. [113] B.Rossen,S.Lind,andB.Lok,“Human-centereddistributedconversationalmodeling:Efcientmodelingofrobustvirtualhumanconversations,”IntelligentVirtualAgents,pp.474,2009.[Online].Available: http://www.springerlink.com/index/10u6284004266247.pdf [114] B.Rossen,Designandevaluationofconversationalmodelingmethodsforinterpersonalsimulation,2011.[Online].Available: http://dl.acm.org/citation.cfm?id=2520229 [115] J.L.McClendon,N.A.Mack,andL.F.Hodges,“TheUseofParaphraseIdenticationintheRetrievalofAppropriateResponsesforScriptBasedConversationalAgents,”ProceedingsoftheTwenty-SeventhInternationalFloridaArticialIntelligenceResearchSocietyConference,pp.196,2014. [116] L.Castillo-Page,“DiversityinthePhysicianWorkforce:Facts&Figures2010,”AssociationofAmericanMedicalColleges,2010.[Online].Available: https://members.aamc.org/eweb/upload/DiversityinthePhysicianWorkforceFactsandFigures2010.pdf [117] L.P.Vardoulakis,L.Ring,B.Barry,C.L.Sidner,andT.Bickmore,“Designingrelationalagentsaslongtermsocialcompanionsforolderadults,”IntelligentVirtualAgents,2012.[Online].Available: http://link.springer.com/chapter/10.1007/978-3-642-33197-8 30 [118] P.Stillman,D.Swanson,M.B.Regan,M.M.Philbin,V.Nelson,T.Ebert,B.Ley,T.Parrino,J.Shorey,A.Stillman,E.Alpert,J.Caslowitz,D.Clive,J.Florek,M.Hamolsky,C.Hatem,J.Kizirian,R.Kopelman,D.Levenson,G.Levinson,J.McCue,H.Pohl,F.Schiffman,J.Schwartz,M.Thane,andM.Wolf,“Assessmentofclinicalskillsofresidentsutilizingstandardizedpatients:Afollow-upstudyandrecommendationsforapplication,”pp.393,1991. [119] V.Benet-MartnezandO.P.John,“LosCincoGrandesacrossculturesandethnicgroups:multitraitmultimethodanalysesoftheBigFiveinSpanishandEnglish.”Journalofpersonalityandsocialpsychology,vol.75,no.3,pp.729,Sep.1998.[Online].Available: http://www.ncbi.nlm.nih.gov/pubmed/9781409 173

PAGE 174

[120] O.John,L.Naumann,andC.Soto,“ParadigmshifttotheintegrativeBigFivetraittaxonomy,”HandbookofPersonality:TheoryandResearch,2008.[Online].Available: http://books.google.com/books?hl=en&lr=&id=8r4KlgerjcUC&oi=fnd&pg=PA114&dq=Paradigm+Shift+to+the+Integrative+Big+Five+Trait+Taxonomy&ots=lbl3vM8G27&sig=2iHp1XaFCSjqF7nkQgfAjqWx0uM [121] J.G.Hollandsworth,J.P.Galassi,andM.L.Gay,“Theadultselfexpressionscale:validationbythemultitrait-multimethodprocedure.”Journalofclinicalpsychology,vol.33,no.2,pp.407,1977. [122] W.W.Chin,W.D.Salisbury,a.W.Pearson,andM.J.Stollak,“PerceivedCohesioninSmallGroups:AdaptingandTestingthePerceivedCohesionScaleinaSmall-GroupSetting,”SmallGroupResearch,vol.30,no.6,pp.751,1999. [123] C.-C.HoandK.F.MacDorman,“Revisitingtheuncannyvalleytheory:DevelopingandvalidatinganalternativetotheGodspeedindices,”ComputersinHumanBehavior,vol.26,no.6,pp.1508,Nov.2010.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S0747563210001536 [124] L.R.Goldberg,J.a.Johnson,H.W.Eber,R.Hogan,M.C.Ashton,C.R.Cloninger,andH.G.Gough,“Theinternationalpersonalityitempoolandthefutureofpublic-domainpersonalitymeasures,”JournalofResearchinPersonality,vol.40,no.1,pp.84,2006. [125] A.Robb,C.White,A.Cordar,A.Wendling,S.Lampotang,andB.Lok,“AQualitativeEvaluationofBehaviorDuringConictwithanAuthoritativeVirtualHuman,”inIntelligentVirtualAgents,2014,pp.1. [126] A.Robb,A.Cordar,S.Lampotang,C.White,A.Wendling,andB.Lok,“TeamingUpWithVirtualHumans:HowOtherPeopleChangeOurPerceptionsofandBehaviorwithVirtualTeammates,”VisualizationandComputerGraphics,IEEETransactionson,vol.21,no.4,pp.511–519,2015. [127] A.Robb,C.White,A.Cordar,A.Wendling,S.Lampotang,andB.Lok,“Acomparisonofspeakingupbehaviorduringconictwithrealandvirtualhumans,”ComputersinHumanBehavior,vol.52,pp.12,2015.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S0747563215004240 [128] R.L.Gorden,BasicInterviewingSkills.Itasca,IL:F.E.Peacock,1998.[Online].Available: http://books.google.it/books?id=LfoVAAAAQBAJ [129] D.Kipnis,S.M.Schmidt,andI.Wilkinson,“Intraorganizationalinuencetactics:Explorationsingettingone'sway.”JournalofAppliedPsychology,vol.65,no.4,pp.440,1980.[Online].Available: http://content.apa.org/journals/apl/65/4/440 [130] E.J.Thomas,J.B.Sexton,andR.L.Helmreich,“Discrepantattitudesaboutteamworkamongcriticalcarenursesandphysicians.”Criticalcare 174

PAGE 175

medicine,vol.31,no.3,pp.956,Mar.2003.[Online].Available: http://www.ncbi.nlm.nih.gov/pubmed/12627011 [131] M.a.Makary,J.B.Sexton,J.a.Freischlag,C.G.Holzmueller,E.A.Millman,L.Rowen,andP.J.Pronovost,“OperatingRoomTeamworkamongPhysiciansandNurses:TeamworkintheEyeoftheBeholder,”JournaloftheAmericanCollegeofSurgeons,vol.202,no.5,pp.746,May2006.[Online].Available: http://www.ncbi.nlm.nih.gov/pubmed/16648014 [132] L.VanDyne,S.Ang,I.C.Botero,andL.V.Dyne,“Conceptualizingemployeesilenceandemployeevoiceasmultidimensionalconstructs,”JournalofManagementStudies,vol.40,no.September,pp.1359,2003.[Online].Available: http://onlinelibrary.wiley.com/doi/10.1111/1467-6486.00384/full [133] G.YuklandC.M.Falbe,“Inuencetacticsandobjectivesinupward,downward,andlateralinuenceattempts.”JournalofAppliedPsychology,vol.75,no.2,pp.132,1990. [134] T.L.ChartrandandJ.a.Bargh,“Thechameleoneffect:theperception-behaviorlinkandsocialinteraction.”Journalofpersonalityandsocialpsychology,vol.76,no.6,pp.893,1999. [135] J.F.Dovidio,K.Kawakami,C.Johnson,B.Johnson,andA.Howard,“OntheNatureofPrejudice:AutomaticandControlledProcesses,”JournalofExperimentalSocialPsychology,vol.33,no.5,pp.510,1997.[Online].Available: http://www.sciencedirect.com/science/article/pii/S0022103197913317 [136] F.J.Bernieri,“Coordinatedmovementandrapportinteacher-studentinteractions,”JournalofNonverbalBehavior,vol.12,no.2,pp.120,1988. [137] R.Buck,“Nonverbalbehaviorandthetheoryofemotion:thefacialfeedbackhypothesis.”Journalofpersonalityandsocialpsychology,vol.38,no.5,pp.811,1980. [138] J.K.BurgoonandD.Buller,“InterpersonalDeception:III.EffectsofDeceitonPerceivedCommunicationand,”JournalofNonverbalBehavior,vol.18,no.2,1994. [139] Y.KimandS.S.Sundar,“Anthropomorphismofcomputers:Isitmindfulormindless?”ComputersinHumanBehavior,vol.28,no.1,pp.241,Jan.2012.[Online].Available: http://linkinghub.elsevier.com/retrieve/pii/S0747563211001993 [140] J.Gratch,R.Artstein,G.Lucas,G.Stratou,S.Scherere,A.Nazarian,R.Wood,J.Boberg,D.DeVault,S.Marsella,D.Traum,S.Rizzo,andL.-P.Morency,“TheDistressAnalysisInterviewCorpusofhumanandcomputerinterviews,”ProceedingsofLanguageResourcesandEvaluationConference(LREC),2014.[Online].Available: http://www.lrec-conf.org/proceedings/lrec2014/pdf/508 Paper.pdf 175

PAGE 176

[141] M.Kipp,“Anvil-AGenericAnnotationToolforMultimodalDialogue,”Proceedingsofthe7thEuropeanConferenceonSpeechCommunicationandTechnology(Eurospeech),pp.1367,2001. [142] P.BoersmaandV.vanHeuven,“Praat,asystemfordoingphoneticsbycomputer,”GlotInternational,vol.5,no.9/10,pp.341,2001. [143] R.Vertegaal,R.Slagter,G.VanDerVeer,andA.Nijholt,“EyeGazePatternsinConversations:ThereisMoretoConversationalAgentsthanMeetstheEyes,”Analysis,pp.301,2001.[Online].Available: http://portal.acm.org/citation.cfm?doid=365024.365119 [144] A.Bloomer,A.,Grifths,P.,Merrison,A.J.,&Merrison,IntroducingLanguageinUse:ACoursebook,2005. [145] S.J.Cowley,“OfTiming,Turn-Taking,andConversations,”JournalofPsycholinguisticResearch,vol.27,no.5,pp.541,1998.[Online].Available: http://dx.doi.org/10.1023/a:1024948912805 [146] K.Jokinen,M.Nishida,andS.Yamamoto,“Oneye-gazeandturn-taking,”Proceedingsofthe2010workshoponEyegazeinintelligenthumanmachineinteractionEGIHMI10,pp.118,2010.[Online].Available: http://dl.acm.org/citation.cfm?id=2002333.2002352 [147] L.J.CronbachandP.E.Meehl,“Constructvalidityinpsychologicaltests.”pp.281,1955. [148] M.Usoh,E.Catena,S.Arman,andM.Slater,“Usingpresencequestionnairesinreality,”Presence,pp.1,2000.[Online].Available: http://discovery.ucl.ac.uk/106368/ [149] I.K.CrombieandH.T.Davies.,“Whatismeta-analysis?”Whatis,pp.1,2009.[Online].Available: http://bmj-ebn.highwire.org/content/16/1/3.extract [150] G.Cumming,Understandingthenewstatistics:Effectsizes,condenceintervals,andmeta-analysis.Routledge,2012. [151] W.Viechtbauer,“Conductingmeta-analysesinfRgwiththefmetaforgpackage,”JournalofStatisticalSoftware,vol.36,no.3,pp.1,2010.[Online].Available: http://www.jstatsoft.org/v36/i03/ [152] J.SeyamaandR.S.Nagayama,“ProbingtheUncannyValleywiththeEyeSizeAftereffect,”Presence:TeleoperatorsandVirtualEnvironments,vol.18,no.5,pp.321,Oct.2009.[Online].Available: http://www.mitpressjournals.org/doi/abs/10.1162/pres.18.5.321 [153] A.Bandura,“Sociallearningtheory,”pp.305,1977. 176

PAGE 177

[154] S.V.Babu,T.Y.Grechkin,B.Chihak,C.Ziemer,J.K.Kearney,J.F.Cremer,andJ.M.Plumert,“Animmersivevirtualpeerforstudyingsocialinuencesonchildcyclists'road-crossingbehavior,”IEEETransactionsonVisualizationandComputerGraphics,vol.17,no.1,pp.14,2011. [155] J.S.Fiorelli,“PowerinWorkGroups:TeamMember'sPerspectives,”HumanRelations,vol.41,no.1,pp.1,1988. 177

PAGE 178

BIOGRAPHICALSKETCHAndrewRobbwasbornin1986inHuntsville,Alabama.HegrewupinOrlando,FloridawherehegraduatedfromWinterParkHighSchoolwithhonors.HeattendedtheUniversityofFlorida,wherehereceivedafullscholarshipthroughtheBrightFuturesprogram.Hegraduatedwithhonorsin2009withaBachelorofScienceinComputerEngineering.HeremainedattheUniversityofFloridatopursuehisPh.D.incomputerengineeringunderthesupervisionofDr.BenjaminLok,andreceivedhisPh.D.inthefallof2015.AndrewwasawardedtheUFAlumniFellowshiptosupporthisresearch,whichfocusedontheuseofvirtualhumansforteamtraining.Hisworkreceivedsignicantinternationalrecognitionbothwithintheeldsofcomputerscienceandmedicinewith9publicationsinjournalsandconferences,includingjournalarticlesinTransactionsonVisualizationandComputerGraphicsandComputersinHumanBehavior. 178