Identifying and Exploring Factors Affecting Embodied Conversational Agent Social Presence for Interpersonal Skills Training

MISSING IMAGE

Material Information

Title:
Identifying and Exploring Factors Affecting Embodied Conversational Agent Social Presence for Interpersonal Skills Training
Physical Description:
1 online resource (153 p.)
Language:
english
Creator:
Chuah, Joonhao
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Doctorate ( Ph.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Computer Engineering, Computer and Information Science and Engineering
Committee Chair:
Lok, Benjamin
Committee Members:
Peters, Jorg
Lampotang, Samsun
Entezari, Alireza
Black, Erik Wade

Subjects

Subjects / Keywords:
eca -- mr -- presence -- virtual -- vr
Computer and Information Science and Engineering -- Dissertations, Academic -- UF
Genre:
Computer Engineering thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
Embodied conversational agents (ECAs) have been used as virtual conversational partners in interpersonal skills training applications such as medical interviews, military decision making, and cultural training. Ideally, in interpersonal skills training users will perceive and treat the ECAs the same as they would real people. The perception and treatment of ECAs as real people is known as social presence. This dissertation identifies and explores approaches to increase an ECA’s social presence. One approach is to use mixed reality technology to bring the ECA into the physical space and increase its physicality. Three studies were run on physicality. The first two suggested that increasing an ECA’s occupancy of the physical space can elicit more realistic treatment of the ECA. These studies also suggested there may be interaction effects between physicality and plausibility, and also matching levels of physicality are important. The third study suggested increasing an ECA’s interaction with the physical space does not have an effect on social presence. A qualitative study was also run to identify other factors affecting social presence. This study suggested factors related to an ECA’s plausibility, including what the ECA says and how it responds, also affect social presence. This study also emphasized the importance of a user's expectations and how those can affect social presence.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Joonhao Chuah.
Thesis:
Thesis (Ph.D.)--University of Florida, 2013.
Local:
Adviser: Lok, Benjamin.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2014-02-28

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2013
System ID:
UFE0045759:00001


This item is only available as the following downloads:


Full Text

PAGE 1

IDENTIFYINGANDEXPLORINGFACTORSAFFECTINGEMBODIEDCONVERSATIONALAGENTSOCIALPRESENCEFORINTERPERSONALSKILLSTRAININGByJOONHAOCHUAHADISSERTATIONPRESENTEDTOTHEGRADUATESCHOOLOFTHEUNIVERSITYOFFLORIDAINPARTIALFULFILLMENTOFTHEREQUIREMENTSFORTHEDEGREEOFDOCTOROFPHILOSOPHYUNIVERSITYOFFLORIDA2013

PAGE 2

c2013JoonHaoChuah 2

PAGE 3

TomyparentsfortheirneverendingsupportforeverythingIdo 3

PAGE 4

ACKNOWLEDGMENTS Ithankmyresearchadvisorandsupervisorycommitteechair,Dr.BenjaminLok.Hisadvice,support,andcountlessone-on-onemeetingshaveshapedmeintotheresearcherthatIamtoday.Hehasalsotaughtmehowtoeffectivelycommunicateandcollaboratewithpeopleinawidevarietyofelds,askillIamsurewillcarrymefarinmycareerandlifeingeneral.Ialsothankmysupervisorycommitteemembers,Dr.ErikBlack,Dr.AlirezaEntezari,Dr.SamsunLampotang,andDr.JorgPetersfortheirfeedbackandsupportofmyresearch.Ithankmycollaboratorsinhealthcareeducationfortheirsupport.Dr.CaseyWhiteprovidedexcellentadviceonstudydesignandmeetingeducationalgoals.Dr.AdamWendling,Dr.MeredithMowitz,Dr.LindsayThompson,andDr.MariaKellypatientlysuppliedsubjectmatterexpertiseandfeedbacktoensuremyscenariosweremedicallyaccurate.ThankyoutothemembersoftheVirtualExperiencesResearchGroup.TheworkofVERGalumniDr.KyleJohnsen,Dr.AaronKotranza,Dr.AndrewRaij,andDr.BrentRossenlaidthefoundationformyresearch.Theadviceofpost-docsDr.RegisKopperandDr.AndreaKleinsmithhelpedmenavigatenewterritoryasIprogressedtowardsearningmyPh.D.Thankyoutomyfellowstudents,AndrewRobb,ShivashankarHalan,DiegoRivera-Gutierrez,MalloryMcManamon,MichaelBorish,AndrewCordar,VaishnaviKrishnan,DoaaElSheikh,andGuangyanHu.Inadditiontocontributingtothesoftwareweallshare,theyhaveservedassoundingboards,pilottesters,studyproctors,editors,andfriends.Lastly,Ithankmyfamily.IthankmyparentsHoeandSoonforencouragingmetopursuemyPh.D.andsupportingmeineverythingIhaveeverattempted.IthankmybrotherJoonYeeforintroducingmetoprogrammingatanearlyage. 4

PAGE 5

TABLEOFCONTENTS page ACKNOWLEDGMENTS .................................. 4 LISTOFTABLES ...................................... 10 LISTOFFIGURES ..................................... 11 ABSTRACT ......................................... 12 CHAPTER 1INTRODUCTION ................................... 13 1.1Motivation .................................... 15 1.1.1MedicalTeamTraining ......................... 15 1.1.2InSituTraining ............................. 15 1.1.3Vision .................................. 16 1.2ThesisStatement ................................ 16 1.3OverviewofApproach ............................. 17 1.3.1DimensionsofPhysicality ....................... 17 1.3.2OccupancyofthePhysicalSpace ................... 17 1.3.3InteractionwiththeEnvironment ................... 18 1.3.4IdentifyingFactorsBeyondPhysicality ................ 18 1.4Innovations ................................... 18 2REVIEWOFLITERATURE ............................. 20 2.1VirtualHumans ................................. 20 2.1.1EmbodiedConversationalAgents ................... 20 2.1.2AgentvsAvatar ............................. 21 2.1.3InterpersonalSkillsTraining ...................... 22 2.2Immersion,Presence,andPlausibilityIllusion ................ 23 2.2.1Immersion ................................ 23 2.2.2Presence ................................ 24 2.2.2.1MeasuringPresence ..................... 25 2.2.3PlausibilityIllusion ........................... 27 2.3SocialPresence ................................ 27 2.3.1DenitionsofSocialPresence ..................... 27 2.3.2MeasuringSocialPresence ...................... 29 2.3.2.1Perception .......................... 29 2.3.2.2SocialResponses ...................... 29 2.3.2.3CombiningPerceptionandSocialResponses ....... 30 2.3.3FactorsAffectingSocialPresence ................... 30 2.3.3.1Agency ............................ 30 2.3.3.2VisualRealism ........................ 32 5

PAGE 6

2.3.3.3Behavior ........................... 33 2.3.3.4UserPersonality ....................... 33 2.4QualitativeResearch .............................. 34 2.4.1WhyQualitativeResearch ....................... 34 2.4.2OverviewofApproaches ........................ 34 2.4.2.1NarrativeResearch ..................... 35 2.4.2.2Ethnography ......................... 35 2.4.2.3CaseStudy .......................... 35 2.4.2.4Phenomenology ....................... 36 2.4.2.5GroundedTheory ...................... 36 2.4.3PhenomenologyandGroundedTheoryProcesses ......... 37 2.4.3.1PhenomenologyAnalysisandRepresentation ...... 37 2.4.3.2GroundedTheoryAnalysis ................. 38 2.4.4QualitativeVirtualHumanResearch ................. 38 3DIMENSIONSOFPHYSICALITY .......................... 39 3.1Overview .................................... 39 3.2OccupancyofthePhysicalSpace ...................... 40 3.2.1SizeFidelity ............................... 40 3.2.2PositionFidelity ............................. 41 3.2.3FormFidelity .............................. 41 3.2.4ConcordancewiththePhysicalSpace ................ 42 3.2.5RangeofValidViewpoints ....................... 43 3.3InteractionwiththeEnvironment ....................... 44 3.3.1AwarenessofChanges ......................... 45 3.3.2AbilitytoManipulate .......................... 46 3.4IntertwiningofTechnologyandDimensionsofPhysicality ......... 47 4OCCUPANCYOFTHEPHYSICALSPACE .................... 50 4.1Overview .................................... 50 4.2DevelopinganECAwithHighOccupancy .................. 51 4.2.1IncreasingOccupancyofthePhysicalSpace ............ 51 4.2.2ComparisontoOtherSystems .................... 52 4.3HolisticEvaluation:ExploringOccupancyasaWhole ........... 53 4.3.1Conditions ................................ 54 4.3.1.1HighPhysicalityNurseandDaughter ........... 54 4.3.1.2LowPhysicalityNurseandDaughter ............ 54 4.3.2WizardofOz .............................. 56 4.3.3Participants ............................... 57 4.3.4Procedure ................................ 58 4.3.4.1Tutorial ............................ 59 4.3.4.2CriticalIncident ........................ 59 4.3.5Metrics .................................. 61 4.3.5.1Self-ReportSocialPresence ................ 61 6

PAGE 7

4.3.5.2RealisticBehavior ...................... 61 4.3.5.3PlausibilityIllusion ...................... 62 4.3.6Results ................................. 62 4.3.6.1Self-ReportSocialPresence ................ 63 4.3.6.2AddressingECAs ...................... 63 4.3.6.3PlausibilityIllusion ...................... 64 4.3.6.4InteractionofPhysicalityandPlausibility .......... 65 4.3.7Discussion ................................ 66 4.3.7.1Self-ReportSocialPresence ................ 66 4.3.7.2AddressingECAs ...................... 66 4.3.7.3PlausibilityIllusion ...................... 67 4.3.7.4InteractionofPhysicalityandPlausibility .......... 67 4.3.8Limitations ................................ 68 4.4IndependentComponentsEvaluation:ExploringIndividualDimensionsofOccupancy .................................. 68 4.4.1Participants ............................... 69 4.4.2Procedure ................................ 70 4.4.2.1Handoff ............................ 70 4.4.2.2CriticalIncident ........................ 71 4.4.3Metrics .................................. 71 4.4.4Results ................................. 72 4.4.4.1Outliers ............................ 72 4.4.4.2Plausibility .......................... 73 4.4.4.3FormFidelityandConcordance .............. 73 4.4.4.4Concordance ......................... 74 4.4.4.5FormFidelity ......................... 75 4.4.5Discussion ................................ 75 4.4.5.1FormFidelityandConcordance .............. 75 4.4.5.2Concordance ......................... 76 4.4.5.3FormFidelity ......................... 77 4.4.5.4Plausibility .......................... 77 4.4.6Limitations ................................ 78 4.5OverallDiscussion ............................... 78 5INTERACTIONWITHTHEENVIRONMENT ................... 89 5.1Overview .................................... 89 5.2MotivationandBackground-RealPediatricDevelopmentalExams .... 91 5.2.1ImportanceofDevelopmentalScreening ............... 92 5.2.2CurrentTrainingMethods ....................... 92 5.2.3SkillsTestsforFour-Year-Old-Children ................ 93 5.3MixedRealityPediatricDevelopmentalExam ................ 94 5.3.1MixedRealityVirtualHumans ..................... 95 5.3.1.1GraphicsandAudio ..................... 95 5.3.1.2OccupancyofthePhysicalSpace ............. 95 7

PAGE 8

5.3.1.3WizardofOz ......................... 96 5.3.2InteractionPhysicality ......................... 97 5.3.2.1HighPhysicalityInterface .................. 98 5.3.2.2LowPhysicalityInterface .................. 99 5.4UserEvaluation ................................. 100 5.4.1Participants ............................... 100 5.4.2Procedure ................................ 101 5.4.2.1Tutorials ............................ 101 5.4.2.2DevelopmentalExams .................... 102 5.4.3Metrics .................................. 103 5.4.3.1Self-ReportSocialPresence ................ 104 5.4.3.2ObservedBehavior ..................... 104 5.4.3.3EducationalBenet ..................... 105 5.5ResultsandDiscussion ............................ 106 5.5.1Self-ReportSocialPresence ...................... 106 5.5.1.1Self-ReportSocialPresenceResults ........... 106 5.5.1.2Self-ReportSocialPresenceDiscussion .......... 107 5.5.2ObservedBehavior ........................... 107 5.5.2.1ObservedBehaviorResults ................. 107 5.5.2.2ObservedBehaviorDiscussion ............... 108 5.5.3FreeResponseFeedback ....................... 109 5.5.3.1FreeResponseFeedbackResults ............. 109 5.5.3.2FreeResponseFeedbackDiscussion ........... 109 5.5.4EducationalBenet ........................... 109 5.5.4.1EducationalBenetResults ................. 109 5.5.4.2EducationalBenetDiscussion ............... 110 5.5.5OverallDiscussion ........................... 111 5.6BroaderImpact-SimulatingVulnerablePopulations ............ 112 6IDENTIFYINGFACTORSBEYONDPHYSICALITY ................ 118 6.1Overview .................................... 118 6.2SystemDescription .............................. 120 6.3StudyDesign .................................. 121 6.3.1Participants ............................... 121 6.3.2StudyProcedure ............................ 121 6.3.3AnalysisProcedure ........................... 124 6.3.4MyPersonalExperiences ....................... 125 6.4ResultsandDiscussion ............................ 126 6.4.1SocialPresence ............................ 126 6.4.2Conversation .............................. 128 6.4.2.1Content ............................ 128 6.4.2.2GettingResponsesBack .................. 129 6.4.2.3SpeedofResponses .................... 131 6.4.3Movements ............................... 131 8

PAGE 9

6.4.3.1Eye/HeadMovement .................... 131 6.4.3.2FacialExpressions ...................... 132 6.4.4Behavior ................................. 132 6.4.5Voice ................................... 133 6.4.6VisualRealism ............................. 133 6.4.7Physicality ................................ 134 6.4.8OverallDiscussion ........................... 134 6.5RelationtoOurOtherStudies ......................... 135 6.5.1HolisticEvaluationofOccupancy ................... 135 6.5.2IndependentComponentsEvaluationofOccupancy ........ 136 6.5.3EvaluationofInteraction ........................ 137 7CONCLUSION .................................... 140 7.1ReviewofResults ............................... 140 7.2Limitations ................................... 141 7.3FutureDirections ................................ 142 7.4BroaderImpact ................................. 144 REFERENCES ....................................... 145 BIOGRAPHICALSKETCH ................................ 153 9

PAGE 10

LISTOFTABLES Table page 3-1Examplesofhowtechnologiesmayaffectmorethanonedimension ...... 49 4-1Physicalitylevelsbydimensionforthenurseanddaughter ............ 80 4-2WordsusedbytheparticipantstoaddresstheECAsandthenumberofparticipantsthatusedthemineachcondition .......................... 82 6-1Numberofparticipantsthatmadestatementsrelatedtoeachtheme. ...... 139 10

PAGE 11

LISTOFFIGURES Figure page 3-1AtaxonomyforthedimensionsofECAphysicality. ................ 48 3-2Positiondelitycomparison ............................. 48 4-1PairofANDIsystems ................................ 80 4-2ANDIrotating ..................................... 81 4-3Lowphysicalitycondition .............................. 81 4-4Approximatetimespentineachstageoftheholisticevaluation ......... 81 4-5Socialpresenceforthelowandhighphysicalitynurse .............. 82 4-6Socialpresenceforthelowandhighphysicalitydaughter ............ 83 4-7InteractioneffectsofconditionandECA ...................... 84 4-8Theindependentcomponentsevaluationconditions ............... 85 4-9Approximatetimespentineachstageoftheindependentcomponentsevaluation 86 4-10Self-reportsocialpresenceresultsforNotRealandComputerizedfortheindependentcomponentsevaluation ............................... 87 4-11Self-reportsocialpresenceresultsforInRoom,Watching,andSentientfortheindependentcomponentsevaluation ........................ 88 5-1Highinteractionphysicalitychild .......................... 113 5-2Lowinteractionphysicalitychild ........................... 114 5-3VirtualchildandMRpaper ............................. 114 5-4Paperapplication ................................... 115 5-5Approximatetimespentineachstageoftheuserevaluation ........... 115 5-6Socialpresenceforthelowandhighphysicalitychild ............... 116 5-7Socialpresencefortheparentinthelowandhighphysicalityconditions .... 116 5-8Averagecondencelevelsfortheexperiencedandinexperiencedgroups ... 117 6-1Socialpresenceforthechildandparent ...................... 127 6-2Participantperformingadevelopmentalexamonthevirtualchild. ........ 138 11

PAGE 12

AbstractofDissertationPresentedtotheGraduateSchooloftheUniversityofFloridainPartialFulllmentoftheRequirementsfortheDegreeofDoctorofPhilosophyIDENTIFYINGANDEXPLORINGFACTORSAFFECTINGEMBODIEDCONVERSATIONALAGENTSOCIALPRESENCEFORINTERPERSONALSKILLSTRAININGByJoonHaoChuahAugust2013Chair:BenjaminLokMajor:ComputerandInformationScienceandEngineeringEmbodiedconversationalagents(ECAs)havebeenusedasvirtualconversationalpartnersininterpersonalskillstrainingapplicationssuchasmedicalinterviews,militarydecisionmaking,andculturaltraining.Ideally,ininterpersonalskillstraininguserswillperceiveandtreattheECAsthesameastheywouldrealpeople.TheperceptionandtreatmentofECAsasrealpeopleisknownassocialpresence.ThisdissertationidentiesandexploresapproachestoincreaseanECAssocialpresence.OneapproachistousemixedrealitytechnologytobringtheECAintothephysicalspaceandincreaseitsphysicality.Threestudieswererunonphysicality.ThersttwosuggestedthatincreasinganECAsoccupancyofthephysicalspacecanelicitmorerealistictreatmentoftheECA.Thesestudiesalsosuggestedtheremaybeinteractioneffectsbetweenphysicalityandplausibility,andalsomatchinglevelsofphysicalityareimportant.ThethirdstudysuggestedincreasinganECAsinteractionwiththephysicalspacedoesnothaveaneffectonsocialpresence.Aqualitativestudywasalsoruntoidentifyotherfactorsaffectingsocialpresence.ThisstudysuggestedfactorsrelatedtoanECAsplausibility,includingwhattheECAsaysandhowitresponds,alsoaffectsocialpresence.Thisstudyalsoemphasizedtheimportanceofauser'sexpectationsandhowthosecanaffectsocialpresence. 12

PAGE 13

CHAPTER1INTRODUCTIONEmbodiedconversationalagents(ECAs)havebeenusedasvirtualconversationalpartnersininterpersonalskillstrainingapplicationssuchasmedicalinterviews[ 2 45 48 69 ],militarydecisionmaking[ 40 ],andculturaltraining[ 5 39 ].Interpersonalskillstrainingallowsuserstolearnandpracticeskillssuchasshowingempathy,demonstratingleadership,andmakingdecisions.Ideally,ininterpersonalskillstraininguserswillperceiveandtreattheECAsthesameastheywouldrealpeople.TheperceptionandtreatmentofECAsasrealpeopleisknownassocialpresence.Inourwork,weidentifyandexploreapproachestoincreaseanECA'ssocialpresence.OnepossibleapproachtoincreasingsocialpresenceistoincreaseanECA'sphysicality.Thephysicalityofuserinteractionshasincreasedinrecentyearsduetotheproliferationofmixedreality(MR)and3Duserinterfacetechnologiessuchassee-throughhead-mounteddisplays(HMDs)andmotioncontrollers(e.g.,NintendoWiiremote)[ 43 ].Thesetechnologiescanalsoincreaseimmersionandpresence,drawingtheuserintothevirtualenvironment[ 60 86 ].Manyofthesetechnologiescanalsobeusedintheoppositedirection.Insteadofdrawingusersintoavirtualenvironment,MRtechnologycanbringECAsintotheuser'sphysicalspaceandallowtheECAstointeractwiththeenvironment.WeusetheconceptsofimmersionandpresenceinvirtualenvironmentstocreateananalogywithECAs.WeproposethatthemixedrealityECAanaloguesofimmersionandpresencearephysicalityandsocialpresence.InthecontextofmixedrealityECAs,wedenephysicalityasthetheobjectivelevelofdelityofanECA'soccupancyofandinteractionwiththephysicalspace.ThisdenitionofphysicalityderivesfromSlater'sdenitionofimmersionastheobjectivelevelofdelityofthesensorystimuliproducedbyatechnologicalsystem[ 77 ].RatherthanstudyingthelevelofimmersionofMR 13

PAGE 14

technologyusedfordisplayingECAs,weproposestudyingthelevelofECAphysicalityenabledbyMRtechnology.Physicality,aswithimmersion,canbeobjectivelyquantiedanddoesnotdependontheuser'ssubjectivefeelings.Physicality,againaswithimmersion,alsohasmanydimensions.Immersion'sdimensionsincludeeldofview,textureresolution,andhapticfeedbackdelity.Physicality'sdimensionsinclude,butarenotlimitedtoanECA'ssize,therangeofvalidviewpoints,andtheECA'sawarenessofchangestothephysicalspace.Theseandotherphysicalitydimensionshavebeenexploredbyseveralresearchers[ 31 44 52 54 71 ].InChapter 3 weformallydescribethedimensionsandcategorizeexistingresearchinthecontextofphysicality.Webelievethatincreasingphysicalitytendstoincreasesocialpresencesimilarlytothetendencythatincreasingimmersionleadstoincreasedpresence[ 60 ].Socialpresence,likepresence,isasubjectivefeelinganddependsontheuser.Aspresencereferstotheuser'ssenseofbeingthere,socialpresencerelatestotheuser'ssenseofanECAbeingthere.Moreformally,Blascovichdenedsocialpresenceastheextenttowhichindividualstreatembodiedagentsasiftheywereotherrealhumanbeings[ 17 ].Physicality,however,isnottheonlyfactorthataffectssocialpresenceandrealisticuserbehavior.AsSlaterproposes,realisticuserbehaviorinvirtualenvironmentsrequiresplausibilityinadditiontoimmersion[ 78 ].Plausibilityinvirtualandmixedrealityenvironmentsdependsonvirtualelementsrespondingandreactingeventhoughtheuserhasnodirectcontroloverthem.WhileSlaterprovidesexamplessuchasavirtualhumansteppingbackwhenapproachedtooclosely,Slaterdoesnotextensivelyexploretheconceptofvirtualhumanplausibility.Partofourworkistoidentifyfactorsthatcontributetoplausibilityandsocialpresence. 14

PAGE 15

1.1MotivationThemotivatingapplicationforthisdissertation'sworkismedicalteamtraining,specicallyinsitutraining.Teamtrainingdrivesthegoalofincreasingsocialpresence,andinsitutrainingmotivatesthefocusonexaminingphysicality. 1.1.1MedicalTeamTrainingTeam-trainingexercisesareextremelyimportantinhealthcare.Theinabilitytofunctionasateamcanleaddirectlytopatientcareerrorsincludingmedicationerrorsorevenpatientdeaths[ 50 70 ].Thesemaybecausedbycommunicationfailuresorpoorcoordination[ 96 ].Simulation-basedteam-trainingisusedtopreparemedicalandhealthsciencesstudentstoworkoninterdisciplinaryteams[ 91 ].However,whenstudentsbecomeworkingprofessionals,theirschedulesbecomemuchbusier.Unpredictableclinicaldemandsmakeitdifculttoscheduleallofthenecessaryteammemberstoattendthesametrainingsession[ 73 ].Missingsometeammemberscanmaketrainingfragmentedandarticial.OnesolutiontothisproblemissubstitutingECAsformissingteammembers,essentiallyallowinginterdisciplinaryteamtrainingondemand.ForECAstobeeffectivesubstitutes,theECAswouldneedtobeconsideredsociallypresentandelicitrealisticbehaviorfromtheusers.Inthisdissertation,weidentifyfactorsthataffectanECA'ssocialpresencethroughuserstudies.Theseuserstudiesalltakeplaceinamedicalcontext,andtwoofthemtakeplacespecicallyinamedicalteamtrainingcontext. 1.1.2InSituTrainingInsitutrainingischaracterizedbytakingplaceintheactualenvironmentwheretheskillswillbeused.Inthecaseofmedicaltraining,theenvironmentsincludepatientrooms,operatingrooms,orrecoverywards.Usingtheactualenvironmentsoffersanumberofcompellingadvantages.Itallowslearnerstopracticewithallthenuancesof 15

PAGE 16

anactualclinicalenvironmentandalsoreducestheneedforadditionalspacededicatedsolelytotraining.Becauseinsitutrainingtakesplaceinrealenvironments,ECA-basedinsitutrainingmustalsobringtheECAsintorealenvironments.ThismeanstheECAshavethepotentialtooccupyandinteractwiththeenvironmenttosomeextent.ThisdissertationexploreshowvariousaspectsofanECA'soccupancyofandinteractionwiththeenvironment,i.e.,it'sphysicality,affecttheECA'ssocialpresence. 1.1.3VisionThisdissertation'sworktakesseveralstepstowardsthefollowingvisionofon-demandtraining.Imagineaninsituteamtrainingexercisehasbeenscheduledinvolvingasurgeon,ananesthesiologist,andtwonurses.Thishastakenalotofplanningduetotheschedulesofboththepersonnelandthedemandfortherealoperatingroomusedfortheexercise.Atthelastminute,thesurgeonandoneofthenursesarecalledawaytoattendtorealpatients.Thesurgeonandnursebothplaymajorrolesintheexerciseanditcannotgoonwithoutthem.Insteadofcancelingtheexercise,ECAsarebroughtintoreplacethemonjustafewminutesnotice.TheseECAsareconstructedsotheycanbeeasilymovedintoarealoperatingroomandappeartooccupythephysicalspace.TheECAsarealsoprogrammedsuchthattheyrespondandbehaveinwaysthatarealsurgeonandrealnursewouldrespond.ThiscombinationoftraitsgivestheECAshighsocialpresencesotherealhumantraineestreatthemexactlyastheywouldrealhumans. 1.2ThesisStatementInthecontextofmedicalinterpersonalskillstraining,embodiedconversationalagentsareperceivedassociallypresentandelicitrealistictreatmentiftheembodiedconversationalagents1)occupythephysicalspaceand2)demonstratebehaviorandresponsesconsistentwiththetrainee'sexpectations.Thesetwoconditionsaresufcient 16

PAGE 17

suchthattheagentwillbeperceivedassociallypresentandelicitrealistictreatmenteveniftheagentisunabletodirectlyinteractwiththeenvironment. 1.3OverviewofApproachToevaluatethisthesis,weconductedaliteraturereviewoffactorsaffectingsocialpresenceandconductedfouruserstudies.Allofouruserstudiestookplaceinthecontextofinterpersonalskillstrainingforhealthsciences.Theparticipantsforallofouruserstudieswerehealthsciencesstudents(e.g.,nursingandmedicalstudents)orhealthsciencesprofessionals(e.g.,practicingnursesanddoctors). 1.3.1DimensionsofPhysicalityWeproposedusingtheterm`physicality'todescribeanECA'soccupancyofandinteractionwiththephysicalspace(Chapter 3 ).WereviewedexistingliteratureonMRECAs,identiedseveraldimensionsofphysicality,andproposedataxonomytoorganizethesedimensions.Thistaxonomycontainstwomaincategories,occupancyofthephysicalspaceandinteractionwiththeenvironment,whicharebothmentionedinthethesisstatement. 1.3.2OccupancyofthePhysicalSpaceWedesignedandconstructedanovelhighoccupancyECAsystemandusedittoconducttwostudiesexploringoccupancyofthephysicalspace(Chapter 4 ).Asarstexplorationoftherelationshipbetweenphysicalityandsocialpresence,weconductedaholisticevaluationthatvariedseveraldimensionsofoccupancyatonce.Thisholisticevaluationwasabetween-subjectsdesignwith18participants.Theresultssuggestedthatincreasingoccupancyofthephysicalspacecanincreasesocialpresence.However,theresultsalsosuggestedincreasingoccupancymayincreaseexpectationsabouttheECA'sbehaviorandreactions.Failingtomeettheseexpectationsmayreducesocialpresence.Inordertobetterunderstandtheroleofspecicdimensionsofphysicalityinincreasingsocialpresence,weconductedanindependentcomponentsevaluation.The 17

PAGE 18

independentcomponentsevaluationindividuallyvariedtwodimensionsofphysicality,formdelityandconcordance.Twentyninenursesparticipatedinthisbetween-subjectsevaluation.Resultsfromthisindependentcomponentsevaluationsuggestednotonlythatindividualdimensionsofphysicalityaffectsocialpresencetodifferentextents,butalsothatmatchinglevelsofphysicalityareimportantforincreasingsocialpresence. 1.3.3InteractionwiththeEnvironmentTocontinueourexplorationofphysicality,weconductedastudyexaminingtheothercategoryofdimensions,interactionwiththeenvironment.Thisstudywasawithin-subjectsdesignwith22participants.Theresultsofthisstudysuggestedanyeffectsofinteractiononsocialpresencearedominatedbytheeffectsofoccupancy. 1.3.4IdentifyingFactorsBeyondPhysicalityResultsandfree-formfeedbackfromourrstthreestudiessuggestedthatadditionalfactorsbesidesphysicalitymayhaveastronginuenceonsocialpresence.ThesefactorsmightberelatedtoexpectationsoftheECA'sbehaviorandresponses.Toformallyidentifythesefactors,weconductedaqualitativestudywith14participants.Wefoundaqualitativeapproachmoresuitablethanaquantitativeapproachforidentifyingcomplex,unknownfactors.Thisstudyidentiedseveralfactors,includingbehaviorandresponses,focusedonbothverbalandnon-verbalcommunication. 1.4InnovationsTheinnovationsofthisworkwereinidentifyingandformalizingspecicfactorsthatcontributetoanECA'ssocialpresence.WeproposedsocialpresenceandphysicalityastheECAanalogsofpresenceandimmersion.Wealsoformalizedthedimensionsofphysicalitybyproposingataxonomy.Thistaxonomymayaideresearchersinconductingfocusedexperimentsanddiscussingresultsinanorganizedmanner.Asimilartaxonomyhasprovidedthesebenetsto3Duserinterfaceresearchers[ 18 ].WhilethistaxonomyfocusesonmixedrealityECAs,itmayalsoprovidebenettoresearchersintheeldsofvirtualrealityECAs,telepresence,andsocialrobots. 18

PAGE 19

Withinthisframework,weconductedstudiesfocusedonthosedimensionsofphysicalitythathaverelativelylittleexistingsocialpresenceresearch.WeexaminedconcordancewiththeenvironmentandformdelityandfoundthatthecombinationofthetwocansignicantlyincreaseanECA'ssocialpresence.WealsoexaminedinteractionwiththeenvironmentanddidnotndanysignicanteffectonECA'ssocialpresence.However,wedidndanECAwithlittledirectinteractionwiththeenvironmentbuthighoccupancyoftheenvironmentcouldstillelicithighsocialpresenceandrealisticuserbehavior.Inordertoconductourstudies,wedesignedandbuiltANDI(ANimatronicDIgitalavatar),anovelMRdisplaysystemforanECAwithhighoccupancyofthephysicalspace.ANDIisarstinstanceoftheideaofcombiningvirtualcomponentswithphysicalcomponentstotakeadvantageoftheexibilityofvirtualcomponentsandtheinherenthighphysicalityofphysicalcomponents.Inaddition,ANDIisportable,low-cost(under$3000),anddoesnotrequiretheusertowearanyglassesormarkers.ThiscombinationoftraitsmakesANDImorepracticalthansystemssuchassee-throughhead-mounteddisplays,animportantrequirementforadoptioninrealtrainingprograms.Inadditiontophysicality,weidentiedseveralfactorsrelatedtobehavioralrealismanduserexpectationsthatalsocontributetosocialpresence.Weusedaqualitativeapproach,specicallyphenomenology,toidentifythesefactors.Qualitativeapproachesarerareincomputersciencebutareapowerfulmethodofidentifyingthemyriadfactorsinuencingasubjectivehumanexperience.Throughthisqualitativeapproach,weidentiedseveralfactorsthatcontributetoanECA'splausibility,includinghowwellwhattheECAsaysanddoesmatchestheuser'sexpectations. 19

PAGE 20

CHAPTER2REVIEWOFLITERATUREThischapterisdividedintofoursections.Therstthreesectionsrepresentthemaincomputerscienceareasthatmyworkbuildsupon:virtualhumans;immersion,presence,andplausibility;andsocialpresence.Thefourthsectionprovidesadidacticoverviewofqualitativeresearchtechniquesforreadersunfamiliarwiththesetechniques.Thisdidacticoverviewisprovidedbecausequalitativeresearchiscommoninanthropologyandsociologybutuncommonincomputerscience.Withincomputerscience,qualitativeresearchissometimesusedinthehuman-computerinteractioneld. 2.1VirtualHumansForthepurposesofthisdissertation,virtualhumansarecomputerprogramsthatsimulatetheappearanceandbehaviorofarealhuman.Virtualhumanshavebeenusedinavarietyofinterpersonalskillstrainers,includingmilitaryleadershiptraining[ 40 ],medicalinterviewtraining[ 2 45 48 69 ],andculturalskillstraining[ 5 39 ].Notethatthetermvirtualhumanalsoencompassesnon-conversationalentitiessuchasvirtualhumansusedinsurgicalsimulations.Whiletheseareimportantapplicationsofvirtualhumans,thisdissertationwillfocusexclusivelyonvirtualhumansusedinconversationalandsocialinteractions. 2.1.1EmbodiedConversationalAgentsThevirtualhumansinourresearchareatypeofembodiedconversationalagent[ 22 ].Embodiedconversationalagentsaresoftwareentitiesthatcanparticipateinaconversationandpossesssomesortof`physical'form.The`physical'formdoesnothavetobealivingthing.Additionally,the`physical'formmaybeapurelyvirtual,on-screenrepresentation.Forexample,anagentcouldbeembodiedinacartoonimageofatalkingvase.Aslongasthecartoonimagecancarryonaconversationwithauser,itisanembodiedconversationalagent.Theconversationcomponentiscritical;life-like 20

PAGE 21

virtualentitiesthatcannotreplicatethefunctionsofface-to-faceconversationarenotembodiedconversationalagents.Thisdissertationwillfocusprimarilyonvirtualhumansbecausetheexperimentsexamineonlyvirtualhumans.However,thetheoriespresentedinthisdissertationmayalsoapplytoembodiedconversationalagentsingeneral. 2.1.2AgentvsAvatarSomedenitionsofvirtualhumansincludebothavatarsandagents.Avatarsdifferfromagentsinthatavatarsarerepresentationsofrealhumansratherthansoftwareagents.Agentsarecontrolledbyanarticialintelligenceofsomesort,whereasavatarsarecontrolledbyrealhumans.Bothavatarsandagentsappearfrequentlyinvirtualhumanresearch.Ourresearchfocusesspecicallyonagentsandhowagentsareperceivedandtreatedbyusers.Wefocusonagentsbecausetherealworldapplicationofourresearchintendstoemployagents.Therealworldapplicationofourresearchisinterpersonalskillstrainersforhealthscienceslearners.However,theagentsinourexperimentsare,unbeknownsttotheparticipants,intruthavatarscontrolledbyahumanoperatorbehindthescenes.Usingahumanoperatorensuresallparticipantsreceiveequivalentexperiencesbyremovingvariancesduetoarticialintelligenceorspeech-to-texterrors.Thisdeceptionisdonebecausesomeexperimentshavefoundavatarsandagentsaretreateddifferently[ 9 36 ],thoughotherexperimentsfailedtonddifferences[ 64 90 ](seesection 2.3.3.1 fordetails).Becausethedifferencesbetweenavatarsandagentsmayexist,wechosetoerronthesideofcaution.Webelievethatimprovementstoarticialintelligenceandspeech-to-textsystemswilleventuallyremovetheneedforahumanoperator.Whenthishappens,theresultsofourexperimentswillstillremainvalid. 21

PAGE 22

2.1.3InterpersonalSkillsTrainingIninterpersonalskillstraining,learnerspracticecommunicationskillsbyinteractingwithotherpeople.Interpersonalskillstrainingfocusesondeveloping`soft'skillsratherthanknowledge.`Soft'skillsincludeshowingempathy,displayingleadership,decisionmaking,andworkingeffectivelyasateam.Interpersonalskillstrainingcanleveragevirtualhumansasconversationalpartners[ 2 5 39 40 45 48 69 ].Virtualhumansofferanumberofcompellingadvantagessuchasincreasedavailabilityandstandardization.SomevirtualhumanimplementationscanbeaccessedovertheInternetandmayallowpeopletotrainfromvariousremotelocationsattheirownconvenience[ 72 ].Additionally,unlikerealhumansvirtualhumanscanalwaysrespondidenticallytoidenticalinputandnevervaryduetofatigueorotherunintendedreasons[ 83 ].Severalstudieshavedemonstratedthevalidityofusingvirtualhumansforinterpersonalskillstraininginapplicationssuchasmedicalinterviews[ 2 45 48 69 ],militarydecisionmaking[ 40 ],andculturaltraining[ 5 39 ].Inthemedicaleld,virtualpatientshavebeenusedtotraindoctorsoncommunicationskills.Johnsenetal.foundperformanceinavirtualpatientinterviewpredictedperformanceinastandardizedpatientinterview(aninterviewwithanactorplayingtheroleofapatient)[ 45 ].Andradeetal.foundmedicalstudentscouldimprovetheirself-efcacyatdeliveringbadnewstoapatientbypracticingwithapatientavatarinSecondLife[ 2 ].However,virtualhumansarenotperfectreplacementsforrealhumansininterpersonalskillstraining.Raijetal.comparedvirtualpatientinterviewstostandardizedpatientinterviewsandfoundthecontentsweresimilarbutparticipantsappearedlessengagedandlesssincereinthevirtualpatientcondition[ 69 ].Engagementandsincerityaremeasuresofsocialpresence(Section 2.3 ).Ourworkexploresfactorsthataffectsocial 22

PAGE 23

presence,waystoencourageuserstotreatvirtualhumansmorerealistically,andwhatrealismmeanstohealthsciencelearners. 2.2Immersion,Presence,andPlausibilityIllusionThissectiondenesthreevirtualrealityconceptsanddenestherelationshipbetweenthem.Thethreeconceptsareimmersion,presence,andplausibilityillusion.Thesecloselyrelatedconceptsconstituteanimportantfoundationforourwork.Inourwork,wedrawparallelsfromtheseconceptstothevirtualhumanconceptsofphysicality,socialpresence,andbehavioralrealism. 2.2.1ImmersionSlaterdenesimmersionastheobjectivelevelofdelityofthesensorystimuliproducedbyatechnologicalsystem[ 77 ].Theword`objective'andthephrase`technologicalsystem'areveryimportantinthisdenition.Immersionisanobjectivemeasurebecauseitdoesnotvarybetweenobservers.Themaximumlevelofimmersionisconstrainedbythelimitsofthetechnologicalsystem.Technologicalsystemshavemanydifferentcomponentsincluding,butnotlimitedto,thedisplay,graphicscard,andtrackingsystem.Asaresult,immersionincludesmanydifferentdimensions.Thesedimensionsinclude,butarenotlimitedto,eldofview(whattheusercanseeatonce),eldofregard(whattheusercanseebyturninghishead),textureresolution,polygoncount,latency,andhapticfeedbackdelity.Thislargenumberofdimensionscanmakeitdifculttocomparedisparatesystems.Toillustratethispoint,considertwohypotheticalsystems,systemAandsystemB.SystemAusesahead-mounteddisplaywithheadtrackingbutonlyrendersgraphicsaswireframes.SystemBusesa17monitorandnoheadtrackingbuthasphoto-realisticgraphics.ClearlysystemAhasamoreimmersivedisplay,butsystemBhasmoreimmersivegraphics.Thismakesitdifculttosayonesystemisdenitivelymoreimmersivethantheother.Evenattemptingtorankthevariousdimensionsbyimportancecanbedifcultbecausetheimportancemayvarybyapplicationoruser,andhenceimmersionwould 23

PAGE 24

ceasetobeanobjectivemeasure.Asaresult,experimentstendtocomparesystemsthatvaryinonlyonedimensionofimmersion.Notethatwhileimmersionisanobjectivemeasure,wemaynotknowhowtoobjectivelymeasuresomedimensionsofimmersion.Forexample,olfactorydelityofasmellgenerationdevicemaybedifculttoobjectivelyquantify.However,intheoryitwouldbepossibletoanalyzeallthechemicalcomponentsoftheoriginalandgeneratedsmellsandmakeanobjectivecomparison.Immersionisverysimilartothevirtualhumanconceptofphysicality(Chapter 3 ).Bothareobjectivemeasuresoftechnologicalsystems,havemanydifferentdimensions,andhavedimensionsthatwemaynotknowhowtoobjectivelymeasure. 2.2.2PresenceSlaterdenespresenceasthepsychologicalsenseof`beingthere'[ 79 ].Presence,unlikeimmersion,isasubjectivefeelingthatdependsheavilyontheuser.Asaresult,twousersthatinteractwiththesamesystemmayreportdifferentlevelsofpresence.Thesedifferentlevelsmaybetheresultofdifferentexpectationsorevendifferentexplorationsofthesystem.Forexample,ifuserAputsonahead-mounteddisplayandsimplylooksaroundaphoto-realisticenvironment,userAmightreportahighlevelofpresence.IfuserBputsonthesamehead-mounteddisplay,looksaroundthesamephoto-realisticenvironment,andalsoattemptstowalkaroundandtouchthingsbutfeelsnohapticfeedback,userBmightreportalowerlevelofpresence.Inordertoachievehigherlevelsofpresence,researchersgenerallyincreaseasystem'simmersion.Studieshavefoundimmersionandpresencegenerallyhaveapositiverelationship[ 60 87 ].However,notallaspectsofimmersionaffectpresenceequally.Severalresearchershavetestedtheeffectsofvariousaspectsofimmersiononpresenceinthepitexperiment.Thepitexperimentisanoftrepeatedexperimentwhereauseristoldtowalkacrossavirtualroomtotheotherside.Themiddleoftheroomisalargepit,sotheusermustwalkonanarrowledgearoundthepit[ 79 ].Usohetal. 24

PAGE 25

foundhavingparticipantsmovetheirbodiestowalk,ratherthanjustpressingabutton,increasedpresence[ 87 ].Meehanetal.foundregisteringaphysicalledgewiththevirtualledgesoparticipantscouldfeeltheirfeetontheedgealsoincreasedpresence[ 60 ].However,ZimmonsandPanterdidnotndasignicantdifferenceinpresencewhenvaryingthevisualrealismoftheroom(textureresolutionandlightingquality)[ 95 ].Theybelievedthevisualpitinallconditionscreatedasenseofpersonaldangerthatovershadowedanydifferencesinpresence.Similarly,weexpectthatdifferentaspectsofphysicalitywillaffectsocialpresencedifferently.Someaspectsofphysicalitymayalsohavestrongeffectsthatovershadowtheeffectsofotheraspectsofphysicality. 2.2.2.1MeasuringPresenceResearchersuseavarietyofmethodstomeasurepresence.Becausepresenceisasubjectivefeeling,commonlyusedinstrumentssuchastheSUS[ 79 ]andWS[ 93 ]assesspresenceusingself-reportratingscales.Forexample,oneitemontheSUSquestionnairereadsWhenyouthinkbackaboutyourexperience,doyouthinkoftheofcespacemoreasimagesthatyousaw,ormoreassomewherethatyouvisited?Theofcespaceseemstometobemorelike...(1)ImagesthatIsaw.(7)SomewherethatIvisited.Presencecanalsobemeasuredusingbehavioralandphysiologicalmeasures.Physiologicalmeasuresincludeheartrate,skinconductance,andskintemperature.ThesemeasureswereusedbyMeehanetal.inaninstanceofthepitexperiment[ 60 ].Meehanetal.foundheartrateandtoalesserextentskinconductancewerebothgoodmeasuresofpresence.Abehavioralmeasureofpresenceinthepitexperimentisiftheuserschoosetowalkaroundthepitorsimplyglideacrossit(someimplementationsoftheenvironmentdonotimplementgravity).Slaterfoundthatthevastmajorityofpeoplechoosetowalkaroundthepit[ 79 ].Anotherbehavioralmeasureofpresenceistheloomingresponse, 25

PAGE 26

e.g.,apersonduckingwhenavirtualobjectiestowardshereventhoughsheknowsnothingisphysicallytheretohither[ 76 ].Notethatwhilebehavioralandphysiologicalmeasuresareobjectivemeasuresofauser'sreaction,thereactionitselfisstillsubjectiveinthatdifferentusersmayreactdifferentlytothesamestimuli.Whileobjectivityisadesirabletrait,behavioralandphysiologicalmeasuresmaynotbeappropriateforallapplications.Applicationssuchasthepitexperimentthatintentionallygeneratearousaloranemotionalresponseareservedwellbyheartrateandskinconductance.Inotherapplicationsthatshouldnotgeneratearousaloremotionalresponses,heartrateandskinconductancemaynotreectpresencelevels.Comparingpresenceresultsacrossdisparateenvironmentsmaynotprovideusefulinformation.Usohetal.performedanexperimentwhereonegroupofsubjectsperformedataskinavirtualofceenvironmentwhileanothergroupofsubjectsperformedthesametaskinanequivalentrealofceenvironment[ 87 ].Accordingtoonepresencequestionnaire(SUS),theaveragepresencescorewasonlymarginallyhigherfortherealcondition.Foranotherpresencequestionnaire(WS),therewasnosignicantdifferencebetweentherealandvirtualenvironments.Usohetal.believedthatinthevirtualenvironment,peoplewerebasingthepresencescoresonsystemfactorsrelatedtoimmersionsuchashowmuchoftheenvironmentwasmodeledandlatency.Bycontrast,intherealenvironmentpeoplebasedpresenceoncomparisonstotheirexpectationsofanofceenvironmentaswellastheirexpectationsofinteractionswithpeopleintheenvironment.Theseresultssuggestpeoplemayhavedifferentexpectationsandstandardswhenexposedtodifferenttechnologiesandlevelsofimmersion.Theseresultsalsosuggestthatpresencemaybeinuencedbyanotherfactor,plausibility,whichwillbediscussedinthenextsection. 26

PAGE 27

2.2.3PlausibilityIllusionPlausibilityillusionistheillusionthatwhatisapparentlyhappeningisreallyhappening(eventhoughyouknowforsurethatitisnot)[ 78 ].Achievingplausibilityillusiondependsonelementsinthevirtualenvironmentrespondingtotheuser'sactionsdespitetheuserhavingnodirectcontrolovertheelements.Forexample,ifausermovestooclosetoavirtualhumanandthevirtualhumanstepsback,thismaintainsplausibilityillusion.Slaterproposedbothplausibilityillusionandplaceillusion(presence)arenecessaryforuser'stodemonstraterealisticbehavior[ 78 ].Becauserealisticuserbehaviorisonegoalofinterpersonalskillstraining,webelieveplausibilityillusionisanimportantaspect.However,littlevirtualrealityresearchhasexploredplausibilityillusion.InthecontextofECAs,plausibilityillusionmaybetightlycoupledwiththeECA'sbehavior.Someresearchhasexaminedtheeffectsofnon-verbalbehavioronsocialpresence(seeSection 2.3.3.3 fordetails).However,thisresearchdoesnotexplorepossiblyimportantaspectsofbehaviorsuchasrespondingtowhattheusersaysanddoes.Chapter 6 describesaqualitativestudyinwhichweidentiedaspectsofanECA'sbehaviorthatmaycontributetoplausibility. 2.3SocialPresenceSocialpresenceisvieweddifferentlybyavarietyofresearchersacrossmanyelds.Thissectionpresentssomeofthedenitionsandmeasuresofsocialpresenceandexplainsourchoiceofdenitionandmetrics.Thissectionalsoexaminesexistingresearchintofactorsaffectingsocialpresence. 2.3.1DenitionsofSocialPresenceBecauseourresearchfocusesonagent-basedtraining,weuseadenitionofsocialpresencethatfocusesspecicallyonagents.Thisdenition,createdbyBlascovich,denessocialpresenceastheextenttowhichindividualstreatembodiedagentsasiftheywereotherrealhumanbeings[ 17 ]. 27

PAGE 28

However,therearemanyotherdenitionsofsocialpresencethatinuencethegeneraltheoryofsocialpresence.AccordingtoBioccaetal.,thetermsocialpresencehasrootsinsymbolicinteractionismandsocialpsychology.[ 16 ].Somesocialpsychologyresearchtreatssocialpresenceasabinarymetric,anotherpersoniseithersociallypresentornot[ 80 ].Ontheotherhand,computer-mediatedcommunicationresearchtypicallyviewssocialpresenceasacontinuumwithdegreesofsocialpresence,i.e,differentsystemscanproducedifferentlevelsofsocialpresence.Withincomputer-mediatedcommunicationresearch,denitionsofsocialpresencefallintoseveralclassications[ 16 ].Theseclassicationsincludeco-presence,psychologicalinvolvement,andbehavioralengagement.Withintheseclassications,severalrelevantdenitionsstandout.Gunawardenadenessocialpresenceasthedegreetowhichapersonisperceivedasa`realperson'inmediatedcommunication[ 37 ].Inourresearch,wefocusontheperceptionofembodiedconversationalagentsas`realpeople'inmixedrealityenvironments.ThedenitionsofMcLeodetal.[ 59 ]andShortetal.[ 75 ]suggestthatphysicalitymayberelatedtosocialpresence.McLeodetal.focusesonthedegreeoftangibilityandproximityofotherpeoplethatoneperceivesinacommunicationsituation[ 59 ].Tangibilityandproximityareaspectsofphysicality.Shortetal.denessocialpresenceasanattitudinaldimensionoftheuser,amentalsettowardsthemedium[ 75 ].Thisdenition'sfocusonthemediumsuggeststhatchangesinthemedium'sphysicalitycanaffectsocialpresence.Behavioralengagementdenitionsemphasizetheimportanceofaspectsotherthanthemedium.Palmerdenessocialpresenceaseffectivelynegotiate(ing)arelationshipthroughaninterdependent,multi-channelexchangeofbehavior[ 66 ].BehaviorisespeciallyimportantwithECAsbecausebehaviormustbeimbuedbythecreatorandsoftware.InChapter 6 ,weexploretherelationshipbetweenbehavioralengagementandsocialpresenceforhealthsciencelearners. 28

PAGE 29

2.3.2MeasuringSocialPresenceAccordingtothereviewofsocialpresenceperformedbyBioccaetal.,thereisasofyetnowidelyacceptedmeasureofsocialpresence[ 16 ].Justastherearemanyvaryingdenitionsofsocialpresence,therearemanyvaryingmeanstomeasuresocialpresence.ThesemeansofmeasuringsocialpresencefallundertwocampsaccordingtoareviewbyBailensonetal.[ 10 ].Onecampfocusesonperceptionwhiletheotherfocusesonsocialresponses.Thisdivisionparallelswaysofmeasuringpresence. 2.3.2.1PerceptionPerceptionistypicallyevaluatedusingself-reportsurveys.Bailensonetal.developedaninstrumentwithveLikert-typesurveyitems[ 9 ].Theitemsassessperceptionofthevirtualhumanasarealpersonandaspresentintheroom.Thissurveywasusedasoneinstrumentinallofourexperiments. 2.3.2.2SocialResponsesSocialresponsesareevaluatedusinganalysisofuserbehaviorsandphysiologicaldata.Forabehavioralmeasureofsocialpresence,Bailensonetal.usedinterpersonaldistance(howcloseapersonstandstoanotherperson)[ 9 ].Interpersonaldistanceisameasureofsocialpresencebecauseindividualstendtorespectaperson'spersonalspacebubble.Moreover,interpersonaldistanceisalow-level,subconsciousresponse[ 34 ].IntheexperimentbyBailensonetal.,participantsweretoldtoapproachvirtualhumansandreadtheirnametags.Theexperimentusedahead-mounteddisplayandrealwalking,soparticipantsphysicallymovedabouttheroomtoapproachthevirtualhumans.Theirresultsindicatedparticipantsmaintainedgreaterdistancefromthevirtualhumanswhenthevirtualhumansdemonstratedmutualgazebehavior(lookingattheparticipant).Participantsalsomaintainedgreaterdistancewhenapproachingthevirtualhumansfromthefrontthantheback.Panetal.usedheartrateandskinconductancedataasphysiologicalmeasuresofsocialpresence[ 67 ].Thisexperimentcomparedsociallyshymalestosociallycondent 29

PAGE 30

malesindividuallyapproachingasociallyforwardvirtualwoman.Thevirtualwomanengagedtheparticipantinaconversation,askingaseriesofquestions.Theirresultsfoundbothheartrateandskinconductancewentupwhentheparticipantinitiallybegantalkingwiththevirtualwoman.Thesearebothsignsofincreasedanxietyorarousal.Interestingly,forthesociallyshymales,theheartrateandskinconductanceovertimefellbelowtheirpre-engagementlevels,whereasforthesociallycondentmalestheyreturnedtotheirpre-engagementlevels.ThissuggeststhatECAscanbesuccessfullyusedtoreducesocialanxietyinaninterpersonalscenario.Ourexperimentsusebehavioralmetrics.Interpersonaldistanceisnotanapplicablemetricinourexperimentsbecausetheparticipantsdonothavetheopportunitytoapproachthevirtualhumans.Instead,weuseaspectsoftheconversationasbehavioralmetrics(Chapters 4 and 5 ).AspectsoftheconversationwerealsonotedasinformalevidenceofhighsocialpresenceinanexperimentbyPanetal.[ 67 ]. 2.3.2.3CombiningPerceptionandSocialResponsesregardlessofwhatisactuallycontrollingtheECA.Bailensonetal.alsocomparedself-reportsurveyswithsocialresponsebehavioralmetrics[ 8 ].Behavioralmetricsshoweddifferencesevenwhenself-reportsurveysdidnot.Asaresult,theysuggestusingbehavioralmetricstocomplementself-reports.Inourstudies,weusebothbehavioralandself-reportmetrics. 2.3.3FactorsAffectingSocialPresenceResearchershaveidentiedseveralfactorsthatmayinuencefeelingsofsocialpresence.Thesefactorsincludethevirtualhuman'sagency,visualrealism,non-verbalbehavior,anduser'spersonality.Thisdissertationidentiesandexploresadditionalfactorsincludingmixedrealitytechnology,verbalbehavior,anduserexpectations. 2.3.3.1AgencyAgencyisifthevirtualhumanisanavatar(human-controlled)oranagent(computer-controlled).Experimentsstudyperceivedagency,i.e.,whattheuserbelieves, 30

PAGE 31

bytellingtheparticipantsthevirtualhumanisanagentoranavatarwhilekeepingthetrueagencyofthevirtualhumanconstant.Experimentalresultsdisagreeonwhetherperceivedagencyhasaneffectonsocialpresence.Bailensonetal.andGuadagnoetal.foundthatusershadhighersocialpresencewithavatarsthanagents[ 9 36 ].TheresultsofBailensonetal.showedaneffectofagencyonsocialpresencewithastrongstatisticalsignicance(p<0.001).Avatarswereratedonaverageslightlyaboveneutral(1.17),whereasagentswereratedonaveragebelowneutral(-3.26)onaself-reportscaleof-15to15.TheresultsofGuadagnoetal.,thoughtheaverageratingswereslightlylower(avatars-0.60,agents-1.31,samescale).Interestingly,Gaudagnoetal.alsoexaminedsocialinuence,i.e.,howpersuasivesomebodyis.Theyfoundmaleagentsweresignicantlymorepersuasivethanmaleavatarsandasimilartrendwithfemaleagentsandavatars.However,vonderPuttenetal.andNowakandBioccafailedtondanysignicanteffectofagencyonsocialpresence[ 64 90 ].VonderPuttenetal.usedbothBailenson'ssocialpresencesurvey[ 9 ]andBiocca'snetworkedmindsscale[ 15 ].Neitherscaleshowedsignicantdifferencescorrelatedwithagency,thoughtheaveragesocialpresenceratingswerelowerthaninBailenson'sexperiments.NowakandBioccaexaminedsocialpresence,co-presence,andtelepresenceandalsofailedtondanysignicanteffectofagency.Becausesomeresearchindicatesbelievingavirtualhumanisanavatarincreasessocialpresence,wechosetopresentthevirtualhumansinourexperimentsasagents.Intruth,allourvirtualhumanswerecontrolledbyahumanoperatorinordertoeliminatespeech-to-textandarticialintelligenceerrorsasfactorsthatmayvarybetweenparticipants.ThisiscommonpracticeandhasbeenusedinmanystudiesinvolvingECAsandnaturallanguageunderstandingsystems[ 28 31 44 58 ]. 31

PAGE 32

2.3.3.2VisualRealismVisualrealismhasmadegreatadvancesinrecentyearsashardwarecontinuestogetfasterandcheaper.Polygoncountshavegottenhigher,andvirtualhumanscanbemorephoto-realisticthaninyearspast.However,muchresearchsuggestslowlevelsofvisualrealismaresufcienttoinvokehighlevelsofpresenceandsocialpresence.Inanexperimentwithoutvirtualhumans,ZimmonsandPanterexamineddifferentlevelsoftextureandlightingqualityandfailedtondanydifferencesinpresence[ 95 ].Theyfoundevensubstitutingalltextureswithablackandwhitegridpatterndidnotsignicantlyreducepresencecomparedtohigherresolutionteturesandusingradiosityforlighting.Inanexperimentwithvirtualhumans,Panetal.notedthatparticipantsrespondedquiterealisticallytothevirtualwomanintheirexperimentdespiteboththevirtualwomanandthevirtualenvironmentnotlookingrealistic[ 67 ].Increasinglevelsofvisualrealismmayalsohurtratherthanhelp.Extremelyhighlevelsofvisualrealismmaycausevirtualhumanstoentertheuncannyvalley.Theuncannyvalleyisaphenomenonwhereincreasingrealismincreasespositiveresponsesuntilitsuddenlycausesafeelingofrevulsion[ 62 ].Evenwithoutenteringtheuncannyvalley,increasingvisualrealismmayresultinlowersocialpresence.Forexample,NowakandBioccafoundparticipantsreportedmoresocialpresenceinteractingwithalessanthropomorphicimagethanamoreanthropomorphicimage[ 64 ].Theyattributedthisdecreaseinsocialpresencetoanincreaseinexpectationsofthemoreanthropomorphicimage.Inadditiontoimprovedvisualrealism,technologicalimprovementshavealsoimprovedmixedrealitydisplays.Verylittleresearchhasexaminedsocialpresenceinthecontextofmixedreality.Mostexistingsocialpresenceresearchusespurelyvirtualenvironments,eitherhead-mounteddisplaysorstandardcomputermonitors.Mixedrealityintroducesseveralunexplorednewfactorssuchashowcloselythevirtualhuman'ssizeistolifesizeandifthevirtualhumancanorcannotaffectobjectsinthe 32

PAGE 33

physicalenvironment.Inourwork,weproposedataxonomyfororganizingthesefactors(Chapter 3 )andconductedexperimentsexploringtheeffectsofthesefactorsonsocialpresence(Chapters 4 and 5 ). 2.3.3.3BehaviorSeveralstudieshavefoundrelationshipsbetweennon-verbalbehaviorandsocialpresence.VonderPuttenetal.exploredifhigherbehavioralrealism(noddinginacknowledgement)orlowerbehavioralrealism(nonodding)affectedsocialpresence[ 90 ].Theyconcludedthevirtualhuman'sbehavioralrealismwasagoodpredictoroftheamountofwordsspokenbytheuser,abehavioralmetricofsocialpresence.Similarly,Bailensonetal.comparedvirtualhumansthatmaintainedeyecontactwithaparticipanttothosethatdidnot[ 7 ].Theyfoundfemaleparticipantsmaintainedmoreinterpersonaldistancetothosethatmaintainedeyecontact.Maintaininginterpersonaldistance(i.e.,notviolatinganother'spersonalspacebubble)isabehavioralmeasureofsocialpresence.Forconversationalvirtualhumans,socialpresencemayalsobeaffectedbytheverbalbehaviorandtheuser'sexpectationsofverbalbehavior.Verbalbehaviorisanaspectofplausibility.Auserhasnodirectcontroloverwhatavirtualhumanwillsay,butauserhasanexpectationsayingsomethingshouldcausethevirtualhumantorespondappropriately.Tothebestofourknowledge,noexistingresearchhasstudiedtheimportanceofverbalbehaviorinperceptionsofsocialpresence.InChapter 6 ,wereportonaqualitativestudythatidentiedverbalbehaviorasanimportantaspectofanECAthatcontributestosocialpresence. 2.3.3.4UserPersonalitySocialpresenceisasubjectivemeasurethatdependsontheuser.Asaresult,theuser'spersonalitymayaffectsocialpresence.VonderPuttenetal.investigatedthispossibility[ 90 ].Theyevaluatedeachparticipant'spersonalityusingavarietyofestablishedpersonalitysurveys[ 13 21 23 55 74 ].Participantsthentalkedwitha 33

PAGE 34

virtualhuman.Theyassessedsocialpresenceusingself-reportsurveysevaluatingtheuser'semotionalstateandperceptionofthevirtualhuman.Theirresultsfoundtheparticipant'spersonalitywasthebestpredictoroffeelingstowardstheexperienceandperceptionofthevirtualhuman. 2.4QualitativeResearchTofurtherexaminesocialpresence,Iuseacombinationofquantitativeandqualitativeresearchtechniques.Becausequalitativeresearchtechniquesarelesscommonincomputerscienceresearch,Iwillexplainwhyqualitativeresearchwasamoreappropriatetechniqueforoneofourstudies,provideabriefintroductiontoavailabletechniques,andexplainmychosentechnique. 2.4.1WhyQualitativeResearchQualitativeresearchprovidesadeeperunderstandingofatopicinrelationtoasmallpopulation.Itisusefulforunderstandingphenomenonwherethevariableshaveacomplexinterplay.Asaresult,qualitativeresearchtypicallydoesnotinvolvemanipulatingasmallnumberofvariablestocreatetwoormoreconditionsthatareotherwiseexpectedtobeequal.Qualitativeresearchdoesrequireinterpretationbytheresearcher,andasaresulttheresearchermustattempttosetasidebiasesandpre-conceivedideas.Thisresearchapproachismorecommoninpsychology,anthropology,andsociology.Inourwork,wesoughtadeeperunderstandingofwhatsocialpresencemeanstohealthsciencesstudents.Particularly,weexploredwhatitmeansforavirtualhumantoberealintheeyesofnursingstudents(Chapter 6 ). 2.4.2OverviewofApproachesCreswellidentiesvemajorapproachestoqualitativeresearch:narrativeresearch,phenomenology,groundedtheory,ethnography,andcasestudy[ 27 ].Theseapproachesdifferinpurpose,procedure,andnumberofsubjects.Inthissection,wewillbrieyexplaineachoftheseapproachesandexplainwhyweconsideredonlytwoapproaches 34

PAGE 35

applicable.InChapter 6 wewillfurtherexplainournalchoiceofapproaches.TheexplanationsoftheseapproachesarederivedfromCreswell'sbook.Creswell'sbookwaspersonallyrecommendedbyseveralqualitativeresearchers,includingmycollaboratorDr.CaseyWhite,asanauthoritativeintroductiontoqualitativeresearchandhasbeencitedover20,000times. 2.4.2.1NarrativeResearchNarrativeresearchisbestforcapturingthedetailedstoriesorlifeexperiencesofasinglelifeorthelivesofasmallnumberofindividuals[ 27 ].Innarrativeresearch,theresearcherperformslong,detailedexaminationsofasinglepersonorasmallnumberofpeoplewhohaveexperiencedtheresearchtopic.Theresearchermayinterviewthesubjectsdirectly,askthesubjectstokeepjournals,orreviewphotographsandpersonalartifacts.Afteranalyzingthesesources,theresearcherorganizesthesourcesintoaframeworktheresearcherbelievesmakessenseandformsacoherentstory.Wechosenottousenarrativeresearchbecauseourresearchtopic,perceptionofandresponsestovirtualhumans,isnotalifeexperience.Mostpeoplecurrentlyhaverare,shortexperienceswithvirtualhumansratherthanlifeexperiences. 2.4.2.2EthnographyEthnographyisaqualitativedesigninwhichtheresearcherdescribesandinterpretsthesharedandlearnedpatternsofvalues,behaviors,beliefs,andlanguageofaculture-sharinggroup[ 27 ].Ethnographersexamineculturalconcepts.Whileourresearchfocusesonspecicculturalgroups(e.g.healthsciencesstudents),ourresearchisnotexamininganaspectthatdenestheculture.Asaresult,ethnographyisnotanappropriateapproachforourresearch. 2.4.2.3CaseStudyAcasestudyisagoodapproachwhentheinquirerhasclearlyidentiablecaseswithboundariesandseekstoprovideanin-depthunderstandingofthecasesoracomparisonofseveralcases[ 27 ].Casestudiesareappropriateforexaminingaspecic 35

PAGE 36

incidentoraboundedsmallnumberofincidents.Forexample,acasestudycouldstudyhowwinningthePowerballlotterychangedthelivesofthewinners.Thishasasmallnumberofcasessoitispossibletodoin-depthanalysisofallofthem.Whileourexperimentscreateasmallnumberofcases,acasestudyisnotagoodapproach.Thecasesareaby-productofourexperimentratherthanatargetforaninvestigation.Additionally,ourresearchismoreinterestedinunderstandingaspecicphenomenonanditscausesratherthanthecasesthemselves. 2.4.2.4PhenomenologyPhenomenologicalresearchdescribesthemeaningforseveralindividualsoftheirlivedexperiencesofaconceptoraphenomenon[ 27 ].Itexaminesaspecicphenomenonwithrespecttoasmallgroupofindividualsanddistillstheessenceofthephenomenonforthatsmallgroup.Inphenomenologicalresearch,theresearchercollectsdatafrommanypersons,primarilythroughinterviews,andextractstheessentialthemes.Inourresearch,aphenomenologicalresearchapproachcouldlookatthephenomenonofperceivingandreactingtoavirtualhumanrealistically.Thiswouldanswerthequestionofwhattheword`real'meanstoasmallgroupofusers. 2.4.2.5GroundedTheoryTheintentofagroundedtheorystudyistomovebeyonddescriptionandtogenerateordiscoveratheory[ 27 ].Groundedtheoryisgroundedincollecteddata,ofteninterviews.Aresearcherdoesnotbeginaprojectwithapreconceivedtheoryinmind...rathertheresearcherbeginswithanareaofstudyandallowsthetheorytoemergefromthedata[ 81 ].Agroundedtheoryapproachwouldallowustodevelopatheoryofwhypeopleperceiveandtreatsomevirtualhumansmorerealisticallythanothers.Thistheorymayincludemanyfactors,bothonthetechnologicalsideandontherealhumanside,andcomplexpathways.AsshownbyVonderPuttenetal.,humanfactorssuchasthe 36

PAGE 37

user'spersonalitycaninuenceperceptionofvirtualhumans[ 90 ].Thehumanandtechnologicalfactorsmayhaveacomplexinterplaythatisbestexploredusingqualitativeresearchratherthanmanipulationofindividualfactorsinaquantitativeapproach. 2.4.3PhenomenologyandGroundedTheoryProcessesBothphenomenologyandgroundedtheoryresearchtechniquestypicallyinvolvesemi-structuredinterviews.Semi-structuredinterviewshaveonlyasmallnumberofxedquestions.Thesequestionsareopen-ended,andbasedontheresponsestheinterviewergeneratesadditionalquestionsduringtheinterview.Bothapproachesfollowacycleofinterviewingandanalysiswhereasmallnumberofparticipantsareinterviewed,thentheirdataareanalyzed,thequestionsaremodiedorexpanded,andmoreinterviewsareconducted.Thiscyclecontinuesuntiltheanalysisreachessaturation,thatisnonewideasarefound.Thetotalnumberofparticipantsneededforsaturationvariesbutistypicallybetween10and20[ 27 ].Phenomenologyandgroundedtheorydifferintheiranalysisprocesses.Theseprocessesaredescribedbelow. 2.4.3.1PhenomenologyAnalysisandRepresentationPhenomenologyanalysisbeginswithtranscribinginterviews,readingthroughtheinterviewsseveraltimes,andmakingnotes.Afterreadingthetranscriptsseveraltimes,theresearcherbeginstoidentifysignicantstatements.Thesestatementsarethenlabeledwithmeaningunits.Meaningunitsarethehighlevelmeaningofastatement,andseveraldifferentphrasingssharethesamemeaningunit.Forexample,thestatementsIthoughtitwasreallyrealthewaytheireyesandheadsfollowedmeandThewaytheykeptlookingatmemadethemfeelrealbothcorrespondtothemeaningunittheeyeandheadmovementsaffectedrealism.Theresearchertheninterpretsthedatabydevelopingrepresentationsofwhathappened,howthephenomenonwasexperienced,anddistillingtheessenceofthephenomenon.Thisinterpretationofteninvolvesgroupingmeaningunitsintocommon 37

PAGE 38

themes.Finally,theresearcherpresentstheessenceofthephenomenonusingtables,gures,anddiscussion. 2.4.3.2GroundedTheoryAnalysisGroundedtheoryanalysisalsobeginswithtranscribinginterviews,readingthroughtheinterviewsseveraltimes,andmakingnotes.However,theresearcherthengoesthroughthreeconsecutivecodingphases-open,axial,andselectivecoding.Opencodinginvolvesidentifyingcommoncategoriesofinformationinthetranscripts.Duringaxialcodingtheresearcherselectsonecentralcategoryofinformation(i.e.thecentralphenomenon)anddetermineshowtheothercategoriesrelatetothecentralcategory.Therelationshipsmaybecausal,context,interveningconditions,strategies,andconsequences.Theresearchertheninterpretsthedataduringselectivecodingbyrelatingthecategoriesintoastoryorpropositions.Finallytheresearcherusesthisstorytopresentatheoryexplainingthecentralphenomenon. 2.4.4QualitativeVirtualHumanResearchWhilequalitativeresearchisrelativelynewtovirtualhumanresearch,itisanestablishedmethodintheliterature.Bearmanperformedaqualitativeanalysisofmedicalstudents'experienceswithavirtualpatient[ 12 ].Thevirtualpatientwasacomputerprogramwherestudentsselectedchoicesfromalistandtheprogramplayedbackvideorecordingsofarealhumanactor.Heranalysisfoundstudentsstronglyself-identiedasstudents(asopposedtodoctors)andtheirgenerallifeexperiencesstronglyinuencedtheirrelationshipwiththevirtualpatient.Shealsosuggestedvirtualpatientscouldbeusedtopresentstudentswithcomplexsituationstheyarelikelytoencounterasprofessionals.Bearman'sstudyinuencedourstudydesignandanalysisprocedure.Wesimilarlyexpectthatthestudents'experienceswillaffecttheirperceptionsofthevirtualhumans,andthemedicaltrainingscenariosusedinmystudiescanbeintroductionstoacomplexsituationforsomelessexperiencedlearners. 38

PAGE 39

CHAPTER3DIMENSIONSOFPHYSICALITYThischapterdescribesourproposedtaxonomyfordimensionsofphysicalityandreviewsrelevantliterature.ThistaxonomywassubmittedaspartofajournalarticletoPresence:TeleoperatorsandVirtualEnvironments,andmuchofthischapter'stextissharedwiththatarticle.Thearticlewasacceptedbuthasnotyetbeenpublished.PersonalContributions:Iperformedtheliteraturesearch,formalizedtheresultsintothesedimensions,andproposedthistaxonomytocategorizethedimensions.Iwrotealltextsharedbetweenthischapterandthejournalarticle.Collaborators:Numerouspeoplediscussedthesedimensionsofphysicalitywithme,renedtheideas,andhelpedshapethenaltaxonomy.ThesepeopleincludeDr.RegisKopper,theanonymousreviewersofourjournalarticle,andmycommitteemembers.RelevancetoThesis:Thischapterdenesthetaxonomyfordimensionsofphysicalitythatweusedtoselectanddesignourexperiments. 3.1OverviewMRtechnologycanincreasethephysicalityofanECAbyblendingrealandvirtualelements.DifferentMRtechnologiesmakevarioustradeoffswithregardstothingssuchasthepositionofvirtualelementsortheuseofhapticfeedback.Toformalizethesetradeoffsandtheirrelationshiptophysicality,weproposeataxonomyofphysicalitydimensions(Figure 3-1 ).Thetaxonomydividesphysicalityintotwocategoriesofdimensions:occupancyofthephysicalspaceandinteractionwiththeenvironment.Forbrevity,wewillrefertothesecategoriesasoccupancyandinteraction.Inthissectionwedetailthedimensionsandreviewexistingresearchthathasexploredthedimensionsindifferentcontexts.Wenotethatthetaxonomyweproposeisnotexhaustiveandthatotherdimensionsmayexist;herewedescribethedimensionswehavegatheredfrompersonalexperienceandreviewsofexistingresearch.Someofthesedimensionsalso 39

PAGE 40

existinpureVRenvironments,andndingsrelatedtothesedimensionsmayapplyinbothMRandpureVR.Notethatvisualrealismandrenderingqualityarenotdirectlydimensionsofphysicality.Weacknowledgethatthesemayaffectsocialpresence,buttheygenerallydonotaffecttheextenttowhichanECAoccupiesthephysicalspaceorinteractswiththeenvironment.However,someaspectsofvisualrealismandrenderingqualitycancontributetodimensionsofphysicality,andweexplainthisinthecorrespondingdescriptionsofdimensionsbelow. 3.2OccupancyofthePhysicalSpaceOccupancyofthephysicalspacedescribeshowaccuratelyanECAappearstosharethephysicalspacewiththeuser.WeidentifyseveraldimensionswithinoccupancyofthephysicalspacerelatedtoanECA'ssize,position,form,andconcordancewiththephysicalspace.Conformingtoourdenitionofphysicality,occupancyofthephysicalspacecanalwaysbemeasuredobjectively,althoughthismaybedoneattheperceptuallevel.Forexample,onasee-throughHMDanECAmayonlybephysicallyaninchtall,butthevisualsizeoftheECAisobjectivelyperceivedatlife-size,andthusithashighsizedelity(seeSection 3.2.1 ). 3.2.1SizeFidelitySizeFidelityreferstohowcloselyanECAconformstolife-size.ExamplesofhighsizedelityECAsincludeECAsdisplayedatlife-sizeonprojectionscreensandsee-throughHMDs.ExamplesoflowsizedelityECAsincludedisplayingECAsatsmallerthanlife-sizeoncomputermonitors.SizedelitywasexploredinastudybyJohnsenetal.[ 44 ],whoexaminedtheeffectsofdisplaysizeonuserrespectshowntowardsavirtualpatient.Thestudycomparedalife-sizevirtualpatientonaTVtoasmaller-than-life-sizevirtualpatientonamonitor.TheirresultssuggestedusersshowedmorerespecttowardsthehighsizedelityECAthanthelowsizedelityECA. 40

PAGE 41

3.2.2PositionFidelityPositionFidelityreferstohowcloselyanECA'spositionconformstoitsexpectedpositioninphysicalspace.Positiondelitycanbeincreasedbyusinghead-coupledperspective(alsoknownassh-tankVR[ 92 ])orstereoscopicdisplaystorenderECAsasiftheywerecloserorfartherthanthedisplay'sphysicalposition.Thedisplay'sphysicalpositioncanalsobechangedtoincreasepositiondelity.Figures 3-2A and 3-2B illustratehighandlowpositiondelityrespectively.InFigure 3-2A thedisplaysareplacedinthephysicalenvironmentwhererealpeoplewouldbestanding.InFigure 3-2B theECAsaredisplayedonthewallfartherbackthanwhererealpeoplewouldbestanding.Tothebestofourknowledge,noresearchershavelookedatpositiondelitywithECAs.However,interpersonaldistanceplaysanimportantroleinthelevelofintimacyofaninteraction[ 3 ].Bailensonetal.demonstratedthatpeoplegenerallyrespectanECA'spersonalspaceandreacttoanECAviolatingtheirownpersonalspace[ 6 ].Asaresult,webelievethatpositiondelitymayplayanimportantroleinanECA'ssocialpresence. 3.2.3FormFidelityFormFidelitydescribeshowcloselythephysicalformofthedisplayconformstotheintendedphysicalformoftheECA'sbody.LowformdelityECAsincludeECAsdisplayedonmonoscopicmonitors,televisions,andprojectionscreens;thesedisplaysandtheimagesonthemareallphysicallyat.Stereoscopicdisplayscanincreaseformdelitybymakingtheimageperceptuallythree-dimensional.Physicallythree-dimensionaldisplaysthatmatchtheECA'sphysicalformincreasingformdelityfurtherbymakingtheECAsolidinadditiontoperceptuallythree-dimensional.Forexample,robotsandmannequinscanmatchtheintendedphysicalshapeoftheECA.Spatialaugmentedreality[ 14 ]canalsobeusedtodisplayimageson3DsurfacesthatmatchthephysicalshapeoftheECA[ 56 ]. 41

PAGE 42

HighformdelityECAshavebeencreatedbyresearchers,andsomehaveexploredtheeffectofformdelityonsocialpresence.Lincolnetal.createdShaderLampAvatars,adisplaysystemthatprojectsanavatarontoastyrofoamhead[ 56 ].Thestyrofoamheadisa3Dprojectionsurfacethatallowsmultipleviewerstoseeaperspective-correct,3Dimagewithoutspecialglasses.Additionally,thestyrofoamheadismountedonamotorizedplatformthatallowsittoturnandlookatpeople.AttheISMAR2009conference,Lincolnetal.addedamannequinbodyanddemonstratedafulllife-sizeavatar,howevertheydidnotrunanyformalstudieslookingatsocialpresenceorperceptionoftheECA.Rivera-Gutierrezetal.appliedShaderLampAvatarsinamedicalcontext,creatingShaderLampVirtualPatients[ 71 ],whichtheyusedinamedicaleyeexamsimulation.MedicalstudentsusedaWiicontrollerasanophthalmoscopeandotherexamtools.ThoughtheydidnotcomparetheShaderLampVirtualPatienttoalowformdelityversion,theyreceivedfeedbacksuggestingthattheShaderLampVirtualPatient'sphysicalpresenceincreasedtherealismofthepatient.Thisisconsistentwithourhypothesisthatphysicality(ofwhichformdelityisacomponent)ispositivelyrelatedtosocialpresence.Leeetal.exploredtheeffectoflowandhighformdelityonsocialpresence[ 54 ].TheauthorscomparedaphysicalSonyAiboroboticdogtovideosoftheAibodisplayedonamonitor.WhiletheAiboisnotaconversationalagent,theAiboisasocialrobotandsharesasimilarrelationshipwithusers.Theirresultsshowedthatpeopleevaluatedthehighformdelityagentmorepositivelyandthatthehighformdelityagentelicitedahighersenseofsocialpresence.Theseresultssuggestthatformdelitymaybeanimportantfactorthataffectssocialpresence. 3.2.4ConcordancewiththePhysicalSpaceConcordancewiththephysicalspacedescribeshowcloselyanECA'senvironmentisassociatedwiththeuser'sphysicalenvironment.LowconcordanceECAsoccupy 42

PAGE 43

distinctvirtualenvironments.ExamplesoflowconcordanceECAsincludetypicalvideogamecharactersthatoccupyapurelyvirtualenvironmentcompletelydissociatedfromtheuser'sphysicalenvironment.HighconcordanceECAsleverageMRtechnologytoblendtheECAwiththeuser'sphysicalenvironment.Forexample,asee-throughHMDcanoverlayanECAontheuser'sphysicalenvironment.Highconcordancecanalsobeachievedusingspatialaugmentedreality[ 14 ].SpatialaugmentedrealitytechniqueswereusedintheFlatWorldproject[ 65 ],inwhichdisplayswereplacedwithinphysicaldoors.Whenusersopenedthedoors,thedisplaysshowedlife-sizeECAsthatappearedtobestandingontheothersideofthedoor.Concordancecanbeincreasedusingrenderingtechniquesthatincorporatethephysicalenvironment'slighting.Bothphysicalandvirtualobjectscancastcorrectshadowsontoeachother.Techniquesthatcapturethephysicallighting,includingdiffusereectionsoffobjects,canblendthephysicalandvirtualelementsbetter.Dowetal.exploredconcordancewiththephysicalspaceinastudycomparinganARversionofthegameFacadetothedesktopversionofthegame[ 31 ].ARFacadeusedasee-throughHMDtooverlaythegame'svirtualcharactersonaphysicalenvironment.TheresultsshowedthatimmersiveARincreasedpresenceandengagement.Althoughtheirstudydidnotmeasuresocialpresence,webelievethattherearesituationsinwhichitisimportanttofocusontheECAsandtostudysocialpresence. 3.2.5RangeofValidViewpointsRangeofvalidviewpointsistheanalogofeldofregardinvirtualreality.TherangeofvalidviewpointscapturestherangeofphysicalpositionsfromwhichtheECAcanbecorrectlyviewed.AnECAonaatdisplaywithnoheadtrackingcanonlybeviewedcorrectlyfromonephysicalposition,theonecorrespondingtothestaticvirtualcamera.Head-coupledperspectiveincreasestherangeofvalidviewpointsbymovingthevirtualcameratomatchtheuser'sphysicalposition.Aphysicallythree-dimensionaldisplaymayenableafull360-degreerangeofvalidviewpoints.Note,stereoscopicdisplays 43

PAGE 44

withouthead-trackingstillaffordonlyonevalidviewpointbecauseanymovementoftheuserwouldcauseaperspectivemismatch.IncreasingtherangeofvalidviewpointsincreasestheoccupancyofthephysicalspacebecauseitallowstheusertoperceivetheECAasathree-dimensionalentitythatcanbeviewedfrommultipleangles.Tothebestofourknowledge,noresearchershaveexaminedtheeffectsoftherangeofvalidviewpointsonperceptionofECAs. 3.3InteractionwiththeEnvironmentWeidentifytwodimensionstotheinteractionwiththeenvironmentcategory:theECA'sawarenessofchangestotheenvironmentandtheECA'sabilitytochangetheenvironment.Weconsidertheenvironmenttoincludethephysicalandvirtualspacesandanythingcontainedinthesespaces,includingentitiessuchasobjects,otherECAs,andusers.Toclassifyentities,weleverageMilgramandKishino'svirtualitycontinuum[ 61 ]andapplyittoindividualentities.Entitiesmaybepurelyvirtual,purelyphysical,oramixofvirtualandphysical.Notethatweconsideronlywheretheentitiesfallonthevirtualitycontinuumanddonotconsidertheuser'sinterfaceformanipulatingentities.TechnologiessuchastheNintendoWiiremote,PlayStationMove,andMicrosoftKinectcanincreasethephysicalityoftheuser'sinteractionwiththeobjectsbutdonotaffectthephysicalityoftheECA'sinteractionwiththeobjects.Interactionwiththeenvironmentonlyincludes`physical'changestotheenvironment,thatischangestopropertiessuchaspositionandorientation.Non-physicalinteractionssuchasconversationarenotincludedasdimensionsofphysicality.WhileconversationisclearlyanimportantpartofanECAandmayalsoaffectsocialpresence,conversationisbeyondthescopeofthiswork.WedrawonresearchintothephysicalityofuserinputinterfacestoformhypothesesaboutECA'sinteractionwiththeenvironmentphysicality.Usohetal.foundapositive 44

PAGE 45

relationshipbetweenuserinterfacephysicalityandpresence[ 86 ].Theirresearchcomparedrealwalking,walkinginplace,andyingastravelmethodsinavirtualenvironment.Theyfoundbothwalkingmethodshadstatisticallyhigherpresencethanying.Theyalsofoundthatrealwalkinghadhigherpresencethanwalkinginplace,thoughthedifferencewasnotalwaysstatisticallysignicant.Similarly,weexpectthatincreasingthephysicalityofanECA'sinteractionwiththeenvironmentmayincreasesocialpresence. 3.3.1AwarenessofChangesAwarenessofchangesreferstoanECA'sabilitytosenseandrespondtochangesinanotherentity'sposition,orientation,oranyotheraspectofphysicalstate.LowawarenessECAsareonlyawareofchangestopurelyvirtualentities.HighawarenessECAsareawareofchangestoatleastsomepurelyphysicalentities.Purelyphysicalentitiesincludetheuserandtrackedphysicalobjects.Researchshowsawarenessofchangestotheenvironmentcanincreasesocialpresenceandenrichcommunication.CreatingECAswithhighawarenesshashistoricallyhadmanybarriers,whichincludesettingupandcalibratingexpensivetrackingsystems,attachingtrackingducialstoobjectsandusers,andinstrumentingobjectswithsensors.However,recentadvancesintrackingtechnologyhaveloweredtheentrybarriers.Forexample,theMicrosoftKinectcantracknotonlytheuser'sphysicalpositionbutalsothepositionofvariousbodypartswithouttheaidoftrackingducials.TheKinectcanbeusedwithgesturerecognitionlibrariessuchasFAAST[ 82 ].Gesturerecognitioncouldfacilitateadditionalnon-verbalcommunicationinsocialinteractionswithECAs.Forexample,Fujieetal.createdaconversationalrobotthatcoulduseheadgestures(e.g.nodding)tosupplementverbalcommunicationandcorrectlyinterpretambiguousverbalinput[ 32 ].Othersensorssuchasaccelerometers,gyroscopes,andcamerashavebecomecheapandcommonplace.Thishasenabledthespreadofmotion-sensingdevicessuchastheNintendoWiiremote,PlayStationMove,andmostsmartphones.Thesedevices 45

PAGE 46

providenewmethodsforuserstomanipulateobjects.Becauseofthesenewlyloweredentrybarriers,webelievemanyfutureECAswillbeawareofchangestotheusersandobjectsanditisimportanttostudytheeffectsofawarenessonsocialpresence.Existingresearchshowsthatawarenessoftheuser'sphysicalpositioncanincreaseanECA'ssocialpresence.Bailensonetal.comparedvirtualavatarsthatmaintainedeyecontactwithusers(demonstratingawarenessoftheuser'sposition)tovirtualavatarsthatlookedstraightahead[ 7 ].Theyfoundthatwhenavatarsmaintainedeyecontact,usersmaintainedalargerinterpersonaldistance(apositivemeasureofsocialpresencerelatedtonotviolatingpersonalspace).ObjectmanipulationcanenrichECAinteractionsandprovidealternativecommunicationchannels.Forexample,Kotranzaetal.usedaNintendoWiiremotetomanipulatevariousvirtualtoolsinavirtualeyeexam[ 53 ].Theyfoundthatthenon-verbalinteractionwaseffective,efcient,andsatisfactory.Additionally,itmitigateddissatisfactionwitherrorsinthevoice-recognitionsystem. 3.3.2AbilitytoManipulateAbilitytomanipulatereferstoanECA'sabilityto`physically'changeanotherentity'sposition,orientation,oranyotheraspectofstate.By`physically'wemeananysortofphysics-basedcontactbetweentheECAandanotherentity,includingapurelyvirtualECAtouchingapurelyvirtualobject.BecausemostECAsexistaspurelyvirtualentities,evenwithinMRenvironments,mostECAscanonlymanipulatevirtualobjectsoratmostthevirtualcomponentsofMRobjects.Manipulatingphysicalobjectstypicallyrequiresactuatorsormotorswhichcanaddsignicantcomplexity.CheapECAswithmanipulationabilitiesequaltoarealhumanarestillalongwayoff,butlimitedmanipulationofphysicalobjectsisachievable.Forexample,Kotranzaetal.createdanECAthatcouldphysicallytouchtheuser[ 52 ].TheECAwasamannequinequippedwithabasicmotorizedarm(twoservomotorsreplacedtheoriginalmannequinarm'sjoints).ThisECAwasusedinaMR 46

PAGE 47

medicalbreastexamwhereanHMDoverlaidavirtualpatientonthemannequin.Tostudytheeffectofthephysicaltouch,Kotranzaetal.comparedapurelyvirtualtouch(wherethemotorizedarmdidnotmove)toavirtualandphysicaltouch(whereboththevirtualandmotorizedarmsmoved).Theyfoundthatthehigherphysicalitytouchcommunicatedmoreeffectivelyandsimulatedthepatientbetter. 3.4IntertwiningofTechnologyandDimensionsofPhysicalityManydimensionsofphysicalityaretightlycoupledbecausethetechnoloigesthatincreasephysicalityinonedimensionmaysimultaneouslyaffectotherdimensions.Insomecases,thiseffectonotherdimensionswillbeanincreasewhileinothercasesitwillbeadecrease.Forexample,avideosee-throughHMDincreasesconcordancebyblendingtherealandvirtual.TheHMDalsosupportsrenderinganECAatlifesizeanywhereintheuser'seld-of-viewandfromanyangle,henceprovidinghighpositiondelity,highsizedelity,andawiderangeofvalidviewpoints.Asanotherexample,considerusingastandarddepartmentstoremannequinasanECA.Themannequinhashighoccupancyofthephysicalspace,butthemannequincannotmoveandisunabletomanipulateeitherphysicalorvirtualobjects.Table 3-1 listsexamplesoftechnologiesandwhichdimensionsofphysicalitytheyaffect.Thisisbynomeansanexhaustivelistofenablingtechnologies,andmanyofthesetechnologiescanbecombined.Theintertwiningoftechnologiesanddifferentdimensionsofphysicalitycanmakeitdifculttoselecttheappropriatetechnologyforanapplication.Webelievethisselectioncanbeaidedbyidentifyingthedimensionsofphysicalityandconductingresearchexaminingeachdimension'seffectsonsocialpresence. 47

PAGE 48

Figure3-1. AtaxonomyforthedimensionsofECAphysicality. A BFigure3-2. Positiondelitycomparison.A)Highpositiondelity.B)Lowpositiondelity. 48

PAGE 49

Table3-1. Examplesofhowtechnologiesmayaffectmorethanonedimensionofphysicality. TechnologySizePositionFormConcordanceViewpointsAbilitytowanipulate See-throughHMD1xxxxx2Head-coupledperspectivexxStereoscopicdisplay3xxSpatialARxxxx Note:Xindicatesatechnologyhasaneffectonadimension.Awarenessofchangestothephysicalspacehasbeenleftoffthischartbecausetechnologiesaffectingawarenesstypicallydonotaffectotherdimensions.1Withheadtracking2Manipulationofvirtualobjects3Withoutheadtracking 49

PAGE 50

CHAPTER4OCCUPANCYOFTHEPHYSICALSPACEThischapterdescribesapairofstudiesexaminingtheeffectsofanECA'soccupancyofthephysicalspaceonsocialpresence.OnestudywaspublishedintheproceedingsoftheIEEEconferenceonVirtualReality2012[ 26 ].AnextendedversionofthatpublicationwassubmittedtothejournalPresence:TeleoperatorsandVirtualEnvironments.ThejournalsubmissionexpandsupontheVirtualRealitypaperbyincludingthesecondstudyandthedimensionsofphysicalitydescribedinChapter 3 .Thejournalsubmissionwasacceptedbuthasnotyetbeenpublished.Muchofthischapter'stextissharedwiththejournalsubmission.PersonalContributions:Idesignedandconductedthestudyandanalyzedthedata.Ialsodesignedandimplementedthesee-throughdisplayandphysicallegcomponentsofANDIthatenablehighconcordanceandhighformdelityrespectively.Iwrotealltextsharedbetweenthischapterandthejournalarticle.Collaborators:Dr.AdamWendling,Dr.CaseyWhite,Dr.SamsunLampotang,andAndrewRobbcontributedtothestudydesignforbothstudies.Dr.AdamWendlingprovidedsubjectmatterexpertiseonthemedicalscenariosusedinbothstudiesandrecruitedparticipants.AndrewRobbhelpedconductbothstudies;inparticular,heremotelycontrolledthevirtualhumansinbothstudies.AndrewalsowiredthemotorandcircuitboardthatenableANDI'sphysicallegstorotate.RelevancetoThesis:Thischapterexplorestheeffectsonecategoryofphysicalitydimensions-occupancyofthephysicalspace-onsocialpresence. 4.1OverviewToexploretherelationshipbetweenoccupancyofthephysicalspaceandsocialpresence,werstdevelopedanECAwithhighoccupancyofthephysicalspacethenrantwouserstudiescomparingECAswithdifferentlevelsofoccupancy.BothuserstudiestookplaceinamedicalteamtrainingcontextandinvolvedvirtualhumanECAs. 50

PAGE 51

Asaninitialexplorationintotherelationshipbetweenoccupancyandsocialpresence,therststudy(n=18)wasaholisticevaluation.TheholisticevaluationvariedseveraldimensionsofphysicalityatonceandcomparedapairofhighphysicalityECAstoapairoflowphysicalityECAs.Theholisticevaluation'sresultssuggestedincreasingphysicalityincreasedsocialpresenceandelicitedmorerealisticbehavior.Theholisticevaluationalsosuggestedphysicalitymayhaveaninteractioneffectwithplausibility(behavingandreactingasarealhumanwould).Inordertobetterunderstandtheroleofspecicdimensionsofphysicalityinincreasingsocialpresence,weconductedanindependentcomponentsevaluation(n=29).Theindependentcomponentsevaluationindividuallyvariedtwodimensionsofphysicality,formdelityandconcordance(seeSection 3 ).Resultsfromthisindependentcomponentsevaluationsuggestednotonlythatindividualdimensionsofphysicalityaffectsocialpresencetodifferentextents,butalsothatmatchinglevelsofphysicalityareimportantforincreasingsocialpresence. 4.2DevelopinganECAwithHighOccupancyToachievehighphysicalityatareasonablecost,wedevelopedanovelhybridvirtual-physicalvirtualhumansystemcalledANDI(ANimatronicDIgitalavatar)[ 24 ].ANDIcombinesMRtechnologywithphysical,animatronicpropstocreateahighphysicalityvirtualhumanthatoccupiesthephysicalspace(Figure 4-1 ).ANDIisnotthehighestphysicalityECApossiblebutratherabalancebetweencost,highphysicality,andexibility.ANDIusesentirelyoff-the-shelfcomponentsandcostsapproximatelyUS$3000includingalaptop. 4.2.1IncreasingOccupancyofthePhysicalSpaceANDIachieveshighsizedelitybydisplayingthevirtualhuman'supperbodyatlife-sizeona40,1920x1080LCDTVinportraitorientation.TheTVismountedtoawheeledcartthatallowsANDItobepositionedasappropriateinthephysicalspace,achievinghighpositiondelity. 51

PAGE 52

Toachievehighconcordancewiththephysicalspace,ANDIcreatestheillusionofasee-throughdisplay.Thesee-throughdisplaycombinesaMicrosoftKinecttrackingsystem,head-coupledperspective,andapanoramicphotooftherealenvironment.ThepanoramicphotocapturestheenvironmentbehindANDIandistakenfromtheuser'sexpectedpositionintheenvironment.Thephotobecomesthetextureonavirtualportionofacylinderplacedbehindthevirtualhuman.Whenthecylinderisrenderedusinghead-coupledperspective,thephotoisapproximatelyregisteredwiththerealenvironmentaroundtheTV.Thiscreatestheillusionofasee-throughdisplayandplacesthevirtualhumaninphysicalspace.Head-coupledperspective(alsoknownasFishTankVR[ 92 ])increasedphysicalitybyprovidinganillusionofdepthonthe2Ddisplays.Head-coupledperspectiveplacedthevirtualcameraattheuser'sphysicalpositionandmodiedtheprojectionmatrix.ThiswarpedtheimageandallowedtheusertoseedifferentpartsoftheECAasheorshemovedaroundtheroom.Stereoscopic3DwasnotusedbecausetheactiveLCDshutterglassesdidnotworkwiththeTVinportraitorientation.ANDIincreasesformdelitybyincludingphysical,animatronicproplegs.Thelegsconsistofapairofpantslledwithpillowstufng.ThepantsalsophysicallymovewhentheECAmoves.WhentheECAturnstoadministerdrugstothepatientortalktotheotherECA,asteppermotorrotatesthetopofthepants.Thisrotationmatchesthephysicallegsmovementstothevirtualupperbody'smovements(Figure 4-2 ).Thephysicallegsandvirtualupperbodywerekeptvisuallyalignedbyhead-coupledperspective. 4.2.2ComparisontoOtherSystemsComparedtoalternativehighoccupancysystemssuchassee-throughHMDsandrobots,ANDIischeaper,moreportable,andmoreexible.ThesecharacteristicsenhanceANDI'ssuitabilityforinsituteamtraining. 52

PAGE 53

ANDIcostslessthan$3000andusesentirelyoff-the-shelfcomponents.AnANDIunitcanbebuiltinafewhours.ThemostexpensivecomponentsofANDIaretheTVandlaptop,bothofwhicharegeneral-purposeequipmentandcanbereusedelsewhere.Bycontrast,fully-articulatedrobotssuchastheRoboThespiancost$80,000,requiremuchmorecomplexassembly,andarespecial-purposeequipment[ 1 ].TomakeANDIportable,alloftheequipmentisself-containedandmountedontoawheeledstand.TheKinectismountedtothetopoftheTVandcanautomaticallycalibrateitspositionandorientationbydetectingthegroundplane.Thiseliminatestheneedforinstallingandcalibratingexternaltrackingsystemsintheenvironment,animportantrequirementfordeploymentinactualclinicalenvironments.ANDI'sdesignalsoeliminatestheneedforrunningpowerandcommunicationcableseverywhere.Cablingcanpresentaserioustriphazardinfunctioningclinicalenvironments.ANDIisexibleinthatitcandisplayawidevarietyofcharacterswithonlyminimalchanges.Likesee-throughHMDsbutunlikerobots,ANDI'sgender,weight,andfacialfeaturescanbechangedinsoftwarebyloadingdifferentvirtualhumanmeshes.Oftentimes,thephysicallegsdonotneedtobechangedbecausedifferentcharacters,e.g.afemalenurseandamaledoctor,sharethesameappearancefromthewaistdown.Ifthephysicallegsdoneedtobechanged,itisasimplematterbecausetheyareattachedbyasingleeasilyaccessiblescrew.Thiscombinationoflowcost,highportability,andhighexibilitymakeANDIsuitableforthethevisionofondemandteamtraining.Onafewminutesnotice,asmanyANDIunitsasneededcouldbeloadedwithappropriatemeshesandrolledintoaninsitutrainingenvironment.Thiswouldhelpwiththeproblemofcoordinatingthedemandingschedulesofextremelybusymedicalprofessionals. 4.3HolisticEvaluation:ExploringOccupancyasaWholeTheholisticevaluationexploredwhetheroccupancyofthephysicalspacehadaneffectonsocialpresence.Asaninitialexploratorystudy,theholisticevaluationvaried 53

PAGE 54

severaldimensionsintheoccupancyofthephysicalspacecategorysimultaneously.Theholisticevaluationusedabetween-subjectsdesignwithtwoconditions,highphysicalityandlowphysicality.Theholisticevaluationtookplaceinthecontextofamedicalteamtrainingexercise.Theteamtrainingexerciseinvolvedananesthesiaprovider(theparticipant)workingwithanurseECAtotreatapatient.TheexercisealsorequiredtheparticipanttointeractwithanotherECA,thepatient'sadultdaughter. 4.3.1ConditionsThetwoconditionsvariedthephysicalityofthenurseanddaughter.Thenurseanddaughteralwayshadmatchingphysicality(i.e.,highandhighorlowandlow).Table 4-1 summarizesthephysicalitydifferencesbetweentheconditions.Notethatwhileweusetheadjectives`high'and`low'todescribetheconditions,theconditionsdonotrepresentthehighestandlowestpossiblephysicalities.Insteadtheconditionsrepresenthighandlowphysicalityrelativetoeachother.Inbothconditionsthepatientwasamannequinpatientsimulator.Amannequinpatientsimulatorissimilartoadepartmentstoremannequinbutconsumesoxygen,producescarbondioxide,andhasapalpablepulseamongotherfeatures.Thesefeaturesallowunmodiedmedicalequipmenttobeuseddirectlywiththesimulator.Mannequinpatientsimulatorsarecommonlyusedtoteachphysiology[ 42 ]buthavealsobeenusedinteam-trainingexercises[ 41 ]. 4.3.1.1HighPhysicalityNurseandDaughterThehighphysicalityconditionusedapairofANDIsystemstocreateasensethatthevirtualhumanssharedthephysicalspacewiththeuserratherthaninhabitingaseparatevirtualspace(Figure 4-1 ).EachANDIsystemwasplacedinanappropriateplacebythepatient'sbed. 4.3.1.2LowPhysicalityNurseandDaughterThelow-physicalityconditionwasdesignedtocreateasensethatthevirtualhumansinhabitedavirtualspace,ratherthantheuser'sphysicalspace(Figure 4-3 ). 54

PAGE 55

Thelow-physicalityconditionalsoattemptedtouseonlyequipmentcommonlyfoundinmedicalteam-trainingenvironments.Toaccomplishthis,thelow-physicalityconditionusedaprojectionscreen,avirtualbackground,andxedperspective.Asingle1024x768frontprojectionsystemdisplayedthetwovirtualhumans,resultinginlowformdelity.Theprojectorscreenmeasured50diagonallyandwasmountedtothewall40abovethegroundand50awayfromwherearealnurseanddaughterwouldhavestood,resultinginlowpositiondelity.Thepatientandprojectionscreenwereparallel.Thenurseanddaughtervirtualhumanseachoccupieda22x30areaoftheprojectionscreen.Thisareawaslargeenoughtodisplaytheupperportionofthevirtualhumanatslightlylargerthanlife-size,resultinginslightlylowersizedelitythanthehighphysicalitycondition.Thevirtualenvironmentcreatedlowconcordancewiththephysicalspacebyusingabackgrounddisjointedfromthephysicalroom.Thebackgroundwasaprivacycurtain,acommonfeatureinpatientcareunitswherethemedicalscenariotypicallyoccursinreallife.However,thephysicalroomwasnotapatientcareunitbutratherasimulationroomwherescenariosaretypicallypracticed.ThisplacedtheECAsinadistinctvirtualenvironmentwithnoconcordancewiththeuser'sphysicalenvironment.Fixedperspective,ratherthanhead-coupledperspective,furtheremphasizedthelowlevelsofalldimensionsofphysicality.Forxedperspectivethevirtualcameraremainedataxedpositionthatdidnotchangeastheusermoved.Fixedperspectivealsodidnotmodifytheprojectionmatrix.Head-coupledperspectivewouldhaveincreasedtherangeofvalidviewpointsandalsowouldhavecreatedtheillusionofvolumebehindthedisplay,whichwouldhaveincreasedphysicality.Additionally,unlikethehigh-physicalitycondition,therewerenophysicalelementsthatneededtobevisuallyalignedwiththevirtualelements. 55

PAGE 56

4.3.2WizardofOzTheECAswerecontrolledviaaWizardofOz(woz)setup.Inawozsetup,aconcealedhumanoperator,ratherthananarticialintelligencesystem,controlstheECAs.WozcontrolhasbeenusedinmanystudiesinvolvingECAsandnaturallanguageunderstandingsystems[ 28 31 44 58 ].Wozwaschosentoeliminateconfoundsthatcouldbeintroducedbyanarticalintelligencesystem.Articialintelligencesystemssometimesproduceincorrectresponsesduetospeech-to-texterrorsornaturallanguageunderstandingerrors.Wozalsoallowedforamorenaturalinteraction,removingtheneedforuserstowearamicrophone.ParticipantsweretoldthattheECAswerecontrolledbyacomputerandnotahumanoperator.ThiswasdonebecauseBailensonetal.foundthatagency(beliefthattheECAiscontrolledbyahumanorcomputer)hadaneffectonsocialpresence[ 6 ].Socialpresencewashigherforavatarsbelievedtobecontrolledbyhumans.ParticipantawarenessofECAcontrolbyhumanscouldpotentiallyhaveraisedsocialpresenceenoughtomasktheeffectsofphysicality.Thehumanoperatorworkedfrombehindascreen,acommonitemintheseenvironments.Theoperatorwasabletoseetheparticipantsthroughastreamingcamerafeedandwasabletoheartheparticipantsdirectly.Theoperatorwasabletorespondtoparticipantspeecheswithinhalfasecondinmostcases.Thisspeedwaspossiblebecausetheresponsecouldusuallybedeterminedbeforetheparticipantnishedspeaking.Insomecases,adelayoftwotothreesecondswasintroducedbytheneedfortheoperatortondthecorrectresponsetoplay.Thisdelaywasreducedbygroupingresponsesbycategory,reducingthenumberofresponsesneedingtobesearchedbytheoperator.Theoperatorchosefromatotalof121possibleresponsesforthenurse,25forthepatient,and36forthedaughter.Theoperatortypicallyissuedthecorrectresponseandissuedatmostoneincorrect,non-sensicalresponseperparticipant. 56

PAGE 57

Theoperatorwasfamiliarwiththescenarioandthesetofresponses.However,theoperatorwasnotamedicalprofessionalanddidnotknowtheanswertoeveryquestion.Inthesecases,thecharactersrespondedwithI'mnotsureorIdon'tknow.Thishappenedonaveragefourtimesperparticipant.Insomecases,theoperatorknewtheanswerbuttherewerenoaudiolesfortheseanswers.Inthesesituationsthecharacterdidnotrespondtothequestion.Duringtheintroduction,userswereinstructedtocontinuewithanotherquestionifthisoccurred.Thisoccurredonaverageonceperthreeparticipants.Somewozsystems,suchasKlemmeretal.'sSuedesystem,purposelyintroducearticialerrors[ 49 ].Thesearticialerrorssimulatetheerrorsanautonomous,non-wozsystemmightproduce.Wechosenottointroducearticalerrorsbecausewecouldnotpredicthowmanynon-articialerrorsaparticularparticipantwouldencounter.Introducingarticialerrorswouldraisetheerrorratebutwouldnotguranteeaconsistenterrorratebetweenparticipants. 4.3.3ParticipantsTheuserpopulationfortheholisticevaluationconsistedofmedicalprofessionalsspecializinginanesthesiaatUF&ShandsHospitalinGainesville,Florida.Theseprofessionalspossessedtheknowledgeandexperiencenecessarytocompletethecriticalincidentscenario(Section 4.3.4.2 ).Dr.AdamWendling,afacultymemberoftheDepartmentofAnesthesiology,recruitedparticipantsoveraperiodoftwoweeks.Recruitedparticipantswereaskediftheywerewillingtoassistwithateamtrainingexercise.Ifso,theyweretemporarilyrelievedoftheirclinicaldutiestoparticipateintheexercise.Intotal,23medicalprofessionalsparticipatedinthestudy.Threeparticipantswereremovedduetotechnicaldifculties.Twoadditionalparticipantswereremovedbecausetheywereoutliersandnotrepresentativeofthepopulation.Atleast47%oftheirself-reportsocialpresenceratingswerebeyondtwostandarddeviationsoftheaverage, 57

PAGE 58

andtheyrepeatedlyexpressedadislikeforsimulationexercises.Removingtheseparticipantsresultedinanalnof18.Thirteenwereresidents(doctorswhograduatedfrommedicalschoolandareintheirsecondorthirdyearofspecializedtraining),onewasanintern(adoctorwhohasgraduatedfrommedicalschoolandisintherstyearofspecializedtraining),twowereattendings(doctorswhohavecompletedtheirspecializedtraining),andtwowerecertiedregisterednurseanesthetists(advancedtrainednursesspecializinginanesthesia).Allparticipantswereexpectedtohaveencounteredrealscenariosintheirprofessionalcareerssimilartothesimulatedone.Residentswerethetargetoftheteam-trainingexercise,andasaresultmadeupthemajorityoftheparticipants.Participantsrangedfrom28to40yearsold,withanaverageageof33.Sixteenparticipantsweremale,andallbutoneparticipantspokeEnglishastheirprimarylanguage.Participantswerealternatelyassignedtothehighandlowphysicalityconditions.Excludingparticipantsremovedduetotechnicaldifcultiesandoutliers,thehighphysicalityconditionhadeightparticipantsandthelowphysicalityconditionhadtenparticipants. 4.3.4ProcedureBothgroupsfollowedidenticalstudyprocedures(Figure 4-4 ).Thestageswere: CONSENT.Completedaninformedconsentformandwerebriefedaboutthegoalsandscopeofthetrainingexercise. TUTORIAL.Virtualnurseexplainshowtointeractwiththevirtualhumans DEMOGRAPHICSSURVEY.Completedademographicsandbackgroundsurvey CRITICALINCIDENT.Virtualnurseanswersquestionsaboutthepatient,administersdrugswheninstructed,andsuggestscoursesofaction.Virtualfamilymemberinterjectswithquestionsaboutthepatient'sstatus. CRITICALINCIDENTSURVEY.Completedafeedbacksurveyandfree-forminterview 58

PAGE 59

4.3.4.1TutorialPilottestingshowedthatmanypeoplewereatrstuncomfortablewiththevirtualhumans.Thisledtodifcultycommunicatingwiththem,confusionaboutthevirtualhumancapabilities,and,inthecaseofthehigh-physicalityvirtualhuman,surprisewhenthemotorizedlegsrstmoved.Toreducethisdiscomfort,theparticipantsrstwentthroughatutorialwheretheyinteractedwiththevirtualhumannurse.Thenursepromptedtheparticipanttoaskaquestion,thenprovidedaresponse.Thenursealsodemonstratedhisabilitytomoveandadministerdrugstothepatientwheninstructed.Afterthis,thenurseexplainedhecannotmovephysicalobjectsandaskedtheparticipanttoconnectanoxygenmasktoanearbyoxygentank.Thisprovidedtheparticipantwithpracticecommunicatingandanunderstandingofthecapabilitiesandlimitationofthevirtualhuman.Familiaritywiththemedicalequipment,includingthemannequinpatientsimulator,wasassumed.Thisexactmannequinwasusedbyallresidentsintheirprioryearasinterns,andallotherparticipantshadexperiencewithsimilarmannequinpatientsimulators.Themannequinwasconnectedtoanactualanesthesiamachineidenticaltotheoneparticipantsusedintheirjobs. 4.3.4.2CriticalIncidentTheproctortoldparticipantstheywouldbeperformingateamtrainingexerciseinvolvingvirtualhumans(nodistinctionwasmadebetweenECAandvirtualhuman).Heintroducedthevirtualhumansasthenurse,thepatient,andthepatient'sfamilymember.Theproctorgavenonamestothevirtualhumansbutdidexplainthenursewasthesamenursefromthetutorial.Finally,theproctorexplainedthemedicalscenarioandinstructedparticipantstheirtaskwastoassessthesituationandtakeanappropriatecourseofaction.Thegoalsofmedicalteamtrainingexercisessuchasthisonearetopracticehowto: 59

PAGE 60

workefcientlywithteammembers identifyamedicalconditionquickly treatthemedicalconditioncalmlyandcorrectlyThisspecicexercisefocusedondiagnosisandearlymanagementofperioperativemyocardialinfarction.Thisconditionoccursinapproximately5%ofpatientsundergoingmajorsurgeryandincreasestherelativeriskofdeathbymorethanfourtimes[ 30 ].Duetothecondition'scriticalnature,medicalprovidersmustrecognizeitandactquickly.Providersmustdirectateamofproviderstotreattheconditionandpreventseveremorbidityandpossiblymortality.Duringtheexercise,theparticipantrstfoundthepatientdisplayingsymptomsthatcouldhaveindicatedavarietyofpossibleconditions.Todiagnosethecorrectcondition,theparticipantshouldhavethengatheredadditionalsymptomsandhistoryfromthepatient,nurse,anddaughter.Next,theparticipantshouldhavedirectedthenursetoperformspecictaskssuchasorderinglaboratorystudiesandadministeringmedications.Iftheparticipantdidnotperformcertainnecessaryactionsbypresettimes,thenursesuggestedthoseactions.Providingthesesuggestionsisoneofthedutiesofanurseinarealmedicalteam.Whiletheparticipantwasdiagnosingandtreatingthepatient,thedaughterinterjectedwithquestionsthatoccurredatpresettimes.Forexample,thedaughteraskedWhyisnothingworking?Questionslikethiscreatedempatheticopportunitieswheretheparticipantcouldcalmorcomfortthedaughter.Theexerciseendedwhenoneoftwothingsoccurred.One,theparticipanttookthenecessaryactionstotreatthepatientortwo,thenursesuggestedallthenecessaryactionsandtheparticipantchosenottofollowthesuggestions.Inbothcases,thenurseinformedtheparticipanttheexercisewascomplete. 60

PAGE 61

4.3.5MetricsWeusedacombinationofself-reportsurveys,observedbehaviorsinvideorecordings,andpost-exerciseinterviewstoassesssocialpresence,realisticbehavior,andplausibility. 4.3.5.1Self-ReportSocialPresenceAvequestionseven-point(-3to3)Likert-typesurveyassessedsocialpresenceforeachECA.ThissurveywastakenfromBailensonetal.[ 6 ]withminormodicationsforclarity.Therstoccurrenceoftheword`person'ineachquestionwasreplacedbytheroleoftheECA(nurseorfamilymember).Weabbreviatedthequestionsforreportingintablesandguresasfollows: INROOM.IperceivethatIaminthepresenceofanurse/familymemberintheroomwithme. WATCHING.Ifeelthatthenurse/familymemberiswatchingmeandisawareofmypresence. NOTREAL.Thethoughtthatthenurse/familymemberwasnotarealpersoncrossedmymindoften. SENTIENT.Thenurse/familymemberappearedtobesentient,conscious,andalivetome. COMPUTERIZED.Iperceivedthenurse/familymemberasbeingonlyacomputerizedimage,notasarealperson. 4.3.5.2RealisticBehaviorWeassessedrealisticandnon-realisticbehaviortowardstheECAsusingvideorecordingsofthecriticalincidentexercise.Behavioralmetricswerecollectedinadditiontoself-reportsocialpresenceontherecommendationofBailensonetal.[ 8 ].Behavioralmetricscouldincludegazebehavior,interpersonaldistance,andconversationalfeatures.Ofthese,weusedonlyconversationalfeaturesbecausetheexercise'sfocusonpatientcarepotentiallyinuencedgazebehaviorandinterpersonaldistance.Specically,wenotedhowtheparticipantsaddressedtheECAs.Theparticipants 61

PAGE 62

addressedtheECAsusingwordssuchasma'amorfamilymember.Toanalyzehowrealistictheparticipantsbehaved,weclassiedwordsusedtoaddresstheECAsintothreemannersofaddress:non-realistic,realisticbutimpersonal,andrealisticandpersonal.Useofwordswasanalyzedasabinarymetric;aparticipanteitherusedthatwordtoaddresstheECAornot.Thisdatawasanalyzedusingalikelihoodratio.Onepossiblebehavioralmetricisiftheparticipantchosetofollowthenurse'sadviceornot.Wechosenottousethismetricfortworeasons.First,someparticipantsdidnotreceiveanyadvicebecausetheyorderedthetreatmentsearlyenough.Second,ourparticipantshadarangeofeducationalstatusesandfamiliaritieswiththisscenario. 4.3.5.3PlausibilityIllusionWeassessedplausibilityillusionusingqualitativepost-exerciseinterviewquestions.WewereunabletousethephysiologicalmetricsusedbySlateretal.becauseoneofthegoalsofmedicalteam-trainingistoreduceanxiety.Slateretal.studiedplausibilityillusioninvirtualenvironmentsdesignedtogenerateanxiety[ 78 ].Lowanxietyinourstudycouldindicatefailuretoachieveplausibilityillusion,butitcouldalsoindicateachievingplausibilityillusionwithanexperiencedclinicianwhoisunstressedbythescenario.Duringthepost-exerciseinterview,weaskedparticipantsWhatdidyouthinkofthesimulation?IfparticipantsdidnotmentionspecicECAs,weaskedaboutthoseECAsspecically(i.e.,Whatdidyouthinkofthenurse?).Thesequestionswereleftopen-endedtogettheparticipant'sstrongestimpressionwithoutbiasingthemtowardsanyonequality.Weacknowledgethatthesequestionscanonlyprovidearoughapproximationofplausibility.InChapter 6 weexaminetheideaofECAplausibilityinmoredetail. 4.3.6ResultsOurresultssuggestedthatincreasingphysicalitycanincreasesocialpresence.However,self-reportsocialpresenceresultsdifferedbetweenthenurseanddaughter 62

PAGE 63

ECAs.Thisdifferencemaybeexplainedbyaninteractioneffectofphysicalityandplausibility.Theresultsforself-reportsocialpresencewereanalyzedusinganon-parametricANOVA-typestatisticforeachECAwithphysicalityasabetween-subjectsfactor.Non-parametricanalysiswaschosenpertherecommendationofKapteinetal.forLikert-typedatawithsamplesizeslessthan50[ 47 ].Forbetween-subjectsfactors,theBoxapproximationwasalsousedforsmallsamplesizespertherecommendationofBrunneretal.[ 20 ],whichresultsinafractionaldenominatordegreesoffreedom.AnalysiswasdoneusingRandthenparLDpackage. 4.3.6.1Self-ReportSocialPresenceThenurseECAsupportedthehypothesisthatincreasingphysicalityincreasedsocialpresence.Thehighphysicalityconditionhadhigheraverageself-reportsocialpresencescoresforallveitems(Figure 4-5 ).Resultsshowthatparticipantsperceivedthenursetobesignicantlymoresentient,conscious,andalive(F(1,14.674)=8.312,p<0.05,highphysicality(M=1.63,SE=0.32),lowphysicality(M=0.56,SE=0.29)).AmarginallysignicantdifferencefavoringthehighphysicalityconditionwasalsofoundfortheInRoomitem(F(1,13.725)=3.316,p=0.09,highphysicality(M=1.63,SE=0.39),lowphysicality(M=0.78,SE=0.32)).Theotherdifferenceswerenotstatisticallysignicant(allp>0.37).ThedaughterECAdidnotsupportthehypothesisthatincreasingphysicalityincreasedsocialpresence.Neitherconditionhadhigherscoresforallveself-reportsocialpresenceitems(Figure 4-6 ).Therewerenosignicantdifferencesbetweenthetwoconditionsforanyoftheveself-reportsocialpresenceitems(allp>0.18). 4.3.6.2AddressingECAsTheresultsformannersofaddresssuggestincreasingphysicalityresultsinmorerealisticbehavior.Table 4-2 summarizesthewordsusedtoaddresstheECAsandhowmanyparticipantsusedeachword.Threeoutoftenlowphysicalitycondition 63

PAGE 64

participantsaddressedthedaughternon-realistically,addressingherbyherroleratherthanbyhername.ForexampleoneparticipantsaidFamilymember,doeshehaveanyheartproblems?Nohighphysicalityparticipantsusedthisnon-realisticmannerofaddress,astatisticallysignicantdifference(p<0.05).Theuseofrealisticbutimpersonalmannersofaddresswiththenursefurthersupportthehypothesisthatincreasingphysicalityelicitsmorerealisticbehavior.Threeoutoftenlowphysicalityconditionparticipantsaddressedthenurseasnursewhilenohighphysicalityparticipantsdidso(p<0.05).Participantsinbothconditionsaddressedthepatientanddaughterinarealisticbutimpersonalmannersuchassir(p=0.81,low=70%,high=75%)andma'am(p=0.50,low=40%,high=25%).Participantsinbothconditionsusedrealisticandpersonalmannersofaddresswithboththenurseandthepatient.ParticipantsaddressedthenurseasMatt(p=0.60,low=50%,high=63%).ThenurseintroducedhimselfasMattduringthetutorial,andhealsoworeanamebadgewithhisrstnameinlargeprint.Furthermore,participantsaddressedthepatientasMr.Starks(p=0.50,low=40%,high=25%).ThepatientwasintroducedasMr.PhilipStarksbythenurseatthebeginningoftheexercise.However,noparticipantsaddressedthedaughterinarealisticandpersonalmanner.Onlyoneparticipantaskedforhername,andnoneaddressedherasMs.Starksoranyothervariantofhername. 4.3.6.3PlausibilityIllusionPlausibilityillusionwasachievedbythenurseECAbutnotthedaughterECA.Thenursedidhavesomeminorproblems;oneparticipantexpectedthenursetorespondwithmoreinformation.However,nonefoundthisdistractingenoughtoconsiderthenursenotplausible.Thenursespokefrequentlyinresponsetotheparticipant'squestionsandinstructions.Further,heplayedtheroleofaclinicianperformingajobandhencewasnotexpectedtobeemotional. 64

PAGE 65

Duringpost-exerciseinterviews,participantscommentedonboththedaughter'sinclusioninthescenarioandthelackofemotioninherresponse.Sixparticipants(33%)foundthedaughternon-plausible.Familymembersarerarelypresentinpost-anesthesiacareunits,wherethescenariotookplace,andthisdiminishedthedaughter'splausibility.Furthermore,participantsexpectedthedaughtertobemoreemotional.Theyexpectedarealdaughtertointerjectmoreoften,expressmoreemotionthroughfacialexpressions,andalsotomoveclosertothepatienttocomforthim.Thedaughteronlyinterjectedbetweentwoandthreetimesperinteraction,didnotexpressanyemotionthroughfacialexpressions,andwasnotcapableofmovingclosertothepatient.Thesefactorscombinedtomakethedaughternotplausible.Differencesinsocialpresencescoresbetweenthenurseandthedaughter,regardlessofcondition,supportthedaughterbeinglessplausible.ThereweresignicantdifferencesforNotReal(p=0.01,nursewas(M=0.65,SE=0.31)better)andWatching(p=0.04,nursewas(M=0.47,SE=0.27)better).TherewasamarginallysignicantdifferenceforSentient(p=0.06,nursewas(M=0.47,SE=0.27)better). 4.3.6.4InteractionofPhysicalityandPlausibilityAninteractioneffectbetweenconditionandECAwouldindicateotherfactorsbesidesphysicalityaffectedsocialpresence.Weusedanon-parametricANOVA-typestatisticwithphysicalityasthebetween-subjectsfactorandECAasthewithin-subjectsfactortodetermineiftherewasaninteractioneffectbetweenconditionandECA(Figure 4-7 ).TheBoxapproximationcouldnotbeappliedtothewithinsubjectsfactor,sothedenominatordegreesoffreedomisinnity.Aninteractioneffectwasfoundwhencomparingthenurseanddaughter.Forallveself-reportsocialpresenceitems,thedifferencesbetweenthehighphysicalitynurseanddaughterwerelargerthanthedifferencesbetweenthelowphysicalitynurseanddaughter.ThesedifferencesweresignicantforNotReal 65

PAGE 66

(F(1,1)=7.192,p<0.01)andmarginallysignicantforComputerized(F(1,1)=3.355,p=0.07).ForNotRealthehighphysicalityvirtualhumanshadadifferenceof(M=)]TJ /F7 11.955 Tf 9.3 0 Td[(1.38,SE=0.39)(nursewasbetter),whilethelowphysicalityvirtualhumanshadadifferenceof(M=0.00,SE=0.33).ForComputerized,thehighphysicalityECAshadadifferenceof(M=)]TJ /F7 11.955 Tf 9.3 0 Td[(0.88,SE=0.19)(nursewasbetter),whilethelowphysicalityECAshadadifferenceof(M=0.22,SE=0.36)(daughterwasbetter). 4.3.7Discussion 4.3.7.1Self-ReportSocialPresenceBailensonetal.usedaCronbach'salphatodetermineifthescoresfromallvestatementscouldbesummedintoanoverallsocialpresencescore.TheCronbach'salphaforthenurse(0.84)indicatedwecoulddoso.Thesocialpresencescoreswere(M=6.00,SE=1.29forhighphysicalityand(M=2.56,SE=1.36)forlowphysicality.Thisdifferencewasnotstatisticallysignicant(p=0.10)butdoesagreewiththetrendofincreasingphysicalityincreasingsocialpresence. 4.3.7.2AddressingECAsIntotal,50%ofthelowphysicalityparticipantsusedeitherfamilymemberornursetoaddressoneoftheECAs(oneparticipantdidboth).Noneofthehighphysicalityparticipantsdidso.Theseresultssuggestincreasingphysicalityincreasessocialpresencebyelicitingmorerealisticbehavior.Addressingthedaughteras`familymembe'wasclearlyunrealistic,andshowedtheparticipantsdidnottreatthelowphysicalitydaughterasifshewasarealperson.Threelowphysicalityparticipantsaddressedthedaughternon-realistically,andstrikingly,noneofthehighphysicalityparticipantsdidso.Unlikewiththedaughter,addressingthenursebyhisrolewasrealistic,butitwasimpersonal.Thismannerofaddresswasacceptablebehaviorinsuchemergencysituationswithunfamiliarteammembers.However,doctorsconsidereditmorerespectfultouseanurse'snameorevennotuseawordatalltoaddressthenurse. 66

PAGE 67

Thislessrespectfulmannerofaddressmayhavebeenusedbecausetheseparticipantsdidnotfeeltheneedtotreatthelowphysicalitynurserealistically. 4.3.7.3PlausibilityIllusionThenurse'ssocialpresencescoresshowedhighplausibilityillusionallowingincreasingphysicalitytoincreasesocialpresence.Fortehnurse,allvesocialpresencestatementstrendedpositively,withonestatementstatisticallysignicantandonestatementmarginallysignicant.Thedaughter'ssocialpresencescoresshowedlowplausibilitylimitingsocialpresence.Therewerenocleartrendsindicatingeithertheloworhighphysicalityconditionaffectedsocialpresence.Thissuggeststhateventhoughthehighphysicalitydaughterhadahigherpotentialsocialpresence,thelackofplausibilitylimitedsocialpresencetoalowerlevel.ThisrelationshipbetweenphysicalityandsocialpresencesupportstheideathattheyaretheECAanaloguesofimmersionandpresence.SlaterbelievesthatimmersionprovidestheboundarieswithinwhichPI[presence]canoccur[ 78 ].Similarly,physicalityprovidestheboundariesofsocialpresence,andtoreachtheseboundariesplausibilityexpectationsmustbemet. 4.3.7.4InteractionofPhysicalityandPlausibilityTheinteractioneffectbetweenphysicalityandECAdemonstratedthatincreasingphysicalityincreasedexpectations.Thuswhenthephysicalitywasincreasedandtheexpectationsweremet(asisthecasebetweenthelowphysicalitynurseandhighphysicalitynurse),theself-reportsocialpresencescorescorrespondinglyincreased.However,whenthephysicalitywasincreasedandtheexpectationswerenotmet(asisthecasebetweenthelowphysicalitydaughterandhighphysicalitydaughter),self-reportsocialpresencewasnotsignicantlyraised.Theinteractioneffectissupportedbyndingsfromseveralotherresearchers.BothGarauandBailensonetal.foundsocialpresencewaslowestwhenlowbehavioral 67

PAGE 68

realismwascombinedwithhighphotographicrealismorviceversa[ 10 33 ].NowakandBioccafoundthatlessanthropomorphicimageswerecorrelatedwithhighersocialpresence[ 64 ].Theybelievedincreasinganthropomorphismincreasedsocialpresenceexpectations.Similarly,inourstudyincreasingphysicalitymayhaveincreasedplausibilityexpectations. 4.3.8LimitationsTheholisticevaluation'sresultssuggestoccupancyofthephysicalspaceaffectssocialpresence.However,theholisticevaluationvariedseveraldimensionsintheoccupancyofthephysicalspacecategorysimultaneously.Howeachdimensionorcombinationofdimensionsaffectedsocialpresencecannotbedeterminedfromtheholisticevaluation.Moreover,theholisticevaluationvarieddisplayresolutionandbrightnesswhicharenotaspectsofphysicality.Toaddresstheselimitations,weconductedtheindependentcomponentsevaluation. 4.4IndependentComponentsEvaluation:ExploringIndividualDimensionsofOccupancyTheindependentcomponentsevaluationlookedspecicallyattheeffectsofformdelityandconcordancewiththephysicalspaceonsocialpresence.Theindependentcomponentsevaluationusedabetween-subjects,2x2designwithhighandlowlevelsofformdelityandconcordance.AllfourconditionsusedthehighphysicalityANDIunits,butsomeofANDI'shighphysicalityfeatureswereremovedinsomeconditions.Thefourconditionswere: LCLF.Lowconcordance(plain,blackvirtualbackground)andlowformdelity(nophysicallegs)(Figure 4-8A ).Eightparticipants. LCHF.Lowconcordance(plain,blackvirtualbackground)andhighformdelity(physicallegs)(Figure 4-8B ).Nineparticipants. HCLF.Highconcordance(see-throughdisplay)andlowformdelity(nophysicallegs)(Figure 4-8C ).Sevenparticipants. HCHF.Highconcordance(see-throughdisplay)andhighformdelity(physicallegs)(Figure 4-8D ).Eightparticipants. 68

PAGE 69

Allfourconditionsmaintainedthesamehighlevelofsizeandpositiondelity.TheECAswerelife-sizeandplacedinthecorrectphysicalplaces.Sizeandpositiondelitywerekeptconstantbecausetechnologiesforhighconcordance,suchassee-throughdisplays,typicallyalsoinherentlyenablehighsizeandpositiondelity.Theindependentcomponentsevaluationusedasimilarscenariototheholisticevaluationbutwithmodicationstoaddressthelimitedpoolofparticipantsandthelow-plausibilitydaughter.Toincreasethepoolofparticipants,theindependentcomponentsevaluationchangedwhichrolewasplayedbytheparticipant.Insteadofrealdoctorsaskingavirtualnursequestions,theindependentcomponentsevaluationusedrealnursesaskingavirtualdoctorquestions.Thepoolofavailablerealnurseswasroughlytwiceaslargeasthepoolofavailableparticipantsfortheholisticevaluation.Thelow-plausibilitydaughterwasexcludedfortheindependentcomponentsevaluation.Instead,theindependentcomponentsevaluationincludedanadditionalvirtualnurse.Thevirtualnurseaidedtheparticipant(alsoanurse)byadministeringdrugstothepatientwhenrequestedtodoso.Delegatingdrugadministrationtoavirtualnurseallowedtheparticipantstofocusonthecommunicationaspectswithoutworryingaboutphysicaltasks.Thisdelegationisalsocommoninrealsituations.Aswiththeholisticevaluation,alloftheindependentcomponentsevaluation'sECAswerecontrolledbyahumanoperatorbehindthescenes.TheoperatorpracticedcontrollingtheECAsaspartofapilotstudyinvolvingtenrealnurses.Thepilotstudyalsoaidedinexpandingthesetofanticipatedquestionssomorepre-programmedresponsescouldbeaddedtotheECAs. 4.4.1ParticipantsThepopulationfortheindependentcomponentsevaluationconsistedofnursesthatworkedinthepost-anesthesiacareunitatUF&ShandsHospitalinGainesville,Florida.Thesenursesallhadthepotentialtoencountercasessimilartothesimulatedcriticalincidentintheirjobs,thoughnotallnurseshaddoneso.Nurseswererecruited 69

PAGE 70

toparticipateinthestudyaspartofatrainingexercise.Participationwasvoluntary,andparticipantswerenotofferedanycompensation.Intotal32nursesparticipatedintheindependentcomponentsevaluation.Participantshadpracticednursingforatleast4yearsandhadanaverageageof45.29(SD=10.57).Allbutoneparticipantwasfemale,andallparticipantsspokeEnglishastheirprimarylanguage. 4.4.2ProcedureTheindependentcomponentsevaluationfollowedasimilarproceduretotheholisticevaluation.Becausetheindependentcomponentsevaluationrequiredtheparticipanttoanswerquestionsaboutthepatient,theindependentcomponentsevaluationincludedanextrastagewhereavirtualdoctorprovidesthenurse(participant)withinformationaboutthepatient.Allfourgroupsfollowedidenticalstudyprocedures(Figure 4-9 ).Thestageswere: 1. CONSENT.Completedaninformedconsentformandwerebriefedaboutthegoalsandscopeofthetrainingexercise. 2. TUTORIAL.Virtualtutorialdoctorandvirtualtutorialnurseexplainhowtotalkwiththevirtualhumansandhavetheparticipantpracticeportionsofthelaterstages 3. DEMOGRAPHICSSURVEY-Completedademographicsandbackgroundsurvey 4. HANDOFF.Virtualhandoffdoctorprovidespatientinformationtotheparticipant.Thevirtualnurse'sdisplaywasturnedoffandthephysicallegs(ifany)werehidden. 5. HANDOFFSURVEY.Completedafeedbacksurveyaboutthehandoffdoctor 6. CRITICALINCIDENT.Virtualcriticalincidentdoctoraskstheparticipantquestionsaboutthepatientandgivestreatmentorders.Participantdelegatesdrugorderstothevirtualcriticalincidentnurse. 7. CRITICALINCIDENTSURVEY.Completedafeedbacksurveyandfree-forminterview 4.4.2.1HandoffDuringthehandoff,thevirtualhandoffdoctorprovidedtheparticipantwithinformationabouttheproceduresdonetothepatientandthepatient'smedicalhistory. 70

PAGE 71

Theparticipantrecordedthisinformationonapaperform.Iftheparticipantfeltanyinformationordetailwasmissing,theparticipantcouldaskthevirtualhandoffdoctorformoreinformation. 4.4.2.2CriticalIncidentInthecriticalincident,theparticipantworkedwiththevirtualcriticalincidentdoctor(adifferentECAthanthevirtualhandoffdoctor)andthevirtualcriticalincidentnursetotreatthepatient.Thevirtualdoctoraskedtheparticipantquestionsaboutthepatient.Theparticipantansweredthequestionsandtypicallyreferredtothenotesshetookduringthehandoffstage.Asthevirtualdoctoraskedquestions,thepatientsvitalsignsdegraded.Whenthevitalsignsdegradedtocertainspeciedpoints,thevirtualdoctorordereddrugstotreatthepatient.Theparticipantthendelegatedthedrugorderstothevirtualchargenurse.Thevirtualchargenurseadministeredthedrugsandverballyconrmedtheorderbyrepeatingit.Thisdelegationofordersiscommoninrealmedicalscenarios. 4.4.3MetricsTheindependentcomponentsevaluationusedthesameveself-reportsocialpresenceitemsastheholisticevaluation(InRoom,Watching,NotReal,Computerized,andSentient).Aswiththeholisticevaluation,thedatawasanalyzedusingatwo-waynon-parametricANOVA-typestatisticwithconcordanceandformdelityasbetween-subjectsfactorsandECAasawithin-subjectsfactor.Non-parametricanalysiswasagainchosenpertherecommendationofKapteinetal.forLikert-typedatawithsamplesizeslessthan50[ 47 ].Forbetween-subjectsfactors,theBoxapproximationwasalsousedforsmallsamplesizespertherecommendationofBrunneretal.[ 20 ],whichresultsinafractionaldenominatordegreesoffreedom.Tothebestofourknowledge,therearenopost-hoctestsavailableforthetwo-waynon-parametricANOVA-typestatisticweusedtoanalyzeourdata.Asanapproximatesubstitute,weusedapair-wiseone-wayANOVA-typestatisticwithconditionasthebetween-subjectsfactor. 71

PAGE 72

Ifnointeractioneffectwasfoundforeitherofthebetween-subjectsfactors,thedatawasre-analyzedusingtwoseparateone-waynon-parametricANOVA-typestatisticswitheachbetween-subjectfactor.RandthenparLDpackagewereusedtoanalyzeallself-reportsocialpresencedata.TheindependentcomponentsevaluationalsousedperceivedintelligenceoftheECAsasaproxyforplausibility.ParticipantsratedeachECA'sintelligenceonascaleof1-10(low-high)basedontheECA'sabilitytoachievetheteamtraininggoals.Wedidnotexpectanyplausibilitydifferencesbetweenconditions,butwedidexpecttheECAstoallhavehighplausibility.Weanalyzedperceivedintelligenceusingarepeated-measuresANOVA.Iftheassumptionofsphericitywasviolated,weusedtheGreenhouse-Geisseradjustment. 4.4.4ResultsForallvesocialpresenceitems,theLCLFconditionconsistentlyreceivedworseaverageratingsthantheHCHFconditionforbothcriticalincidentvirtualhumans.Comparingallfourconditionsshowedtwodifferenttrendsamongtheitems.Twoitemswereaffectedbybothformdelityandconcordance,whilethreeitemswereaffectedbyonlyconcordance. 4.4.4.1OutliersThreeparticipantshadparticularlybadexperienceswiththetrainingsystem.Thesebadexperienceshadtwocauses.TherstcausewasparticipantsaskingtheECAsalargenumberofunanticipatedquestions.UnanticipatedquestionsresultedinthehumanoperatorchoosingresponsessuchasIdon'tknow,you'llhavetocheckhischartorsilenceifnosuitablepre-recordedresponsewasavailable.Acompetentrealdoctorornursewouldneverrespondinsuchamannerorcompletelyfailtorespond.ThesefailuresdiminishedtheplausibilityoftheECAsandwerereectedintheperceivedECAintelligenceratings.BasedontheperceivedECAintelligenceratings,weidentiedtwostatisticaloutliers.Bothparticipantsratedtheintelligenceofthecriticalincident 72

PAGE 73

nurseasa1.Oneparticipantratedtheintelligenceofthecriticalincidentdoctora1whiletheotherratedita3.Thecriticalincidentnurseanddoctorreceivedratingsof(M=7.16,SD=2.07)and(M=7.45,SD=1.91)respectively.Thesecondcauseofbadexperienceswasextremediscomfortwithcomputer-basedsimulation.WhilemanyparticipantswereatrstmildlyuncomfortablewiththeECAs,onlyoneparticipantshowedextremediscomfort.Sherepeatedlystatedthattheexercisemadeherverynervousandshewasnotagoodcandidateforcomputer-basedsimulation.Further,sheresponded`Stronglydisagree'toallpositivestatementsand`Stronglyagree'toallnegativestatementsforallthreeECAs.Onlyoneotherparticipantrespondedsimilarly,andthisotherparticipantwasalreadyremovedasanoutlierduetoperceivedECAintelligenceratings.Theseextremeresponsesemphasizethatunlikethemajorityofthepopulation,thisoutlierwasunabletosuspenddisbelief.Afterremovingoutliers,theLCLFconditionhad8participantsandtheotherthreeconditionshad7participantseach. 4.4.4.2PlausibilityThesubjectiveintelligenceoftheECAresultssuggestedthetwocriticalincidentECAswerehighlyplausible,butthehandoffdoctorECAwassignicantlylessplausibleaccordingtoarepeated-measuresANOVA(F(1.286,34.735)=79.063,p<0.001).Thecriticalincidentnurseanddoctorreceivedsimilarintelligenceratingsof(M=7.16,SD=2.07)and(M=7.45,SD=1.91),butthehandoffdoctorreceivedaratingof(M=4.52,SD=1.61).Notably,thehandoffdoctor'shighestratingof6waslowerthantheaverageratingforthecriticalincidentECAs.Aone-wayANOVAdidnotndanysignicantdifferencesbetweenconditionsinintelligenceratingsforthehandoffdoctor(p=0.82),criticalincidentnurse(p=0.49),orcriticalincidentdoctor(p=0.63). 4.4.4.3FormFidelityandConcordanceTheresultssuggestbothhighformdelityandhighconcordancewerenecessarytoimproveratingsfortheComputerized(Figure 4-10A )item.ForthisitemtheHCHF 73

PAGE 74

conditionreceivedanaveragescorebetween`Slightlydisagree'and`Neutral'forboththecriticalincidentvirtualhumans.Theotherthreeconditionsreceivedanaveragescorebetween`Neutral'and`Slightlyagree'(theComputerizedsurveyitemwasanegativestatement,sodisagreeisbetter).Becausethehandoffdoctorshowedlowerplausibilityandplausibilitymayhaveaninteractioneffectwithphysicality,weincludedonlythemoreplausiblecriticalincidentECAsintheanalysis.However,wedoreporttheresultsforthehandoffdoctorECAfordiscussionpurposes.ForComputerized,theresultsshowedamarginallysignicantinteractioneffectbetweenconcordanceandformdelity(F(1,20.655)=3.674,p=0.07)butnosignicanteffectsofconcordance(F(1,20.655)=2.603,p=0.12),form(F(1,20.655)=0.451,p=0.51),orECA(F(1,1)=0.386,p=0.53).Pair-wiseanalysisfoundtheHCHFconditionwassignicantlybetterthantheHCLFcondition(F(1,1)=4.581,p<0.05)andLCHF(F(1,1)=7.559,p<0.01)butnottheLCLFcondition(F(1,1)=1.827,p=0.18)).Therewerenoothersignicantdifferencesbetweengroups(allp>0.43).ForNotReal(Figure 4-10B ),theresultsshowedasimilartrendtoComputerized.However,thetwo-wayANOVA-typestatisticfailedtondasignicantinteractioneffectsbetweenformdelityandconcordance(F(1,20.211)=1.393,p=0.25).NointeractioneffectswerefoundforInRoom(F(1,22.092)=0.000,p=0.98),Watching(F(1,22.181)=0.564,p=0.46),andSentient(F(1,23.863)=0.020,p=0.89). 4.4.4.4ConcordanceTheresultssuggestthatconcordance,regardlessofformdelity,affectedratingsfortheInRoom(Figure 4-11A ),Watching(Figure 4-11B ),andSentient(Figure 4-11C )items.Forallthreeitems,bothhighconcordanceconditionstendedtoreceiveanaverageratingof`SlightlyAgree'whilebothlowconcordanceconditionsreceivedan 74

PAGE 75

averageratingof`Neutral'forbothcriticalincidentvirtualhumans.However,thehandoffdoctordidnotshowthesametrendfortheWatchingandSentientitems.Consideringconcordanceastheonlybetween-subjectsfactor,theresultsshowedmarginallysignicanteffectsforInRoom(F(1,25.185)=4.209,p=0.05),Watching(F(1,24.538)=3.080,p=0.09),andSentient(F(1,25.952)=3.195,p=0.09).NosignicanteffectswerefoundforNotReal(F(1,25.511)=0.471,p=0.50). 4.4.4.5FormFidelityConsideringformdelityastheonlybetween-subjectsfactor,theresultsshowednosignicanteffectsforNotReal(F(1,25.164)=1.185,p=0.29),InRoom(F(1,25.703)=0.055,p=0.82),Watching(F(1,26.930)=0.041,p=0.841),andSentient(F(1,26.949)=0.036,p=0.85). 4.4.5Discussion 4.4.5.1FormFidelityandConcordanceThecombinationofhighformdelityandhighconcordanceappearstoincreasethesenseofthevirtualhumanasarealperson.TheComputerizeditemreadIperceivedthenurseasbeingonlyacomputerizedimage,notasarealperson.Thephrase'Realperson'alsoappearedintheNotRealitemasThethoughtthatthenursewasnotarealpersoncrossedmymindoften.ThoughtheNotRealitemdidnotshowsignicantinteractioneffectsbetweenformdelityandconcordance,itdidshowasimilartrendtoComputerized.Theeffectofformdelityinconjunctionwithhighconcordancewasapparentdespitethephysicallegsoccupyingaverysmallportionoftheparticipant'svisualeld.Theparticipant'sviewofthelegswasblockedbythepatientsimulatorandbed.Onlyafewverticalinchesofthephysicallegsremainedvisible.ThissuggeststhatevengivingasmallportionofanECAhighformdelitycansubstantiallyimprovefeelingsoftheECAasarealperson. 75

PAGE 76

Thehandoffdoctor'slowerplausibilitymayexplaininconsistenciesinthetrends.ThetwocriticalincidentECAsreceivedsimilarself-reportsocialpresencescorestoeachother,butthehandoffdoctorsometimesreceiveddifferentscores.FortheComputerizedandNotRealitems(Figures 4-10A and 4-10B ),thescoresforthehandoffdoctorshowadifferenttrendthanthecriticalincidentECAs.IntheLCLFconditionthehandoffdoctorreceivedbetterscoresthanthecriticalincidentECAs.ThismaybebecauseLCLFconditionparticipantshadlowerinitialexpectationsofrealismandthehandoffdoctor'splausibilitymatchedtheseexpectations.ThecriticalincidentECAshadbetterplausibility,andasaresultparticipants'expectationsofrealismincreasedbutthephysicalitydidnotmeettheirexpectations. 4.4.5.2ConcordanceTheresultsfortheInRoom,Watching,andSentientitemsshowtheimportanceofconcordance.Thoughthedifferencesforallthreeitemswereonlymarginallysignicant,togethertheyestablishaconsistenttrend.Increasingconcordancebyaddingaphotographofthephysicalenvironmentincreasedtheratingbyafullpointonaseven-pointLikertscaleforInRoom(Figure 4-11A )andSentient(Figure 4-11B ).Thisfullpointalsomadethedifferencebetweenaneutralratingandapositiverating.TheresultsfortheInRoomitemspecicallysuggestthatthesee-throughdisplaydoesnoticeablyaffectconcordance.TheInRoomitemstatesIperceivethatIaminthepresenceofanurse/doctorintheroomwithme.Bothhighconcordanceconditionsreceivedaverageratingsonthepositivesideofthescalewhilebothlowconcordanceconditionsreceivedaverageratingsofneutral.Thesee-throughdisplaymadeasignicantdifferencedespitetheillusionbeinglessthanperfect.Thebackgroundphotosometimesisnotregisteredcorrectlywiththesurroundingrealbackgroundbecausethephotolackscorrectdepthinformation.However,ithasaverylowcostandstillmakessubstantialimprovements. 76

PAGE 77

TheresultsfortheInRoom,Watching,andSentientitemswereconsistentbetweenthetwocriticalincidentECAsbutnotconsistentwiththehandoffECA.Thismaybebecauseduringthehandofftheparticipantwastakingnotesandfocusedonherclipboardratherthanthevirtualhuman. 4.4.5.3FormFidelityTheindependentcomponentsevaluationfailedtondasignicanteffectforformdelity.ThisisinconsistentwithLeeetal.'sndingsofformdelityaffectingsocialpresence[ 54 ].Thismayhavebeencausedbytheinteractioneffectbetweenformdelityandconcordance.TheLCHFconditionlikelydidnotreceivethebenetsofformdelitybecausetheconcordanceoftheECA'supperandlowerhalvesconicted.Theupperhalfwasdisplayedagainstaplainblackbackgroundandhadlowconcordancewiththephysicalspace.Thelowerhalf,however,hadhighconcordancebecausethephysicallegshadtherealbackgroundandhencenaturallyhadhighconcordancewiththephysicalspace.Thehighformdelitymayhavemadethevirtualhumanmorerealhaditnotbeenforthemismatchedlevelsofconcordancewiththephysicalspace. 4.4.5.4PlausibilityThelowerintelligenceratingsforthehandoffdoctormayhavebeencausedbythehandoffdoctortalkingtoofast,givingexcessivelylonganswers,oroccasionallynotknowingtheanswertoaquestion.Manyparticipantsaskedthehandoffdoctortoslowdownbecausetheycouldnotwritenotesfastenough.However,thehandoffdoctorignoredthisrequestandcontinuedspeakingataxedratebecauseallofthespeecheswerepre-recorded.Asaresult,manyparticipantsaskedthehandoffdoctortorepeatpiecesofinformation.Thehandoffdoctorsreplyoftencontainedfarmoreinformationthanneededbecausetherequestedinformationwasattheendofalongpre-recordedspeech.Forexample,iftheparticipantaskedWhatdrugdidhehavebolusesof?thehandoffdoctor 77

PAGE 78

mightrespondHehaddexamethasoneandketaminethismorningforanalgesiaandhehashadintermittentbolusesoffentanylthroughoutthesurgery.Insomecases,participantsaskedthehandoffdoctorunanticipatedquestions.Forexample,iftheparticipantaskedWhichantibioticisheonandthedoctorECAansweredYou'llhavetocheckhischartMostrealdoctorswouldknowthisinformation,butthisquestionwasunanticipatedandhenceitdidnothaveaspecicpre-recordedanswer. 4.4.6LimitationsOnelimitationoftheindependentcomponentsevaluationwasthatonlyasmallportionofthephysicallegswerevisible.Theresultssuggestparticipantsatleastsubconsciouslynoticedthephysicallegseventhoughonlytwoparticipantscommentedonthephysicallegs.Also,theinclusionofphysicallegschangedmorethanjustformdelity.ThephysicallegsalsocompletedtheECA'sbody.Withoutlegs,theECAsinthelowformdelityconditionsappearedasoatingECAsfromthewaistup.ThisoatingisanalternativeexplanationforthelowformdelityconditionsratingworsethantheHCHFcondition.Totestthisalternateexplanation,futurestudiescomparingthephysicallegswithvirtuallegsonasecondTVneedtobeconducted. 4.5OverallDiscussionTheholisticevaluationsuggestedthatphysicalityhasapositiverelationshipwithsocialpresence.Thisledustodelvedeeperintospecicdimensionsofphysicalityandconducttheindependentcomponentsevaluation.Theindependentcomponentsevaluationrepeatedmanyoftheresultsfromtheholisticevaluation.TheholisticevaluationfoundasignicantdifferencefortheSentientitemandamarginallysignicantdifferencefortheInRoomitem.Thesetwoitemsbothshowedmarginallysignicantdifferencesintheindependentcomponentsevaluation.Thedifferencesmayhavebeenlesspronouncedintheindependentcomponentsevaluationbecauseallconditionsin 78

PAGE 79

theindependentcomponentsevaluationincludedhighsizedelityandhighpositiondelity.Theresultsoftheindependentcomponentsevaluationsuggestthatthedifferencesseenintheholisticevaluationwerenotsolelycausedbydifferencesinsizedelity,positiondelity,orfactorsunrelatedtophysicalitysuchasdisplayresolutionandbrightness.Hadthatbeenthecase,theindependentcomponentsevaluationshouldhaveshownnodifferencesbetweenconditions.However,thisdoesnotindicatesizedelityandpositiondelityareunimportant.Rather,highsizedelityandpositiondelitytendtoaccompanyhighconcordance.HighconcordancerequiresMRtechnologysuchassee-throughHMDsorothersee-throughdisplays.Thesesee-throughdisplaysusuallysupporthighsizedelityandhighpositiondelitywithoutanyaddedcost.BothevaluationsdemonstratedtheimportanceofplausibilitywhendevelopingECAsfortrainingapplications.ImplausibilitylimitsboththesocialpresenceoftheECAandtheusefulnessofthetrainingapplicationasawhole.EffortmustbespentdevelopingandtestingtheconversationalcomponentofECAs.Withoutthiseffort,increasedphysicalitycanbewasted.InChapter 6 ,weexploretheconceptofECAplausibilityinmoredepthandidentifyspecicfactorsthatmayaffectsocialpresence.Theoutliersinbothevaluationsdemonstratethatdespitebestefforts,someusersarelikelytoalwayshaveissuessuspendingdisbelief.Theseusersmaybeuncomfortablewithallcomputer-basedsimulationsregardlessofphysicality.However,thisdoesnothaveanyrelationtotheuser'sabilitiestoperformtheirjobsinreality.Inboththeholisticevaluationandindependentcomponentsevaluations,wemadethetradeoffofincreasedecologicalvalidityatthecostoflowerstatisticalpower.Theevaluationsbothusedrealhealthscienceprofessionalsandmedicalcasesthatmedicalteamsreallypractice.Bothevaluationswerealsoconductedintheapplication'stargetenvironment,speciallyequippedroomsdedicatedtomedicalsimulation.Thesechoicesprovidehighecologicalvalidity.However,thesechoiceslimitedboththeparticipantpool 79

PAGE 80

andtimesthestudiescouldberun.Fortheindependentcomponentsevaluation,theentireparticipantpool(post-anesthesiacareunitnursesatUF&ShandsHospital)wasexhaustedthroughpilottestingandconductingthestudy.Usingalargerparticipantpool,suchasundergraduatepsychologystudents,couldincreasethestatisticalpower.However,undergraduatepsychologystudentsdonotpossessthenecessaryexperiencetocompletethetrainingexercise.Moreover,undergraduatepsychologystudentsdonotaccuratelyreectthedemographicsofthetargetpopulation. Figure4-1. PairofANDIsystems.EachANDIdisplaysthecharactersatlife-sizewheretheyshouldstandanduseacombinationofasee-throughdisplayandphysicallegstocreatethesensethattheECAssharethephysicalspace. Table4-1. Physicalitylevelsbydimensionforthenurseanddaughter HighphysicalityLowphysicality SizeLife-sizeNearlife-sizePositionCorrectpositionOnwallConcordanceSee-throughdisplayVirtualenvironmentFormTVandphysicallegsProjectionscreenViewpointsMany(head-tracked)One 80

PAGE 81

Figure4-2. ANDIrotating.NotethephysicallegsrotatetoapproximatelymatchtheorientationoftheECAonscreen. Figure4-3. Lowphysicalitycondition.Theprojectionscreendisplaysthecharactersatlarger-than-life-sizeagainstawallandusesavirtualbackgroundtocreatethesensethattheECAsoccupyadistinctvirtualspace. Figure4-4. Approximatetimespentineachstageoftheholisticevaluation.Totaltimewasapproximately25minutes. 81

PAGE 82

Figure4-5. Socialpresence(meanandstandarderror)forthelowandhighphysicalitynurse.Higherindicatesbetter.Scoresfornegativestatements(NotRealandComputerized)havebeeninvertedforeaseofcomparison. Table4-2. WordsusedbytheparticipantstoaddresstheECAsandthenumberofparticipantsthatusedthemineachcondition. WordsCategoryLowPhysicalityHighPhysicality Familymember,wifeNon-realistic30NurseRealisticbutimpersonal30Ma'amRealisticbutimpersonal42SirRealisticbutimpersonal76MattRealisticandpersonal55Mr.StarksRealisticandpersonal42 Note,oneparticipantmistakenlythoughtthefamilymemberwasthepatient'swifeinsteadofdaughter. 82

PAGE 83

Figure4-6. Socialpresence(meanandstandarderror)forthelowandhighphysicalitydaughter.Higherindicatesbetter.Scoresfornegativestatements(NotRealandComputerized)havebeeninvertedforeaseofcomparison. 83

PAGE 84

Figure4-7. InteractioneffectsofconditionandECA.SignicantinteractioneffectforNotReal(p=0.01)andmarginallysignicantinteractioneffectforComputerized(p=0.07).Higherindicatesbetter.Scoresfornegativestatements(NotRealandComputerized)havebeeninvertedforeaseofcomparison. 84

PAGE 85

A B C D E FFigure4-8. Theindependentcomponentsevaluationconditions.Conditionsonthetopuseaplain,blackvirtualbackgroundforlowconcordancewiththephysicalspace.Conditionsonthebottomuseaphotographoftherealbackgroundforhighconcordance.ConditionsonthelefthaveonlyaTVforlowformdelity.Conditionsontherightincludephysicallegsforhighformdelity.A)LCLF-lowconcordance,lowformdelity.B)LCHF-lowconcordance,highformdelity.C)HCLF-highconcordance,lowformdelity.D)HCHF-highconcordance,highformdelity.E)CloseupofnophysicallegsinHCLF.F)CloseupofphysicallegsinHCHF. 85

PAGE 86

Figure4-9. Approximatetimespentineachstageoftheindependentcomponentsevaluation.Totaltimewasapproximately30minutes. 86

PAGE 87

A BFigure4-10. Self-reportsocialpresenceresults(meanandstandarderror)forNotRealandComputerizedfortheindependentcomponentsevaluation.Lightanddarkgraybackgroundsindicatelowandhighconcordancerespectively.Solidanddottedllsindicatelowandhighformdelityrespectively.*Scoresfornegativestatementshavebeeninvertedforeaseofcomparison.A)Computerized*.B)NotReal*. 87

PAGE 88

A B CFigure4-11. Self-reportsocialpresenceresults(meanandstandarderror)forInRoom,Watching,andSentientfortheindependentcomponentsevaluation.Lightanddarkgraybackgroundsindicatelowandhighconcordancerespectively.Solidanddottedllsindicatelowandhighformdelityrespectively.A)InRoom.B)Watching.C)Sentient. 88

PAGE 89

CHAPTER5INTERACTIONWITHTHEENVIRONMENTThischapterdescribesastudyexaminingtheeffectsofanECA'sabilitytointeractwiththeenvironmentonsocialpresence.ThestudywaspublishedintheproceedingsoftheIEEEconferenceonVirtualReality2013andalsoinaspecialissueofthejournalIEEETransactionsonVisualizationandComputerGraphics[ 25 ].Thischaptersharestextwiththatpaper.PersonalContributions:Idesignedandconductedthestudyandanalyzedthedata.IalsodesignedandimplementedboththehighandlowphysicalityinterfacesfortheECAtointeractwiththeenvironment.Collaborators:Dr.ErikBlackhelpedwithstudydesign,recruitedsomeparticipants,andalsoprovidedgiftcardsasparticipantcompensation.MedicaldoctorsDr.MeredithMowitz,Dr.MariaKelly,andDr.LindsayThompsonprovidedsubjectmatterexpertiseonpediatricdevelopment.RelevancetoThesis:Thischapterexplorestheeffectsonecategoryofphysicalitydimensions-interactionwiththeenvironment-onsocialpresence.Ourpreviousstudieskeptphysicalitydimensionsinthiscategoryconstant.Theresultsfromthischapter'sstudysuggestthatECAswithlowinteractionphysicalitycanstillbeperceivedassociallypresent. 5.1OverviewInourproposedtaxonomyfordimensionsofphysicality(Chapter 3 ),onecategoryofdimensionsisinteractionwiththeenvironment.Interactionwiththeenvironmentincludestwodimensions,awarenessofchangestotheenvironmentandtheabilitytochangetheenvironment.Ourpriorstudiesfocusedonoccupancy(Chapter 4 ),andtheECAsinthosestudieshadverylimitedinteractionwiththeenvironment.TheonlychangestotheenvironmenttheECAswereawareofwerechangestotheuser'sposition,andthey 89

PAGE 90

onlydemonstratedthisawarenessbyturningtheirheadstolookattheuser.TheECAswereunabletovisiblymanipulateanyobjectsintheenvironment,thoughthenurseECAwasabletoadministertreatmentstothemannequinpatientsimulator,andthepatientsimulatorrespondedaccordingly.Toexploretheeffectofinteractionwiththeenvironmentonsocialpresence,wesoughtascenariowiththefollowingqualities: anECAvisiblymanipulatesanobjectintheenvironment anECAmustbeawareofauser'schangestoanobjectintheenvironment manipulationoftheobjectiscriticaltothescenario thephysicalityoftheECA'sinteractioncouldbevariedUnfortunately,themedicaltrainingscenariousedinourtwopreviousstudiesonoccupancywasnotsuitablefortworeasons.First,thescenariohasverylimitedinteractionwiththeenvironmentbecauseinteractionwiththeenvironmentisnotakeypartofthescenario.Theonlymanipulationofobjectsisadministeringdrugs,andtheECAneverneedstobeawareofchangestotheenvironment.Whileadministeringdrugsisimportant,seeingtheactofadministeringthedrugisnotasimportant,andinsimulationexercisespeopleoftensimplystatetheyareadministeringdrugsinsteadofphysicallyadministeringadrug.Second,thetechnologyweuseddidnotsupportmultiple,distinctlevelsofinteraction.Asimplemented,thetechnologyonlysupportedonelevelofinteraction-abstractlymanipulatingobjects.ThenurseECAcouldverballysayheorshewasadministeringadrugandthesimulatorrespondedaccordingly,buttheuserneversawtheECAinteractwithevenavirtualIV.Asanalternatescenario,wechoseapediatricdevelopmentalexamonavirtualfour-year-oldchild.Inthisscenario,interactionwiththeenvironmentisacriticalcomponentoftheexam.Theexaminerandthechildbothdrawonasheetofpaper,pointatdrawings,andtalkaboutthedrawings.ThisrequirestheECAtobothchangean 90

PAGE 91

objectintheenvironment(drawingonthepaper)andbeawareofchangestotheobject(seeingtheexaminer'sdrawings).Wedesignedandimplementedtwodifferentphysicalityinteractions: Highphysicality-ThechilddrewdirectlyonanMRsheetofpaperandcouldalsomovethepaper.Thepaperwasdisplayedusingacombinationofamonitorandatablet(Figure 5-1 ).Thetabletalsoallowedtheparticipanttodrawdirectlyonthepaper. Lowphysicality-Thechildabstractly`drew'onanapplicationonamonitor(Figure 5-2 ).Inadditiontobeingpurelyvirtual,theapplicationwasnotintegratedintothemixedrealityenvironment.Instead,theapplicationwasastandarddesktopapplicationwithamousepointer,buttons,andatitlebar.Whenthechild`drew'onthepaper,abuttonappearedandtheparticipantclickedthebuttontorevealthechild'sdrawing.Notethataswithourpriorstudies,theterms`high'and`low'donotnecessarilyindicatethehighestorlowestphysicalitypossible.Instead,theyareadjectivesdescribingthephysicalityofthetwoconditionsrelativetoeachother.Weusedthisexaminawithin-subjectsstudywith22participants.Wehypothesizedthatthehighphysicalityconditionwouldhavehigherratingsofself-reportsocialpresenceandelicitrealisticresponsesfrommoreparticipants.InadditiontoexploringtheeffectofanECA'sinteractionphysicalityonsocialpresence,thisstudyexaminedtheeducationalbenetofamixedrealitypediatricdevelopmentalexam.wehypothesizedthattheexamwouldprovideaneducationalbenetregardlessofphysicality. 5.2MotivationandBackground-RealPediatricDevelopmentalExamsScreeningchildrenthroughpediatricdevelopmentalexamsisanimportantmethodofdetectingdelaysandcorrectingthematanearlyage.However,studentsreceivelittleopportunitytopracticetheseexams.Unlikewithadultcases,pediatriccasestypicallydonothavestandardizedpatients.Usingvirtualpediatricpatientsgivesstudentsadditionaltrainingopportunitiesthattheymightotherwisenotget. 91

PAGE 92

5.2.1ImportanceofDevelopmentalScreeningMultiplestudieshavedeterminedthatroutinedevelopmentalscreeningsofchildrenwithavalidandreliableassessmentleadtoimprovedoutcomesinchildren[ 11 88 ].Despitethisdataandthegeneralrecognitionthatdevelopmentalassessmentisanimportantcomponentofprimarycarepediatrics,healthsciencepractitionershavefrequentlyreportedlowlevelsofself-efcacyrelatedtothetask[ 84 ].Additionallycomplicatingmattersisalackofconsistencyamonginsurersandpractitionersinthecommunityalongsideincreasingpatientloads,resultingindecreasedtimespentwithpatients.Differinginsuranceplansofferdifferentlevelsofreimbursementfordevelopmentalscreening,whichrangefrom$0-$30.Thus,generalpediatriciansselectivelyscreenthosewhotspeciccriteria,over-estimatingtheirabilitytoidentifyat-riskchildrenandprovidinginadequaterolemodelingfortrainees[ 38 ].Thoughmanydifferentdevelopmentalscreenersarecurrentlyinuseindifferentclinicalsituations,threecommonscreenersinclude:theDenverII[ 29 ],theAgesandStagesQuestionnaire(ASQ)[ 4 ]andtheParents'EvaluationofDevelopmentalStatus(PEDS)[ 68 ].BoththeASQandPEDSassessmentsrequireparentalorfamilymemberresponses,meaningthesescreenersaretypicallycompletedpriortoavisitwithahealthcareprovider.Atthetimeofthevisit,theprovidermayusethescreenertoaskmorein-depthorclarifyingquestions.TheDenverIIisaphysicianadministeredassessmentandispresentlythemostcommonlyuseddevelopmentalassessmentbygeneralpediatricians.Eachscreenerhasdifferentpsychometricproperties,butallthreehavebeenrecognizedasvalidandreliableforageneralpediatricpopulation[ 84 ]. 5.2.2CurrentTrainingMethodsEvidenceindicatesthatacomprehensiveapproachtodevelopmentalscreeningresultsinincreasedretentionamonghealthsciencetrainees[ 84 ].Unfortunately,fewmethodshavebeenempiricallyestablishedasaneffectivemeansofteachinghealthcareprofessionalshowtoeffectivelyadministeranassessmentanddiscusstheresults 92

PAGE 93

withfamilymembers[ 63 ].Accordingtoeducatorsandstudentsweinterviewed,currenttrainingmethodsfocusmainlyonteachingstudentswhatthedevelopmentalmilestonesare(e.g.,four-year-oldchildrenshouldbeabletodrawcircles).Moststudentsreceivelittletonotrainingonhowtoactuallyperformtheexampriortotheirclinicaltrainingrotations(hands-ontraininginrealclinics).However,clinicalrotationsfocusmainlyonhealthissuesratherthandevelopmentalissues.Studentsreceiveverylittleexposuretoperformingandcorrectlyscoringdevelopmentalexamsonrealchildren.Somestudentspracticedevelopmentalexamsonthechildrenoffriendsorfamily.Thisprovidesvaluableexperiencebutitoftenlacksthechallengesassociatedwithexamsonrealchildreninclinic.Mostimportantly,thechildrenoffriendsandfamilytypicallydonothailfromeconomically-disadvantagedbackgrounds.Moreoftenthannot,thesechildrenarefreeofdevelopmentaldelays.Studentsnormallyreceivenotrainingorpracticedeliveringtheresultsofexamstoparents[ 63 ].Whenthesestudentsgoontoperformtheirrstrealexams,theyareoftenunsurehowtoproperlyaddressparents'concerns.Forexample,aparentmightaskwhytheirfour-year-oldchildcannottiehisshoesyetwhenanotherfour-year-oldchildcan.InexperiencedexaminersmightanswerwithsomethinggeneralsuchasEverychildlearnsatadifferentpace.Suchgenericanswersdolittletorelieveaconcernedparent.Experiencedexaminersmightinsteadpointoutthattyingshoesisnottypicallymastereduntilalaterage,andthattheparentshouldtheirchildtoothersbecauseevenafewmonthsagedifferencemakesabigdifferenceatsuchanearlyage. 5.2.3SkillsTestsforFour-Year-Old-ChildrenTheMRpediatricdevelopmentalexamfocusedonskillsforfour-year-oldchildren.Four-year-oldchildrenareofparticularinterestbecausetheirexamsfocusheavilyonpersonal-social,language,andne-motorskills.Testingtheseskillsrequiresbothverbalandnon-verbalcommunication,andtheresultsarehardertointerpretthanexamson 93

PAGE 94

infants.Examinersmustlearnnotonlyhowtoproperlyadministerthetestsbutalsohowtoproperlyscoretheresults.Personal-socialskillsincludeitemssuchaspreparingcerealandgettingdressedwithnohelp.Languageskillsincludespeakinginunderstandable,completesentencesandnamingcolors.Assessingboththeseskillscategoriestypicallyrequirestheexaminertoaskthechildquestions.IfthechildisshyandanswersIdon'tknow,theexaminermayhavetoengagethechildmoreorasktheparentifthechildcanperformtheskill.Assessingne-motorskillsinvolvesextensiveinteractionswithapenandpaper.Theexaminertypicallydrawsasimpleshape,suchasacircle,andasksthechildtonametheshape.Theexaminerthenpassesthepenandpapertothechildandasksthechildtocopytheshape.Thisprocessrepeatswithothershapessuchassquaresandcrosses.Anothertestrequiresthechildtoidentifythelongeroftwolines.Thene-motorskillsexamalsorequiresthechildtodrawapersonwithfourtosixparts(e.g.head,eyes,arms,andlegs).Theexaminerwillpointtopartsandaskthechildtonameorcountthem.Thenumberofpartsisanimportantsignofachild'sdevelopment.Asachilddevelops,shewillbegintoaddmorepartssuchasabody,mouth,andhair. 5.3MixedRealityPediatricDevelopmentalExamThemixedrealitypediatricdevelopmentalexamallowedauser(e.g.medicalornursingstudent)topracticeperforminganexamonavirtualchild.Thevirtualchildwasaccompaniedbyhisvirtualfather.Thevirtualchildandfatheralwayshadthesameoccupancyofthephysicalspace,butthevirtualchild'sinteractionphysicalitywasvariedbetweenlowandhighphysicalityinterfaces. 94

PAGE 95

5.3.1MixedRealityVirtualHumansThevirtualchildandparentleveragedthehigh-occupancyECAsdescribedinChapter 4 .Wemadesomechangestomeettherequirementsofthepediatricdevelopmentalexam. 5.3.1.1GraphicsandAudioHealthsciencepractitionersaretrainedtoconstantlylookandlistenforsubtleproblemswithpatients.Practitionersmaydiagnoseamedicalproblemifthevirtualhumanhaslow-delityanimationsandspeech.Anearlyversionofthevirtualchildhadlow-delity,jerkyanimation,andduringpilottestingpediatriciansthoughtthechildhadmuscularproblems.Thenalversionsofthevirtualhumansinteractedusingasetofhigh-delity,pre-programmedanimationsandspeeches.Toensurehigh-delityanimations(e.g.drawingshapesonthepaper),wehiredaprofessionalanimatortocreatealloftheanimations.Forhighquality,naturalspeech,wepre-recordedrealhumansreadingallofthevirtualhumans'pre-programmedspeeches.Adifferenthumanrecordedeachvirtualhuman.Eachvirtualchildincluded53pre-recordedspeeches,andeachvirtualparentincluded42pre-recordedspeeches.Thevirtualchildrecordingswereperformedbythefourandve-yearoldchildrenoftwocolleagues.Thevirtualhumanswere60,000trianglemeshesgeneratedusingtheEvolver.comwebsite.BecauseEvolverdidnothavechildmeshes,athin,short(140cm),young-lookingadultmeshwasusedasabaseforthechild.Thebasemesh'sboneswereshortenedsotheresultingmeshhadanappropriateheight,torsolength,armlength,andleglengthforafour-year-oldchild[ 57 ]. 5.3.1.2OccupancyofthePhysicalSpaceBecausethevirtualchildismuchsmallerthananadult,thevirtualchildwasdisplayedona24LCDmonitorinportraitorientationinsteadofa40LCDTV.The 95

PAGE 96

virtualchilddidnothavephysicallegsbecausehewastooshortforthelegstobevisible.Anadditionalmonitorwasplacedonadeskbetweenthevirtualchildandtheuser.Inthelowphysicalitycondition,thismonitordisplayedthepaperapplication.Inthehighphysicalitycondition,thismonitordisplayedthepaper,partofthedesk,andpartofthechild(Figure 5-3 ).Toensurethehighphysicalitycondition'svirtual/physicalpaperwasproperlyregisteredwiththerealenvironment,therealenvironmentwasmanuallymodeledinGoogleSketchUp,andthemodelanddisplaylocationsweremanuallycalibratedusingatapemeasure.ThismodeloftheenvironmentreplacedthepanoramicphotousedasabackgroundinthestudiesinChapter 4 .ThepanoramicphotowasnotsuitablebecauseportionsoftheenvironmentinfrontoftheECAwerevisible(e.g.,thedesk),andthepanoramicphotowasnotassociatedwithaccuratedepthinformation. 5.3.1.3WizardofOzAswithourpriorexperiments,weusedaWizardofOzcontrolinterfacetoavoidproblemswitharticialintelligence.Icontrolledthevirtualhumansduringalloftheinteractions.Ilisteneddirectlytotheparticipant,butIcouldnotphysicallyleavetheroomforlogisticreasons.Instead,Isatoutsideoftheparticipant'seldofviewandtoldtheparticipantIwasthereonlytotroubleshoot.Whentheparticipantnishedspeaking,Ipickedanappropriateresponsefromalistofpre-programmedresponses.Theresponsesweredisplayedonatablet(aMotorolaXoom),allowingmetosilentlyselectaresponse.Ihadextensiveknowledgeofthepre-programmedresponses,havingcreatedalltheresponseswiththeaidofmedicalpractitioners.Ialsopracticedwithovertenpilottestparticipantsbeforebeginningthestudy.Iftheparticipantaskedthefatheraquestionwithoutapre-programmedresponse,thefatheransweredMywifewouldknowtheanswertothatorMywifeusuallydoesthat.Iftheparticipantaskedthechildaquestionwithoutapre-programmedresponse(e.g.Whohelpedyougetdressedthismorning?),thechildrespondedIdon'tknow 96

PAGE 97

orshruggeditsshoulders.Thisisnaturalshybehaviorforachild,andstudentsmustlearntorephrasequestionstodrawouttheanswer(e.g.Didmommyhelpyougetdressedthismorning?)orredirectthequestiontotheparent.Iftheparticipantaskedthechildtoperformanactionitwasnotpre-programmedtodo(e.g.drawaletter),thechildrespondedNo.Refusingtodosomethingisalsonaturalbehaviorforachild,especiallywheninteractingwithunfamiliarpeople,andhealthsciencespractitionersoftenencounterthisbehaviorwhenperformingexams. 5.3.2InteractionPhysicalityThepediatricdevelopmentalexamforfour-year-oldchildreninvolvedfrequentinteractionswithasheetofpaper.Theexamusedthepapertotestthechild'sabilitiesto: Nameandcopyshapesdrawnbytheexaminer Pointtothelongeroftwolinesdrawnbytheexaminer DrawapersonandnameandcountthebodypartsThesetaskscreatedseveralopportunitiestointeractwiththepaper.Boththechildandexaminerdrewonthepaperandpoinetedatdrawingsorpartsofdrawings.Inarealexam,thechildandexaminerwouldalsoconstantlypassthepenandpaperbackandforth.Wecreatedtwodifferentphysicalityinterfacestotesthowperceptionofthevirtualchildwasaffectedbythevirtualchild'sabilityorinabilitytointeractwithobjects.OneinterfacewasahighphysicalityinterfacewhereboththevirtualchildandhumanuserdirectlyinteractedwithanMRsheetofpaper(Figure 5-3 ).Theotherinterfacewasalowphysicalityinterfacewherethevirtualchildandhumanuserabstractlyinteractedwiththepaperthroughbuttonclicks(Figure 5-4 ).BothinterfacesusedthesameMRvirtualhumansandenabledboththeexaminerandchildtocommunicatewhichshapestheydrew. 97

PAGE 98

5.3.2.1HighPhysicalityInterfaceThehighphysicalityinterfaceallowedboththevirtualchildandtheexaminer(arealhumanuser)todirectlyinteractwithasharedsheetofMRpaper.TheMRpaperspannedtwodisplays(Figure 5-3 ).Therstdisplay,a10.1AsusTransformertablet,wasplacedatonthetableanddisplayedonehalfofthepaper.Thepapervisuallycontinuedontoa17LCDmonitorplacedverticallyontheedgeofthetablet.Themonitor'sverticalorientation,coupledwithsh-tankVR,allowedthemonitortocorrectlyrenderthechild'shandwhenhedrewonthepaper.Apurelyhorizontaldisplay,suchasthepaperandprojectorusedbyJunuzovicetal.[ 46 ],wouldhaveresultedinavisualdisconnectbetweenthechild'sarmonthevertical24LCDandhandonthehorizontalpaper.Theuserphysicallyinteractedwiththepaperusingthetabletandacapacitivestylus.Notethatwhiletheuser'sinterfacefordrawingshapesdoesnotdirectlyimpacttheECA'sphysicality,thisinterfacedoesemphasizethatthepaperisasharedobjectthatoccupiesthephysicalspace.Todrawshapes,theusertraceddashedoutlinesoftheshapeswiththestylus.Theoutlinesconsistedoftwosquares,twocircles,twocrosses,alongline,andashortline.Tracingoutlineswasusedinsteadoffree-formdrawingtokeeptheinterfaceequivalenttothelowphysicalityinterface.Thelowphysicalityinterfacedisplayedbuttonsfordrawingeachshape,whichgavetheusersareminderofwhichshapestodraw.Thebuttonsalsopreventedtheuserfromdrawingshapesthesystemwouldnotrecognize(e.g.trianglesorletters).Boththechildandtheuserslidthepaperbackandforth.Thechildslidthepapertowardshimselftopointtothelongeroftwolinesdrawnbytheuser.Theuserslidthepaperclosertoherselftogetabetterlookatthechild'sdrawingsorslidthepapertowardsthechildtolethimdrawonit.Toslidethepaper,theuserslidtwongersonthetablet.Thismotionmimickedusingtwongerstoslidearealpieceofpaperinastraightline. 98

PAGE 99

Whenthechild'sdrawingofapersonslidoffthemonitorandontothetablet,shadedcirclesappearedoverbodyparts(similartoFigure 5-4 ).Thesecirclesindicatedhotzoneswheretheusercould`point'usingthestylus.Theusercouldtaponabodypartandaskthechildtonamethebodypartorcolorinthedrawing.Theusercouldalsoaskthechildtocountthebodypartsastheusertappedthemonebyone. 5.3.2.2LowPhysicalityInterfaceThelowphysicalityinterfaceabstractedtheobjectinteractionintobuttonsinamouse-basedapplication(Figure 5-4 ).ThelowphysicalityapplicationcommunicatedwiththerestofthesystembutwasnotintegratedintotheMRenvironment.TheapplicationwasaC#applicationdisplayedonthesame17LCDusedfortheMRpaper,andtheusercontrolledtheapplicationusingastandardwirelessmouse.Thetabletwasphysicallyremovedwhenthelowphysicalityinterfacewasused.Theapplicationpresentedbuttonsfordrawingeachshapelabeled`Drawshape'.Whentheuserclickedabuttontodrawashape,thebuttonwasreplacedbyanimageoftheshape(Figure 5-4 ).Theimageswereperfectrepresentationsoftheshaperatherthanhand-drawnimages.Theperfectrepresentationsemphasizedthattheuserwasnotdirectlymanipulatinganobjectintheenvironment.Whenthechilddrewashape,abuttonlabeled`ShowChild'sDrawing'appeared.Clickingthebuttonreplacedthebuttonwithanimageofthechild'sdrawing.ThisadditionalstepemphasizedthedisconnectbetweenthelowphysicalityinterfaceandtheMRenvironment.Thechildcouldnotdirectly`draw'onthepaperapplication.However,thechildstillmadedrawingmotions,andthebuttondidnotappearuntilthechildnishedthedrawingmotion.Becausethelowphysicalityinterfacedidnotinvolvedirectmanipulationofthepaper,therewasnoequivalentforslidingthepaperbackandforth.Thepositionofthe17LCDintentionallyblockedthelowerportionofthechild.Thispreventedtheuserfromseeingthechild'shandwhenthechilddrewshapes.Seeingthe 99

PAGE 100

child'shanddrawingdirectlyonthevirtualtablewithoutvirtualpaperwouldbeodd,andincludingthevirtualportionofthepaperwouldcreatetheimpressionthatthechildcoulddirectlymanipulateobjectslikeinthenaturalinterface. 5.4UserEvaluationTheuserstudy'sprimarygoalwastoexaminetheeffectoftheECA'sinteractionphysicalityonsocialpresence.Thestudyalsoexploredtheexam'seffectivenessasatrainingtoolanditsusability.Thestudyusedawithin-subjectsdesignwhereparticipantsperformedtwoexams,onewitheachphysicalityinterface.Toavoidparticipantsassessingthesamechildmorethanonce,wecreatedtwochild-parentpairs.Bothofthesechildrenhadsimilardevelopmentalstatusesandbothparentshadsimilarconcerns;onlyappearance,voice,andminordetailsdifferentiatedthepairs.Wecounter-balancedboththeorderofphysicalityandtheorderofchild-parentpairs,resultinginfourgroups.Wealsocreatedathirdchild-parentpairthatinstructedparticipantsonhowtousethesystemasapartofaninteractivetutorial. 5.4.1ParticipantsTheuserpopulationfortheevaluationconsistedofhealthprofessionsstudentswithpediatricdevelopmentalexamtrainingandpediatricresidents.Participantswererecruitedbye-mailannouncementoveraperiodofthreeweeks.Theannouncementofferedparticipantstheopportunitytopracticeperformingadevelopmentalexamonavirtualchildaspartofatrainingexercise.Participationwasvoluntary,andparticipantswerecompensatedwitha$20giftcard.Somestudentsalsoreceivedextracreditforparticipating,butnoparticipantsweregradedaspartoftheuserevaluation.Intotal22peopleparticipatedintheevaluation.Theaverageagewas25.45(SD=5.11)and18participantswerefemale.Tenparticipantswereundergraduatenursingstudents,sevenweremedicalstudents,threewerepediatricsresidents(physiciansthathavenishedtheirdegreesbutarestillinspecializedpediatricstraining),onewasa 100

PAGE 101

graduatenursingstudent,andonewasaphysician'sassistantstudent.Allbutonehadpriorexperienceeitherperformingorobservingpediatricdevelopmentalexams.Participantswereevenlydistributedintothefourcounter-balancedgroups.Allgroupshadsimilargenderandstudent-typecompositions. 5.4.2ProcedureAllfourgroupsfollowedidenticalstudyprocedures(Figure 5-5 ).Thestageswere: 1. (Consent)Completedaninformedconsentformandwerebriefedaboutthegoalsandscopeofthetrainingexercise. 2. (Tutorial1)Interactedwiththetutorialchild-parentpairandlearnedtouseonephysicalityinterface 3. (Pre-ExerciseSurvey)Completedademographicsandbackgroundsurvey 4. (Exam1)Performedadevelopmentalexamwithonephysicalitychild-parentpair 5. (Mid-ExerciseSurvey)Completedafeedbacksurvey 6. (Tutorial2)Interactedwiththetutorialchild-parentpairandlearnedtousetheotherphysicalityinterface 7. (Exam2)Performedadevelopmentalexamwiththeotherphysicalitychild-parentpair 8. (Post-ExerciseSurvey)Completedafeedbacksurveyandafree-formfeedbackinterview 5.4.2.1TutorialsThetutorialstaughttheparticipanthowtointeractwiththevirtualhumans.Thetutorialsalsogavetheparticipantachancetopracticethedifferentphysicalityinterfaces.Duringthetutorial,avirtualparentinstructedtheparticipanthowtoverballycommunicatewiththevirtualhumansanduseonephysicalityinterface.Theparentspokethemajorityofthetimeandpromptedtheparticipantwithwhattosayanddo.Theparent'sinstructionswerethesameinallconditionsexceptwhenreferringspecicallytotheobjectinteractioninterface.Tutorial1hadfourphases: 101

PAGE 102

1. Introduction-Theparentrstintroduceshimselfandthechild,thenexplainsthegoalsandscopeoftheexercise. 2. Namingcolors-Theparentexplainshowtousetherstphysicalityinterfacetopointatdifferentcoloredpartsinthechild'sdrawingandaskthechildtonamethecolor. 3. Drawingshapes-Theparenttoldtheparticipanttodrawasquare,askthechildtonametheshape,andaskthechildtocopytheshape.Thechildthencopiedtheshapeincorrectly(drawingatriangleinsteadofasquare),andtheparenttoldtheparticipanttodrawtheshapeagainwhiletellingthechildwhattopayattentionto.Askingthechildtorepeatafailedtestcommonlyoccursindevelopmentalexams;onlyrepeatedfailuresshouldbeconsideredtruefailures. 4. Closing-Theparentinstructedtheparticipanthowtoaskboththeparentandchildquestions,andhowtodelivertheexamresultstotheparent.Tutorial2wasasubsetofTutorial1andonlyincludedthenamingcolorsanddrawingshapesphases.Bothtutorialsusedthesamechild-parentpair,andthetutorialchild-parentpairwasnotusedineitherofthedevelopmentalexams. 5.4.2.2DevelopmentalExamsThedevelopmentalexamsfollowedfourstepsoutlinedexplicitlyonasheetofpapergiventotheparticipants: Findoutwhattheparent'sconcernsare Assessthechild'spersonal-social,nemotor-adaptive,andlanguageskills Deliveryourassessmenttotheparent AddressanyquestionsandconcernstheparenthasTheparentbegantheexambyintroducinghimselfandthechildandexpressingconcernthatthechildisbehindcomparedtootherchildren.Iftheparticipantaskedformoreinformation,theparentwouldelaborateonhisconcernswithspecicexamplesofthingsthechildcannotdoorisnotgoodat.Afterinterviewingtheparent,participantswentontoassessingthechild'sskills.Tohelpassessthechild'sskills,participantsreceivedaprintedlistofskillstakenfromtwostandarddevelopmentalscreeners,theDenverII[ 29 ]andASQ[ 4 ].These 102

PAGE 103

skillsincludebothne-motorandlanguageskills.Forne-motorskillssuchascopyingacircle,thesheetalsoincludeddescriptionsofwhattopassandfail(e.g.foracircle,passanyenclosedshapeandfailcontinuousroundmotions).Forlanguageskills,thesheetlisteditemstoaskthechild(e.g.phonenumber)butlefttheparticipantfreetophrasethequestionhowevershedesired.Topassthelanguageskillstest,thechildshouldknowatleastfourofthesixitemsonthislist.Bothchildrenonlyknewfouroftheitems,andthechildrenanswered`No'or`Idon'tknow'fortheothertwoitems.Thiscreatesanopportunityforambiguity.Participantscouldinterpretthisasthechildtrulynotknowingtheanswer,thechildbeingshy,ortheapplicationmakingamistake.Whentheparticipantwasdoneassessingthechild,shedeliveredtheresultsofherassessmenttotheparent.Bothchildrenwerecreatedtobeinthe50thpercentileandhencenormalfordevelopment.However,someparticipantsassessedoneorbothofthechildrenasslightlybehind.Thishappenedwhenaparticipantincorrectlyfailedthechildforaskill(e.g.failingacirclebecauseitisnotroundenoughwhenanyenclosedformshouldpass).ThisalsohappenediftheparticipantaskedalargenumberunexpectedquestionsandreceivetoomanyIdon'tknowresponses.Afterreceivingtheresults,eachparentagainexpressedconcernthatthechildwasbehindhispeers(e.g.,Mysister'skidsallknewthealphabetathisage.Whydoesn'the?).Similarquestionscommonlyariseinrealexams,andexaminersmustlearntorespondinawaythatsatisestheparent.However,studentsdonottypicallyreceiveinstructiononthisskillandmaynothaveencounteredthissituationyet. 5.4.3MetricsThestudyevaluatedtheMRexamontwoaspects:socialpresenceandeducationalbenet.Forbothaspects,thestudyusedbothquantitativemeasuresintheformofself-reportsurveysandqualitativemeasuresintheformofpost-exercisefree-responsequestions. 103

PAGE 104

5.4.3.1Self-ReportSocialPresenceBothself-reportsurveyitemsandobservedbehaviorsmeasuredhowsociallypresentparticipantsperceivedthechild.Forself-reportsurveyitems,thestudyusedthesameveLikert-typeitemsadaptedfromBailensonetal.[ 9 ]asthestudiesinChapter 4 .BecausetheMRexaminvolvedtwovirtualhumans,thesurveysaskedeachitemtwiceandreplacedtheword`person'with`child'or`parent.'Thesurveyitemswereonaseven-pointscalefrom`Stronglydisagree'to`Stronglyagree.'Theveitems,withabbreviationsforreportingresultsinparentheses,were: (InRoom)IperceivethatIaminthepresenceofachild/parentintheroomwithme. (Watching)Ifeelthatthechild/parentiswatchingmeandisawareofmypresence. (NotReal)Thethoughtthatthechild/parentwasnotarealpersoncrossedmymindoften. (Sentient)Thechild/parentappearedtobesentient,conscious,andalivetome. (Computerized)Iperceivedthechild/parentasbeingonlyacomputerizedimage,notasarealperson.Asinpreviousstudies,weanalyzedtheself-reportsocialpresenceresultsusinganon-parametricANOVA-typestatistic. 5.4.3.2ObservedBehaviorAccordingtofurtherresearchbyBailensonetal.[ 8 ],behavioralmeasuresofsocialpresencemaybemoresensitivethanself-reportsurveyitems.Thebehavioralmeasureinourexperimentwasthetoneofvoicetheparticipantused.Peopletypicallytalktoyoungchildrenusingadistinct,child-specictoneofvoice.Anaudio-onlyrecordingofeachparticipantspeakingtothevirtualchildwasplayedtoveindependent,third-partycoders.EachcoderansweredthequestionIstheadultusingatoneofvoicereservedforchildren(e.g.,higherpitch,extraemphasisoncertainwords)?withyes,no,orunsure.Ifatleastthreecodersansweredyes,thentheparticipantwasconsideredtohaveusedthechild-specictoneofvoice. 104

PAGE 105

Codersalsolistenedforanothersignoftalkingtoachild,acknowledgingwhatthechildsaid.EachcoderansweredthequestionDidtheadultacknowledgethechild'sanswersatleastonce(e.g.,repeatingtheanswer,sayinggoodjob)?Ifatleastthreecodersansweredyes,thentheparticipantwasconsideredtohaveacknowledgedwhatthechildsaid.WeanalyzedbothbehavioralmetricresultsusingMcNemar'stest.McNemar'stestissimilartoaChi-squaredtestbutforpaired(within-subjects)data. 5.4.3.3EducationalBenetChangesinaparticipant'scondencelevelswereusedasindicatorsoftheexam'seducationalbenet.Participantsusedsurveyitemstoratetheircondenceonthreeexamskills:interviewingaparent,interactingwithachild,andassessingachild'sdevelopment.Thesurveyitemsusedave-pointscalerangingfrom`Notatallcondent'to`Verycondent.'Thethreesurveyitemsappearedinthepre-exercise,mid-exercise,andpost-exercisesurveys.Thethreesurveyitems,withabbreviationsforreportingresultsinparentheses,were: (Interviewing)Interviewingaparentabouthis/herchild'sdevelopment. (Interacting)Interactingwiththechildduringadevelopmentalexam (Assessing)Assessingachild'sdevelopmentBecauseexperiencelevelmayplayaroleincondence,theparticipantsweredividedintotwogroupsforanalysis-inexperiencedandexperienced.Groupswerebasedontheestimatednumberofexamsperformedpriortothestudy.Themeanestimateofeightexamswasusedasthedividinglinebetweentheinexperiencedandexperiencedgroups.Theinexperiencedgroupcontained14participants,allofwhomestimatedhavingperformedveorlessexams.Theexperiencedgroupcontainedeightparticipants,allofwhomestimatedhavingperformed10ormoreexams.Noparticipantsestimatedhavingperformedmorethanvebutlessthantenexams.Theexperiencedgroupcontainedallthreeresidentsandveofthesevenmedicalstudents. 105

PAGE 106

Correctlyassessingthechild'sdevelopmentalstatuswasnotusedasametricfortworeasons.First,participantsthatperformedamorethoroughexamreceivedmoreIdon'tknoworNoresponsesbecausetheyaskedquestionsorgaveinstructionswithoutpre-programmedresponses.Theseresponsesinuencetheassessmentandcanbeinterpretedasasignofdevelopmentaldelays.Second,participantsdidnothaveaccesstoacompletescoringsheet.SomeparticipantswerefamiliarwiththeDenverIIscoringsheet,whereasotherswerefamiliarwiththeASQscoringsheet.Forfairness,neitherscoresheetwasprovided,soparticipantshadtoformanassessmentbasedsolelyontheirownjudgment. 5.5ResultsandDiscussionOverall,weobservedsignsofhighsocialpresenceinbothconditionsbutdidnotndanydifferencescorrelatedwithinteractionphysicality.Thissuggeststhatoccupancyofthephysicalspacemayhaveastrongereffectthaninteractionwiththeenvironment. 5.5.1Self-ReportSocialPresence 5.5.1.1Self-ReportSocialPresenceResultsTheECA'sinteractionphysicalitywasnotconsistentlycorrelatedwithadifferenceinself-reportsocialpresence.Onlyonesurveyitemshowedastatisticallysignicantdifference,theWatchingitemfortheparent(p=0.01),favoringthehighphysicalitycondition.Therewerealsomarginallysignicantdifferencesfortwootheritems,theNotRealitemfortheparent(p=0.07)andtheInRoomitemforthechild(p=0.08).Boththesemarginallysignicantdifferencesfavoredthelowphysicalitycondition.Statisticaltestsconrmedthattherewerenostatisticallysignicantdifferencesbetweenthechild-parentpairs.Additionally,thephysicalityorderandchild-parentpairorderwerenotcorrelatedwithanystatisticallysignicantdifferences. 106

PAGE 107

5.5.1.2Self-ReportSocialPresenceDiscussionTheresultsforself-reportsocialpresencesuggesttheECA'sinteractionphysicalitydoesnotaffecttheECA'ssocialpresence.Unlikewiththestudiesonoccupancy,wedidnotndanyconsistenttrendsofoneconditionratedbetterthantheother.WhiletherewasastatisticallysignicantdifferencefortheWatchingitemfortheparent,themagnitudeofthisdifferencewassmall,only0.28(Figure 5-7 ).Additionally,theparent'sinteractionphysicalityremainedconstantinbothconditions,sophysicalityisunlikelytohavecausedanydifference.Thechild'sinteractionphysicalitywasvariedbetweenconditions,buttherewerenosignicantdifferencesforanyoftheself-reportsocialpresenceitems.TherewasamarginallysignicantdifferencefortheInRoomitem,butthemagnitudewassmall,only0.36favoringthelowphysicalitycondition(Figure 5-6 ). 5.5.2ObservedBehavior 5.5.2.1ObservedBehaviorResultsWeobservedbehavioralsignsofhighsocialpresenceinbothconditions,butfailedtondanycorrelationwiththeECA'sinteractionphysicality.Recordingsfromtwoparticipantswerelostduetotechnicaldifculties,soobservedbehaviorresultsonlyinclude20participants.Therewasnosignicantdifference(p=0.13)betweenthetwoconditionsfortheuseofthechild-specictoneofvoice.Fourparticipantsusedthechild-specictoneofvoiceinonlythehighphysicalitycondition.Fourmoreparticipantsdidnotusethechild-specictoneofvoiceineithertheloworhighphysicalitycondition.Theremainingtwelveparticipantswithvideorecordingsusedthechild-specictoneofvoiceinbothconditions.Therewasalsonosignicantdifference(p=1.00)betweenthetwoconditionsforacknowledgingthechild.Eighteenparticipantsacknowledgedthechildinbothconditions.Thiseighteenincludesthefourparticipantsthatdidnotusethechild-specic 107

PAGE 108

toneofvoiceineithercondition.Oneparticipantonlyacknowledgedthechildinthehighphysicalitycondition.Onemoreparticipantdidnotacknowledgethechildineithertheloworhighphysicalitycondition.Wealsoobservedotherbehaviorthatsuggestedhighsocialpresence.Someparticipantstookextrastepstoengageandencouragethechild.Forexample,oneparticipantaskedthechildIlikeyourouttyouhaveon.Wereyouabletoputthatonyourself?insteadofdirectlyaskingtheparentCanhedresshimselfinthemorning? 5.5.2.2ObservedBehaviorDiscussionTheobservedbehaviorresultssuggestthatparticipantsperceivedbothconditions'virtualhumansassociallypresent.Allparticipantsdemonstratedatleastoneofthetworealisticbehaviors,and11demonstratedbothbehaviorsinbothconditions.Whiletherewasnostatisticallysignicantdifferenceinuseofthechild-specictoneofvoice,itisinterestingthatfourparticipantsusedthetoneonlyinthehighphysicalityconditionbutnoneuseditonlyinthelowphysicalitycondition.OnepossibleexplanationforthelackofstatisticalsignicanceisthattheECA'sinteractionphysicalityhasarelativelyweakeffectonsocialpresence,andthisweakeffectmasmaskedbythestrongereffectsofoccupancy.Bothconditionsusedvirtualhumanswithhighoccupancyofthephysicalspace.Thevirtualhumanswerelife-size,positionedcorrectly,supportedawiderangeofvalidviewpoints,andwereblendedwiththerealenvironment.However,theresultsalsosuggestthatnotallparticipantsperceivedthechildassociallypresent.Fourparticipantsacknowledgedthechild'sresponsesbutdidnotspeaktoeitherchildinthechild-specictoneofvoice.Itispossiblethatthefourparticipantsalsodonotusethattoneofvoiceevenwithrealchildren.However,forlogisticalreasonswewereunabletorecordparticipantsinteractingwitharealchildforcomparison. 108

PAGE 109

5.5.3FreeResponseFeedback 5.5.3.1FreeResponseFeedbackResultsSeveralparticipantswithdevelopmentalexamexperienceconsideredthechildmorerealisticthantheparent.Participantscommentednotontheparent'sappearanceoranimationbutinsteadonhisbehavior.Specically,theparentneverinterjectedintheexamandoverallseemedlessinvolvedthanarealparent.Oneexperiencedparticipantcommented,Theparentneedsalittlework,notasrealistic.Thekidwasgood,veryinteractive...Usuallyparentsaremoreoverbearing,somethingyouhavetoworkwithinreallife.AnotherexpectedParentswouldbeabletogivelongeranswersorfollowupquestions.Also,alotofparentswillencouragethechildduringtheexaminationprocess. 5.5.3.2FreeResponseFeedbackDiscussionThefeedbackcomparingtherealismofthevirtualchildandvirtualparentsuggestthatbehavioralrealismmayhaveaneffectthatisnotnecessarilyreectedinself-reportsocialpresencesurveys.Boththechildandparentreceivedsimilarself-reportsocialpresenceratings.ThesendingsreectsimilarndingsinthestudiesinChapter 4 whereoneECAwasperceivedaslessplausiblethantheothers.Chapter 6 furtherexplorestheideaofplausibilityandhowitmayaffectsocialpresence. 5.5.4EducationalBenet 5.5.4.1EducationalBenetResultsThecondencescoresshowedtheinexperiencedandexperiencedgroupsstartedwithdifferences,butthedifferenceshrankafterusingtheMRexam(Figure 5-8 ).Intheinexperiencedgroup,71%ofparticipantsreportedacondencechangeforatleastoneofthethreesurveyitems,andcondencechangeswerecommonforallthreesurveyitems.Intheexperiencedgroup,condencechangeswerecommonforonlyonesurveyitem. 109

PAGE 110

Pre-exercise,theexperiencedgrouphadstatisticallyhigherscoresforInterviewing(p<0.05)andInteracting(p<0.05),butthedifferencewasnotstatisticallysignicantforAssessing(p=0.05)accordingtoapaired,one-tailedT-test.Post-exercise,thedifferenceswerenolongerstatisticallysignicantforanyoftheitems(Interviewingp=0.09,Interactingp=0.14,Assessingp=0.44).Intheinexperiencedgroup,someparticipantscondenceincreasedwhileothersdecreased.Eightofthe14inexperiencedparticipantsreportedacondenceincreaseforatleastoneitemandfourofthe14reportedacondencedecreaseforatleastoneitem(tworeportedanincreaseforoneitemandadecreaseforanotheritem).Onlyfourreportednocondencechangeforallitems.Condencechangeswereaboutequallycommoninallthreesurveyitems(veinInterviewing,seveninInteracting,andseveninAssessing).However,asawhole,theinexperiencedgroupdidnotshowastatisticallysignicantchangeforanyofthethreeitemsaccordingtoarepeated-measuresANOVAanalysis.Intheexperiencedgroup,63%ofthecondencechangeswereforAssessing.Threeparticipants'self-reportedscoresforAssessingdecreasedandtwoincreased.TheInterviewingitemhadonlyoneparticipantchange(adecrease),andtheInteractingitemhadonlytwoparticipantschange(bothincreases). 5.5.4.2EducationalBenetDiscussionTheresultssuggesttheMRexamprovidedaneducationalbenettomostoftheinexperiencedgroupandsomeoftheexperiencedgroup.Intheinexperiencedgroup,71%oftheparticipantsshowedsomechangeincondence.Thechangewasnotstatisticallysignicantbecausesomeparticipants'condenceincreasedwhileothersdecreased.Thisvariancecouldbebecausesomestudentswerewell-preparedbutunder-condentwhileotherstudentswereless-preparedbutover-condent.Forwell-preparedstudents,theadditionalpracticeopportunityincreasedtheircondence.Forless-preparedstudents,theMRexamservedasawakeupcallthatemphasized 110

PAGE 111

theareastheylackedpracticein.Thisinterpretationissupportedbyfree-responsefeedback.OneparticipantsaidtheMRexamwasa`GoodideabecauseIdidn'trealizehowlittleIknewaboutpedsassessment.AnothersaidIthinkitwouldbeveryhelpfulbecauseI'mscaredtodeathofworkingwithpediatricpatients...itwouldbegreattobeabletositinhereforanhour.Formembersoftheexperiencedgroup,theMRexamcouldserveasarefresherondevelopmentalexams.Changesincondenceweremainlyseenfor`Assessingachild'sdevelopment'whichsuggestssomeparticipantsrealizedtheyhadforgottendetailsofdevelopmentalexams.Condenceincommunicationskills(InterviewingandInteracting)startedhighandremainedhighbecausethesearemeasuresofinterpersonalskills,notexamskills. 5.5.5OverallDiscussionTakentogether,theself-reportsocialpresencesurveyandobservedbehaviorresultssuggestthatoccupancyofthephysicalspacehasastrongereffectthaninteractionwiththeenvironment.Combinedwiththeeducationalbenetresults,thissuggestshighoccupancyECAscanbeusedassociallypresentconversationalpartners.MostlearnerstalkedtotheECAnaturallyandrealisticallydespitereceivingnoexplicitinstructionstouseadifferenttoneofvoiceoracknowledgethechild'sanswers.LoweroccupancyECAsmightnotelicitthesamerealisticinteractions,asobservedintherststudydescribedinChapter 4 .Thelowimpactofinteractionphysicalitymayberelatedtothenatureofthetask.Thepediatricdevelopmentalexamislargelyacommunication-focusedtask.Bothinterfacesallowedtheparticipantstosendandreceivetheneededinformation.Howeverinsomeotherscenarios,alowphysicalityinteractionmightnotcommunicatethesameamountofinformation.Forexample,Kotranzaetal.foundthatapurelyvirtualtouchcommunicatedlesseffectivelythanavirtualtouchaugmentedwithaphysicaltouchbyamechanicalarm[ 52 ].Thismaybebecausethetouchinthatscenariocommunicated 111

PAGE 112

fearandanxiety,emotionalcontentthatwasnotpresentinthepediatricscenario'sdrawings.Forscenarioswhereinteractingwiththeenvironmentcommunicatesfactualinformation,theresultsfromourstudysuggesteffortsdonotneedtobefocusedonincreasinginteractionphysicality.IncreasinginteractionphysicalitywithhighoccupancyECAscanbeaverychallengingtaskbecauseitmayrequiretheECAtophysicallymoveitsbodyandphysicallymanipulaterealobjects.Ifabstractmanipulationscanstillproducesimilarsocialpresenceresults,thencreatingeducationallybenecialECAinteractionsbecomessimplerandfaster. 5.6BroaderImpact-SimulatingVulnerablePopulationsOurresultssuggestvirtualhumanscansuccessfullysimulateonecase,ahealthypediatricspatient.Withinpediatricstherearemanyotherpotentialcases.Pediatricpatientsarealsopartofabroaderclassofpatients,vulnerablepopulations.Vulnerablepopulationsincludetheelderly,thephysicallydisabled,andtheintellectuallydisabled.Mostvulnerablepopulationsdonothavestandardizedpatientsavailable[ 85 94 ].Withinpediatrics,theMRdevelopmentalexamopensthepossibilityofexposingstudentstosensitivecasessuchassignsofchildabuse.Potentialchildabusecasesare,unfortunately,encounteredwithrealchildren,andexaminersmustpracticeproperlyhandlingthesituation.Examinersmustalsolearntodistinguishbetweensignsofchildabuseorneglectandnormalchildhoodaccidents.Suchcasescanbesafelypracticedwithvirtualchildrenandvirtualparents.Outsideofpediatrics,thereareotherprotectedpopulationssuchastheintellectuallydisabled.Anintellectuallydisabledpersoncouldpossiblybesimulatedbyastandardizedpatientwithespeciallygoodactingskills,butmoststandardizedpatientsarenottrainedactors.However,onegoodactorcouldbemultipliedusingMRvirtualhumantechnology.Ifagoodactorisrecordedonce,hervoicecouldbereusedhundredsoftimesaspartofavirtualhuman. 112

PAGE 113

Figure5-1. Highinteractionphysicalitychild TheWizard-of-Ozcontroltechniquecanleveragetheexistinglargepooloftrainedstandardizedpatients.Wizard-of-Ozcontrolallowsstandardizedpatientstoplayrolesthatdonotmatchtheirappearance.Forexample,atwenty-year-old,healthy,femalestandardizedpatientcancontrolafty-year-old,malevirtualpatientwithanamputatedleg.Thestandardizedpatientcaneasilytakeonthisdisparaterolebecausethevirtualpatienthasasetofpre-programmedresponseswithhigh-delityaudioandanimation. 113

PAGE 114

Figure5-2. Lowinteractionphysicalitychild.Note:lowerscreenimagesimulatedforvisibility. Figure5-3. VirtualchildandMRpaper.Thechildspansthetopandmiddledisplays(bothLCDmonitors),andthepaperspansthemiddleandbottomdisplays(anLCDmonitorandatablet).Fish-tankVRensuresthechildandpaperremainvisuallyconsistentacrossdisplays. 114

PAGE 115

Figure5-4. Paperapplication.Thelefthalfcontainsbuttonsfortheusertodrawshapes.Therighthalfdisplaysthechild'sdrawings.Thebuttoninthebottomrightappearswhenthechilddrawsashape.Clickingthebottomrightbuttonwillrevealthechild'sdrawing. Figure5-5. Approximatetimespentineachstageoftheuserevaluation.Totaltimewasapproximately45minutes.Participantscouldchoosetospendasmuchtimeastheywantedineachstage. 115

PAGE 116

Figure5-6. Socialpresence(meanandstandarderror)forthelowandhighphysicalitychild.Higherindicatesbetter.*Scoresfornegativestatementshavebeeninvertedforeaseofcomparison. Figure5-7. Socialpresence(meanandstandarderror)fortheparentinthelowandhighphysicalityconditions.Higherindicatesbetter.*Scoresfornegativestatementshavebeeninvertedforeaseofcomparison. 116

PAGE 117

Figure5-8. Averagecondencelevelsfortheexperiencedandinexperiencedgroups.Forreadability,standarddeviationsareshownaspositiveerrorbarsfortheexperiencedgroupandnegativeerrorbarsfortheinexperiencedgroup. 117

PAGE 118

CHAPTER6IDENTIFYINGFACTORSBEYONDPHYSICALITYThischapterdescribesaqualitativestudyexploringtheexperienceofarealisticECAinteractionfromtheperspectiveofnursingstudents.PersonalContributions:Idesignedandconductedthestudy,recruitedalltheparticipants,transcribedalltheinterviews,andanalyzedthedata.Iwroteallthetextforthepaper.Collaborators:Dr.ErikBlackandDr.CaseyWhiteadvisedmeonstudydesignandanalysisprocedure.Dr.ErikBlackalsoprovidedgiftcardsasparticipantcompensation.RelevancetoThesis:TheresultsfromthischapterprovideevidencethatanecessaryconditionforsocialpresenceisanECAdemonstratingbehaviorconsistentwiththetrainee'sexpectations. 6.1OverviewOurpreviousstudiessuggestedthathighphysicalityECAscanbeperceivedandtreatedrealistically,butthereareotherqualitiesbesidesphysicalitythatmayalsocontributestronglytoperceptionsofrealism.Inourholisticevaluationofoccupancy(Chapter 4 ),onesuchqualitywasshowinganappropriatelevelofconcernandemotion.ThefamilymemberECAdidnotappeardistraughtenough,andasaresultwasperceivedaslesssociallypresentthanthenurseECAeveninthehighphysicalitycondition.Theadditionalqualityintheindependentcomponentsevaluationofoccupancyappearedtobebeingabletorespondappropriatelytoquestions.OnedoctorECAwasunabletosatisfactorilyanswermanyoftheparticipants'questions,andthesameECAwasunabletoslowdownhisspeech.Inourevaluationofinteraction(Chapter 5 ),allECAswereperceivedassociallypresent,butsomeparticipantscommentedthatarealparentwouldinterjectmoreoften.Theseidenticationsofotherqualitieswerebasedonfree-formfeedbackprovidedbystudyparticipants.Whilethisfree-formfeedbackprovidedusefulhintsastowhatthe 118

PAGE 119

ECAsmightbemissing,thefree-formfeedbackwasnotdesignedtospecicallycapturethisinformation.Thefree-formfeedbacktooktheformoftypedfree-responsesurveyquestionsandunstructureddebrienginterviewquestions.Toformallyexplorethesemissingqualities,wechosetoconductaqualitativestudyfocusedspecicallyontheexperienceofanECAinteraction.Aqualitativeapproachwasmoreappropriatethanaquantitativeapproachbecausethisstudysoughttoidentifythemyriadcomplexfactorsotherthanphysicalitythatmayaffectsocialpresence.QuantitativestudiessuchastheonesweconductedinChapters 4 and 5 wereappropriateforquantifyingtheeffectofcertaindimensionsofphysicalitythatwehadalreadyidentied.Qualitativestudiesinsteadprovideadeeperunderstandingofanexperienceinrelationtoanindividualorasmallpopulation.Therearemanyapproachestoqualitativeresearch.Inourliteraturereview,weidentiedphenomenologyandgroundedtheoryasthetwoqualitativeapproachesmostsuitabletomyresearch.Thetwoapproacheshavemuchincommonbutdifferintheirgoalsandanalysis.Phenomenologyseekstoexplainwhatanexperienceis,whereasgroundedtheoryseekstoexplainwhysomethinghappens.Afterexaminingthetranscriptsfromthethreeparticipantsontherstdayofmystudy,wedeterminedaphenomenologicalapproachwasmoreappropriate.Intheinterviews,theparticipantsexplainedwhatconstituted`real'tothem.However,theparticipantswereunabletoarticulatewhytheyhadrealisticresponsestotheECAsorperceivedoneECAasmorerealisticthantheother.Thereasonsforthisappearedtobedifculttoexpress,possiblybecausetheparticipantsthemselveswerenotconsciousofthereasons.ThoughthisstudyisunabletoexplainthemechanismsbywhichECAselicitrealisticbehavior,thisstudyisabletoidentifyfactorsthatcontributetoECAselicitingrealisticbehavior.Aspartofthediscussion,wecanspeculateastowhythefactorsweidentiedinthisstudyelicitfeelingsofsocialpresenceandrealisticuserbehavior. 119

PAGE 120

Becausefewpeopleinteractwithvirtualhumansonaregularbasis,wechosetoimplementauserstudywhereparticipantsrstinteractedwithvirtualhumansthenwereinterviewedabouttheirexperience.TheexperienceforthisstudywasperformingapediatricdevelopmentalexamonavirtualchildandhisvirtualparentusingthesystemdescribedinChapter 5 .UnliketheexperimentsinChapters 4 and 5 ,thisstudyusedonlyasingleconditionsoallparticipantswouldhaveequivalentopportunitiesintheirexperiences.Aftereachparticipantunderwenttheexperience,weconductedasemi-structuredinterview.Weanalyzedtheparticipantinterviewsandidentiedseveralqualitiesthatconstitutetheword`real'totheseparticipants.Resultssuggestthatparticipantsfocuslargelyontheconversationandotherchannelsofinformation.Theyfoundtheconversationrealbecauseitmatchedtheparticipants'expectationsofwhatarealchildandparentwouldsayduringadevelopmentalexam.However,theECAsfailedtocommunicateemotionsthroughfacialexpressionsandbodylanguage,andthisdetractedfromrealism. 6.2SystemDescriptionThisstudyusedtheMRpediatricdevelopmentalexamdescribedinChapter 5 .Wemadethefollowingmodications: PositionedbothECAsassittinginsteadofstandingandplacedthedisplaysinphysicalchairs(Figure 6-2 ).Theparticipantalsosatdown.Severalparticipantsandmedicalprofessionalsclariedthatsittingismorecommonthanstandingfordevelopmentalexams. Replacedthe17LCDinaverticalpositionwitha12LCDatonthedesk(Figure 6-2 ).Thiswasnecessarybecausetheseatedchild'sdisplaywaslowerthanthepriorstudy,andtheverticalLCDwouldhavecompletelyblockedtheseatedchild'sdisplay. Removedthehighphysicalitypaperinterfaceandonlyusedthelowphysicalityinterface.ThiswasnecessarybecausetheverticalLCDwasremoved,andtheresultsfrommypriorstudysuggestedthelowphysicalityinterfacecouldelicitequallyhighsocialpresenceasthehighphysicalityinterface. 120

PAGE 121

Addedpre-recordedspeechesfortheparenttoencouragethechildtocompletethetest.Forexample,hecanaskthechildtoPleasetrythat.Thisadditionwasbasedonfeedbackfromtheparticipants. Recordedaprofessionalvoiceactorfortheparent.ThepriorvoiceactorsweremycolleaguesAndrewRobbandAndrewCordar.Onlyonevoiceactorwasneededforthisstudybecausethisstudyonlyincludedonedevelopmentalexam.Aswithmypreviousstudies,weusedaWizardofOzcontroltechnique.Icontinuedtobetheoperatorcontrollingthecharactersandagainsatoutsidetheparticipant'seld-of-view. 6.3StudyDesign 6.3.1ParticipantsTheparticipantsforthisstudyconsistedofnursingstudentsattheUniversityofFlorida.Participantswererecruitedbyane-mailsenttoamailinglist.Thee-mailsoughtstudentswithclassroomexperiencewithpediatricdevelopmentalexams.Participationwasvoluntaryandparticipantseachreceiveda$20giftcardbutdidnotreceiveagradeorextracredit.Intotal,14students(13undergraduate,onegraduate)participatedinthestudy.All14werefemalesintheagerangeof18-23.Fourteenparticipantsisatypicalsamplesizeforphenomenologicalstudies[ 27 ].Ratherthanseekingasamplesizelargeenoughtodetectstatisticallysignicantdifferences,phenomenologicalstudiesseeksaturation.Saturationisachievedwhenaddingadditionalparticipantsnolongeraddsnewmeaningunits(seeSection 6.3.3 ).Weachievedsaturationat12participantsandrecruitedtwomoretoconrmthis. 6.3.2StudyProcedureThestudyconsistedoftwosessionsofapproximately30minuteseach.FirstSessionIntherstsession,theparticipantcompletedthefollowing: 1. Informedconsentform 2. Interactivetutorialwhereavirtualfatherandvirtualchildinstructtheparticipantonhowtoaskquestionsanddrawshapes 121

PAGE 122

3. Demographicsandbackgroundsurvey 4. Developmentalexamonavirtualchildwithavirtualfatherpresent 5. Feedbacksurveyincludingquantitativeself-reportsocialpresenceLikert-typeitems[ 6 ].Thoughthisstudywasprimarilyqualitativeinnature,wecollectedquantitativedatatocomparewithourpriorstudywithasimilarsystem(Chapter 5 ) 6. Semi-structuredfeedbackinterviewDuringthedevelopmentalexam,participantswereprovidedwiththefollowingoutlineonasheetofpaperthattheykeptforreference: 1. Findoutwhattheparent'sconcernsare 2. Assessthechild'spersonal-social,nemotor-adaptive,andlanguageskills 3. Deliveryourassessmenttotheparent 4. Asktheparentaboutwhathe/shedoestohelpthechild'sdevelopment 5. AddressanyquestionsandconcernstheparenthasToencourageparticipantstoaskabasicminimumsetofquestions,participantswereprovidedwithachecklistcontainingtheage-appropriateskillstestsfromstandarddevelopmentalscreeners.Participantswerefreetoaskadditionalquestionsastheyfeltappropriate.Thesemi-structuredfeedbackinterviewcontainedthefollowingquestions: Didyouexperienceanyglitchesortechnicalproblems? Howwelldidthechildsimulateachild? Howwelldidtheparentsimulateaparent? Whatdoesthewordrealorrealisticmeantoyouinthiscontext? Weretherethingsthatmadethecharactersseemmorerealistic? Weretherethingsthatmadethecharactersseemlessrealistic?Toencourageparticipantstoelaborateonanswers,additionalrelatedquestionssuchasWhatdoyoumeanbyverywell?wereaddedasneeded. 122

PAGE 123

SecondSessionInthesecondsession,theparticipantwasshownvideoclipsoftheassessmentsheperformedduringtherstsessionandaskedwhatshewasthinkingatthetime.Thevideoclipsincludedseveralscriptedmomentsinthedevelopmentalexam.Manyofthesemomentscouldbeinterpretedaseitherashychild'snaturalbehaviororarticialintelligenceerrors,andwewereinterestedtoseehowparticipantsinterpretedthem.Thesescriptedmomentswere: Childinitiallyrefusingtodrawashape Fatheraskingthechildtotrydrawingtheshapethenthechildcooperates Childnotanswering`Whatisyourphonenumber' Childanswering`Whatisyourlastname'withhisrstname FatheransweringaquestionwithavariantofI'mnotsure,I'dhavetoaskmywifeSomeparticipantsexperiencedonlyasubsetofthescriptedmoments.Thescriptedmomentsalldependedontheparticipantsaskingappropriatequestionsorgivingappropriateinstructions.Someparticipantschosenottotestoneormoreoftheskillsincludedontheprovidedlistoftests,sotheydidnotencountersomescriptedmoments.Afterwatchingthevideoclips,participantswereasked: Howwelldidthechildsimulateachild? Howwelldidtheparentsimulateaparent? Wasthereanythingwecouldhavedonetomaketheexperiencebetter?Thesecondsessionwasconductedbetweenonetothreedaysaftertherstsession,dependingontheparticipant'savailability.Thisdelaygavemetimetoreviewthevideorecordingsandselecttherelevantvideoclips.Thisdelayalsogavetheparticipanttimetoreectontheexperience.Participantswerenotexplicitlyinstructedtoreectonexperiencebutcouldifdesired. 123

PAGE 124

6.3.3AnalysisProcedureOuranalysisfollowedCreswell'sphenomenologicalmethods[ 27 ].First,Itranscribedtheaudiointerviewsintowritteninterviews.Thistranscriptionprocessimprovedmyfamiliaritywiththecontentoftheinterviews.Ifurtherimprovedmyfamiliaritybyreadingthetranscriptsseveraltimesandformedanoverallimpression.Ithenreadthrougheachtranscriptlinebylinelookingforsignicantstatementsdescribingsomequalityoftheexperience.ThesesignicantstatementswereextractedandtaggedwiththeparticipantIDandwhichinterview(rstorsecond)thestatementcamefrom.Ithenre-readthetranscriptsandextractedsignicantstatementsasecondtimetoensurenoneweremissed.Ithenreadthroughallthesignicantstatementsandidentiedmeaningunits.Meaningunitsarethehighlevelmeaningofastatement,andseveraldifferentphrasingssharethesamemeaningunit.Forexample,thestatementsIthoughtitwasreallyrealthewaytheireyesandheadsfollowedmeandThewaytheykeptlookingatmemadethemfeelrealbothcorrespondtothemeaningunittheeyeandheadmovementsaffectedrealism.'Itaggedeachstatementwithoneormoremeaningunits.Multiplemeaningunitswereneededforlongerstatementsincludingcompoundsentencesorlists.Afteridentifyingallofthemeaningunits,Igroupedsimilarmeaningunitsintothemes.ThisgroupinghelpedmeorganizetheresultssoIcouldpresentacoherentpictureofthephenomenon.Finally,Iassembledtablessummarizinghowmanyparticipantsmentionedeachthemeandmeaningunit.Thislaststepisnotpartofstandardphenomenologicalprocedures.IaddedthisstepbecauseIfounditusefulforreportinghowprevalentathemeormeaningunitwas.Thisallowedmetoidentifyareaswheresmallimprovementsmighthavethemostimpact.However,eventhemesormeaningunitsexpressedbyonlyoneparticipantstillprovideavaluableinsightintotheexperienceofaparticipant.This 124

PAGE 125

conceptisexpressedinhorizontalization,akeytenantofphenomenologicalresearch.Horizontalizationistheideathateverymeaningunitisofequalvalueregardlessofhowfrequentthemeaningunitis[ 27 ]. 6.3.4MyPersonalExperiencesInphenomenologicalresearch,theresearcherdescribeshisorherpersonalexperienceswiththephenomenoninordertosetthoseexperiencesasideasmuchaspossible.Theresearcherdoesnotformhypothesespriortoanalysis,butinsteadallowsmeaningsandthemestoemergenaturallyfromthedata.MypersonalexperiencewiththephenomenonoftreatinganECArealisticallyisbasedonobservinghundredsofusersinteractwithECAsoverthepastveyears.Inadditiontoconductingthestudiesdescribedinthepreviouschapters,Iconductedthreemorestudieswithatotalof84participants.IalsoassistedinconductingseveralotherstudiesinvolvingECAs.Outsideofstudies,IhavedemoedECAstohundredsofpeople,someofwhominteractedwithanECAforafewminutes.Basedontheseexperiences,InotedthreethingsbesidesphysicalitythatappearedtoimpactifauserwouldtreatanECArealistically.First,theusermusttaketheinteractionseriously.Inthecontextofvirtualpatients,serioususerswilltreattheinteractionasaneducationalopportunity.Theseuserswillattempttocollectallthenecessaryinformationtoformacorrectdiagnosis.Userswhoarenotseriousmayinsteadtaketheinteractionasanopportunitytotesttheboundsofthesystemandtrytobreakit.Thistestingofthesystemismostcommonamongpeopleseeingademoofthesystemandrareamongparticipantsinastudy.Second,theECAsmustshownanappropriatelevelofemotion.Asobservedinmyholisticevaluationofoccupancy(Chapter 4 ),ifanECAdisplayslessemotionthanexpected,increasingphysicalitymayactuallydecreasesocialpresence.IhavealsoreceivedfeedbackinotherstudiesthattheECAdidnotexpressenoughemotion 125

PAGE 126

throughfacialexpressionsortoneofvoice.ThismaybeexplainedastheECAlookinglikearealpersonbutnotactinglikearealperson.Lastly,theECAmustbeabletosupplyanappropriateresponsetoquestions.Intheindependentcomponentsevaluation,thehandoffdoctorECAwasratedlowerforintelligencethantheothertwoECAs.Thismayhavebeenbecausehewasunabletorespondappropriatelytomanyquestions.First,hewasunabletoslowdownwhenaskedtobyparticipants.Second,heansweredsomequestionswithI'mnotsureyou'dhavetocheckhischartwhenanyrealdoctorwouldknowtheanswers.Third,heansweredotherquestionswithtoomuchinformation,repeatingwholestatementswhenonlyafewwordsofthestatementwereneeded.Ihavealsoseensimilarbehaviorindemos.PeoplewillloseinterestquicklyiftheECAisunabletosatisfactorilyanswermultiplequestions. 6.4ResultsandDiscussionIntotal,weextracted283signicantstatementsandassigned57uniquemeaningunitstothesestatements.Wegroupedthe57meaningunitsinto10themes(Table 6-1 ).Sevenofthesethemesarerepresentedhereassubsections.Meaningunitsfromtheotherthreethemesarealsoincludedinthediscussionofotherthemes.Thissectionfollowsatypicalformatforreportingphenomenologicalresults.Weexplainwhatthethemesandmeaningswefoundareandprovideseveralexamplesintheformofrepresentativequotes. 6.4.1SocialPresenceSocialpresenceistreatingthevirtualhumansasiftheywererealpeople.Observationalandquantitativeself-reportmeasuresofsocialpresenceconrmthatthevirtualhumansinthedevelopmentalexamwereperceivedashighlyrealbytheparticipants.Eventheparticipantswhodidnotexplicitlymentionsignsofhighsocialpresenceintheirinterviewsdemonstratedrealisticbehaviorsimilartomypriorstudy(Chapter 5 )whileconductingthedevelopmentalexam.Allparticipantsspoketothe 126

PAGE 127

childinahigherpitchandusedwordstoencouragethechildsuchascongratulatinghim.Theself-reportsurveyscoresforsocialpresence[ 6 ]werealsopositive(Figure 6-1 )andsimilartoourpriorstudy(Chapter 5 ).Becauseofthesesocialpresenceresults,weincludedall14participantsinouranalysis. Figure6-1. Socialpresence(meanandstandarderror)forthechildandparent.Higherindicatesbetter.*Scoresfornegativestatementshavebeeninvertedforeaseofcomparison. Theobservedbehavioroftalkingtothechildinahigherpitchandusingwordstoencouragehimmaybepartiallyinuencedbytheparticipants'attitudes.FiveparticipantsspecicallymentionedthattheyweretalkingtotheECAsinarealisticmannerbecausetheyweresupposedtobeplayingtheroleofanurse.P1describedit,Imeanhewasakid,helookedlikeakid,IknewhewasakidandI,like,knowIwassupposedtobeplayingtheroleandactinglikeitwasarealkid.Duringinterviews,nineparticipantsreportedatleastoneoftwosignsofsocialpresence-feelinganxiety[ 67 ]andforgettingthatthevirtualhumanswerecomputers.Feelingsofanxietyincludedfeelingpressuretorespondtothevirtualparent'squestionsinatimelymannerandfeelingnervousaboutsayingtherightthings.Six 127

PAGE 128

participantsexplicitlymentionedfeelingsomekindofanxiety.P5describedit,Whenhe[theparent]startedaskingmorequestions,Iwasbeginningtofeellikeahh.Thesefeelingssuggesttheparticipantsperceivedthevirtualhumansasrealpeoplethathadfeelingsandcouldjudgetheparticipantifshemademistakes.Webelieveparticipants'anxietywasduetoperceptionsofthevirtualhumansasrealpeople.Fouroftheparticipantsexplicitlymentionedthatattimestheyforgottheyweretalkingtocomputersorvirtualpeople.P8describeditas,IforgotthechildwasacomputerwhenIwastalkingtothechild.However,itisalsopossibleparticipantsfeltthispressurebecausetheyknewanotherrealhuman(myself)waspresent,thoughIwasoutofviewbehindtheparticipantsandtheywereawaretheexamwasungraded.Onlyoneparticipantexplicitlymentionedmypresenceasaffectingperformance.P6said,Havingyoutherewasinterestingbecausewhenyougoinwiththeinstructorandyou'retryingtowatchwhatyou'regoingtosayandit'skindaputonagoodshowfortheinstructor. 6.4.2ConversationAllfourteenparticipantsmentionedaspectsoftheconversationascontributingtotheinteractionbeing`real.'Themostfrequentlymentionedaspectsofrealismwerethecontentofwhatthechildandfathersaid(86%),gettingresponsesbacktoquestions(57%),andthespeedofresponses(64%).Overall,itappearsthatwhenaskedwhattheword`real'meansinthiscontext,thecontextparticipantsconsideredwastheconversation,nottheappearanceofthevirtualhumans.ThisissupportedbytheveparticipantsthatfeltoneECAwasmorerealisticthantheotherbecauseshetalkedorinteractedwiththatECAmore. 6.4.2.1ContentTwelveparticipantsmentionedthatthecontentofwhatthechildandfathersaidmatchedwhattheyexpectedarealchildandfatherwouldsay.AsP8describedit,Theanswerstheygivetoquestionswereanswersthatyou'dexpecttohearfromahuman. 128

PAGE 129

Theseexpectationswerebasedonpastexperiencesworkingwithchildrenandparents.Allparticipantshadpriorexperienceworkingwithchildrenineitherclinicalsettingsornon-clinicalsettingssuchasbabysitting.Sixparticipantsfocusedinparticularonthefather'sconcernsaboutthechild.P6commented,WhenIaskedwhathisconcernswere,Ididn'tknowwhathewasgoingtosaybutheactuallysaidrealthingsandthatwascoolP3commented,Imeanthedadwasconcernedwhetherhischildwasdevelopingproperlyhewaslikeoh,theotherkidslearnedthatatschoolandIfeellikethat'showanormalpersonwouldact.However,participantsalsoreportednegativefeelingswhentheyencounteredthelimitationsofthesystem.Thefatheroftendidnothavepre-recordedanswerstospecicquestions,andasaresultgavedefaultresponsessuchasI'mnotsure,mywifewouldknowtheanswertothat.Theuseofdefaultanswerswiththechild(e.g.,Idon'tknow)didnotbothertheparticipants.P9stated,BecauseI'vemetchildrenlikethatandIhaven'tmetverymanyparentslikethatIguess.Likehe,theparentalmostseemstomethatIwouldbeconcernedabouthissocialskills.Buthe'sacomputer,so...Thissuggestsamismatchbetweenthecontentoftheconversationandherexpectationsofthecontent.ThemismatchemphasizesthattheECAisacomputerwithlimitedcapabilities. 6.4.2.2GettingResponsesBackEightparticipantsmentionedtheabilitytoaskquestionsandreceiveresponsesasanaspectofrealism.Forexample,whenaskedwhatrealmeantinthiscontext,P6responded,Beingabletofocusontryingtotalktothemandgettheiranswersback.Thisfocusongettingresponsesbackmayberelatedtotheparticipant'sexpectations.NoparticipantshadexperiencedasimilarECAinteractionbefore.Fiveparticipantsspecicallynotedthatthefreeformnatureofaskinganyquestionandreceivinganappropriateresponsecontributedtotherealism. 129

PAGE 130

Realismcouldbeemphasizedwhenthevirtualhumansfarexceededtheparticipants'expectations.Fiveparticipantsnotedtimeswherethevirtualhumansexceededtheirexpectations.Inoneinstance,P1askedthechildifhewasafanofalocalsportsteamandthechildsaidyes.Thisisunrelatedtotheexam,andwhenaskedaboutitP1saidIwasverysurprisedbecauseassoonasIaskeditIwaslikeoh,whydidIaskit,he'snotgoingtobeabletoanswerme.Ourvirtualhumansrespondedwithextremelyhighaccuracytoawidevarietyofquestionsbecausetheywerecontrolledbyahumanoperator.Noparticipantsindicatedtheywereawareofthehumanoperator,thoughsomeexpressedsurpriseathowadvancedthecomputersystemwas.WhilenocurrentautonomousECAsystemshavetheaccuracyofourhumanoperator,severaldoallowfreeformconversationsusingspokenortypedinput[ 40 48 69 ].Insomeinstances,aECAsystemmaybeunabletogenerateanappropriateresponsetoaspecicquestion.Onestrategyfordealingwiththisproblemistofallbacktodefaultresponses.Forthefather,thedefaultresponsetounexpectedquestionswasavariantofMywifewouldknowtheanswertothat.Inmostcases,participantsacceptedthisanswer.However,participantsalsoremarkedthatthefatherwasnotveryinvolvedinhischild'slife.Insomecases,participantsdecidedthisanswerwasunreasonableanddetractedfromthefather'srealism.Forthechild,thedefaultresponsetounexpectedquestionswasIdon'tknow.Inallbutoneinstance,participantsacceptedthisanswerasananswerarealchildwouldgive.InthisinstanceP10noted,IthinkitwasacomputerproblemcauseIthinkachildwouldunderstandhowtodrawapepperoni.Onlyfourparticipantsnotedinsomewaythattheseresponsesweredefaultresponses.P12commented,IthinkthatmyquestionmighthavebeenalittleconfusingsoIthinkhemighthavegottenjustagenericanswerinresponse. 130

PAGE 131

6.4.2.3SpeedofResponsesParticipantscommentedbothnegativelyandpositivelyonthedelaybetweentheparticipantnishingherquestionandanECAresponding.Whenaskediftherewereanyotherthingsthatmadethecharactersseemmorerealistic,P10responded,Thewaytheyrespondedtothequestions,andquicklytoo.Therewasn'talagoranything.Occasionalslowresponsesdetractedfromtherealism.WhenaskedDidtheyansweryourquestionslikearealparentandchildwouldhave?P3responded,Yeah.Iwouldsayforthemostpart.Theonlythingwasthepauses,ifitwaslongerthannormal.Longerpauseswereduetomesometimesneedingtondandselectanappropriateresponse. 6.4.3MovementsElevenofthe14participantsmadecommentsthatsomeaspectofmovement,orlackthereof,affectedhowrealtheyperceivedvirtualhumanstobe.Manymovementsprovideimportantnon-verbalcuesaboutwhatapersonisthinking. 6.4.3.1Eye/HeadMovementSixofthefourteenparticipantsmadecommentsaboutthevirtualhumans'eyeandheadmovements.Thevirtualparentalwayslookeddirectlyattheparticipant.Thevirtualchildalsolookeddirectlyattheparticipantexceptforwhileheisdrawingashape.Thisbehaviorwasnotedbyfourparticipants.P10noted,ThewaytheymovedwhenImovedandkepttheireyesonme,thatwasveryrealistic.However,otherparticipantsfoundtheheadandeyemovementslackingbecauseheadmovementsnormallyprovideimportantinformation.Forexample,whenparticipantsaskedthechildhisphonenumber,thechilddidnotrespondinanyway.P3thoughtitwouldhavebeenbetterifhe[thechild]hadturnedhisheadtowardhim[thefather]orlookeddownlikeno,Idon'tknowthat.Insteaditwasjustthesamefacetheentiretime. 131

PAGE 132

6.4.3.2FacialExpressionsThevirtualhumans'facialmovementswerelimitedtospeaking;thevirtualhumansdidnotexpressanyemotionsintheirfaces.However,facialexpressionsoftenprovideimportantnon-verbalcues.P11said,Imeanifit'sacomputerit'shardtosee-orevenlikefacialexpressions.Someparticipantsheardemotioninthevoices(e.g.interpretingsofterspeechassadness),andnotedthedisparitybetweenthevoiceandtheface. 6.4.4BehaviorTwelveofthe14participantsnotedatleastoneoftwospecicbehaviors,thechildnotcooperatingandtheparentinterveningtohelpthechildpasstheexam.Asascriptedmoment,thechildinitiallyrefusestodrawashapeasecondtime.Eightoftheparticipantsnotedthatthisrefusalisrealisticbehaviorforachild.Thisrefusalmadeastrongimpressiononsomeparticipants.P10mentioned,`Thenostage,that'sthethingthatreallystuckouttome,himrefusingtodothingsisverytypicalofchildrenthatage.Insomecases,thismadethechildseemmorerealbecausetheparticipantsexpectedacomputertoalwayscooperate.Inanotherscriptedmoment,thefathereitherasksthechildtopleasefollowthenurse'sinstructionsoranswerthenurse'squestion.Eightoftheparticipantsmentionedthisinterventionbehaviorcontributedtotherealism.However,oneoftheseparticipantsalsonotedthefatherhelpedtooearly.P5said,I'venoticedalotofparentspitchinininterviewsindifferentproviderinterviews.Idon'tthinkit'snormalwhenthechildcouldanswerforme.P3alsoencounteredaninstancewheresheexpectedthefathertointerveneevenmore.Shecommented,Iaskedthechildtodosomethingandthenifhedidn'tlikeknowwhatIwassayingorknowwhataplussignwas,thenmaybetheparentwouldhavelikeoh,no,youknowwhatthisis,makeitwiththeirhands.ThesendingsareconsistentwithBearman'sndingsthatstudents'lifeexperiencesinuencetheirrelationshipwithvirtualpatients[ 12 ]. 132

PAGE 133

6.4.5VoiceNineofthe14participantscommentedonthevoices.Sevenofthesementionedthatthetoneofvoiceconveyedemotionalinformationsuchasconcernorsadness.P5describeditasYoucouldsensetheconcernin,actuallyineithervoice,sothatwasveryrealtome.Fourparticipantscommentedspecicallyonhownaturalthevoiceswere.Participantswerenotexplicitlytoldthevoiceswererecordingsofrealpeople,andIamunsurehowmanyrealizedthis.Thissuggeststhatusingrecordingsofrealpeopleimprovesrealismintwoways.Onewayisthatthevoicessoundnaturalasopposedtoarticialorrobotic.Moreimportantly,thevoicescancarryemotionalinformationthatautomatictext-to-speechsystemsgenerallydonotconvey.Thisisimportantincommunication-orientedtaskslikethepediatricdevelopmentalexambecauseitisanadditionalinformationchannel. 6.4.6VisualRealismThethemeofvisualrealismencompassedstatementsabouthow`real'or`good'thevirtualhumanslookedtotheparticipants.Noparticipantsmentionedvisualrealismintheirinitialdenitionofrealism,andonlysixmentioneditatanypoint,usuallywhenaskedwhatmadethecharactersmoreorlessrealistic.Fourdescribedthevisualrealismpositively(goodorrealistic).Theothertwodescribedthevisualrealismsomewhatnegatively.P13describedthevirtualhumansascartoonishandP1saidtheywereLikeSimsonthecomputer,sonotsorealistic.Notably,thevirtualhumanswerenotphotorealistic.Themodelshadrelativelylowpolygoncounts(60,000triangles)comparedtomanycurrentmodels,andIdidnotleveragetechniquestoincreasedetailsuchasbumpmappingornormalmapping.Despitethis,thevisualrealismwassufcientsuchthatfouroftheparticipantsconsidereditgood.Eightothersdidnotmentionvisualrealismatallduringtheinterviews,suggestingitwasfarlessimportanttotheirideaof`real'thanconversational 133

PAGE 134

andbehavioralrealism.ThismatcheswithKotranzaet.al'sndingsthatmedicalstudentsrankedimprovementsinverbalcommunicationasmorebenecialthanimprovementsinvisualrealism[ 51 ]. 6.4.7PhysicalitySixoftheparticipantsmadecommentsrelatedtodimensionsofphysicality.ThreementionedthatthepositionsoftheECAsmattered.P3saidonethingthatmadeitrealwasthefactthatthey'residebysideliketheywouldbeinanofce.Thisemphasizesnotonlytheimportanceofpositiondelitybutalsomatchingtheparticipant'sexpectations.P13appreciatedtheuseofaseparatedisplayforeachECA,statingyoucanlookatoneandthenlookbackattheotherinsteadoflookingallatonescreen.That'sveryrealistic.Thisstatementalsohighlightsthatpositiondelity,awarenessoftheenvironment,andbehavioralrealismcanplayimportantrolesincommunication.Whereapersonlooksispartofnon-verbalcommunicationandcanprovidecluesaboutemotionalstate.Gazeisalsoapredictorofconversationalattention[ 89 ]andaregulatorofconversationalow[ 35 ].ECAsthatarepositionedcorrectlyandareawareoftheuser'spositioncancommunicateextrainformationbylookingornotlookingattheuser.Fourparticipantsmentionedthesizedifferencebetweenthechildandfathercontributedtorealism.Whilethesestatementsseemrelatedtosizedelity,theymayalsoberelatedtoformdelity.ThedisplaysarebothrectangularbutroughlyconformtothesizeofeachECA.ThesizedifferencemayhavebeenimportantiftheparticipantsperceivedtheECAsasembodiedinthedisplaysthemselves,ratherthanjustinthemeshes.ThelikelihoodofthisperceptionisincreasedbytheuseoftwoseparatedisplaystosimulatetwoseparateECAs. 6.4.8OverallDiscussionThesignsofhighsocialpresencecombinedwiththedescriptionsofvisualrealismsuggestvisualrealismhasreachedasufcientlevelforthevirtualhumanstobe 134

PAGE 135

perceivedasrealdespitebeingfarshortofphotorealistic.Attheselevelsofvisualrealism,conversationalandbehavioralrealismbegintomattermore.ThesendingsareconsistentwithPanetal.,whoalsoattributedhighsocialpresencetohighECAbehavioralrealismdespitelowvisualrealism[ 67 ]Tothisgroupofparticipants,theword`real'meanstheparticipantsareabletocarryonaconversationinamannertheyareusedtoandthevirtualhumansbehaveastheyexpect.Thisfocusonconversationalandbehavioralrealismmayberelatedtothepopulationandtask.Nursingstudentsaretrainedtocollectinformationthroughbothinterviewandobservation.Thetaskofassessingthechild'sdevelopmentalleveldependeddirectlyonconversationbutnotdirectlyonvisualrealism.Otherpopulationsandtasksmayshowdifferentresults,butIbelievetheseresultswillholdformosttaskswithamajorconversationalcomponent.ConversationalandbehavioralrealismmaybestrongfactorsinaECA'splausibility.Slaterproposedthatplausibilityisneededinadditiontopresenceforuserstobehaverealistically[ 78 ].Presenceappearsachievablewithcurrenttechnology.However,creatingplausiblevirtualhumansrequiresadditionalworkinmodelingtheconversationandtheECA'spersonality.Therearecurrentlynoestablishedinstrumentsformeasuringplausibility.Ourresultscouldbeusedtocreateself-reportsurveysforevaluatingplausibility.SuchsurveyscouldincludeLikert-typeitemssuchasIfeltliketheparentsaidthingsaparentwouldsay.Thesesurveyscouldbeusedtoevaluatemodelsofconversationalandemotionallycommunicativebehavior. 6.5RelationtoOurOtherStudiesTheresultsfromthisqualitativestudymayfurtherexplainsomeofthendingsfromourpriorstudies(Chapters 4 and 5 ). 6.5.1HolisticEvaluationofOccupancyTheholisticevaluationofoccupancycontainedtwoECAs,anurseandapatient'sdaughter.Inthisevaluation,wefoundincreasingphysicalitypositivelyaffected 135

PAGE 136

self-reportsocialpresenceratingsforthenursebutnegativelyaffectedtheratingsforthefamilymember.Thisdifferencemightbeexplainedbythedaughterfailingtomeettheparticipants'expectationsofbehavior.Participantsmayhaveexpectedthedaughtertodemonstratemoreconcerninherspeechandmovements.Someparticipantsmayhaveexpectedhertospeakmoreoften,perhapsinterrupting.Whenthedaughterdidspeak,shemaynothaveconveyedenoughemotioninhervoice.Thedaughter'sspeecheswererecordedbymycolleagueMalloryMcManamonwhodidnothaveanyvoiceactingexperienceortraining.Aswiththechildinthequalitativestudy,thedaughtermaynothavemovedenough.ThedaughteralwayslookedattheparticipantexceptwhenthedaughterturnedtospeaktothenurseECA.Thedaughterdidnotlookdownatherfatherormakeanymovementstocomforthim.ThenurseECA,ontheotherhand,mayhavemettheparticipants'expectationsbecausetheyhaddifferentexpectationsofthenurse.Theparticipantslikelyexpectedthenursetoactprofessionalandcalmratherthanemotional.Asaresult,thepre-recordedvoice'slackofstrongemotiontexpectations. 6.5.2IndependentComponentsEvaluationofOccupancyTheindependentcomponentsevaluationofoccupancycontainedthreeECAs,ahandoffdoctor,anurse,andacriticalincidentdoctor.ThehandoffdoctorwasratedsignicantlylowerforintelligencethantheothertwoECAs,suggestingthehandoffdoctormayhavehadplausibilityissues.Theseissuesbecamemoreevidentinlightofthendingsofthisqualitativestudy.Thehandoffdoctormayhavebeenhamperedbyhisinabilitytoprovidesomeinformationparticipantsexpectedandsometimesprovidinginformationinanunnaturalmanner.Occasionally,theparticipantsaskedforinformationwedidnothaveaspecicpre-recordedresponsefor.Inthosecases,weemployedthedefaultresponsestrategyandhadthehandoffdoctorrespondwithI'mnotsure,you'llhavetocheckhischart. 136

PAGE 137

Mostparticipantsdidnotndthisareasonableanswerforadoctorinthisscenario.Similarly,inthequalitativestudysomeparticipantsdidnotnditreasonableforthefathertoknowabsolutelynothingaboutthingssuchaswhatspecicfoodshischildate.Inothercases,thehandoffdoctorprovidedinformationinanunexpectedorunnaturalmanner.Oneexamplewasreportingthetimesdrugswereadministered.Doctorsnormallyreporttimesusingabsolutetimessuchas8:00am.However,becauseparticipantscameinthroughoutthedayandwewereusingpre-recordedaudio,wehadthehandoffdoctorreporttimesusingrelativetermssuchas`one-hourago.'Severalparticipantsdislikedthisandaskedthehandoffdoctorforabsolutetimes,butthehandoffdoctorwasunabletoprovidethoseasarealhumanwouldhavebeenableto.Anotherexampleofanunnaturalmannerofprovidinginformationwasrepeatingdetailssuchasthedosagesofdrugs.Ifaparticipantaskedthehandoffdoctortorepeatadosage,thehandoffdoctorwouldrepeatthepre-recordedaudioclipfromthebeginninguntilthedosage.Thisoftenmeantsayingoneormorecompletesentencesbeforegettingtotheinformationtheparticipantwanted.Besidesbeinganunnaturalwaytosupplythissmallpieceofinformation,there-useoftheexactsameaudiorecordingagainemphasizedthatitwasnotarealhuman.Asimilarre-useofanaudioclipwasnotedasarticialbyaparticipantinthequalitativestudyaswell. 6.5.3EvaluationofInteractionThendingsofthequalitativestudymayprovideadditionalexplanationfortheresultsoftheevaluationofinteraction(Chapter 5 ).Intheevaluationoftheeffectofinteractionwiththeenvironmentonsocialpresence,Ifailedtondaneffect.Thismaybeexplainedbythequalitativestudy'sndingthatparticipantsfocusedlargelyoncommunicationandcontent.Intheevaluationofinteraction,thehighandlowphysicalityinterfacesservedasfunctionallyequivalentcommunicationchannelsandcarriedthesamecontent. 137

PAGE 138

UnliketheECAsinevaluationsofoccupancy,alltheECAsintheevaluationofinteractionwereperceivedassimilarlysociallypresent.ThismaybebecausetheevaluationofinteractionECAsallhadsimilarlevelsofmeetingtheparticipants'behaviorexpectations.Thetwochildrenandtwofatherssaidthingsthatweresimilartowhatparticipantshadexperiencedbeforeorimaginedachildandfatherwouldsay.Theirawsweresimilarto.Noneofthemmovedquiteasmuchasparticipantsexpectedrealpeopletomove.Childrentypicallydonotsitasstill,andadultstypicallygestureastheytalk.Similarly,thechildandfathermaynothavespokenquiteasmuchasparticipantsexpected.Someparticipantsnotedfour-year-oldchildrenoftentalkatlengthabouttopicstheylike,thoughitisnotunusualforachildtobeshyduringadevelopmentalexam.Someparticipantsalsocommentedthatparentsareoftenmoreinvolvedintheexam,interjectingandtryingtohelptheirchildren. Figure6-2. Participantperformingadevelopmentalexamonthevirtualchild. 138

PAGE 139

Table6-1. Numberofparticipantsthatmadestatementsrelatedtoeachtheme. ThemeNumberofparticipants Conversationcontributedtorealism14Behaviorcontributedtorealism12Attributionofresponsetopersonalitytrait11Movementscontributedtorealism11Signsofhighsocialpresence10Voicecontributedtorealism9Discussionofvisualquality6Discussionofphysicality6Reasonsforparticipantactingrealistic5Noticingcomputererrors3 139

PAGE 140

CHAPTER7CONCLUSION 7.1ReviewofResultsInthisdissertation,weclaimedthat: Inthecontextofmedicalinterpersonalskillstraining,embodiedconversationalagentsareperceivedassociallypresentandelicitrealistictreatmentiftheembodiedconversationalagents1)occupythephysicalspaceand2)demonstratebehaviorandresponsesconsistentwiththetrainee'sexpectations.Thesetwoconditionsaresufcientsuchthattheagentwillbeperceivedassociallypresentandelicitrealistictreatmenteveniftheagentisunabletodirectlyinteractwiththeenvironment.TheresultsofourevaluationsdemonstratethatECAsthatmeetthesetwoconditionscanbeperceivedassociallypresentandelicitrealistictreatment.Inallfourofourevaluations,ECAselicitedthemostrealisticuserresponseswhentheECAsdemonstratedbehaviorandresponsesconsistentwiththetrainee'sexpectations.Ourholisticevaluationofoccupancysuggestedthathigheroccupancyofthephysicalspaceelicitedrealisticbehaviorwhereasloweroccupancyofthephysicalspaceelicitedsomeunrealisticbehavior.Theholisticevaluationalsosuggestedthatincreasingoccupancycouldincreaseexpectationsofbehavioralrealism.OurindependentcomponentsevaluationofoccupancysuggestedthatincreasingbothconcordancewiththeenvironmentandformdelitycansignicantlyincreasethefeelingoftheECAasarealperson.Ourevaluationofinteractionwiththeenvironmentfailedtondaneffectonsocialpresence.However,bothconditionselicitedrealisticbehaviorfromtheparticipants.Thissuggeststhatanyeffectsofinteractionwiththeenvironmentaredominatedbytheeffectsofoccupancyofthephysicalspace. 140

PAGE 141

Ourqualitativestudy'sndingsemphasizedtheimportanceofbehaviorandresponsesinadditiontophysicality.Non-verbalbehaviorcancommunicateimportantinformation,andhealthsciencelearnersmayexpecttheseinformationchannels.Meetingexpectationsofbehaviorisalsoimportant,thoughitispossibletoelicitrealisticuserresponsesevenifsomeexpectationsarenotmet. 7.2LimitationsOurstudiesallowedustoestablishtheefcacyofECAsatelicitingrealisticuserbehaviorunderidealconditions.Theseidealconditionsusedwizardofozcontrol,specicallydesignedscenarios,andhighlyskilledpopulations.Underreal,non-idealtrainingconditionsdifferentfromourstudies,theeffectivenessofECAsatelicitingrealisticuserbehaviormaybereduced.Efcacyandeffectivenessdifferinthatefcacydescribeshowwellsomethingworksinidealconditionswhereaseffectivenessdescribeshowwellsomethingworksinreal,non-idealconditions.LargerscaledeploymentsofECA-basedtrainingapplicationswilllikelyusearticialintelligencesystemsinsteadofwizardofozcontrol.WizardofozcontrolallowedtheECAstorespondwithveryhighaccuracytoawidevarietyofquestions.Articialintelligencesystemsusuallyrespondwithsignicantlyloweraccuracy,andloweraccuracywilllikelyresultinlowersocialpresence.Thisissuggestedbycommentsinthequalitativestudymentioningthatrespondingtoawidevarietyofquestionsincreasedrealism.However,webelievethataslongastheconversationremainfreeform,theECAscanstillelicitrealisticuserresponses.Someexistingarticialintelligencesystemsarecapableofansweringfreeformquestionswithhighenoughaccuracytocarryonaconversation[ 72 ].Articialintelligencesystemswillcontinuetoimproveandmayonedayapproachtheaccuracyofwizardofoz.Wizardofozcanalsobecombinedwitharticialintelligencesystemsinhybridapproachesthatprimarilyusearticialintelligencetoanswerquestionsbutfallbacktowizardofozforquestionswherethearticialintelligencesystemhaslowcondence. 141

PAGE 142

Thescenariosinallofourstudiesfocusedoninterpersonalskillsthatmainlyinvolvedconversationandcommunication.Someofthesescenariosalsoinvolvedphysicalinteractions,butnoneofthesescenariosincludeddifcultphysicalinteractions.ScenarioswheredifcultphysicalinteractionsareacriticalcomponentmaybeaffecteddifferentlybyECAphysicality.Forexample,ascenariothatinvolvesintubatingapatient(insertingatubeintothepatient'strachea)maybeaffectedbythephysicalityoftheECA'sinteractionswiththeenvironment.Suchscenariosmaybebettertrainedusingrealhumansintheroleswithdifcultphysicalinteractions.TheparticipantsforourstudiesconsistedofmedicalandnursingstudentsattheUniversityofFloridaandnursesatUF&Shandshospital.Bothofthesearehighlyrankedinstitutions,andtheparticipantsmaybeaboveaveragecomparedtootherpotentialtrainees.Othertraineesmaynotadapttothenewtechnologyasquickly,andasaresultsocialpresenceandtrainingeffectivenessmaybeaffected.ItmaybepossibletoincreasetrainingeffectivenessthroughrepeatedexposurestoECA-basedtrainingsystems.Peoplewithmorefamiliaritywith3Dvirtualenvironmentsmayexperiencedifferentlevelsofsocialpresenceorfocusondifferentaspectsofrealism.Veryfewparticipantshadexperiencewith3DvirtualenvironmentssuchasSecondLifeorrst-person-perspectivevideogames.Thismayhaveinuencedtheirfocusonconversationalandbehavioralrealismasopposedtovisualrealism.Visualrealismmaybemoreimportanttovideogameplayersfamiliarwiththelatestcuttingedge3Dgames.However,webelievethattheleveloffamiliaritywith3Dvirtualenvironmentsinourparticipantswasrepresentativeofthatoftypicalmedicaltrainees. 7.3FutureDirectionsThisdissertationhasidentiedseveralofthemanyfactorsthatcanaffectanECA'ssocialpresence.ThenextstepsincludeusingtheresultsfromthequalitativestudytocreateguidelinesforplausibleECAsandinstrumentsformeasuringanECA's 142

PAGE 143

plausibility.Theseinstrumentsmaybeself-reportLikert-typesurveys.Studiescouldalsobeconductedtoquantifytheeffectsofcertainaspectsofplausibilityonsocialpresence.OneunexploredfactorthatmayeffectsocialpresenceiscomfortwithECAs,especiallyafterrepeatedexposurestoECAs.ECAtechnologyisoftenunfamiliartolearners.LearnersmaybecomemorecomfortableandfamiliarwithECAtechnologyastheyuseitmoreoftenfortraining.Thescenarioandparticipantpoolfromtheindependentcomponentsevaluationaffordssomeopportunityforthis.Thescenarioinvolvedthreesuccessivestageswhereparticipant'sworkedwithECAs,anditwouldbepossibletomeasuretheircomfortwiththeECAtechnology.Itmayalsobepossibletoexaminerepeatedsessions.Wehavealreadyconductedafollowupstudywheresomeparticipantsfromtheindependentcomponentsevaluationwentthroughthetrainingsessionasecondtime.However,wehavenotlookedspecicallyatcomfortlevelswiththeECAtechnology.AnotherstepistoevaluatetheeffectivenessofsociallypresentECAsforinterpersonalskillstraining.ResearchhasshownECA-basedinterpersonalskillstrainingispredictiveofperformance[ 69 ],butresearchhasnotdirectlyexaminedtherelationshipbetweensocialpresenceandrealworldperformance.Thequestionremainshowmuchsocialpresenceisnecessaryforeffectivetraining.Thenecessarylevellikelyvariesdependingonthespecicskillsbeingtrainedaswell.Researchcouldalsobeconductedintotechniquesformanagingexpectations.Expectationscanplayanimportantroleinperceptionofsocialpresence.Theholisticevaluationsuggestedincreasingphysicalityraisedexpectations.Thequalitativestudysuggestedmeetingplausibilityexpectationsisveryimportantforsocialpresence.Techniquessuchasreectivelearningcouldbeused.Reectivelearningasanexperientiallearningtechniquewherelearnersinternallythinkaboutanissue[ 19 ].Havinglearnersreectonthetrainingscenarioandskillsbeingtrainedmayhelp 143

PAGE 144

manageexpectationsbyputtingthelearnerinthesamemindsetastheECAcreatoraboutwhattheECAwillsayanddo. 7.4BroaderImpactThisdissertation'sworktakesseveralimportantstepstowardsthevisionofon-demand,insitu,medicalteamtraining.ANDIisapracticalimplementationofanECAdisplaysystemthatcanbeliterallyrolledinonamoment'snotice.WhileANDIhaslimitationssuchasbeingunabletophysicallyinteractwithobjectsintheenvironment,ourworksuggeststhisdoesnotlimitANDI'ssocialpresenceinmanycontexts.ANDIiscurrentlybeingusedaspartofanNSFgrantexploringmixedrealityECAsforteamtraining.Successfulteamtrainingexercisescouldimprovepatientcare,potentiallysavinglivesandreducingcostlymistakes.WhilethisdissertationfocusedonmixedrealityECAs,thendingscouldalsoinformresearchintheeldsofsocialrobotsandtelepresence.SocialrobotsandECAshaveagreatdealofoverlap.ManysocialrobotsareECAs,andevensocialrobotsthatdonotconversestillattempttoconnecttousersonasociallevel.Thisconnectionmaybeaffectedbytheresponsespeed,abilitytonon-verballycommunicatethroughwholebodymovementsandfacialexpressions,andtheuser'sexpectations.Theuser'sexpectationsmaybeaffectedbydimensionsofphysicalitythatalsoapplytosocialrobots-positiondelity,sizedelity,abilitytochangetheenvironment,andawarenessofchangestotheenvironment.Inmanyways,telepresenceattemptstobringremotepeopleintothelocalspacesotheycanoccupyandinteractwiththeenvironmentandotherpeople.Thisisverysimilartotheideaofphysicality.Ourresultsaboutwhichdimensionsofphysicalityincreasesocialpresencethemostmayalsoincreasecopresenceofareal,remoteperson.Someoftheresultsfromthequalitativestudymayalsobeinformativetotelepresenceresearch.Responsespeedandtheavailabilityofnon-verbalcommunicationchannelssuchasbodymovementsarealsoimportantintelepresence. 144

PAGE 145

REFERENCES [1] AFP.RoboThespiancan'tmakehorizontalwristmovements,resortstounwantedadvances.2011.URL http://www.news.com.au/technology/robothespian-cant-make-horizontal-wrist-movements-resorts-to-unwanted-advances/story-e6frfro0-1226015171685 [2] Andrade,AllenD,Bagri,Anita,Zaw,Khin,Roos,Bernarda,andRuiz,JorgeG.Avatar-mediatedtraininginthedeliveryofbadnewsinavirtualworld.Journalofpalliativemedicine13(2010).12:1415. [3] Argyle,M.Thepsychologyofinterpersonalbehavior.Baltimore:PenguinBooks,1967. [4] ASQ.Ages&StagesQuestionnaires.http://agesandstages.com/,2012.Sept.7,2012. [5] Babu,S.,Suma,E.,andBarnes,T.CanImmersiveVirtualHumansTeachSocialConversationalProtocols?ProceedingsofIEEEVirtualReality2007.2007,215. [6] Bailenson,J.N,Blascovich,J.,Beall,A.C,andLoomis,J.M.Interpersonaldistanceinimmersivevirtualenvironments.PersonalityandSocialPsychologyBulletin29(2003).7:819. [7] Bailenson,JeremyN,Blascovich,Jim,Beall,AndrewC,andLoomis,JackM.Equilibriumtheoryrevisited:Mutualgazeandpersonalspaceinvirtualenvironments.Presence:Teleoperators&VirtualEnvironments10(2001).6:583. [8] Bailenson,J.N.,Aharoni,E.,Beall,AC,Guadagno,RE,Dimov,A.,andBlascovich,Jim.ComparingBehavioralandSelf-ReportMeasuresofEmbodiedAgentsSocialPresenceinImmersiveVirtualEnvironments.7thAnnualInternationalWorkshoponPresence,Valencia.Citeseer,2004. [9] Bailenson,J.N.,Blascovich,J.,Beall,A.C.,andLoomis,J.M.Interpersonaldistanceinimmersivevirtualenvironments.PersonalityandSocialPsychologyBulletin29(2003).7:819. [10] Bailenson,J.N.,Swinth,K.,Hoyt,C.,Persky,S.,Dimov,A.,andBlascovich,J.Theindependentandinteractiveeffectsofembodied-agentappearanceandbehavioronself-report,cognitive,andbehavioralmarkersofcopresenceinimmersivevirtualenvironments.Presence:Teleoperators&VirtualEnvironments14(2005).4:379. 145

PAGE 146

[11] BaileyJr.,DB,Hebbeler,K,Spiker,D,Scarborough,A,Mallik,S,andNelson,L.Thirty-six-monthoutcomesforfamiliesofchildrenwhohavedisabilitiesandparticipatedinearlyintervention.Pediatrics116(2005).6:1155. [12] Bearman,Margaret.Isvirtualthesameasreal?Medicalstudents'experiencesofavirtualpatient.Academicmedicine:journaloftheAssociationofAmericanMedicalColleges78(2003).5:538. [13] Benet-Martnez,VandJohn,OP.LosCincoGrandesacrossculturesandethnicgroups:multitraitmultimethodanalysesoftheBigFiveinSpanishandEnglish.JournalofPersonalityandSocialPsychology75(1998).3:729. [14] Bimber,O.andRaskar,R.SpatialAugmentedReality:MergingRealandVirtualWorlds.Peters,2005. [15] Biocca,FrankandHarms,Chad.Deningandmeasuringsocialpresence:Contributiontothenetworkedmindstheoryandmeasure.ProceedingsofPRESENCE2002(2002).517:1. [16] Biocca,Frank,Harms,Chad,Burgoon,JudeeK,Interface,Media,andLansing,East.TowardsAMoreRobustTheoryandMeasureofSocialPresence:ReviewandSuggestedCriteriaTowardsAMoreRobustTheoryandMeasureofSocialPresence:ReviewAndSuggestedCriteriaAbstract.Presence:Teleoperators&VirtualEnvironments12(2003):456. [17] Blascovich,Jim.SocialInuencewithinImmersiveVirtualEnvironments.TheSocialLifeofAvatars.ed.RalphSchroeder.London:SpringerLondon,2002. [18] Bowman,DougA.,Kruijff,Ernst,LaViola,JosephJ,andPoupyrev,Ivan.3DUserInterfaces:TheoryandPractice.Addison-WesleyProfessional,2004. [19] Boyd,E.M.andFales,a.W.ReectiveLearning:KeytoLearningfromExperience.JournalofHumanisticPsychology23(1983).2:99. [20] Brunner,Edgar,Domhof,Sebastian,andLanger,Frank.NonparametricAnalysisofLongitudinalDatainFactorialExperiments.NewYork,NY,USA:Wiley,2002. [21] Burgoon,JK.TheUnwillingnesstoCommunicateScale:DevelopmentandValidation.CommunicationMonographs43(1976).1:60. [22] Cassell,Justine.Nudgenudgewinkwink:Elementsofface-to-faceconversationforembodiedconversationalagents.MITPress,2000. [23] Cheek,J.TheRevisedCheekandBussShynessScale(RCBS).1983. [24] Chuah,JoonHaoandLok,Benjamin.HybridVirtual-PhysicalEntities.Interna-tionalSymposiumonMixedandAugmentedReality.IEEE,2012,275. 146

PAGE 147

[25] Chuah,JoonHao,Lok,Benjamin,andBlack,Erik.Applyingmixedrealitytosimulatevulnerablepopulationsforpracticingclinicalcommunicationskills.IEEEtransactionsonvisualizationandcomputergraphics19(2013).4:539. [26] Chuah,JoonHao,Robb,Andrew,White,Casey,Wendling,Adam,Lampotang,Samsun,Kopper,Regis,andLok,Benjamin.Increasingagentphysicalitytoraisesocialpresenceandelicitrealisticbehavior.2012IEEEVirtualReality(VR).IEEE,2012,19. [27] Creswell,John.QualitativeInquiry&ResearchDesign:ChoosingAmongFiveApproaches.SagePublications,2007,seconded. [28] Dahlback,N,Jonsson,A,andAhrenberg,L.WizardofOzstudieswhyandhow.Knowledge-BasedSystems6(1993).4:258. [29] Denver.DenverDevelopmentalMaterials,Inc.http://denverii.com/,2012.Sept.7,2012. [30] Devereaux,PJ.Effectsofextended-releasemetoprololsuccinateinpatientsundergoingnon-cardiacsurgery(POISEtrial):arandomisedcontrolledtrial.TheLancet371(2008).9627:1839. [31] Dow,S,Mehta,M,Harmon,E,MacIntyre,B,andMateas,M.Presenceandengagementinaninteractivedrama.ProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystemsCHI07.ACMPress,2007,1475. [32] Fujie,S.,Ejiri,Y.,Nakajima,K.,Matsusaka,Y.,andKobayashi,T.Aconversationrobotusingheadgesturerecognitionaspara-linguisticinformation.RO-MAN2004.13thIEEEInternationalWorkshoponRobotandHumanInteractiveCommunication(IEEECatalogNo.04TH8759)(2004):159. [33] Garau,Maia,Slater,Mel,Vinayagamoorthy,Vinoba,Brogni,Andrea,Steed,Anthony,andSasse,M.Angela.Theimpactofavatarrealismandeyegazecontrolonperceivedqualityofcommunicationinasharedimmersivevirtualenvironment.ProceedingsoftheconferenceonHumanfactorsincomputingsystems-CHI'03.NewYork,NewYork,USA:ACMPress,2003,529. [34] Gifford,R.EnvironmentalPsychology.Boston:Allyn&Bacon,1996,2nded. [35] Gu,E.Visualattentionandeyegazeduringmultipartyconversationswithdistractions.IntelligentVirtualAgents.2006,193. [36] Guadagno,R.E.,Blascovich,J.,Bailenson,J.N.,andMccall,C.Virtualhumansandpersuasion:Theeffectsofagencyandbehavioralrealism.MediaPsychology10(2007).1:1. [37] Gunawardena,CharlotteNandZittle,FrankJ.Socialpresenceasapredictorofsatisfactionwithinacomputer-mediatedconferencingenvironment.AmericanJournalofDistanceEducation11(1997).3:8. 147

PAGE 148

[38] Hamilton,Sutton.Screeningfordevelopmentaldelay:Reliable,easy-to-usetools.JournalofFamilyPractice55(2006).5:415. [39] Henderson,J,Fishwick,P,andFresh,E.Animmersivelearningsimulationenvironmentforchineseculture.TheInterservice/Industry...(2008). [40] Hill,R.,Gratch,J.,Marsella,S.,Rickel,J.,Swartout,W.,andTraum,D.Virtualhumansinthemissionrehearsalexercisesystem.KnunstlicheIntelligenz4(2003).03:5. [41] Hunziker,Sabina,Johansson,Anna,Tschan,Franziska,Semmer,Nobert,Rock,Laura,Howell,Michael,andMarsch,Stephan.Teamworkandleadershipincardiopulmonaryresuscitation.JournaloftheAmericanCollegeofCardiology57(2011).1:2381. [42] Hyatt,Jon-PhilippeKandHurst,StephenA.Novelundergraduatephysiologylaboratoryusingahumanpatientsimulator.MedicalEducation44(2010).5:523. [43] Izadi,Shahram.ANewEraofHumanComputerInteraction.InternationalSymposiumonMixedandAugmentedReality.2012,Keynote. [44] Johnsen,KyleandBeck,Diane.Theimpactofamixedrealitydisplaycongurationonuserbehaviorwithavirtualhuman.IntelligentVirtualAgents.2010,1. [45] Johnsen,Kyle,Dickerson,R.,Raij,A.,Harrison,C.,Lok,B.,Stevens,A.,andLind,D.S.Evolvinganimmersivemedicalcommunicationskillstrainer.Presence:Teleoperators&VirtualEnvironments15(2006).1:33. [46] Junuzovic,Sasa,Inkpen,Kori,Blank,Tom,andGupta,Anoop.IllumiShare.Proceedingsofthe2012ACMannualconferenceonHumanFactorsinComputingSystems-CHI'12.NewYork,NewYork,USA:ACMPress,2012,1919. [47] Kaptein,MauritsClemens,Nass,Clifford,andMarkopoulos,Panos.Powerfulandconsistentanalysisoflikert-typeratingscales.Proceedingsofthe28thinternationalconferenceonHumanfactorsincomputingsystems-CHI'10(2010):2391. [48] Kenny,P.,Parsons,T.,andGratch,J.Evaluationofjustina:AvirtualpatientwithPTSD.IntelligentVirtualAgents(2008):394. [49] Klemmer,ScottR,Sinha,AnoopK,Chen,Jack,Landay,JamesA,Aboobaker,Nadeem,andWang,Annie.SUEDE:AWizardofOzPrototypingToolforSpeechUserInterfaces.Proceedingsofthe13thannualACMsymposiumonUserinterfacesoftwareandtechnology.vol.2.2000,1. [50] Kohn,LT,Corrigan,J,andDonaldson,MS.ToErrIsHuman:BuildingaSaferHealthSystem.NationalAcademiesPress,2000. 148

PAGE 149

[51] Kotranza,A.,Lok,B.,Deladisma,A.,Pugh,C.M.,andLind,D.S.MixedRealityHumans:EvaluatingBehavior,Usability,andAcceptability.IEEETransactionsonVisualizationandComputerGraphics15(2009).3:369. [52] Kotranza,A.,Lok,B.,Pugh,C.M.,andLind,D.S.VirtualHumansThatTouchBack:EnhancingNonverbalCommunicationwithVirtualHumansthroughBidirectionalTouch.2009IEEEVirtualRealityConference.Lafayette,LA:IEEE,2009,175. [53] Kotranza,Aaron,Johnsen,Kyle,Cendan,Juan,Miller,Bayard,Lind,D.S.,andLok,Benjamin.Virtualmulti-toolsforhandandtool-basedinteractionwithlife-sizevirtualhumanagents.2009IEEESymposiumon3DUserInterfaces.Lafayette,LA:IEEE,2009,23. [54] Lee,KwanMin,Jung,Younbo,Kim,Jaywoo,andKim,SangRyong.Arephysicallyembodiedsocialagentsbetterthandisembodiedsocialagents?:Theeffectsofphysicalembodiment,tactileinteraction,andpeople'slonelinessinhumanrobotinteraction.InternationalJournalofHuman-ComputerStudies64(2006).10:962. [55] Lennox,RDandWolfe,RN.Revisionoftheself-monitoringscale.JournalofPersonalityandSocialPsychology46(1984).6:1349. [56] Lincoln,Peter,Welch,Greg,Nashel,Andrew,Ilie,Adrian,State,Andrei,andFuchs,Henry.AnimatronicShaderLampsAvatars.20098thIEEEInternationalSymposiumonMixedandAugmentedReality.Orlando,FL,USA,2009,27. [57] Loomis,Andrew.FigureDrawingforAllIt'sWorth.TitanBooks,2011. [58] Maulsby,David,Greenberg,Saul,andMander,Richard.PrototypinganintelligentagentthroughWizardofOz.ProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystems-CHI'93.NewYork,NewYork,USA:ACMPress,1993,277. [59] McLeod,PoppyLauretta,Baron,RobertS,Marti,MollieWeighner,andYoon,Kuh.Theeyeshaveit:Minorityinuenceinface-to-faceandcomputer-mediatedgroupdiscussion.JournalofAppliedPsychology82(1997).5:706. [60] Meehan,Michael,Insko,Brent,Whitton,Mary,andBrooks,FrederickP.Physiologicalmeasuresofpresenceinstressfulvirtualenvironments.ACMTransactionsonGraphics21(2002).3:645. [61] Milgram,P.andKishino,F.Ataxonomyofmixedrealityvisualdisplays.IEICETransactionsonInformationandSystemsEseriesD77(1994).12:1321. [62] Mori,Masahiro.TheUncannyValley.Energy7(1970).4:33. [63] Nicol,Pam.UsingtheAgesandStagesQuestionnairetoteachmedicalstudentsdevelopmentalassessment:adescriptiveanalysis.BMCMedicalEducation6(2006):29. 149

PAGE 150

[64] Nowak,KristineL.andBiocca,Frank.TheEffectoftheAgencyandAnthropomorphismonUsers'SenseofTelepresence,Copresence,andSocialPresenceinVirtualEnvironments.Presence:TeleoperatorsandVirtualEnviron-ments12(2003).5:481. [65] Pair,J.,Neumann,U.,Piepol,D.,andSwartout,B.FlatWorld:combiningHollywoodset-designtechniqueswithVR.IEEEComputerGraphicsandAp-plications23(2003).1:12. [66] Palmer,Mark.Interpersonalcommunicationandvirtualreality:Mediatinginterpersonalrelationships.Communicationintheageofvirtualreality.eds.FBioccaandMLevy.Mahwah,NJ:LawreneceErlbaumAssociates,1995.277. [67] Pan,Xueni,Gillies,Marco,Barker,Chris,Clark,DavidM,andSlater,Mel.Sociallyanxiousandcondentmeninteractwithaforwardvirtualwoman:anexperimentalstudy.PloSone7(2012).4:e32931. [68] PEDS.Parents'EvaluationofDevelopmentalStatus.http://pedstest.com/,2012.Sept.7,2012. [69] Raij,Andrew,Johnsen,Kyle,Dickerson,Robert,Lok,Benjamin,Cohen,Mark,Duerson,Margaret,Pauly,Rebecca,Stevens,Amy,Wagner,Peggy,andLind,D.Scott.Comparinginterpersonalinteractionswithavirtualhumantothosewitharealhuman.IEEETransactionsonVisualizationandComputerGraphics13(2007).3:443. [70] Reader,TomW,Flin,Rhona,Mearns,Kathryn,andCuthbertson,BrianH.Developingateamperformanceframeworkfortheintensivecareunit.Criti-calcaremedicine37(2009).5:1787. [71] Rivera-Gutierrez,Diego,Welch,Greg,Lincoln,Peter,Whitton,Mary,Cendan,Juan,Chesnutt,Davida,Fuchs,Henry,andLok,Benjamin.ShaderLampsVirtualPatients:thephysicalmanifestationofvirtualpatients.MedicineMeetsVirtualReality19-Nextmed,StudiesinHealthTechnologyandInformatics.vol.173.2012,372. [72] Rossen,B.,Lind,S.,andLok,B.Human-CenteredDistributedConversationalModeling:EfcientModelingofRobustVirtualHumanConversations.IntelligentVirtualAgents.2009,474. [73] Rothschild,StevenKandLapidos,Stanley.Virtualintegratedpractice:integratingteamsandtechnologytomanagechronicdiseaseinprimarycare.Journalofmedicalsystems27(2003).1:85. [74] Scheier,MichaelFandCarver,CharlesS.Theself-consciousnessscale:Arevisedversionforusewithgeneralpopulations.JournalofAppliedSocialPsychology15(1985).8:687. 150

PAGE 151

[75] Short,John,Williams,Ederyn,andChristie,Bruce.Thesocialpsychologyoftelecommunications.Wiley,1976. [76] Slater,Mel.MeasuringPresence:AResponsetotheWitmerandSingerPresenceQuestionnaire.Presence:TeleoperatorsandVirtualEnvironments8(1998).5:560. [77] .Anoteonpresenceterminology.Presenceconnect3(2003). [78] .Placeillusionandplausibilitycanleadtorealisticbehaviourinimmersivevirtualenvironments.PhilosophicaltransactionsoftheRoyalSocietyofLondon.SeriesB,Biologicalsciences364(2009).1535:3549. [79] Slater,Mel,Usoh,Martin,andSteed,Anthony.Takingsteps:theinuenceofawalkingtechniqueonpresenceinvirtualreality.ACMTransactionsonComputer-HumanInteraction2(1995).3:201. [80] Soussignan,RandSchaal,B.Children'sFacialResponsivenesstoOdors:InuencesofHedonicValenceofOdor,Gender,Age,andSocialPresence.DevelopmentalPsychology32(1996).2:367. [81] Strauss,AnselmandCorbin,Juliet.BasicsofQualitativeResearch:TechniquesandProceduresforDevelopingGroundedTheory.SagePublications,1998. [82] Suma,EvanA,Lange,Belinda,Rizzo,AlbertSkip,Krum,DavidM,andBolas,Mark.FAAST:TheFlexibleActionandArticulatedSkeletonToolkit.VirtualReality(2011):247. [83] Tamblyn,RM,Klass,DJ,Schnabl,GK,andKopelow,ML.Theaccuracyofstandardizedpatientpresentation.MedicalEducation25(1991).2:100. [84] Thompson,Lindsaya,Tuli,SanjeevY,Saliba,Heidi,DiPietro,Meredith,andNackashi,Johna.Improvingdevelopmentalscreeninginpediatricresidenteducation.Clinicalpediatrics49(2010).8:737. [85] Tsai,T-C.Usingchildrenasstandardisedpatientsforassessingclinicalcompetenceinpaediatrics.Archivesofdiseaseinchildhood89(2004).12:1117. [86] Usoh,Martin,Arthur,Kevin,Whitton,MaryC.,Bastos,Rui,Steed,Anthony,Slater,Mel,andBrooks,FrederickP.Walking>walking-in-place>ying,invirtualenvironments.Proceedingsofthe26thannualconferenceonComputergraphicsandinteractivetechniques-SIGGRAPH'99(1999):359. [87] Usoh,Martin,Catena,Ernest,Arman,Sima,andSlater,Mel.UsingPresenceQuestionnairesinReality.Presence:TeleoperatorsandVirtualEnvironments9(2000).5:497. 151

PAGE 152

[88] VanAgt,HeleenME,VanDerStege,HeleenA,DeRidder-Sluiter,Hanneke,Verhoeven,LudoTW,andDeKoning,HarryJ.Acluster-randomizedtrialofscreeningforlanguagedelayintoddlers:effectsonschoolperformanceandlanguagedevelopmentatage8.Pediatrics120(2007).6:1317. [89] Vertegaal,Roel,Slagter,Robert,vanDerVeer,Gerrit,andNijholt,Anton.Eyegazepatternsinconversations.ProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystems-CHI'01.NewYork,NewYork,USA:ACMPress,2001,301. [90] VonderPutten,AKramerandGratch,Jonathan.HowOurPersonalityShapesOurInteractionswithVirtualCharacters-ImplicationsforResearchandDevelopment.IntelligentVirtualAgents(2010). [91] Wallin,CJandMeurling,L.Target-focusedmedicalemergencyteamtrainingusingahumanpatientsimulator:effectsonbehaviourandattitude.MedicalEducation41(2007).2:173. [92] Ware,C,Arthur,K,andBooth,KS.Fishtankvirtualreality.ProceedingsoftheSIGCHIconferenceonHumanfactorsincomputingsystemsCHI93.ACMPress,1993,37. [93] Witmer,BobGandSinger,MichaelJ.MeasuringPresenceinVirtualEnvironments:APresenceQuestionnaire.Presence:Teleoperators&VirtualEnvironments7(1998).3:225. [94] Woodard,LaurieJ,Havercamp,SusanM,Zwygart,KiraK,andPerkins,Elizabetha.Aninnovativeclerkshipmodulefocusedonpatientswithdisabilities.Academicmedicine87(2012).4:537. [95] Zimmons,PaulandPanter,Abigail.Theinuenceofrenderingqualityonpresenceandtaskperformanceinavirtualenvironment.ProceedingsofIEEEVirtualReality,2003.vol.2003.IEEE,2003,293. [96] Zwarenstein,MerrickandReeves,Scott.Workingtogetherbutapart:barriersandroutestonursephysiciancollaboration.TheJointCommissionJournalOnQualityImprovement28(2002).5:242,209. 152

PAGE 153

BIOGRAPHICALSKETCH JoonHaoChuahwasbornin1983inNorthampton,Massachusetts.JoongrewupinHouston,TexaswherehegraduatedsummacumlaudefromLanghamCreekHighSchoolin2001.HeattendedtheUniversityofTexasatAustinasaNationalMeritScholar.Hegraduatedwithhighhonorsin2006withabachelorofscienceincomputersciencesandabachelorofartsinAsianculturesandlanguages.In2008,JoonbeganhisPh.D.incomputerengineeringattheUniversityofFloridaunderthesupervisionofDr.BenjaminLok.Hewasawardedafour-yearUniversityofFloridaAlumniFellowshiptosupporthisresearch,whichfocusedontheuseofvirtualhumansocialpresence.Specically,heexaminedtheroleofmixedrealitytechnologyandidentiedfactorsrelatedtoplausibilityandexpectations.Hisworkreceivedsignicantrecognitionwithsixpublicationsinjournalsandconferences,includingarticlesinIEEETransactionsonVisualizationandComputerGraphicsandPresence:TeleoperatorsandVirtualEnvironments.HereceivedhisPh.D.inthesummerof2013.Inthenearfuture,JoonwillbeworkingatAmazon.cominSeattle,Washington. 153