Citation
Self-Determined Gamification in an Online Web Portal

Material Information

Title:
Self-Determined Gamification in an Online Web Portal Employee Performance and Perceptions
Creator:
Wolff, Paul B
Place of Publication:
[Gainesville, Fla.]
Florida
Publisher:
University of Florida
Publication Date:
Language:
english
Physical Description:
1 online resource (190 p.)

Thesis/Dissertation Information

Degree:
Doctorate ( Ed.D.)
Degree Grantor:
University of Florida
Degree Disciplines:
Curriculum and Instruction
Teaching and Learning
Committee Chair:
KUMAR,SWAPNA
Committee Co-Chair:
RITZHAUPT,ALBERT D
Committee Members:
BEAL,CAROLE R
SMITH,SONDRA LORI

Subjects

Subjects / Keywords:
achievements -- gamification -- management -- motivation -- performance -- self-determination
Teaching and Learning -- Dissertations, Academic -- UF
Genre:
bibliography ( marcgt )
theses ( marcgt )
government publication (state, provincial, terriorial, dependent) ( marcgt )
born-digital ( sobekcm )
Electronic Thesis or Dissertation
Curriculum and Instruction thesis, Ed.D.

Notes

Abstract:
This study sought to address a problem of practice by incentivizing job requirements through the addition of achievements in an online web portal. To do this, a workplace analysis was conducted, along with a thorough review of relevant literature. The result was the creation of a framework for designing gamified systems entitled Self-Determined Gamification. Using this framework as a guide, an attempt was made to design achievements with Self-Determination Theory (SDT) in mind, paying particular attention to organizational goals, equity in attaining achievements, employee autonomy in interacting and engaging with the achievements, and individual goals. Design best practices were incorporated through the use of the MDA Design Framework in addition to drawing upon design considerations from the SAPS Reward System, the Four Player Types by Bartle, the Four Types of Fun by Lazzaro, the Five Levels of Mastery by Dreyfus and Dreyfus, and Flow Theory by Czikszentmihalyi. With the design of the system as a major focus of this research, determining the success of the design in aligning with SDT considerations was important and this was accomplished through the use of the Intrinsic Motivation Inventory (IMI) Survey within SDT. Additionally, looking at performance indicators along with employee feedback about the achievements provided insight into the overall success of the design and future considerations. The gamification implementation had positive results with staff reporting enjoying the achievements and the impact they had on improving work habits. Results from the IMI survey also indicated that the design of the achievements in the online web portal aligned with autonomous forms of motivation on the autonomy continuum within SDT. Analysis of performance metrics indicated improved performance across multiple performance indicators. This was particularly true in areas where employees had control in completing a job requirement and where employees had the opportunity to exceed performance expectations. The Self-Determined Gamification framework developed in this study provides both a model for designing gamification rooted in design best practices and relevant theory, and a method for evaluating that design. ( en )
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Thesis:
Thesis (Ed.D.)--University of Florida, 2017.
Local:
Adviser: KUMAR,SWAPNA.
Local:
Co-adviser: RITZHAUPT,ALBERT D.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2018-02-28
Statement of Responsibility:
by Paul B Wolff.

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Embargo Date:
2/28/2018
Classification:
LD1780 2017 ( lcc )

Downloads

This item has the following downloads:


Full Text

PAGE 1

SELF DETERMINED GAMIFICATION IN AN ONLINE WEB PORTAL: EMPLOYEE PERFORMANCE AND PERCEPTIONS By PAUL BENJAMIN CAUSEY WOLFF A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION UNIVERSITY OF FLORIDA 2017

PAGE 2

2017 Paul Benjamin Causey Wolff

PAGE 3

To my wife Courtney for her constant support, patience and understanding throughout the doctoral process

PAGE 4

4 ACKNOWLEDGMENTS I would like to start by expressing my extreme appreciation for my wife Courtney Throughout the entire doctoral process, Courtney was a constant source of encouragement and support. Without her standing by me every step of the way, I would never have pushed through the difficult times. When I wanted to quit, she listened and let me have that moment, day when I decided to keep going. Thank you for listening to me when I needed to vent, for encouraging me when I needed a little push, and for helping me take time to step away from everything when I needed a break. Dr. Kumar was also a constant source of support, both as my advisor during the first two years of the program and as my dissertation chair over the past year. Her advice and guidance steered me throug h the various IRB processes, helped me scale back when I tried to do too much, and kept me grounded as I moved through the entire process. She maintained the perfect balance of being firm and direct about my writing and deadlines, while also being understa nding regarding life and career challenges. Above all, she kept me on track and made sure I continued writing as we moved through the qualifying exams, dissertation proposal, dissertation writing, a nd the actual defense. I could not ask for a better person to help guide me through this process. My dissertation committee challenged me to look at things differently, offered support when I was unsure, and asked questions that made me think more deeply about my research. Being new to this process, I was nervous that a committee would feel like advers aries when it came to completing the dissertation. Instead, they were masters of their craft, who pushed me to be better as a researcher, as a professional,

PAGE 5

5 and a student. My dissertation is complete because of their guidance and advice. It is also complet e because they challenged me and pushed me to be at my best To my friends and family, who were incredibly understanding every time I had to cancel plans or miss out on an event. For all the times you looked at me with sympathy in your eyes, but also with pride, encouragement, and hope. all the lost time over the past three years and hear about the new jobs, new loves, new I am looking forward to re connecting To my colleagues and mo st importantly my team at work For understanding when I n eeded to take time off to write and for giving me the space to do that. For your continued support as I balanced the demands of school, a change in job function, and all the little life events that we never anticipate. journey.

PAGE 6

6 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ .......... 11 LIST OF FIGURES ................................ ................................ ................................ ........ 13 LIST OF ABBR EVIATIONS ................................ ................................ ........................... 15 ABSTRACT ................................ ................................ ................................ ................... 16 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 18 Research Context ................................ ................................ ................................ ... 18 Professional Context ................................ ................................ ............................... 19 Problem of Practice ................................ ................................ ................................ 21 Research Questions ................................ ................................ ............................... 29 Key Terminology ................................ ................................ ................................ ..... 29 Gamification Research ................................ ................................ ............................ 31 Purpose of Study ................................ ................................ ................................ .... 32 Research Design ................................ ................................ ................................ .... 32 Limitations, Ethical Considerations and Bias ................................ .......................... 33 Significance ................................ ................................ ................................ ............ 33 Summary ................................ ................................ ................................ ................ 35 2 LITERATURE REVIEW ................................ ................................ .......................... 36 Overview ................................ ................................ ................................ ................. 36 Key Terminology and Definitions Related to Gamification ................................ ...... 36 Game Based Learning and Serious Games ................................ ..................... 36 Gamification ................................ ................................ ................................ ..... 37 Game Elements ................................ ................................ ................................ 38 Motivation and Engagement through Gamification ................................ ................. 40 Theories and Frameworks Used in Gamification Studies ................................ ....... 42 Self Determination Theory ................................ ................................ ................ 43 MD A Game Design Framework ................................ ................................ ........ 45 Flow Theory ................................ ................................ ................................ ...... 45 Four Player Types ................................ ................................ ............................ 46 Four Types of Fun ................................ ................................ ............................ 47 Five Level s of Mastery ................................ ................................ ...................... 48 SAPS Reward System ................................ ................................ ...................... 49 Excluded Theory ................................ ................................ .............................. 49 Gamification Design and Findings ................................ ................................ .......... 50

PAGE 7

7 Conceptual Frameworks in Gamification Studies ................................ ............. 51 Gamification system design in empirical studies ................................ ........ 53 Research designs in Gamification studies ................................ ................. 55 Unintended results and design issues ................................ ....................... 57 Workplace and Performance Improvement Studies ................................ ................ 60 Workplace Motivational Studies ................................ ................................ ....... 60 Performance Improvement ................................ ................................ ............... 62 Lite rature Review Summary ................................ ................................ .................... 63 3 WORKPLACE ANALYSIS AND GAMIFICATION DESIGN ................................ .... 67 Techworks Workplace Analysis ................................ ................................ .............. 67 Human Performance Technology Improvement Model ................................ .... 67 Behavior Engineering Model ................................ ................................ ............ 68 Analysis of Techworks Work Environment : Summary ................................ ...... 70 Gamification Design Using the Self Determined Gamification Framework ............. 70 Self Determined Gamification Framework ................................ ........................ 70 Self Determination Theory (SDT) Alignment ................................ .................... 72 SDT alignment: organizational goals ................................ ......................... 72 SDT alignment: autonomy ................................ ................................ .......... 73 SDT alignment: equity ................................ ................................ ................ 74 SDT al ignment: individual goals ................................ ................................ 75 MDA Game Design Framework ................................ ................................ ........ 75 MDA framework: mechanics overview ................................ .................... 75 MDA framework: mechanics SAPS reward system ................................ 76 MDA framework: mechanics game element selection ............................ 76 MDA framework: dynamics overview ................................ ...................... 77 MDA framework: dynamics five levels of mastery ................................ ... 77 MDA framework: dynamics four player types ................................ .......... 78 MDA framework: dynamics four types of fun ................................ ........... 78 MDA framework: dynamics flow theory ................................ ................... 79 MDA framework: dynamics achievement structure ................................ 79 MDA framework: aesthetics overview ................................ ..................... 81 MDA framework: aesthetics user experience design .............................. 82 Techworks Portal Gamification ................................ ................................ ............... 83 Summary ................................ ................................ ................................ ................ 89 4 METHODOLOGY ................................ ................................ ................................ ... 90 Research Questions ................................ ................................ ............................... 90 Conceptual Framework ................................ ................................ ........................... 90 Research Design ................................ ................................ ................................ .... 91 Participants ................................ ................................ ................................ ....... 93 Data Collection ................................ ................................ ................................ 94 Data Collection: Self Determination Survey ................................ ..................... 95 Data Collection: Performance Data ................................ ................................ .. 97 Specific performance indicators ................................ ................................ 98

PAGE 8

8 Data collection: research journal ................................ ................................ 98 Data Analysis ................................ ................................ ................................ ... 99 Data analysis: self determination survey ................................ .................. 100 Data analysis: performance data ................................ ............................. 100 Rigor ................................ ................................ ................................ ............... 101 Limitations ................................ ................................ ................................ ...... 102 Ethical Considerations and Bias ................................ ................................ ..... 103 Summary ................................ ................................ ................................ .............. 103 5 RESULTS ................................ ................................ ................................ ............. 105 Data Analysis for Research Question 1: How does a gamification implementation designed using the Self Determined Gamification framework align with the motivation continuum within Self Determination Theory? ............ 105 Data Analysis for Research Question 2: How does the presence of a gamification layer in an online web portal impact college student employee perceptions of th e online workplace environment? ................................ ............ 107 Open ended Feedback ................................ ................................ ................... 107 Theme 1: Positive Experience ................................ ................................ ........ 107 Theme 2: Motivation and Impact ................................ ................................ .... 108 Setting goals and targeting achievements ................................ ............... 109 Job specific impact ................................ ................................ .................. 109 Incentiv izing work and feelings of legitimacy ................................ ............ 1 10 Theme 3: Design Considerations ................................ ................................ ... 111 Attainability and Requirements ................................ ................................ ....... 111 Design Suggestions ................................ ................................ ....................... 112 Data Ana lysis for Research Question 3: Does college student employee performance change after the implementation of a gamification layer within an online web portal used to complete job requirements? ................................ ...... 113 Performance Indicators ................................ ................................ .................. 113 Performance Indicators Quadrant 1 Employee Can Go Above and Beyond Expectations ................................ .. 115 Performance indicators quadrant 1 performance fe edback submissions ................................ ................................ .......................... 115 Performance indicators quadrant 1 proactive equipment checklists ..... 116 Performance indicators quadrant 1 proactive hall sweeps .................... 117 Performance indicators quadrant 1 office shifts completed ................... 119 Performance indicators quadrant 1 late notices received ...................... 120 Performance indicators quadrant 1 critical notices received ................. 122 Performance indicators quadrant 1 mobile support site events attended ................................ ................................ ................................ 123 Performance indicators quadrant 1 portal logs by day .......................... 124 Performance Indicators Quadrant 2 Employee Can Go Above and Beyond Expectations ................................ .. 126 Performance indicators quadrant 2 positive cus tomer service surveys 126 Performance indicators quadrant 2 negative customer service surveys 127 Performance indicators quadrant 2 event assists completed ................ 129

PAGE 9

9 Performance Indicators Quadrant 3 Employee Cannot Go Above and Beyond Expectations ............................. 130 Performance indicators quadrant 3 monthly evaluations completed ..... 130 Performance indicators quadr ant 3 area meetings attended ................. 132 Performance indicators quadrant 3 staff meetings attended ................. 133 Performance Indicators Quadrant 4 Employee Cannot Go Above and Beyond Expectations ............................. 134 Performance indicators quadrant 4 technical forum posts .................... 134 Performance indicators quadrant 4 assigned reactive support tickets .. 136 Performance indicators quadrant 4 ad hoc reactive support tickets ...... 137 Performance indicators quadrant 4 mobile support site reactive support tickets ................................ ................................ ....................... 139 Performance Indicators Summary ................................ ................................ .. 140 Performance indicators summary 2017 compared to 2016 ................... 140 Performance indicators summary 2017 compared to 2015 ................... 141 Performance indicators summary 2016 compared to 2015 ................... 142 Performance indicators summary key findi ngs ................................ ...... 143 Summary ................................ ................................ ................................ .............. 144 6 DISCUSSION ................................ ................................ ................................ ....... 146 Overview ................................ ................................ ................................ ............... 146 Discussion of Findings ................................ ................................ .......................... 147 Gamification Design ................................ ................................ .............................. 148 Gamification Design Overview ................................ ................................ ..... 148 Gamification Design Staff Perceptions and SDT Alignment ........................ 148 Gamification Design Additional Theory Alignment Considerations .............. 150 Gamification Design Iterative and Continuous Improvement ....................... 151 Gamification Design Summary ................................ ................................ ..... 153 Employee Performance ................................ ................................ ........................ 154 Implications and Significance ................................ ................................ ................ 156 Context Specific Implications and Significance ................................ .............. 156 Broader Implications and Significance ................................ ........................... 158 Suggestions for Future Research ................................ ................................ ......... 159 Summary ................................ ................................ ................................ .............. 161 APPENDIX A ORGANIZATIONAL GOALS ................................ ................................ ................. 163 B MAPPING ORGANIZATIONAL GOALS TO JOB REQUIREMENTS .................... 164 C ACHIEVEMENT LIST AND DYNAMICS MAPPING ................................ ............. 165 D ABBREVIATION KEY FOR ACHIEVEMENT LIST AND DYNAMICS MAPPING 176 E IMI SURVEY ................................ ................................ ................................ ......... 177

PAGE 10

10 F MODIFIED IMI SURVEY ................................ ................................ ....................... 178 G IMI SURVEY RESPONSES MEANS AND STANDARD DEVIATIONS ................ 179 LIST OF REFERENCES ................................ ................................ ............................. 181 BIOGRAPHICAL SKETCH ................................ ................................ .......................... 190

PAGE 11

11 LIST OF TABLES Table page 4 1 Techworks staff demographics over time. ................................ .......................... 94 4 2 Research questions, data sources and data analysis summary ......................... 99 5 1 IMI factor alignment ................................ ................................ .......................... 106 5 2 Performance indicators performance feedback submissions ........................ 115 5 3 Performance indicators proactive equipment checklists ................................ 116 5 4 Performance indicators proactive hall sweeps ................................ .............. 118 5 5 Performance indicators office shifts completed ................................ ............. 119 5 6 Performance indicators late notices received ................................ ................ 121 5 7 Performance indicators critical notices received ................................ ............ 122 5 8 Performance indicators mobile support site events attended ........................ 123 5 9 Performance indicators portal logs by day ................................ ..................... 125 5 10 Performance indicators positive customer service surveys ........................... 126 5 11 Performance indicators negative customer service surveys .......................... 127 5 12 Performance indicators event assists completed ................................ ........... 129 5 13 Performance indicators monthly evaluations completed ................................ 130 5 14 Performance indicators area meetings attended ................................ ........... 132 5 15 Performance indicators staff meetings attended ................................ ............ 133 5 16 Performance indicators technical forum posts ................................ ............... 135 5 17 Performance indicators assigned reactive support tickets ............................. 136 5 18 Performance indicators ad hoc support tickets ................................ .............. 138 5 19 Performance indicators mobile support site tickets ................................ ........ 139 5 20 Performance indicators summary 2017 compared to 2016 ........................... 141 5 21 Performance indicators summary 2017 compared to 2015 ........................... 142

PAGE 12

12 5 22 Performance indicators summary 2016 compared to 2015 ........................... 143 A 1 Organizational goals ................................ ................................ ......................... 163 B 1 Mapping organizational goals to job requirements ................................ ........... 164 C 1 Achievement list and dynamics mapping ................................ .......................... 165 D 1 Abbreviation key for achievement list and dynamics mapping ......................... 176 G 1 IMI survey responses means and standard deviations ................................ .. 1 79

PAGE 13

13 LIST OF FIGURES Figure page 1 1 Example of automatic reminder email a Techworks student employee might receive e ach week. ................................ ................................ ............................. 25 1 2 Techworks portal landing page providing a seven day rolling window of requirements. ................................ ................................ ................................ ...... 26 1 3 Proactive checklist tool within Techworks portal. ................................ ................ 27 1 4 Technical forum within the online Techworks portal where student staff connect. ................................ ................................ ................................ .............. 28 3 1 Self det ermined gamification framework ................................ ............................ 71 3 2 Techworks portal screenshot notification icon ................................ ................. 83 3 3 Techworks portal screenshot viewing options ................................ ................. 84 3 4 Techworks development portal screenshot rarity and iconography ................. 86 3 5 Techworks portal screenshot complete and incomplete achievements ........... 87 3 6 Techworks portal screenshot achievement progress bars ............................... 88 4 1 Conceptual framework workplace gamification ................................ ................ 92 5 1 Performance indicators grouped by employee control and ability to go above and beyond. ................................ ................................ ................................ ...... 114 5 2 Bar graph average performance feedback submissions per staff member. ... 116 5 3 Bar graph average proactive equipment checklists com pleted per staff member. ................................ ................................ ................................ ........... 117 5 4 Bar graph average proactive hall sweeps completed per staff member. ....... 119 5 5 Bar graph average office shifts completed per staff member. ........................ 120 5 6 Bar graph average late notices received per staff member. .......................... 121 5 7 Bar graph average critical notices received per staff member. ...................... 123 5 8 Bar graph average mobile support site events attended per staff member. .. 124 5 9 Bar graph average portal logs by day per staff member. ............................... 125

PAGE 14

14 5 10 Bar graph average positive customer service surveys received per staff member. ................................ ................................ ................................ ........... 127 5 11 Bar graph average negative customer service surveys received per staff member. ................................ ................................ ................................ ........... 128 5 12 Bar graph average event assists completed per staff member. ..................... 130 5 13 Bar graph average monthly evaluations completed per staff member. .......... 131 5 14 Bar graph average area meetings attended per staff member. ..................... 133 5 15 Bar graph average staff meetings attended per staff member. ...................... 134 5 16 Bar graph average technical forum posts per staff member. ......................... 136 5 17 Bar graph average assigned reactive support tickets per staff member. ....... 137 5 18 Bar graph average ad hoc reactive support tickets per staff member. .......... 138 5 19 Bar graph average mobile support site reactive support tickets per staff member. ................................ ................................ ................................ ........... 140

PAGE 15

15 LIST OF ABBREVIATIONS Achievements Digital awards for completing a task or series of tasks within a system. Behavior Engineering Model The behavior engineering model was developed by Gilbert (2007). The model evaluates the workplace environment and employee behavior to determine if sufficient supports and skills are present to optimize employee performance. Game Elements Aspects or chara cteristics of modern video games that are distinct enough to be repurposed in a gamification context. Gamification The use of game elements in non game contexts, through a web interface, software application, or comparable system. Mechanics Dynamics Aest hetics (MDA) Game Design Framework The MDA Game Design Framework outlines best practices and approaches in game design. Specifically, the framework looks at intentionally designing games with a mind towards diverse characteristics of players, along with th e overall user experience within a game. Self Determination Theory Self Determination Theory (SDT) is a motivational theory developed by Deci and Ryan (2000). The theory looks at extrinsic and intrinsic motivation on a continuum and advocates moving away from controlling and high pressure forms of motivation. This can be achieved by taking into account equity, organizational goals, individual goals, and autonomy. Web Portal In the context of this study, a web portal is an online environment where employee s engage with a variety of tools to complete and document completion of job responsibilities. All employee performance is tracked within the web portal.

PAGE 16

16 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Education SELF DETERMINED GAMIFICATION IN AN ONLINE WEB PORTAL: EMPLOYEE PERFORMANCE AND PERCEPTIONS By Paul Wolff August 2017 Chair: Swapna Kumar Major: Curriculum and Instruction This study sought to address a problem of practice by incentivizing job requirements through the addition of achievements in an online web portal. To do this, a workplace analysis was conducted, along with a thorough review of relevant literat ure. The result was the creation of a framework for designing gamified systems entitled Self Determined Gamification. Using this framework as a guide, an attempt was made to design achievements with Self Determination Theory (SDT) in mind, paying particula r attention to organizational goals, equity in attaining achievements, employee autonomy in interacting and engaging with the achievements, and individual goals. Design best practices were incorporated through the use of the MDA Design Framework in additio n to drawing upon design considerations from the SAPS Reward System, the Four Player Types by Bartle the Four Types of Fun by Lazzaro the Five Levels of Mastery by Dreyfus and Dreyfus and Flow Theory by Czikszentmihalyi With the design of the system as a major focus of this research, d etermining the success of the design in aligning with SDT considerations was important and this was accomplished through the use of the Intrinsic Motivation Inventory (IMI) Survey within SDT. Additionally, looking at

PAGE 17

17 performance indicators along with emplo yee feedback about the achievements provided insight into the overall success of the design and future considerations. The gamificat ion implementation had positive results with staff reporting enjoying the achievements and the impact they had on improving work habits Results from the IMI survey also indicated that the design of the achievements in the online web p ortal align ed with autonomous forms of motivation on the autonomy continuum within SDT A nalysis of perform ance metrics indicate d improved performance across multiple performance indicators This was particularly true in areas where employees had control in completing a job requirement and where employees had the opportunity to exceed performance expectations The Self Determined Gamification framework developed in this study provides both a model for designing gamification rooted in design best practices and relevant theory, and a method for evaluating that design.

PAGE 18

18 CHAPTER 1 INTRODUCTION Research Context Video games have become increasingly pervasive within society and with the advent of video games on mobile devices and embedded within online social networks, this pervasiveness has begun to transcend demographic boundaries (Barab, Gresalfi, & Ingramp Noble, 201 0; McGonigal, 2011; Squire, 2006; To bias, Fletcher, & Wind, 2014). Though video games were primarily seen as a source of entertainment, their popularity has led to an interest in repurposing the video game model within other contexts such as military train ing, education, and i ndustry (Tobias et al., 2014). The use of video games and video game components in other contexts resulted in a number of strategies and a consistent theme throughout these various game based strategies was a recognition that video gam es excel at engag ing end users. As of 2011, it was estimated that more than 180 million people in the United States played video games for 13 hours or more per week (McGonigal, 2011; as cited in Tobias et al., 2014). With this level of engagement, research ers and designers are attempting to harness the motivat ional components within games. One of the game based strategies that emerged through reviewing the literature is conceptualized under the umbrella term, gamification. Gamification involves the use of game elements in non game contexts. As an example, Chore Wars is an application designed to encourage users to complete household chores by awarding experience points and the ability to level up an avatar after completing specific user defined, real world tasks (McGon igal, 2011). In this case, the application is not a game, but instead represents a method of tracking the completion of chores. G ame elements are incorporated in the form of points, avatars,

PAGE 19

19 and leveling as a method of engaging users to partici pate in completing those chores. Chore Wars is one of many examples of a gamified system in the emerging field of gamification representing a relatively new field of study with the majority of research takin g place betw een 2013 and 2017 Much of the early research consisted of conference proceedings and is exploratory in nature (Ha mari, Koivisto & Sarsa, 2014). In addition, many of these early studies found positive results regarding motivation, engagement and user satis faction (Hamari et al., 2014). These early results are encouraging and indicate that the incorporation of game elements in a system has the p otential to motivate end users. At the same time, as a new field of study, there is a need within the current body of literature to incorporate design best practices and relevant theory when designing gamified systems (Hamari et al., 2014). More recent research is beginning to see this transition take place (Nacke & Deterding, 2017). This dissertation builds on the efforts of existing gamification implem entations in an attempt to identify the most appropriate theory and design best practices, resulting in the Self Determined Gamification framework for the incorporation of game elements within a system At the same time, this dissertation investigates a pr oblem of practice within an online work environment and looks at the potential use of game elements based on the Self Determined Gamification framework to address that problem of practice Professional Context Techworks provides on site technology support and strategic planning to the Housing Department at a major university In total, Techworks supports approximately 8,2 00 residents and 500 employees within the department Clients are s pread out across approximately 3 0 buildings, with departmental staff occupying 19 office locations. Techworks also supports approximately 500 permanent technology

PAGE 20

20 installations, in addition to the thousands of devi ces residents bring to campus. Specific examples of permanent technology installations supported within the hal ls include departmental desktop computers, digital signage, communal printers, HDTVs, and collaborative workstations. As the director of the Techworks program, the researcher oversee s a staff of three full time employees and 50 part time college student em ployees. The part time college student employees are comprised of 46 student techs who provide on site technical support, and 4 s enior techs who supervise and ment or a subsection of their peers. Techworks student employees go through a comprehensive traini ng program and have access to training materials, documentation, and a variety of too ls within an online web portal. Student employees live within the residence halls and help mainta in technology within the halls. Student employees are organized into regio ns, and are responsible for supporting both personal technology brought by residents and the permanent department owned technology installed within those regions. Support of technology within the halls includes both proactive and reactive support. Proacti ve support is accomplished through weekly status checks of permanently installed technology, along with visiting each room on campus within a semester to check for any unreported technology issues. As an example, a proactive check of an HDTV in the halls w ould involve verifying functionality of HDMI and mini Display cables used to connect a laptop to the screen, checking to ensure that the cable channels are clear, verifying the presence of appropriate signage, and p hysically cleaning the screen. Any issues with a proactive check of this kind are reported through the online web portal so they can be addressed befo re a client reports the issue.

PAGE 21

21 Reactive support involves responding to client technology issues after they are reported. This support frequently in cludes helping clients set up devices to connect to the network or troubleshooting technolog y that is behaving irregularly. As part of the Techworks Service Level Agreement (SLA) with the Housing Department student employees working for Techworks contact clients to schedule appointments within 24 hou rs of an issue being reported. Student employees put on two technology support events each week that rotate between the halls. Here, clients can get devices physically cleaned or report issues they are experien cing. Techworks student staff also provide technology for departmental events, consulting with clients about the event and se tting up technology as needed. Supporting technology at departmental events typically includes setting up screens, projectors, spea kers, or video game systems. Problem of Practice Within Techworks student employees complete their work remotely and autonomously with minimal direct supervision and with communication ofte n taking place asynchronously. Work is either completed or recorde d in an online web portal and all progress in performing job requirement s is tracked within the portal. Examples of student employee performance indicators completed within the web portal include posts to a technical forum and completion of assigned tasks such as updating the employee profile within the st aff directory. Examples of student employee performa nce indicators that are recorded within the web portal include documenting reactive support tickets, documenting proactive equipment checklists, attendance at office shifts and attendance at required Techworks events. For each performance area tracked in t he web portal, an example of both required performance and exemplary performance is identified and comm unicated to student employees. Exemplary performance is based on student

PAGE 22

22 employee performance in previous semesters and represents the best example of em ployee performance seen within a respective category over time (Gilbert, 2007) As an example, Techworks organizes two mobile technology support sites each week that are rotated within the halls. Student employees are required to participate at one support site during each two week interval, but can attend more than the required amount. Required performance is defined as attendance at seven events per semester, and exemplary performance is defined as attendan ce at ten events. Other performance indicators re flect the quality of work, such as the number of positive customer service survey respons es a student employee receives. Specific details about each performance indicator are provided in Chapter 5 The remote and unsupervised nature of the work required in Techworks creates unique challenges and student employee performance, in s ome cases, has become an issue. Here, the organization has historically seen a bell curve of performance where most student employees meet the required performance expectations, a s mall number of student employees exceed those expectations, and certain student employees fail to meet the re quired performance indicators. Addressing the needs of student employees who fail to meet required performance indicators, where possible, in order to improve the overall customer experience is a primary concern within Techworks Each year, Techworks management attempts to improve the workplace environment to better align with employee needs in an environment where stude nt employees are unsupervised. Existing student employees who do not meet required performance indicators represent the first problem of practice within Techworks

PAGE 23

23 Within Techworks all employees must be active full time students in good academic standing. Good academic standing is de fined as maintaining a 2.5 GPA or higher. In addition, student employees are required to be on campus resident s in order to work for Techworks Due to the transient nature of a college campus, Techworks student staff rotate on a cyclical basis of every thr ee to four years. Each year Techworks hires 10 to 15 new employees as veteran sta ff graduate or move off campus. Training new staff each year and preparing staff for their job responsibilities represents a secondary problem of prac tice within Techworks A comprehensive training program has been developed over the course of Techworks year history that involves face to face instruction, mentor pairings, a nd regular on the job feedback. All training materials are available online to employees within t he web portal. Open ended survey responses and face to face interviews with student employees who have failed to meet performance expectations have indicated issues of motivation, feelings of disconnect, and a need for more structure within the Techworks environment. Over several years, multiple tools were developed within the online web portal in an attempt to address feelings of disconnect and in an ef fort to provide more structure. One example of a tool created to address these issues includes a reminde rs tool that automatically sends students a list of individual job requirements at the start of each week, along with du e dates for those requirements. Another tool created provides students with an interactive, step by step checklist of what should be com pleted at each permanent technology installation during w eekly proactive status checks. A technical forum provides student employees the opportunity to ask questions of all Techworks staff. Here veteran staff can mentor new employees asynchronously, and fu ll time

PAGE 24

24 Techworks staff can help guide this process by offering expertise and pointing student staff to resourc es for troubleshooting issues. A new landing page within the online portal was developed, that gives students a seven day, rolling window of upcoming job requirements. In addition to the training program and tools mentioned above, the evaluation process for all staff was adjusted to provide more timely feedback to employees. Each student employee is evaluated on a monthly basis based on their performance and both exemplary performance along with clear areas of improvement are communicated. Each student employee is also required to attend a face to face mid semester assessment to review remaining job requirements and ensure staff understand expe ctations. A summative semester evaluation takes place after the comp letion of exams each semester. The summative evaluation incorporates feedback from student supervisors, employees, customers and full time staff from the entire seme ster. Employees can vie w their prior monthly evaluations and summative semester evaluations within the web portal at any time. Figures 1 1, 1 2, 1 3, and 1 4 show examples of som e of the tools discussed above. Figure 1 1 represents an example of an automated weekly email a Tech works s tudent employee might receive. Here, all job requirements for the next seven days are communi cated to the student employee. Figure 1 2 shows th e new web portal landing page. Here students can see job requirements for the next seven days or expand th is view to see all job require ments for the entire semester. The landing page in Figure 1 2 also provides links to al l tools within the web portal. Figure 1 3 represents an example of a proactive checklist a Techworks student emplo yee may complete. Here, s pecific

PAGE 25

25 instructions are provided to student employees and the interface allows employees to indicate whether each checklist it em was performed successfully. Figure 1 4 shows the Techworks Technical Forum where student employees ask que stions about technic al issues. Within the technical forum, more experienced staff can provide guidance and additional resources to one another as student staff engage in the troubleshooting process. Figure 1 1 Example of automatic reminder email a Techworks student empl oyee might receive each week.

PAGE 26

26 Figure 1 2. Techworks p ortal landing page providing a seven day rolling window of requirements.

PAGE 27

27 Figure 1 3. Proactive checklist tool within Techworks p ortal.

PAGE 28

28 Figure 1 4. Technical f orum within the online Techworks p ortal where student staff connect. The changes discussed above resulted in more interaction between student employees and provided student employees with clearly communicated ex pectations on a regular basis. Feedback from student staff, regardle ss of prior performance, was positive and indicated increased feelings of connection to fellow employees and a mo re structured work environment. The number of students who completed mini mum job requirements increased. Despite these positive results, studen t employee performance issues continued with a subset of students and Techworks management began looking for ways to better motivate all employees within the web portal, in order to improve job performance overall and en courage exemplary performance. An an alysis of the online workplace environment, along with trending technology innovations, resulted in a decision in 2013 to investigate and eventually implement gamification elements within

PAGE 29

29 the Techworks web portal in an effort to improve performance and inc reas e employee motivation. The gamification layer was implemented in Spring of 2017. Leading up to this implementation the goal of this dissertation wa s to ( a) design gamification elements according to best practices in prior gamification research ( b) im plement the gamification layer within the Techworks Portal, and ( c) evaluate the impact of the gamification layer on overall emplo yee perfo rmance This context, along with the initial problem of practice, led to the following research questions: Research Questions 1. How does a gamification implementation designed using the Self Determined Gamification framework align with the motivation continuum within Self Determination Theory ? 2. How does the presence of a gamification layer in an online web portal impact college student employee perceptions of the online workplace environment ? 3. Does college student employee performance change after the implementation of a gamification layer within an online web portal used to complete job requirements? Key Terminology Web p ortal: In the context of this study, a web portal is an online environment where employees engage with a variety of tools to complete and document compl etion of job responsibiliti es. All employee performance is tracked within the web portal. Gamification: Gamification involves the incorporation of game ele ments within non game contexts. The intent of incorporating game elements is to motivate and engage users. Achievements: A spec ific game element, achievements represent digital awards for completing a task or series of tasks within a system and specifically within the Techworks Portal.

PAGE 30

30 Self Determination Theory: Self Determination Theory is a motivational theory deve loped by Deci and Ryan (2000). The theory looks at extrinsic and intrinsic motivation on a continuum and advocates moving away from controlling and high pressure forms of motivation. This can be achieved by taking into account equity, organizational goals, individual g oals, and autonomy. Identified and Integrated Regulation: Within Self Determination Theory, Identified Regulation and Integrated Regulation represent less controlling forms of extrinsic motivation. Here, individuals begin to internalize organizational goa ls and values through properly designed extrinsic motivational structures. MDA Game Design Framework: The MDA Game Design Framework outlines best practices and approaches in game design. Specifically, the framework looks at intentionally designing games w ith a mind towards diverse characteristics of players, along with the overall user experience within a game. Behavior engineering m odel: The behavior engineering model w as developed by Gilbert (2007). The model evaluates the workplace environment and empl oyee behavior to determine if sufficient supports and skills are present to optimize employee performance. Environmental s upports: Within the behavior engineering model, environmental supports represent the aspects of a workplace setting that help improve employee perform ance. The focus of environment al supports is on the improvement of workplace settings.

PAGE 31

31 Repertory of b ehavior: repertory of behavior represents the skills and behavior that an employee po ssesses. This includes the training an employee receives. Gamification Research Gamification represents an exciting new field of research with great potential to motivate and engage end users. Early results indicate that incorporating certain game elements resulted in improved user perceptions of their experience increased motivation and increased engagement (Cheong, Cheong & Filippou, 2013, Conaway & Garay, 2014; Frith, 2013 ; Gooch, Vasalou, B enton, & Khaled, 2016; Hamari, 2015 ). Where there are mixed results in the existing body of literature, the gamified treatment frequently increases motivation in a subset of the population, while having no impact on other users (Cruz & Penley, 2014; Gianne tto, Chao & Fontana, 2013; Goehle, 2013; Osipov, Nikulchev, Volinsky & Prasikova, 2015 ). At the same time, Dom nguez et al. (2013) found that while users self report being more engaged and motivated, their performance on certain course components decline, while others improve These primarily positive results are understandable given the early stat e of the research in this field. There is also a growing call in recent reviews of the existing body of literature for the incorporation of theory and game design best practices as the field of study matures ( Dale, 2014; Ha mari, Koivisto & Sarsa, 2014). As research around gamification matures, theory and design considerations are beginning to appear more frequently in gamification literature, but there is still lit tle consensus (Nacke & Deterding, 2017). Early gamification studies have also focused on educational or commercial environments with few studies examin in g gami fication within the workplace. Given the lack of formal studies examining gamification in workpla ce settings, the primarily positive results in

PAGE 32

32 other fields where gamification has been studied, and the recognition of the potential use of gamification within the workplace there is an opportunity to fill a gap in the literature (Bajdor & Dragolea, 2011 ; B edard, 2015; Brownhill, 2013). Purpose of Study Given the lack of gamification studies in the workplace and the call for the incorporation of theory and game design best practices, t his study wa s intended to synthesize and apply best practices in desig ning gamified systems. The study outlined and implemented a model for designing gamified systems that wa s cyclical and reflective with the intention of continuous system improvement. At the same time, this study explore d the potential utility of specific g ame elements in improving the performance of college student employees who work in an unsupervised and autonomous environment. Research Design This study follow ed a t hree phased research approach. Phase one of this study consist ed of a review of existing r esearch to identify both best practices in gamification design and the most appropriate theory when considering gamification implementations. In addition, phase one looked at workplace motivational studies and performance improvement literature to help inf orm the specific implementation of a gamificatio n layer in a workplace setting. Phase one was the focus of Chapter 2 and result ed in a framework for the design of gamification. Phase two of this study involved an analysis of the Techworks workplace enviro nment looking at the gap in performance and environmental supports currently in place. In addition, phase two utilized researched best practices to design a gamification layer within the Techworks workplace that was grounded in appropriat e theory and

PAGE 33

33 design principles. Phase two wa s the focus of Chapter 3 and result ed in the design and implementation of the gamification layer in the Techworks Portal. Phase three of this research involved the evaluation of the gamification design, an analysis of employe e perceptions of the gamification implementation, and an evaluation of any changes in employee performance during t he gamification implementation. This involve d the collection and analysis of both quantitative and qualitative data, which were converged dur ing a discussion of that data. Chapter 4 focuse d on the overall research design and provide d additional context through the presentation of a conceptual framework that inform ed each of the three phases. Limitations, Ethical Considerations and Bias Due to convenience sampling, findings from this study may not generalize to the greater populat ion. Despite this limitation, some recommendations were transferrable to similar po pulations in similar settings. This is consistent with current gamification research which is exploratory in nature. It is important to note that there we re additional variables that may have impact ed employee performance metrics. A number of ethical considerations related to workplace studies and this specific context were also addressed in the study Significance This study wa s significant personally, within my professional practice, and within the f ield of gamification research. On a personal level this research addresse d an area of professional and personal interest through the incorpo ration of game elements in a workplace setting. Gamification is increasingly used in a variety of contexts and I have been personally motivated through interacting with game elements both within g ames and in non game contexts. I also hope to continue to st udy gamification beyond this

PAGE 34

34 dissertation and there is a great deal of potential for longitudinal studies in this context. This research may open up professional opportunit ies on a personal level and at a minimum, contribute d to my own understanding of performance improvement and employee motivation as a supervisor. Within my professional practice, this research helped address performance issues with employees who work in a remote, online environment w ith minimal direct super vision. Each year fulltime staff at Techworks attempt to improve the tools and resources provided to the student staff. While there will likely always be some areas for performance improvement in any organization, this research resulted in performance impr ovements within Techworks in several key areas resulting in an improved service to our client population. Several researchers have pointed to the potential application of gamification within workplace environments but there are few studies that have investigated this possibility (Bajdor & Dragolea, 2011; B edard, 2015; Brownhill, 2013 ; Cardador, Northcraft, & Whicker, 2017 ; Perryer, Celestine, Scott Ladd, & Leighton, 2016 ). This dissertation filled a gap in the existing literature by intentionally desi gning a gamified soluti on within a workplace setting. The intentionally designed nature of this solution should not be u nderstated due to the need within the existing body of research for the incorporation of theory and best practices in gamification desig n T his research attempt ed to address this need by incorporating relevant theory and design principles in imp lementing a gamified solution. By using a formal design process, incorporating relevant theory, and aligning the gamification elements with organiz ational and individual goals, this study provided a f ramework for gamification design A framework

PAGE 35

35 of this kind could be used to guide future gamification design and implementation even outside the context of this study Moreover, this study outlined a met hod of evaluating and continually improving gamification implementations. S ummary Supervising a workplace environment where student workers work remotely with minimal supervisi on presents unique challenges. Over many years, an online web portal was develo ped which houses a variety of tools that help address many of these challenges though some performance and motivation issues persist At the same time, g amification represents an exciting new field of study with the potential to motivate users within a system. As part of the continued improvement of the Techworks portal, a decision was made to implement a gamified achievemen t structure within the portal. This decision was intended to provide employees with more structure within the portal, provide clear expectations through incentivizing key job responsibilities, and increase employee performance with respect to the desired performan ce on the part of the employer. This study sought to incorporate best practices from gamification literature, workplace stud ies, and perfo rmance improvement literature. The intent wa s to design an achievement system within the portal that wa s rooted in Self Determination Theory and the MDA Game Design Framework. Performance of current employees who use the achievement system wa s compared to past employee performance who used the same tools without a gamification layer. The gamification design was also assessed, along with an analysis of employee perceptions of the gamification implementation.

PAGE 36

36 CHAPTER 2 LITERATURE REVIEW Overv iew In approaching the creation of a gamification layer within the Techworks Portal, a thorough evaluation of the existing gamif ication literature was needed. By examining gamification studies, themes emerged regarding design considerations and the incorp o ration of appropriate theory. In addition, a review of workplace motivational studies point ed to key considerations about the design of a gamification layer in a workplace context. Gamification is a relatively new field of study. Of the 67 empirical studi es reviewed in Chapter 2 dealing specifically with gamification, 2 4 were conference proceedings and 43 were p eer reviewed journal articles. Of these, 50 took place between 2013 and 201 7 and only 17 studies were identified that too k place between 2008 and 2012. 14 of these early studies we re conference proceedings Gamification research is primarily exploratory in nature at this stage, with some research designs using a quasi experimental approach. Key Terminology and Definitions R elated to Gamification Game Based Learning and Serious Games Much of the literature dealing with the application of video games or video game elements for motivational and engagement purposes involved a discussion of three main areas: game based learning, se rious games, and gamification. While the focus of Chapter 2 is gamification, it is important to distinguish between the various areas of study that deal with the use of video games and game elements in order to provide

PAGE 37

37 clarity and avoid confusion. Additi onally, defining specific game elements as they relate to gamification will help provide clarity throughout this discussion of the literature. Game based learning involves the creation of a video game environment that engages cognitive processes in the pl ayer with the purpose of teaching or instructi ng ( Tobias et al., 2014 ). Frequently, game based learning environments attempt to maintain the aspects of fun and engagement that are characteristic of video games (Barab Gresalfi, & Arici, 2009; Barab et al., 2010). Within the literature, this is commonly referred to as play, a concept that will be discussed in great er depth later in Chapter 2 Serious games are distinguished from game ba sed learning in several areas. Serious games, sometimes referred to as al ternate reality games, attempt to solve real world problems by engaging users in a game environment ( Kirkley, Tomblin, & Kirkley, 2005; McGonigal, 2011; Nacke, Drachen, & Goebel, 2010) In this sense, serious games encompass a broader category of games conceptually, and both game based learning and gamification might fall into the category of serious games, depending on the nature and pur pose of the respective system. Serious games frequently focus on the real world problem, context and purpose of the ga me (Nacke et al., 2010). Gamification Gamification involves the application of features from video gam es in non gaming environments. While this is not an entirely new concept, the flexibility and pervasiveness of modern technologies like the Internet, sma rt phones, and laptops, have resulted in a revision in the conceptual definition of the term (Deterding, Dixon, Khaled, & Nacke 2011; Ham ari, Koivisto, & Sarsa, 2014). Several definitions emerged in the literature in an attempt to provide clarity within t he discourse One definition

PAGE 38

38 non et al. 2011, 10; as cited in Domnguez et al., 2013; Hakulinen, Auvinen, & Korhone n, 2013; Hamari et al., 2014). Wh ile this broad definition served the purpose of many researchers, another attempt to narrow the definition gaming software et al., 2013, 381). (xiv; as cited in Kim, 2015b). All of these definitions involve the use of game elements or mec hanics outside of a traditional game environment. For the purpose of this dissertation, the following definition was used for gamification: Gamification : The use of game elements in non game contexts, through a web interfa ce, software application or comparable system Game Elements A variety of specific game elements were discussed in the literature, and it is useful to define some of the key game elements that emerged prior to discussing t heir use in empirical studies. It is important to note that th ese elements exist in modern video games and a gamification process attempts to repurpose these el ements outside of video games. Some elements are not unique to games or gamified systems, but their conceptual definition within this context is worth noting. Game element selection focused on those that appeared in multiple sources, though additional elements wer e discussed in the literature. Excluded elements that appeared infrequently within the gamification literature include story/narrative, economies, and three dimensional

PAGE 39

39 environments ( Chapman & Rich, 2017; Deterding et al ., 2011; Hamari et al., 2014). Common game elements within the literature include: Achievements or t rophie s : Digital awards for completing a task or series of tasks within a system ( Chapman & Rich, 2017; Dale, 2014; Dickey, 2007; Hamari et al., 2014; Rapp, 2017; Sailer, Hense, Heinz, & Klevers, 2013; Seaborn & Fels, 2015; Zichermann & Cunningham, 2011 ) Badges : Similar to achievements in that they are frequently awarded for completi n g a task or series of tasks, b adges are distinct in that they tend to have a degree of permanence and are frequently displayed on a nameplate or in some other manner to convey status ( Chapman & Rich, 2017; Deterding et al., 2011; Hamari et al., 2014; Maan, 2013 ; Sailer et al, 2013; Seaborn & Fels, 2015; Zichermann & Cunningham, 2011 ) Points : A numeric measure that quantifie s some action within a system. Often points are used to track progress within a gamified system ( Chapman & Rich, 2017; Dickey, 2007; H amari et al., 2014; Maan, 2013 ; Rapp, 2017; Sailer et al, 2013; Seaborn & Fels, 2015; Zichermann & Cunningham, 2011 ) Levels: Numeric benchmarks that convey expertise, skill o r proficiency within a system. Typically, levels are obtained by accruing a pred efined number of points or completing specific tasks ( Chapman & Rich, 2017; Deterding et al., 2011; Dickey, 2007; Hamari et al., 2014; Maan, 2013 ; Rapp, 2017; Seaborn & Fels, 2015; Zichermann & Cunningham, 2011 ) Progress b ars: Visual representations of a ( Chapman & Rich, 2017; Dale, 2014; Dickey, 2007; Hamari et al., 2014 ; Sailer et al, 2013; Seaborn & Fels, 2015 ) Leaderboards: A rank order listing of users within a system based on some metric. Leaderboards are frequently used to foster a sense of competition within a gam i fied system ( Chapman & Rich, 2017; Deterding et al., 2011; Hamari et al., 2014; Maan, 2013 ; Sailer et al, 2013; Seaborn & Fels, 2015; Zichermann & Cunningham, 2011 ) Avatars: Digital representations of se lf within a gamified system ( Blohm & Leimeister, 2013; Chapman & Rich, 2017; Deterding et al., 2011 ; Sailer et al, 2013 ) Quests: A task or series of tasks that users unde rtake within a gamified system. Quests typically involve some form of narrative structure and are part of a larger progression model ( Blohm & Leimeister, 2013; Chapman & Rich, 2017; Dale, 2014; Dickey, 2007; Rapp, 2017; Sailer et al, 2013; Zichermann & Cunningham, 2011 )

PAGE 40

40 Bosses: Bosses within games represent parti cularly challenging obstacles. Within a gamified system, these obstacles represent challenging goals that will require effort to achieve ( Chapman & Rich, 2017; Hamari et al., 2014; Zichermann & Cunningham, 2011 ) Real time f eedback: Within a gamified system, real time feedba ck deals wit h the overall user experience. When certain benchmarks are reached within the system (a level, an achievement, etc.), real time feedback is the indication to the user, typically through both audio and visual indicators, that the benchmark was a chieved ( Chapman & Rich, 2017; Deterding et al., 2011; Hamari et al., 2014; Zichermann & Cunningham, 2011 ) Rules: The rules govern the structure of the gamified application and define th e processes within the system. The rules dictate how the user will i nteract with the various other game elements within the system ( Chapman & Rich, 2017; Deterding et al., 2011; Maan, 2013 ; Zichermann & Cunningham, 2011 ) Motivation and Engagement through Gamification In reviewing the gamification literature, most studies looked at gamification as a way to motivate and engage end users. In the literature, this is variously referred to as incentive approach, incentive mechanisms, or incentive centered design (A nderson et al., 2013; Deterding et al. 201 1; Farzan & Brusilovsky, 2011). Frequently this manifest ed itself in the form of achievement points or badges that we re awarded to users after completing some task within a system (Anderson et al., 2013). In the case of St ackOverflow, a popular question and answer site a badge system was implemented that awarded users with badges based on activity on the site. This activity typically involved users asking questions, providing answers to questions and rating answers. Here users were observed to increase activity rewarded by a badge while the researchers noted no change in behavior surrounding activities that were not rewarded by a badge (Anderson et al., 2013; Grant & Betts, 2013) Interestingly, user activity wa s accelera ted further when nearing the threshold for attaining the badge indicating that users were paying attention to the badge system and

PAGE 41

41 adjusting behavior in order to meet the requirements to attain a badge (Anderson et al ., 2013; Grant & Betts, 2013). Anderso n et al. (2013) also observed that providing users with multiple small incentives at progressively larger increments preserved the incentive structure over long periods of activity. When adding a plug in incorporating trophies as a reward system to the lea rner management system Blackboard, researchers observed a similar effect as users engag ed in activities that awarded achievements (Domng uez et al., 2013). In addition to observing this pattern of behavior, Domnguez et al. (2013) had participants complete a questionnaire dealing with their experience using t he site and motivation levels. Self reported results from participants indicated higher motivation when using Blackboard and higher motivation towards learning with the achievement structure in place (D omnguez et al., 2013) Another study with similar positive results added an incentive system to CourseAgent, a community based course recommendation system Users with access to the gamified incentives, within CourseAgent rated and recommended more cours es within the system when compared to users who were not using the gamified approach (Farzan & Brusilovsky, 2011). Farzan and Brusilovsky (2011) conducted a second study which opened the gamified aspects of CourseAgent to all users and observed that those who actively participated in the gamified components contributed more to the site. Similarly, in looking at the application Foursquare, Frith (2014) found that participants interacted and engaged with surroundings differently and participants reported inc reases in motivation and engagement du e to the gamified application. Numerous additional studies focused on motivation and engagement with primarily

PAGE 42

42 positive results ( A Cheong et al., 2013; Conaway & Garay, 2014; Cruz & Penley, 2014; De Schutter & Abeele, 2014; Denny, 2013; Giannetto et al., 2013; Hamari & Koivisto, 2013 ; Liebe roth, 2014; Su & Cheng, 2014). These primarily positive results surrounding motivational increases introduced by gamification are pro mising and it is clear that gamification has the potential to motivate. Theories and Frameworks Used in Gamification Studies In a meta analysis of existing gamification literature, Hamari et al. (2014) point to the primarily positive results surrounding e ngagement, motivation, participation, and enjo yment in gamification studies. At the same time, Hamari et al. (2014) point to the need for the incorporation of theory and formal design frameworks in many gamification studies. This is understandable consider ing gamification is still a new field of study and early exploratory studies often la ck a theoretical underpinning. T he use of theory is becoming more prevalent in more recent gamification studies. In addition, several sources present potential theoretical or design frameworks when designing a gamified system even if they did not conduct a formal study The following section outlines the most relevant theory and design frameworks from empirical studies, literature reviews, and conce ptual articles related to gamifica tion and workplace motivation. Specifically, the following theories are discussed in greater detail in this section : Self Determination Theory, the MDA Game Design Framework, Flow Theory, the Four Player Types, the F our Types of Fun, the Five Levels of Master y, and the SAPS Reward System. The rationale behind the exclusion of certain theory from the literature is also explained at the end of this sectio n. Each theory discussed below is relevant within the c ontext of

PAGE 43

43 g amification design. A synthesis of these theories is presented in a framework used to design a gamification implementation in Chapter 3 Self Determination Theory Developed by Deci and Ryan, Self Determination Theory (SDT) looks at motivation as a concep t, specifically focusing on intrinsic and extrinsic motivation ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). The theory places motivation on an autonomy continuum with amotivation on one extreme. Amotivation is defined as a lack of motivation and is considered a state of apathy ( Gagn & Deci, 2005 ; Ryan & Deci, 2000 ) SDT places intrinsic motivation on the o ther extreme of the continuum. Intrinsic k or activity ( Gagn & Deci, 2005 ; Ryan & Deci, 2000 ) According to SDT, everything between these two extremes is extrinsic motivation since a person experiencing amotivation would need some external motivation in order to engage in an activity and a perso n who is intrinsically motivated would not need an external motiva tor to engage in the activity. SDT defines four distinct types of extrinsic motivation which vary in the degree to which the extrinsic motivator attempts to control the individual on the amo tivation side of the continuum, or the degree to which the extrinsic motivator results in autonomous motivation on the intrinsic motivation side of the continuum ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). The four types of extrinsic motivation within SDT are External Regulation, Introjected Regulation, Identified Regulati on, and Integrated Regulation. External regulation deals with highly controlling environments that often present an individual with rewards and punishments that place a great deal of pressure on the individual ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). Introjected regulation is similar to external regulation, but where the latter deals with reward and punishment as the motivation contingency, introjected regulation deals

PAGE 44

44 w orth being tied to performance as the motivation contingency. While not as controlling as external regulation, introjected regulation is still considered a controlling form of extrinsic motivation ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). Identified regulat ion involves extrinsic motivation where the importance of organizational goals, values and regulations are clearly defined for the individual, resulting in the individual feeling greater freedom and volition. With identified regulation, meeting both organi zational and personal goals is the motivation contingency and we see a shift away from controlling forms of extrinsic motivation towards more autonomous motivation ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). Integrated regulation involves the individual fully internalizing the goals, values and regulations of the extrinsic motivation, making those goals central With integrated regulation there is coherence between the organizational goals, values and regulations and those of the i ndividual, making this the motivation contingency ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). Due to the internalization of the organizational goals, integrated regulation is the only form of autonomous extrins ic motivation. Both integrated regulation and int rinsic motivation are autonomous forms of motivation, but they differ in that intrinsic motivation does not need an external motivation to elicit behavior ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). SDT appeared in numerous workplace motivational studies di s cussed later in Chapter 2 The focus on extrinsic motivational structures and workplace autonomy made it an ideal candidate for this study, particularly given the autonomous nature of the workplace environment studied in this dissertation. In addition, Chapman and Rich (2017) point to SDT as a strong theoretical option for gamification design. Gamification

PAGE 45

45 is an extrinsic motivational structure and SDT offer ed both design considerations and evaluation methods for aligning an intervention wit h the more autonomous forms of extrinsic motivation on the autonomy continuum. MDA Game Design Framework The MDA Game Design Framework looks at the mechanics, dynamics and aesthetics within a game (Deterding et al., 2011; Z ichermann & Cunningham, 2011). The theory outlines best practices and approaches to game design, but was repurposed within gamified system design due to the similar considerations it identifies. Within the context of gamification, the mechanics phase represents the game elements defined above and the selection of those mechanics will be contingent upon a variety of factors (Deterding et al., 2011; Zi chermann & Cunningham, 2011). The d ynamics phase within the MDA framework looks at how users interact with the game elements (Deterding et a l., 2011; Zi chermann & Cunningham, 2011). The a esthetics phase within the MDA framework involves the user experience and how the interaction with game elements makes the user feel (Deterding et al., 2011; Zichermann & Cunningham, 2011). When used in gamifi cation design, the MDA Game Design Framework allows designers to appropriately select game ele ments within specific contexts. It also outlines considerations for addressing a variety of user characteristics through a cyclical p rocess of review and revision Within the context of this dissertation, the MDA Game Design Framework provide d an overarching model for gamification design. Flow Theory Developed by Czikszentmihalyi, flow theory attempts to identify the optimal level of engagement within any activit y (Czikszentmihalyi, 1989; Z ichermann & Cunningham, 2011). When applied to video games or gamification, flow theory looks at the interplay

PAGE 46

46 between the challenge of an activity and the ability of the player to perform the activity (Z ichermann & Cunningham, 2011). Flow operates as the ideal level of difficulty within a boredo m (Czikszentmihalyi, 1989; Zichermann & Cunningham, 2011). Flow theory is incorporated in gamification design in an effort to address the various skill levels users possess when performing a task or a ctivity. Within the context of this dissertation, flow theory made particular sense when aligning game elements with a variety of user skill levels. Within the larger context of SDT and the MDA Game Design Framework, providing opportunities that accommodate a variety of user skill levels was particularly impor tant. Four Player Types video game environment (Bartle, 1996; Z ichermann & Cunningham, 2011). The player type categories examine what motivates individuals to engage in activities within the game. The four player types are placed on an x/y axis with the x continuum examining the extent to which players are engaged in acting versus interacting and the y continuum examining whether that acting or interacting involves other players or the environment within the game (Bartle, 1996; Z ichermann & Cunningham, 2011). The four types are killers, achieve rs, socializers and explorers. Killers represent the quadrant involvi ng acting with other players. They are motivated by engaging with other players in competition and defeating others in that competition (Bartle, 1996; Zichermann & Cunningham, 2011). Achievers represent the quadrant involving acting with the environment. They are motivated by completing tasks or quests and meeting g oals

PAGE 47

47 within the game (Bartle, 1996; Z ichermann & Cunningham, 2011). Socializers represent the quadrant involving i nteracting with other players. They are motivated by collaborating with others within the game (Bartle, 1996; Z ichermann & Cunningham, 2011). Explorers represent the quadrant involving interact ing with the game environment. They are motivated by immersing themselves in the game environment and exploring different aspects of the game (Bartle, 1996; Zichermann & Cunningham, 2011). The four player types are not mutually exclusive and players frequently exhibit characteristics of all four types. Looking at different user characteristics within a game, including what motivates users to participate and engage with content, is an important component of any game design. When looking at gamification, many of the same u ser characteristics will exist. environments, they certainly serve to illustrate a variety of perspecti ves a us er might bring to an activity. decisions when building a gamified system. Four Types of Fun Developed by Lazzaro, the four types of fun look at emotion and fun within games (Lazzaro, 2004; Zichermann & Cunningham, 2011). the four types of fun are placed on an x/y axis with the x continuum tracking fun that exists in ga mes versus life and the y continuum examining fun that is go al oriented versus open ended. The resulting four types of fun are hard fun, easy fun, people fun and serious fun (Lazzaro, 2004; Z ichermann & Cunningham, 2011). Hard fun is goal oriented fun with in a game and results in feelings of pers onal triumph over a challenge. Easy fun is open ended fun within a game and results i n curiosity within the player.

PAGE 48

48 People fun is goal oriented fun within a game and the resulting emotion is amusement. Serious fun i s open ended fun within a game and results in relaxation and excitement (Lazzaro, 2004; Zichermann & Cunningham, 2011). According to this theory, game designers should consider all four types of fun when designing games, and the creation of more emotions through the use of these four types of play results in a more captivating and engaging game experience. Within gamification design, providing opportunities for users to engage in different types of play will create a more well rounded system with considera tions for a broader user base. Five Levels of Mastery Developed by Dreyfus and Dreyfus, the five levels of mastery look at the different stages a user experiences when interacting with a system (Dreyfus & Dreyfus, 1980; Z ichermann & Cunningham, 2011). At the first level of mastery, a user is a novice who is completely new to the system (Dreyfus & Dreyfus, 1980; Z ichermann & Cunningham, 2011). Over time, a user moves into the second level of mastery and is considered a problem solver as they gain minimal e xperience within the system and possess more understanding than the novice (Dreyfus & Dreyfus, 1980; Z ichermann & Cunningham, 2011). At the third level of mastery, experts know the system in ways that are not obvious to the novice or problem solver and are beginning to understand the system (Dreyfus & Dreyfus, 1980; Zichermann & Cunningham, 2011). Masters, the fourth level of mastery, truly understand the system and frequently identify with the system on a personal level due to the time spent engaging with the system (Dreyfus & Dreyfus, 1980; Z ichermann & Cunningham, 2011). At the fifth level of mastery, visionaries understand the system so well, they start to identify ways to improve the system (Dreyfus & Dreyfus, 1980; Zichermann & Cunningham, 2011).

PAGE 49

49 When designing any system, the five levels of mastery are important considerations due to the variety in user expertise A well designed system should be able to acco mmodate each level of mastery. Particularly when considering a game or gamification implementa tion, it is important to provide opportunities to users with each level of mastery to avoid a loss of interest by users at any level of mastery. SAPS Reward System The SAPS reward system looks at different types of extrinsic rewards in a tiered structure (Zicher mann & Cunningham, 2011). The extrinsic rewards are arranged from most to least motivating and, in that order, include sta tus, access, power, and stuff. Status, the most motivating extrinsic reward, conveys prestige upon the user (Zichermann & Cunni ngham 2011). Access gives a user the ability to engage with content that another user cannot access (Z ichermann & Cunningham, 2011). Power gives the user control over some aspect of a gamified system that other users do not have (Zichermann & Cunningham, 201 1). Stuff, the least motivating reward, constitutes physical or tangible rewards (Zichermann & Cunningham, 2011). In looking at potential game elements that could be used within a gamified system, each aligns with either status, access, power or stuff w ithin the SAPS reward system. Selecting game elements that provide status or access within a system is worth considering, as this may result in greater motivation. Excluded Theory It is worth noting that additional theories emerged within the literature that were excluded from this literature review for a variety of reasons. Of particular note, two motivational theories appeared on multiple oc casions within the literature. Goal Setting Theory explored many of the same aspects discussed in Self Determinati on Theory,

PAGE 50

50 particularly when looking at how organizational and personal goals align with motivators (Farzan & Brusilovsky 2011; Locke & Latham, 2002). ARCs motivational design also dealt with many of the same concepts SDT discuss ed (Huang, 2011; Keller, 1 987). In both cases, Self Determination Theory aligned more closely with a gamification design process and additional motivational theories were excluded t o avoid unnecessary redundancy. Cognitive Load Theory also appeared in several studies dealing with g ame based learning and serious games, but was not as prevalent in the gamification literature (Ang, Zaphiris, & Mahmood, 2007; Ayres & Paas, 2007; Chandler & Sweller, 1991; Huang, 2011 ; Turan, Avinc, Kara, & Goktas 2016 ). While cognitive load considerations are important, other theories within the dynamics phase of the MDA framework did a better job of capturing u ser experience considerations. Finally, two theories represent recent attempts to provide a gamification fram ework in educational set tings. The Dynamic Model for Gamification of Learning (DMGL) has not gained much traction in the literature since its introduction in 2012 (Kim & Lee, 2012; Kim & Lee, 2013). The Theory of Gamified Learning, introduced in 2015, has also failed to gain trac tion in the literature (Landers, 2015). In both cases, the focus was specifically on the gamification of learning and due to the early stages of development, more established and tested theories within the literature were utilized Gamification Design and Findings The theories that emerged within the gamification literature represent important considerations wh en designing a gamified system. Before discussing that theory wit hin the context of the gamification design in this dissertation, it wa s worthwhile t o also look at trends that emerged while reviewing the empirical st udies related to gamification. In the following section, gamification literature is discussed as it relates to conceptual and

PAGE 51

51 theoretical frameworks, system design, and research design. Whi le many gamification studies yielded positive results this section also looks at unintended negative results that took place in several studies where game elements were not properly aligned with the behavior they intended to m otivate. All of these trends can help inform the design of gamification implementations both in the specific context of the Techworks Portal and more generally when designing a system incorporating game elements. Conceptual Frameworks in Gamification Studies Two gamification models have emerged within recent literature that focus specifically on gamification considerations within an educational environment and provide overa rching theoretical frameworks. The Dynamic Model for Gamification of Learning (DMGL) builds on existing theories and attempts to incorporate various aspects of the MDA Game Design framework, ARCS motivational theory and key characteristics of learning games ( Kim & Lee, 2012; Kim & Lee, 2013) Similarly, the Theory of Gamified Learning attempts to explain the relatio nship between instructional content, learning outcomes, learner characteristics, game elements and overall instructional effectiveness (Landers, 2015). Both models are very new and apply specifically to learning environments The attempt to formalize a con ceptual framework for the design of gamified systems will likely increase as researchers attempt to standardize around a specific framework or guide to gamification design. With this in mind, several researchers have called for the incorporation of theory and design principles in future gamification research through a discussion of the limitations of existing literature ( Blohm & Leimeister, 2013; Dale, 2014; Hamari & Eranti, 2011; Nicholson, 2015; Oprescu, Jones, & Katsikitis, 2014; Osipov, Vol insky, & Gris hin, 2015; Sailer et al. 2013) Other sources have made an attempt to outline best practices

PAGE 52

52 and theories that help guide the design and impleme ntation of gamified solutions. Zichermann and Cunningham (2011) outline numerous theories and best practices in addition to a guide to implementing game elements within code in their book, Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps While not a formalized conceptual framework, they offer several theories and strategies to consider wh ile designing a gamified system. Many of the empirical studies that look at gamification did not include a conceptual framework or formal motivational theory in designing their system ( Anderson et al., 2013; Attali & Arieli Attali, 2014; Cheong et al., 201 3; Cruz & Penley, 2014; Denny, 2013; Domnguez et al., 2011; Farzan & Brusilovsky, 2011; Farzan et al., 2008a; Farzan et al., 2008b; Fitz Walter, Tjondronegoro, & Wyeth 2011; Giannetto et al., 2013; Goehle, 2013; Grant & Betts, 2013; Gustafsson & Bng, 200 8; Hakulinen et al., 2013; Hamari, 2013; Hanus & Fox, 2015; Jung et al., 2010; Li et al., 2012; Lieberoth, 2014; Montala Nummenmaa, Lucero, Boberg, & Korhonen, 2009; Osipov, Nikulchev, et al., 2015; Thom Millen & DiMicco 2012; Witt, Scheiner, & Robra Bi ssantz 2011; Zuckerman & Gal Oz, 2014 ). The absence of conceptual or theoretical frameworks is not surprising due to gamification being a new field of study, but it does point to a need for conceptual frameworks in future research. Without conceptual frame works in place, often studies relied on users self reporting their experience with the system or completing a survey of some form without reporting the validity of the assessment (Attali & Arieli Attali, 2014; Cheong et al., 2013; Cruz & Penley, 2014; Denny, 2013; Domnguez et al., 2011; Frith, 2013; Goehle, 2013; Hamari & Koivisto, 2013; Jakobsson, 2011; Koivisto & Hamari, 2014; Li et al., 2012; Lieberoth, 2014)

PAGE 53

53 Indirect measures of user behavior were informative and encouraging given the primarily po sitive results, but were insufficient on their own to e stablish a causal relationship. This speaks to the potential of a mixed methods approach in gamification research which could provide a more holistic view of a gamification implementation. Here, lookin g at self reported data in addition to a validated form of assessment would help build a case for the treatment or the study as a whole. The need to identify the most appropriate theory when designing a gamified system is clear within the literature and r epresents a natural progression as the field of study matures. Similarly, there is a need to incorporate a conceptual framework that is grounded in theory and out lines a clear research process. This dissertation sought to identify the most appropriate theo ry from the existing literature that c ould guide the design of gamification, while providing a conceptual framework that incorporate d both self reported user experiences and validated forms of assessment. Gamification system design in empirical studies B oth IT system design and game design are characterized by iterative an d cyclical review and revision. This is a critical component in the process of continuing to improve a nd adapt a system of this kind. One of the best examples in the gamification literat ure of this iterative process of system design is characterized by two studies conducted by Farzan et al. (2008 a; 2008b ). The researchers conducted an initial exploratory study which identified issues with the design of their gamified system (Farzan et al. 2008a). Specifically, they noted that the points and status used within the system did not dynamica lly adjust or decay over time. As a result, they noticed contributions decline over time after an initial spike i n usage (Farzan et al., 2008a). In a follo w up study, the researchers addressed these design issues and conducted an

PAGE 54

54 experiment to test the revised system (Farzan et al., 2008b). The follow up study involved a larger sample population over a longer period of time and saw increased participation an d user discussion over time. This is a wonderful example of the importance of continually monitoring a system of this type to address issues as users engage with a system. Outside the above example, there were few instances of follow up studies in the lite rature when potential issues were identified wi th the design of gamification. Instead, researchers frequently noted the problems within the design of the gamified solution but did not address the issue or conduct a follow up study (Attali & Arieli Attali, 2014; Cruz & Penley, 2014; De Schutter & Abeele, 2014; Farzan & Brusilovsky, 2011; Farzan et al., 2008a; Fitz Walter et al., 2011; Hakulinen et al., 2013; Hanus & Fox, 2015; Montola et al., 2009; Osipov, Nikulchev, et al., 2015; Witt et al., 2011) The lack of follow up studies is likely due to the early stage of research within this field, and we will likely see the number of studies of this kind increase over time. In addition, a description of the design considerations when implementing a gamified so lution would be beneficial to the field of study both to help guide researchers in designing future gamified systems and to pr ovide context for any findings. Many gamification studies incorporated game elements in existing systems or built systems from the ground up, but did not reference specific design considerations or a formal design process ( Attali & Arieli Attali, 2014; Cheong et al., 2013; Cruz & Penley, 2014; Denny, 2013; Domnguez et al., 2011; Farzan & Brusilovsky, 2011; Fitz Walter et al., 2011; Giannetto et al., 2013; Goehle, 2013; Gustafsson & Bng, 2008; Hakulinen et al., 2013; Hanus & Fox, 2015; Li et al., 2012; Lieberoth, 2014; Thom et al., 2012; Witt et al.,

PAGE 55

55 2011; Zuckerman & Gal Oz, 2014 ). As an example, Montola et al. (2009) describe addin g achievements to a photo sharing service but do not describe the design process or reference aligning the achievements with desired be havior (Montola et al., 2009). This resulted in unintended user behavior discussed in more detail later in Chapter 2 Sim ilarly, in a study that added game elements including achievements and badges to an open education platform for learning foreign language from native speakers, Osipov, Nikulchev, et al. (2015) used a large sample size of 8,000 users but did not describ e ho w the system was designed. While results were positive, a discussion of game element alignment or articulation of system design considerations in building the gamified system would have helped provide context for the results. Research designs in g amificat ion studies Due to the nature of implementing a gamified solution and testing that system within a specific context, many gamification studies were designed as case studies. Case studies are we ll suited for testing theoretical frameworks or models in real world situations using small sampl e populations (Creswell, 2013). As such, case study research is particularly appropriate when testing gamification implementations In one case study, Cheong et al. (2013) used a gamified software tool for quizzes cal led Quick Quiz with a group of 76 college student s ranging in age from 18 to 48. Participants reported increased engagement, happiness and enhanced learning effect iveness (Cheong et al., 2013). Conaway & Garay (2014) used voluntary convenience sampling to identify 189 participants who completed a questionnaire that focused on factors of gamification that attract customers to company websites. They identified several key factors that characterize successful gamification implementations with a focus on commer cial businesses considering gamified solutions. Hakulinen et al. (2013) added

PAGE 56

56 achievements to TRAKLA2, an online learning environment and used convenience sampling with a group of 281 computer science students taking Data Structures and A lgorithims at Aalt o University. The study found that achievements impacted student behavior, with students targeting activities that rewarded achievements and a small subset of students were particularly motivated to pursue them (Hakulinen et al., 2013). Small sample popula tions were present in several other gamification studies, which is typical of early research and case study research (Li, Grossman, & Fitzmaurice, 2012 ; Osipov, Nikulchev, et al. 2015; Su & Cheng, 2014; Witt et al., 2011). Case studies, particularly withi n a workplace or classroom setting, often relied on convenience sampling (Cruz & Penley, 2014; De Shutter & Abeele, 2014; Denny, 2013; Goehle, 2013; Jung Schneider, & Valacich, 2010; Su & Cheng, 2014; Witt e t al., 2011). While convenience sampling is disc ouraged in more experimental research designs, the use of convenience sampling in gamification research can provide informa tion about the case population. This is particularly relevant in the context of this dissertation, which look ed at a very specific po pulation. M any of the studies within the gamification literature failed to sufficiently isolate the gamified system as a variable or did not include a control group (Cruz & Penley, 2014; De Shutter & Abeele, 2014; Fitz Walt er et al, 2011; Goehle, 2013). Witt et al. (2011), for example, conducted a questionnaire with 30 voluntary participants at an idea competition that incorporated points and leaderboards as game mechanics at the event. They found that users did not engage with the gamified components, th ough they did note that earning points was a positive experience for users overall. The researchers attribute the lack of engagement to the poor design of the leaderboards and point

PAGE 57

57 system, which did not adjust to the individual viewing the website and wer e difficult to find (Witt et al., 2011). Both internal and external validity concerns were present in a large portion of the existing empirical studies, calling into question the generalizability and/or transfer ability of many of the results. While many r esearchers explicitly addressed this in suggesting areas for future research, there are very few examples of follow up studies or longitudinal studies in the gamification body of literature. Overall, the research design s in ex isting studies point to the im portance of isolating gamification as a variable and, where possible, comparing non gamified examples in the form of a control group. Unintended results and design issues As previously stated much of the gamification research findings were positive regarding user motivation, engagement and participation. Where results were mixed, results tended to be primarily positive but with some negative results In particular, several implementations of gamification resulted in the game element causing unintende d user behavior and the researchers frequently attributed this to design issues when implementing game elements This was one of the most interesting trends in the gamification research and points to the importance of following a formal design process. In using the incentive system within the course recommendation websit e CourseAgent, users found the motivational component to be rewarding on an individual basis and researchers saw an increase in the number of courses being recommended and rated ( Farzan & Brusilovsky, 2011) At the same time, they noted that users exhibit ed a positive rating bias, since the system rewarded them personally for positive ratings (Farzan & Brusilovsky, 2011). This was a particularly telling design consideration

PAGE 58

58 when implementin g a system that incentivizes a specific behavior and the researchers noted that there may be a need to adjust the system to incentivize rating courses, rather than positi vely rating courses. When using the smart phone app Orientation Passport, users report ed being more engaged and researchers noted increased attendance and participation overall during orientation for new students in a university setting (Fitz Walter et al., 2011) Users also reported that the gamified system was fun and added value to their orientation experience. At the same time, some users were found to stop using the application after receiving the associated achievements (Fitz Walter et al. 2011). In one case, this resulted in students potentially missing required components of the ori entation program since an achievement was rewarded for attending three events and students were required to attend six events as part of their campus orientati on (Fitz Walter et al., 2011). Some students in the Orientation Passport study also reported gues sing at answers until they earned an achievement in a question response portion of the gamified application, rather than actually learning the requisite informati on (Fitz Walter et al., 2011). This unintended behavior points to the need to properly align g ame elements wi th the intended user behavior. Creating an achievement system that stops rewarding behavior or that fails to reflect the required behavior can result in users not meeting expectations. In the online learning environment, TRAKLA2, researcher s implemented an achievement system that resulted in statistically significant differences in user behavior when compared to a control group for certain achievements, while other achievements seemed to have no impact (Hakulinen et al., 2013) At the same t ime, researchers were

PAGE 59

59 concerned that some students may have been less careful in assignment submissions due to badges being awarded for turning in assignments early (Hakulinen et al., 2013). This example points to the need to carefully observe user behavio r in relation to gamification and the need for flexibility in making adjustments in future iterations. In the photo sharing service, Nokia Image Space, researchers implemented an achievement system that resulted in friendly competition amongst users as th ey compared achievements (Montala et al., 2009) At the same time, some users felt that achievements that were awarded for posting a large number of photos encouraged quantity at the expense of quality and were thus potentially harmful to the community (Montala et al., 2009). This unintended usage pattern points to the need for some quality metric in addition to quantity in an implementation of this kind if at all possible. All of the unintended usage patterns point to important design considerations wh en implementing achievements, badges, or other game elements. It is imperative that thorough research is conducted in order to align the behavior the achievements are intended to produce with the goals of the organization, community, or environment within which the game e lements are being implemented. Thorough testing is also necessary with any system of this type. Most video games, software, and applications go through alpha and beta testing periods as part of the development cycle in order to fin d errors or potential exploits. User behavior is also tracked during this type of testing to verify that the game components are eliciting the intended results (Zichermann & Cunningham, 2011) As Squire (2006) noted when looking at video games as designed experienc es, there is a need to study what participants are actually doing in games and not solely what game designers intended. With most video games, players will find ways

PAGE 60

60 to exploit rules and structures within a game and any attempt to implement a gamified solu tion will need to be constantly monitored, reassessed, and adjusted accordingly (Zichermann & Cunningham, 2011) Workplace and Performance Improvement Studies A review of gamification studies was informative in identifying appropriate theory, design consi derations, and research approaches when considering a gamification implementation. Many of the gamification studies looked at implementations in commercial applications or in educational settings and while some gamification studies occurred in the workplac e, this was uncommon. Since this dissertation focuse d on a workplace environment, it was important to review literature co ncerning workplace motivation. In addition, this study s ought to improve employee performance through gamification and a review of som e performance improvement literature was warranted to guide the research design. The following section reviews relevant workplace motivation studies regarding Self Determination Theory and outlines two models to analyze a workplace environment that guide t he research design of this dissertation. Aspects of these models were utilized in Chapter 3. Workplace M otivational S tudies Several motivational studies within the workplace make use of Self Determination Theory (Baard, Deci, & Ryan, 2004; Deci, Connell, & Ryan, 1989; Deci et al., 2001; Ilardi, Leone, Kasser, & Ryan, 1993; Kasser, Davey & Ryan 1992, Ryan & Connell, 1989). Given the nature of gamification as an extrinsic motivation tool, these studies were of particular interest in approaching this disser tation Some motivational studies have found that extrinsic motivators can undermine intrinsic motivation (Eden, 1975; Deckop & Cirka, 2000; Shirom, Westman, & Melamed, 1999). Gagn and Deci (2005)

PAGE 61

61 suggest that these studies may not have considered some ex traneous variables and that these variables may hav e contributed to the findings. Specifically, the supportive and autonomous considerations that are laid out within SDT were not controlled within many studies of the effects of extrinsic motivation ( Gagn & Deci, 2005). Several studies making use of SDT have found that extrinsic motivation within a workplace environment can bolster intrinsic motivation and lead to autonomous extrinsic motivation when those extrinsic motivators are seen as equitable and when the work environment is supportive of worker autonomy (Baard et al., 2004; Deci et al., 2001; Ilardi et al ., 1993; Kasser et al., 1992). On the other hand, when extrinsic motivators and rewards are not seen as equitable, or when they are seen as controlli ng, resulting in high pressure work environments, extrinsic motivation tends to undermine intrinsic motivation and can sometimes demotivate employees (Baard et al., 2004; Deci et al., 2001; Ilardi et al ., 1993; Kasser et al., 1992). Ryan and Deci (2000) al so point to the importance of communicating organizational goals and allowing employees the opportunity to set perso nal goals within that context. Over time, this can lead to employees internalizing the organizational goals. Though not utilizing SDT, Cord ador et al. (2017) point to the potential application of gamification in a workplace setting and the importance of setting performance goals in that context In particular, they note that gamification of a workplace environment gives employees better acces s to performance data resulting in more immediate feedback on how an individual is performing (Cordador et al., 2017). Cordador et al. (2017) suggest that gamification in a workplace setting will result in increased motivation for employees and increased e ffectiveness at work. Perryer et al. (2016)

PAGE 62

62 make a similar claim regarding gamification in a workplace setting and suggest that SDT would be an appropriate theory to guide the design of a gamified workplace solution. They highlight the focus on personal an d organizational objectives in SDT and the potential of gamification to support achieving those objectives (Perryer et al., 2016). Within the context of gamification within a workplace setting these findings suggest that a gamified solution that was guide d by considerations laid out through Self Determination Theory, might help to better align game elements with the behavior they are intended to motivate. Specifically, these studies highlight the importance of employee autonomy when engaging with game elem ents and equity between employees regarding their ability to attain any gamified incentives. They also point to the importance of clearly defining the goals of the organization so that employees understand the value of their work and can set personal goals related to the ove rarching organizational goals. When designing a gamified solution, ensuring that these goals are clear is another important consideration. Performance I mprovement Van Tiem, Moseley and Dessinger (2012) outline a human performance technology improvement model that focuses on a performance gap with in a workplace setting. Here, the desired performance of employees, on the part of the employer, is compared to the actual performance of emp loyees (Van Tiem et al., 2012). If a gap is iden tified, the cause of the performance gap is analyzed and an inter vention is selected. After designing and implementing the intervention, it is evaluated in tandem with employee perfor mance (Van Tiem et al., 2012). This model was particularly useful within the context of this dissertation since it outlines a structured approach to evaluating a performance gap in a workplace setti ng and implementing a solution.

PAGE 63

63 Aspects of this model were used to examine Techworks employee performance and are detailed in Chap ter 3. Gilbert (2007) lays out a behavior engineering model to evaluate both the ior within a workplace setting. Environmental supports include providing relevant and frequent feedback about emplo yee performance, descriptions of what performance is expected, and guides to achieving adequa te performance (Gilbert, 2007). In addition, environmental supports include providing tools that are designed to match the need of both the organization and employ ees (Gilbert, 2007). Finally, environmental supports provide opportunities for professional development along with both monetary and non monetary incentives for co mpleting work (Gilbert, 2007). an emp loyee possesses and includes the training an employee receives along with the alignment of that training with exemplar y performance (Gilbert, 2007). The behavior engineering model is used to evaluate whether there are areas within either the workplace envi ronment or employee skillsets that could be improved in order t o improve overall performance. Gilbert (2007) also advocates defining and communicating exemplary performance based on historic performance that represents the best instance of performan ce on t he part of an employee. The behavior engineering model was a valuable tool in analyzi ng the Techworks work environment. Like the human performance technology improvement model, this analysis is detailed in Chapter 3, and revealed a need for additional non monetary incentives within the Techworks portal. Literature Review Summary A review of gamification and workplace literature revealed a number of trends that help guide the research in this dissertation. The positive results from gamification

PAGE 64

64 studies point to clear potential in this field to motivate and engage users, and gami fication represents an exciting incentive approach when designed properly. Three key areas emerged from the literature that this dissertation attempt ed to address. The review of literature points to a need for gamification studies to be grounded in appropriate theory, the need for more formal gamification design practices, and the need for gamification studies outside of educational and commercial applications. Earl y gamification studies lacked grounding in theory, resulting in a more recent call for the incorporation of theoretical foundations in gamification research ( Hamari et al., 2014 ; Kim & Lee, 2012; Kim & Lee, 2013; Landers, 2015). A number of relevant theori es have emerged in recent literature both through conceptual articles offering recommendations and formal studies attempting to incorporate theory in gamification research. A review of gamification research also points to the n eed for more formal design pr actices when designing gamification implementations. This is seen through the lack of system design specifications provided in many articles and the resulting unintended usage patterns that emerged in se veral studies. Through utilizing a formal game design framework and incorporating relevant theory at each stage of the framework, this dissertation can help guide the design of gamified systems beyond the scope of the research in question. Though there were some gamification studies that occurred in workplac e settings, there were few concrete examples in this area. The positive results in educational and commercial settings point to a clear opportunity for additional research. With the potential of gamification to motivate, examining a workplace setting and t he application of game elements to motivate employees is an exciting research area. It is,

PAGE 65

65 however, important to incorporate models from workplace studies to help guide the design of the gamification implementation. While there is great potential in the u se of gamification to motivate, it is important to recognize that gamification is a trending topic and temper expectations (Ferrara, 2013; Hamari et al., 2014; Kalinauskas, 2014). representation of the expectations and adoption cycle of an information technology solution (Ferrara, 2013) G enerally, this cycle displays a new technology going through a sharp peak of initial interest, resulting in infl ated expectations. This is followed by a period of disillusionment when the technology fails to meet the inflated expectations. In light of the realities of the technology solution, more reasonable expectations lead to a plateau within the graph where the new technology is used in production within the context of its limitations (Ferrara, 2013) In a discussion of gamification as it relates to Ferrara (2013) discussed gamification as being at the apex of the cycle, with inflated expect ations surrounding gamification. All of the findings within the empirical studies should be looked at critically, with this in mind. The importance of researching well designed gamification implementations that are grounded in appropriate theory is even mo re important within this context As both the concept of gamification and the experiments attempting to study the concept mature, there is a need for solid design principles and best practices in designing gamified solutions for research purposes. At pres ent, there are a variety of studies using Self Determination Theory to study motivation in the workplace, but few gamification studies grounded in this theory. Because gamification deals with extrinsic motivation and at present, the literature is lacking i n theory, SDT is a strong candidate

PAGE 66

66 to provide the theoretical underpinnings of gamified experiences. SDT provides a tested theoretical approach to classifying and evaluating extrinsic motivation. Careful design of the gamified system will also play a sign ificant role in aligning game elements with what they intend to motivate within the system. The MDA Game Design Framework provides the cyclical and recursive design process that is needed when designing a system of this kind to optimize the user experience In conjunction with numerous other theories that look at user characteristics and user motivation, SDT and MDA have the potential to reshape how gamified experiences are designed and how gamification research is conducted. A synthesis of relevant theory and design best practices resulted in a gamification design framework used to guide the design of the gamification implementation within Techworks Chapter 3 involves an analysis of the Techworks work environment and the design o f a gamification layer using this framework. The specific theories discussed in this framework and their application in designing the gamification layer in the Techworks Portal are unpacked in greater detail in Chapter 3

PAGE 67

67 CHAPTER 3 WORK PLACE ANALYSIS AND GAMIFICATION DESIGN The analysis of the Techworks work environment was based on two performance models discussed in Chapter 2: the Human Performance Technology Improvement Model developed by Van Tiem et al. (2012) and the Behavior Engineering Model developed by Gilbert (2007). The overarching design of the gamification within the Techworks Portal was guided by the Self De termined Gamification Framework (Figure 3 1 ) Here, Self Determination Theory (SDT) and the Mechanics Dynamics Aesthetics (MDA) Framework provide impor tant considerations when designing an incentive system in a workplace setting using game elements. Within Chapter 3 the analysis of the Techworks work environment is examined in detail. In addition, each aspect of the Self Determined Gamification Framework is presente d as it pertained to the design within the Techworks Portal. After a discussion of considerations, the actual design is presented, as it occurred within the Techworks Development Portal a s part of a standard design process prior to any new tool being implemented in the production environment Specific achievements are also provided prior to a summary of the overall design process. Techworks Workplace A nalysis Human Performance Technology I mprovement Model The Human Performance Technology Improvement M odel developed by Van Tiem et al. (2012) presents an overarching analysis of employee performance that includes assessing the organization and workplace environment to identify the desired perf ormance of employees and their actual performance. If a gap is identified, the cause of the performance gap is investigated resulting in the selection and design of an

PAGE 68

68 intervention (Van Tiem, et al., 2012). The Intervention is then implemented and maintain ed. An important component of this model is a continuous process of evaluation at each stage within the process and recognition that the intervention represents a change to the workplace environment that needs to be managed. While the entire model was not used in this evaluation, elements of the model were incorporated in identifying a performance gap. Using components of this model, an analysis of the online workplace e nvironment within Techworks revealed a performance gap by comparing the desired perform ance of employees to their actual performance. Here, specific performance requirements were compared to actual performance in a number of key areas. These areas included positive and negative customer service survey responses, posts to a technical forum, r eactive support ticket completion, ad hoc support ticket completion, mobile support site support ticket completion, submission of performance feedback, completion of proactive checklists on equipment, completion of office shifts at the Techworks office, te chnology assistance at events within the residence halls, completion of monthly evaluations, completion of the overall semester evaluation, receipt of late notices, receipt of critical notices, and attendance at a variety of events. Behavior Engineering M odel Initial student employee feedback pointed to a lack of connection, motivation and structure within a work environment where student employees complete job requirements remotely and autonomously within an online portal with minimal supervision Although the decision was already made to implement the gamification layer, analyzing the online workplace environment at Techworks using th e behavior engineering model was a useful exercise to identify potential areas for improvement.

PAGE 69

69 Gilbert (2007) lays out a behavior engineering model to evaluate both the workplace Conducting an analysis of the Techworks organization and workplace environment revealed that while other environmental and individual supp orts we re met, there we re very few non monetary incentives in the online environment at Techworks Specifically, the Techworks environment provided employees with relevant and frequent feedback about performance, set clear expectations about performance in cluding methods for meeting what Gilbert (2007) calls exemplary performance, and provided staff with clear guidelines and training to achieve both adequate and exemplary performance. Training was available throughout the year in the online portal and emplo yees were strategically placed throughout the residence halls so that they had access to experienced staff with diverse skillsets. Techworks full time staff created tools to meet the needs of student staff in completing their work and the tools were speci fically designed to meet the needs of staff and the work being completed. Work took place according to student staff schedules and they were given flexibility in setting that schedule. Staff were strategically selected during the hiring process for their a bility to perform the job, in addition to their ability to adapt to different situations. Financial compensation was high when calculated at an hourly rate and was performance based. A variety of professional development opportunities were offered to staff throughout the year both within Techworks and as part of the overall IT community on campus. Staff recruitment was targeted to meet the needs of the organization and staff were routinely asked for feedback about ways to improve the organization, the workp lace environment, and ways to improve employee motivation in the workplace. This analysis revealed that

PAGE 70

70 non monetary incentives were somewhat lacking in the organization, despite meeting other factors within the Behavior Engineering Model. Analysis of Tec hworks Work Environment : Summary The results of the performance gap analysis and the Behavior Engineering Model, when taken in tandem, suggest ed that the selected intervention, the incorporation of gamification elements in the online web portal, ha d the potential to incentivize work and increase motivation. With a need identified and the intervention selected, the gamification layer needed to be designed, developed and implemented within the online web portal (Van Tiem, et al., 2012). The design of t he gamification layer is detailed in the remainder of Chapter 3 Gamification Design Using the Self Determined Gamification Framework Self Determined Gamification Framework The review of relevant theory from gamification research resulted in a framework th at wa s grounded in Self Determination Theory (Figure 3 1 ) and that portray ed a gamification design process that s ought to create autonomous user experiences. This framework was used to design the gamification implementation in the Techworks Portal. Based o n the findings of this research, this framework also has the potential to guide the design of other gamified systems intended to motivate users.

PAGE 71

71 Figure 3 1 Self determined g amification f ramework

PAGE 72

72 S elf D etermination T heory (SDT) Alignment The overall design of the game elements within the Techworks Portal was rooted in Self Determination Theory (SDT). By focusing on features highlighted in SDT, the hope was to design an incentive structure that did not result in a controlling or high pressu re work environment. To do this, SDT recommends designing motivational components or incentives that are closely tied to organizational goals ( Gagn & Deci, 2005 ; Ryan & Deci, 2000) After aligning the extrinsic motivational components, in this case game elements, with organizational goals, SDT stresses the importance of allowing employees the ability to choose where they will focus their efforts when engaging in work ( Gagn & Deci, 20 05 ; Ryan & Deci, 2000) In addition, the game elements should be attainable to all employees, should they choose to pursue a specific area of work ( Gagn & Deci, 2005 ; Ryan & Deci, 2000) Aligning the game elements with organizational goals and designing t hem so that employees can engage in autonomous work that is attainable to any employee presents employees with a work environment conducive to internalization of organizational goals. In the end, the hope is that employees begin to internalize the goals of the organization, resulting in the completion of meaningful work. SDT a lignment: o rganizational g oals Prior to selecting the specific game elements or designing the actual system, it was important to identify organizational goals. Within Techworks four overarching goals guide p ractice when completing work: (a ) enhancing the educational environment within the reside nce halls through technology, (b ) enhancing the entertainment opportunities within the reside nce halls through technology, (c ) providing quali ty customer service to every client through timely and reliable on site technology support, and (d ) supporting

PAGE 73

73 fellow Techworks staff through timely communication, clear expectations, and mentoring. The four Techworks goals a re also articulated in Appendix A For Techworks student staff, these goals are accomplished through the completion of specific job requirements. These job requirements are discussed in greate r detail in Chapter 5 but they include: customer service surveys received, technical forum pos ts, assigned reactive support tickets completed, ad hoc reactive support tickets completed, mobile support site support tickets completed, performance feedback submissions, proactive equipment checklists completed, proactive hallsweeps completed, office sh ifts comp leted, event assists completed monthly evaluations completed, late notices received, critical notices received, mobile support site events attended, area meetings attended, staff meetings attended, and portal logs by day Each job requirement is mapped to a specific category and each category is mapped to one or more organizational goal. The Techworks job categories include: customer service, evaluations, events, support tickets, other support, tech forum, and general. The job category, job requirements and organizationa l goal are included in Appendix B SDT a lignment: a utonomy For the job requirements listed above, some are strictly defined for staff to complete, while others are more flexible. As an example, staff must complete six office shifts each semester, but there is no required amount of technical forum posts. For most of the job requirements, staff are free to go above and beyond the minimum job requirement. For example, a staff member could complete as many office shifts as she wou ld like, though she must complete the required six per semester. Exceptions include mandatory events, where there are a finite number of requirements with no opportunity to go above and beyond. Examples of this type of requirement include staff meetings

PAGE 74

74 an d regional area meetings. While it was important to clearly convey the minimum job requirement, it was also important in designing the gamification components to allow staff the freedom to pursue areas of interest beyond the required work. Within SDT, this is referred to as providing employees with autonomy and where possible, the gamification design allowed for employee autonomy when engaging in work ( Gagn & Deci, 2005 ; Ryan & Deci, 2000) SDT a lignment: e quity Each employee within Techworks has the oppor tunity t o complete each job requirement and to engage in work above and beyond the required amount. When looking at job requirements and considering gamification options, it was important that each staff member would be able to be successful in obtaining t he gamified incentives. Within SDT, this is conceptualized under the term equity ( Gagn & Deci, 2005 ; Ryan & Deci, 2000) Within the eventual achievement structure, detailed later, multiple achievements were identified within each job category and each is attainable by an employee during their tenure with the organization To test this, it was confirmed that each achievement would have been earned by at least one past employee who exhibited exemplary work. It is important to note that the initial design of the achievements looked at all employees over time. Since most employees work an average of three years for Techworks there were outliers that skewed the initial mapping of certain achievements. For example, an employee that worked for Techworks in both u ndergraduate and graduate school would have completed significantly more work over a five to seven year period than an employee working for Techworks the average tenure. For this reason, outliers were removed when determining if a prior employee would have earned each achievement and

PAGE 75

75 achievements were designed around employees who worked 3 4 years for the organization. SDT a lignment: i ndividual g oals Within Techworks each region sets goals for a semester at the outset of a semester. These goals are set wi thout the involvement of full time Techworks staff. In addition, staff meet once per semester with a supervisor to review goals and talk about progress. When selecting game elements and the eventual design of the gamification layer, it was important to rev iew prior staff goals and ensure they were reflected in the system. In addition, it was important for game elements to be transparent so that staff could see what could specifically be obtained and set goals within the system. While staff may internalize o rganizational goals, it was important for them to be able to set their own goals in relation to the game elements. MDA Game Design Framework The MDA Game Design Framework presents an iterative and cyclical process to follow when designing games. Within th e context of the SDT considerations already presented, the MDA Game Design Framework offers specific design considerations for a gamification environment. These considerations help ed guide the process of selecting game elements, establishing the rules and parameters for the game elements within the overarching system, and designing the system with the end user experience in mind (Deterding et al., 2011; Zichermann & Cunningham, 2011) In the following sections, each of these aspects of designing the gamific ation layer is explored in greater detail. MDA f ramework: m echanics o verview Within the MDA Game Design Framework, the m echanics phase is focused on the selection of game elements within the context of gamification design (Deterding et

PAGE 76

76 al., 2011; Ziche rmann & Cunningham, 2011) The review of literature revealed a number of possible game elements that could be incorporated in this study and several that were inappropriate for a workplace environment. Particularly within the context of SDT, leaderboards a ppeared to create high pressure environments where the focus becomes competition and this did not map well to the Techworks environment (Hanus & Fox, 2015). Similarly, obtaining points that allowed an employee to level created problems both from a design s tandpoint, in mapping job requirements to a point structure, and in creating a somewhat rigid and controlling work environment. Studies have also indicated that points and levels in a workplace setting can create issues relative to veteran employees who have had access to the system longer than new employees, in addition to a decline in participation over time (Farzan et al., 2008a). MDA f ramework: m echanics SAPS r eward s ystem The SAPS Reward System also influenced the selection of game elements. Here, each game element was mapped to a specific reward structure and assessed within the context of the Techworks Portal. Within this context, it was difficult to provide employees with tangible rewards. It was also difficult to grant additional access or powe r to employees, though allowing staff to become forum moderators was a consideration at one point. Ultimately game element selection was tied to those elements that could offer status within the Techworks Portal. Not only was status the most motivating rew ard structure within SAPS, it also was the easiest to implement within the Techworks Portal (Zichermann & Cunningham, 2011) MDA f ramework: m echanics g ame e lement s election Within the existing portal structure, and in the context of the job requirements mapped to specific organizational goals, it was determined that designing an

PAGE 77

77 achievement system would provide employees an opportunity to engage with an incentive structure that would map well to the existing job requirements. This would also minimize dev elopment time since t he achievements could be tied to existing job requirements as a reinforcement, rather than designing a new method of measuring work completed. Achievements could also be designed within the system to offer employees status, without the competitive aspects introduced by a leaderboard. Here, obtaining more difficult achievements could provide status, but all employees have the opportunity to attain those achievements. Though it would be decided later, progress bars were also incorporated to help visually display completion of different achievements within categories. By selecting a single primary game element in achievements, this also minimized the number of compounding variables a variety of game elements would have introduced. MDA f ram ework: d ynamics o verview Within the MDA Game Design Framework, Dynamics represents the design of the system along with how users will interact with the system. With the selection of achievements as the primary game element, this stage involved identifyi ng potential achievements, identifying the specific requirements for obtaining the achievement, and ensuring that a variety of user characteristics were taken into consideration. MDA f ramework: d ynamics f ive l evels of m astery Dreyfus and Dreyfus identif ied five levels of mastery to consider when designing a system of this type. These levels of mastery speak to user expertise when interfacing with a system (Dreyfus & Dreyfus, 1980; Zichermann & Cunningham, 2011) As job requirements were mapped to achievements, it was important to accommodate each level of mastery. Simple achievements, such as an achievement earned for completing

PAGE 78

78 your first support ticket would engage novice users, for example. Beyond achievements tied to specific job requirements, considering mastery levels pointed to additional achievements, such as earning an achievement for logging into the Techworks Portal 100 times, an achievement that might appeal to master users who were extremely familiar with the system. Each achievement is mapped to one of the five mastery levels in Appendix C in the column ML labeled as N, PS, E, M, V to represent novices, problem solvers, experts, masters and visionaries respectively. These mastery levels were also used to identify how difficult an achie vement was to attain, resulting in a corresponding color mapping in the aesthetics phase. MDA f ramework: d ynamics f our p layer t ypes designing the achievements to ensure di fferent interests were accommodated. Bartle identifies users who are variously motivated by competing with other users as killers, completing tasks within the system as achievers, collaborating with other users as socializers, and exploring different aspec ts of the system as explorers (Bartle, 1996; Zichermann & Cunningham, 2011) Each achievement was mapped to a specific player type under the column PT represente d as K, A, S, and E in Appendix C MDA f ramework: d ynamics f our t ypes of f un Lazzaro identifi ed four types of fun categorized as hard fun, easy fun, people fun, and serious fun (Lazzaro, 2004; Zichermann & Cunningham, 2011) The types of fun represent the feelings and emotions users experience when engaging with each type of fun. Hard fun represen ts goal oriented fun and is associated with triumphing over a challenge. Easy fun is open ended and is associated with curiosity. People fun is goal oriented and is associated with interacting and amusement. Serious fun is open ended

PAGE 79

79 and is associated with excitement. In designing achievements within the Techworks portal each achievement was mapped to a specific type of fun. This is displayed in Appendix C and each type of fun is labeled under the column F as H, E, P or S. MDA f ramework: d ynamics f low t h eory The final theory influencing the design of the achievements was within an activity. Here, it was important to stage achievements so that they would scale in difficulty (Czikszentmihalyi, 1989; Zichermann & Cunningham, 2011). Though early achievements are easy to obtain and more difficult achievements require staff to work for Techworks for multiple semesters, the achievements within the portal were designed so that staff should always have something they could strive for that is within their current skill level. MDA f ramework: d ynamics a chievement s tructure The r esulting achievement structure wa s comprised of 2 40 achievements, organized into categories and m apped to mastery levels, player types, and types of fun. In doing so, multiple user characteristics were considered and achievements could frequently be categorized as appealing to multiple player types. Though not formal job requirements, some additional achievements were created to accommodate different activities a user might engage in while working for Techworks and to create achievements to accommodate different user characteristics. This resulted in an additional categor y to those outlined in Appendix B titled awards, to reflect the monthly and semester awards that are distributed at staff meetings. Staff are encouraged to both live in different regions and continue working for Techworks throughout their undergraduate and graduate school experience, r esulting in the addition of

PAGE 80

80 achievements related to where staff live and the number of semesters an employee works for Techworks as well. In addition, some achievements were created related to interfacing with the Techworks Portal, such as updating a user profile or logging into the system. When considering different levels of mastery, 40 achievements were asso ciated with novice employees, 43 wi th problem solving employees, 62 with expert employees, 66 with master employees, and 29 with visionary employees Here, the achievements that would require the most time working with Techworks along with the level of experience required with the program helped map achievements to mastery levels. When looking at player types, where th ere was significant overlap, 157 achievements were identified as potential areas of competition that could appeal to the killer player type, 2 28 were associated with completion of tasks that might appeal to the achiever player type, 95 were associated with collaboration and helping others which could appeal to the social player type, and 119 were associated with either exploring the Techworks Portal or the physical residence hall spaces, which might appeal to the explorer player type. When looking at the types of fun associated with each a chievement or corresponding requirement, 77 aligned with the open ended curiosity associated with easy fun, 23 aligned with the open ended relaxation and excitemen t associated with serious fun, 5 5 aligned with the goal oriented social amusement as sociated with people fun, and 85 aligned with the goal oriented triumph over a challenge associated with hard fun. The achievements identified correlate with specific job requirements over time and we re mapped to organizational goals. In defining these achievement s, a Techworks

PAGE 81

81 It was important to include achievements that appealed to multiple user characteristics including easy, moderate and difficult challenges. More difficult achievements were de fined based on what Gilbert (2007) refers to as exemplary performance, by looking at the best examples of employee performance in prior semesters for each job requirement. Again, outliers were removed from this assessment, when looking at achievements that map to 4 years of the average employee. The full list of achiev ements is presented in Appendix C along with a key for abbr eviations presented in Appendix D MDA f ramewo rk: a esthetics o verview Within the MDA Game Design Framework, Aesthetics represents the overall user experience within the system. Here, the design of the gamification layer focused on the look and feel of the system. The Techworks Portal wa s modularly designed so that different tools and interfaces have different user group permissions, and data from each tool can be accessed and processed by other tools within the portal. The reminder tool discussed in greater detail in Chapter 1, for example, draws on all the other tools in the portal to alert staff to upcoming job requirements. In order to tie each reminder notification to a job requirement, this tool needed to be linked to each corresponding data point. Following this model, a gamification layer was designed that drew upon existing data and notified staff when they reached the threshold to attain an achievement. The portal itself is coded in PHP and uses an on click notification system to notify users they received an achievement after taking an actio n within the system. In addition, upon logging into the portal, a script was created to check for any achievements earned since the last time the user logged in. This is necessary due to

PAGE 82

82 the data for support tickets coming from an external system. Because most achievements build on one another, the check is not processor intensive, as the system In addition, some achievements can only be received after a semester is finalized within the Techworks Por tal. These achievements are automatically processed upon transitioning to the next semester. MDA f ramework: a esthetics u ser e xperience d esign In designing the user experience, a variety of existing games were examined that provide an achievement structu re. The intent was to replicate core functionality of existing achievement systems within games. In addition, recommendations laid out by Zicherman and Cunningham (2011) in designing the user experience were followed. Of particular note, the importance of real time feedback when a user receives an achievement was an important consideration. In addition, allowing users to see future achievements that were not yet attained was identified as an important feature. After multiple design iterations, a version of the gamification layer was tested within the development portal. The development portal is not available to student staff and is used only for testing purposes. A final version was implemented prior to the Spring 2017 semester. After three months of use, the achievement structure was assessed for its alignment with the motivation continuum within SDT by using a vetted SDT questionnaire. In addition, student employee perceptions of the achievement structure were assessed through open ended survey questions. Similar surveys are used with staff each time a new tool is implemented in the Techworks Portal.

PAGE 83

83 Techworks Portal Gamification The resulting gamification layer within the Techworks Portal integrated seamlessly with the existing tools within the portal. U pon logging in, a new notification icon in the upper right section of the portal notifies users if they have any unseen achievements. By clicking this icon, users can see a list of recently received achievements or visit the achievement s page, where they c an navigate existing achievements. Note that staff are not required to view these notifications and can ignore them if they do not wish to see the achievement interface. The notification icon can be seen in Figure 3 2 Figure 3 2 Techworks portal scre enshot notification i con After navigating to the achievement central page, an employee has options to view different achievements according to category. Each category section indicates the number of achievements an employee has earned that have not been viewed. An overall summary page allows employees to see all recent achievements earned. Categories include those listed in Appendix A with the addition of the awards category and include: general, support tickets, awards, customer service, tech forum, events, evaluations, and other support. Employees have the option to show all achievements, or they can toggle between achievements they have completed or those that are incomplete. If an employee views a page with unseen achievements, the notification number will dynamically adjust, or an employee has the option to mark all achievements as read. These o ptions can be seen in Figure 3 3

PAGE 84

84 Figure 3 3 Techworks portal screenshot viewing o ptions

PAGE 85

85 Each achievement consists of an image which signifies the achievement category, a title for the achievement, a description of the achievement requ irement or threshold, either a banner with an associative count (e.g. the number 10 for completing 10 reactive support tickets) or an icon for achievements without a count structure (e.g. the Retire My Jersey Achievement in which a student employee receive s any combination of Tech of the Semester, Rookie of the Semester or S enior Tech of the Semester during their time at Techworks ), and one of five colors corresponding to the presumed rarity of obtaining the achievement. The presumed rarity was determined b y the level of mastery associated with the achievement in question. Rarity was defined as uncommon, common, rare, epic, and legendary and the corresponding colors are grey, green, blue, purple, and orange respectively. The rarity and color display is consi stent with numerous video games on the market, in an effort to present staff with a format they might be familiar with already. Each rarity is also associated with a different background icon. All iconography was identified using the open source icons avai lable through Font Awesome ( http://fontawesome.io/icons/ ). A sample of icons according to rarity can be s een in Figure 3 4 As stated previously, users can expand and collapse each category and can look at diff erent achievements associated with different job requirements. As an example, within the Other Support category, a user could look at achievements associated with Housing Assists. Here a user could see both achievements they had earned and achievements to strive for. Achievements that have not yet been earned are slightly

PAGE 86

86 Figure 3 5 Again, users can toggle whether they want to see incomplete achievements within the inter face. Figure 3 4 Techworks d evelopment portal screenshot rarity and i conography

PAGE 87

87 Figure 3 5 Techworks portal screenshot complete and incomplete a chievements

PAGE 88

88 On the summary page, an employee can view progress bars related to each achievement category. Here they can see how many achievements they have completed overall, and how many are left to complete. This same view is replicated for each achievement category. A graphical progress bar dynamically fills as staff earn achie vements in the portal. A sample progre ss bar can be seen in Figure 3 6 Progress bars were developed using the open source options available through Twitter Bootstrap ( http://getbootstrap.co m/components/#progress ). Figure 3 6 Techworks portal screenshot achievement progress b ars In testing this system, multiple users were replicated or spoofed within the Techworks Portal development environment. User achievements earned were compared to the database of user data tracking completion of job requirements to confirm that each user received appropriate achievements. This data was adjusted to test functionality of all achievements, along with the dynamic aspects of the system such as the notifi cation number adjusting and the progress bars adjusting.

PAGE 89

89 Summary The design of the gamification layer within the Techworks Portal involved an attempt to identify an incentive mechanism that was aligned with SDT and followed best practices outlined through the MDA Game Design Framework. Multiple theories were utilized in considering different user characteristics and this in combination with an analysis of organizational goals and job requirements, led to the decision to implement the specific game element of achievements within the Techworks Portal. In total, 2 40 achievements were identified that map to organizational goals and a variety of user characteristics. In addition, a user interface was designed to allow student employees to interact with the incen tive structure. By following best practices for designing a system of this kind, and utilizing relevant theory, it wa s hoped that the achievement structure w ould align with more autonomous forms of motivation within the SDT autonomy continuum. Assessing this alignment to determine the fidelity of the design, along with user perceptions of the system as it relates to their work was a major focus of this dissertation and i s explored in greater detail in Chapt er 4

PAGE 90

90 CHAPTER 4 METHODOLOGY A three phased approach was used to explore the use of gamification incentives in a workplace setting and their potential to improve employee performance. The overarching research design is the primary focus of Chapter 4 First a conceptual framework is presented that inform ed each of the three phases in this study. Second, the research design is provided including research participants, data collection methods, and data analysis methods. Chapter 4 concludes with a discussion of rigor, potential limitations, and e thical considerations. T hree research questions guided the study. Research Questions How does a gamification implementation designed using the Self Determined Gamification framework align with the motivation continuum within Self Determination Theory? How does the presence of a gamification layer in an online web portal impact college student employee perceptions of the online workplace environment? Does college student employee performance change after the implementation of a gamification layer within an o nline web portal used to complete job requirements? Conceptual Framework The Workplace Gamification conceptual fr amework presented in Figure 4 1 guide d the three phased research design in this study. Phase one of the research involved the analysis and review of research and current literature to identify best practices in designing gamification in addition to an approach to analyze employee performance. Phase two of the research involved an analysis of the workplace environment based on performance mana gement literature. This analysis pointed to a gap in performance and an absence of non monetary incentives within the Techworks

PAGE 91

91 work environment. Phase two also involved the design of a gamification application layer within the Techworks Portal following t he Self Determined Gamification Framework. Phase three of this research involve d the evaluation of the gamification design, analysis of employee performance, and evaluation of employee perceptions about the gamification implementation. T he fidelity of the design was assessed using a vetted SDT questionnaire to determine the extent to which the gamification elements align with the motivation continuum within SDT In addition, this phase involve d the comparison of retrospective performance data with the perf ormance data gathered during the gamification implementation Summative performance data totals for the entire staff were used in this comparison. Research Design The overarching three phased research approach outlined in the conceptual framework above in volved research analysis to identify best practices in the literature, workplace analysis to identify performance gaps and environmental supports, the design of the gamification layer, the implementation of the gamification layer, and evaluation of the gam ification implementation. The evaluation of the implementation occurred after three months of use and involved the collection of both quantitative and qualitative data, the analysis of that data and the convergence of the data in a discussion of results. I n the following sections information is detailed about participants, data collection, data analysis, rigor, limitations, ethical considerations, and bias.

PAGE 92

92 Figure 4 1. Conceptual f ramework workplace g amification

PAGE 93

93 Participants Convenience sampling was necessary due to the study taking place in an actual work setting, however participants could choose whether or not to interact with the gamified aspects of the portal. Participants in this study we re part time student staff known as Student Techs Studen t Tech s in the study live d and work ed in the residence halls at a major university and provide d on site technical support to residents and full time Housing Department staff. Participants work ed independe ntly and their work often occurred outside of normal business hours without management supervision. All Student Techs in the study wer e students at the university in good academic standing. Participants we re fluent English speakers and we re comfortable with the use of computers, web based instructional mate rials and web based tools Student staff receive d one week of intensive training prior to the start of the Fall semester. This training program wa s designed to prepare staff for the majority of work they were asked to complete while working for Techworks and included comprehensive training on the Techworks Portal, completion of job responsibilities, and expectations regarding job responsibilities For some, this is the only training they receive d prior to troubleshooting client technical problems. As a res ult, staff we re selected for their technical knowledge and analytical troubleshooting skills. Student staff also receive d monthly training at staff meetings and ha d access to all training materials online through the Techworks Portal. All participants worked for Techworks for at least one full semester prior to the study taking place. Techworks staff demographics var y from semester to semester. Specific demographic information is provided below for each of the retrospective summative da ta sets used in this study, in addition to the current staff population. Specific

PAGE 94

94 semesters are broken into subsections below. A summative table displaying this information follows in Table 4 1 2017 ( Current Staff) : The Spring 201 7 Techworks student staf f was composed of 43 individuals ranging in age from 18 to 2 7 The staff was 35% new employees (n=15 ) who began working in August 2016. The remaining 65% (n=28 ) were returning staff who had worked for Techworks in at least one previous semester. 2016: The Spring 2016 Techworks student staff was composed of 46 individuals ranging in age from 18 to 25. The staff wa s 43% new employees (n=20 ) who began working in August 2015 The remaining 57% (n=26) were returning staff who had worked for Techworks in at least one previous semester. 2015: The Spring 2015 Techworks student staff was composed of 48 individuals ranging in age from 18 to 2 4 The staff wa s 48% new employees (n=23) who began working in August 2014. The remaining 52% (n=25) were returning staff who had worked for Techworks in at least one previous semester. Table 4 1 Techworks s taff d emographics over t ime. Year Total s taff New e mployee Returning e mployee 2017 43 15 (35%) 28 (65%) 2016 46 20 (43%) 26 (57%) 2015 48 23 (48%) 25 (52%) Data Collection Data collection involved two primary data sources detailed below. Results of a Self Determination survey helped determine the fidelity of the design by assessing achievements as they relate to the autonomy continuum within Self Determinatio n

PAGE 95

95 Theory. The survey also include d o pen ended questions to help identify employee perceptions regarding the gamification implementation. Finally, performance data during the gamification implementation was gathered and compared to retrospective performance data gathered prior to the gamification implementation. Each data source is explained in greater detail below. Data c ollection: self d etermination s urvey T he fidelity of the design was assessed using the SDT Intrinsic Motivation Indicator (IMI) survey. Th e survey was issued to all participants after the gamified portal wa s in use for three months in April of 2017. Participants receive d an email with a link to the survey, hosted in the Qualtrics Research Suite, a survey tool available through a license agre ement with the University of Florida at Gainesville In total, 15 individuals submitted feedback out of the 43 staff members representing a 34.88% response rate. The IMI survey consists of 22 items with Likert responses on a 7 point scale The 22 items are mapped to fo ur subscales listed as interest enjoyment, percei ved competence, perceived choice and pressure tension (Deci & Ryan, n.d.). The four subscales align with autonomous extrinsic and intrinsic motivation on the autonomy continuum within SDT and helped determine the extent to which the gamification implementation aligns with more autonomous motivation on the autonomy continuum Different versions of the IMI survey have been found to be both reliable and valid within different settings (Deci & Ryan n.d.) McAuley, Duncan and Tammen (1989) conducted a study to specifically test the validity and reliability of the IMI McAuley et al, (1989) reported the following alpha coefficients indicating reliability for each of the four s ub effort They also

PAGE 96

96 (McAuley et al., 1989). The researchers utilized a chi square goodness of fit test and confirmed the factor structure of the IMI survey (McAuley et al., 1989). The standard 22 item version of th e IMI is provided in Appendix E Deci and Ryan (n.d.) also note that it ems on the IMI can be modified to address specific activities being studied without impa cting validity or reliability. The modified version used in this study is provided in Appendix F Based on the results of these surveys, specific achievements, the inte rface within the portal, or any other aspect of the gamification implementation can be adjusted to better align with more intrinsic motivation along the autonomy continuum in future design iterations Though beyond the scope of this dissertation, the conti nued improvement of this system will allow for the potential of more longitudinal studies in this environment. Three open ended survey questions were appended to the Self Determination Survey. Open ended r esponses from participants explore d employee percep tions about the gamification elements in relation to their work. This data was free text with no word count limitation to allow employees to leave as much feedback as they would like. Specifically, student employees were asked about their perceptions of the addition of achievements and progress bars within the portal, whether they felt the achievements impacted their performance, and how the system could be improved further. This is consistent with prior semester surveys used to gather student employee feedback about web portal improvements that followed a similar format Typically, after adding a new feature or tool to the web portal, staff are asked to provide feedback after a semester of use. Data will be gathered conti nuously over future semesters in a process of continual

PAGE 97

97 improvement, however the focus of this dissertation study was data from the first three months of use in order to evaluate the design and initial impact. The specific open ended survey questions are d etailed below. Please provide any feedback regarding the achievements added to the Techworks web portal this semester. If any of the achievements impacted the work you completed in the Techworks web portal this semester, please describe how. If possible, please provide examples. What changes would you make to the achievements to improve this feature in future semesters ? Data c ollection: performance d ata In addition to design considerations and employee perceptions, this study also sought to evaluate the i nitial impact of the inter vention on employee performance. After the gamification implementation and three months of use, performance data from previous semesters was compared to current performance data. Here, overall performance in a number of key catego ries was compared to performance in previous semesters, when the web portal did not contain game elements. Retrospective data from all student employees from previous semesters was compared to current staff totals. Re trospective performance data was alread y collected over several years. This data wa s primarily numeric ratio data about employee performance and participation within the online portal. Examples of data collected include the number of reactive tickets completed, proactive technology checks compl eted, and positive customer service survey responses received. Current comparative data about employee performance for these same indicators was collected within the online web portal during the semester the gamification components were in use. This data s et was summative in nature and included totals for all staff rather than individual metrics. These summative

PAGE 98

98 totals included students who met communicated performance expectations, exceeded those expectations, or failed to meet expectations. A more detailed description of each employee performance indicator is provided in Chapter 5 but a list of indicators is included below. Specific p erformance i ndicators Positive customer service surveys r eceived Negative customer service surveys r eceived Te chnical forum p osts As signed reactive support t ickets Ad hoc r eactive support t ickets Mobile support s ite r eactive support t ickets Performance feedback s ubmissions Proactive equipment checklists c ompleted Proactive h all sweeps c ompleted Office shifts c ompl eted Event assists c ompleted Monthly evaluations c ompleted Late n otic es r eceived Critical notices r eceived Mobile support site events a ttended Area meetings a ttended Staff meetings a ttended Portal logs by d ay Data c ollection: r esearch j ournal According to Creswell ( 2013 ), an important component of qualitative research is reflexivity. This allows the researcher to better situate themselves contextually within the Because this study incorporates qualitative data, and because the study focuses on the design of a gamified system, keeping a journal to track design considerations in addition to thoughts around the implementation may provide valuable context for both the current study and future implementations. With this in mind, a research journal was kept during the study that tracked design considerations, events, and thoughts about the implementation that

PAGE 99

99 occurred throughout the study. In total, 34 journal entries we re recorded between November of 2016 and March of 2017. Of these, fourteen dealt with design considerations and were more expansive in nature. The remaining twenty focused on thoughts around the implementation, but were often shorter regarding content. Da ta Analysis T hree types of data were collected during this research from the two data sources described above These data types include d Likert IMI survey responses, free text responses to open ended survey questions, and numeric ratio data tracking summat ive employee performance over time in several job performance categories. For each data set, a different method of analysis was used. Analysis methods for each data type are described below. T he specific data sources and analysis methods for each research q uestion are outlined in Table 4 2 T able 4 2 Research questions, data sources and data a nalysi s s ummary Research q uestion Data s ource Data a nalysis How does a gamification implementation designed using the Self Determined Gamification framework align w ith the motivation continuum within Self Determination Theory? Survey data from IMI (Intrinsic Motivation Indicator) Descriptive s tatistics How does the presence of a gamification layer in an online web portal impact college student employee perceptions of the online workplace environment? Open ended survey responses Open coding Does college student employee performance change after the implementation of a gamification layer within an online web portal used to complete job requirements? Numeric ratio me trics from the web portal Descriptive s tatistics

PAGE 100

100 Data a nalysis: s elf d etermination s urvey Likert data from the IMI survey responses w ere analyzed to determine the extent to which the gamification incentivizing intervention aligns with the autonomy continuum within Self Determination Theory (SDT). The Likert responses align with four subscales that map to autonomous extrinsic and intrinsic motivation on the autonomy continuum within SDT Calculating the results of the IMI survey responses involve d re verse scoring specific items and then averaging items across subscale scores (Deci & Ryan, n.d.). Because the gamification implementation was designed with SDT in mind, this data set help ed determine the fidelity of the design. How the ga mification impleme ntation aligned with the autonomy continuum also influence d interpretations of other data sources during the convergence of data analysis. Open ended responses were reviewed and coded using an open coding process. Here, a process of emergence was utilized as phrases, activities and behaviors emerge from the data (Creswell, 2013; LeCompte & Schensul, 2013) These initial codes were grouped around concepts and ideas prior to determining if responses were weighted or skewed in any direction. Finally, responses were analyzed for specific suggestions and perceptions pertaining to the achievements. Free responses were analyzed for them atic elements based on this open coding process Data analysis: performance data Retrospective summative performance data from stu dent staff was compared to corresponding performance data for staff using the gamification layer. Because returning staff existed in all three groups and because this research did not track those individuals between the groups, inferential statistics were not possible. Instead, means and standard deviations for each of the performance indicators were compared for the

PAGE 101

101 three time periods to determine if any changes to performance occurred. Because descriptive statistics were used, statistical significance cou ld not be determined. This analysis helped to determine if any change in performance occurred during the semester that the achievements and progress bars were in use within the web portal. Rigor As was stated previously, t he IMI survey used in this study consists of questions rated on a 7 point Likert scale and various versions have been found to be both reliable and valid within different settings (Deci & Ryan, n.d.). Summative performance data from prior years was compared to comparable data from the s e mester the gamification layer wa s in use. In addition, employees had the opportunity to leave anonymous feedback through open ended survey responses. This data was analyzed using open coding, looking for themes about participant perceptions of the gamifica tion elements. By using a vetted questionnaire and bringing both qualitative and quantitative data together for analysis, this study leverage s the strengths of both forms of research, yielding high quality meta inferences consistent with relevant theory an d other research findings (Onwuegbuzie & Johnson, 2006; Tashakkori & Teddlie, 2006). Creswell (2014) describes mixed methods research as involving the rigorous collection and analysis of both quantitative and qualitative data that is then merged or connect ed. Either form of data on its own may not be sufficient to paint the entire picture when conducting research and the two methods can complement each other when used in conjunction appropriately (Ivankova, Creswell & Stick, 2006). When identifying the specific mixed method design, it was important to consider the way data would be merged, the timing of the data collection, and the relative emphasis placed on the two types of data. Due to time constraints created by the dissertation process data needed

PAGE 102

102 to be gathered as quickly as possible during the gamification implementation The goal of the evaluation was to use both forms of data in tandem and compare and contrast results. It was important in this case to place equal weight on the two data types. Wi th this in mind, a triangulation approach was best suited to the needs of the study. Within the triangulation model, three common variants were described in the literature that varied in terms of when the data was merged in the research process and the int ent behind merging the data (Creswell & Clark, 2007). Because comparing and contrasting the two forms of data was the focus, the convergence model made the most sense, where data is merged during interpretation. Limitations While a formal control group wa s not possible, comparison of overall staff performance to prior staff who utilized the web portal without a gamification layer had the potential to reveal interesting trends. Without random sampling or random assignment, there were clear threats to both i nternal and external validity. While findings may not generalize to the greater population as a result, recommendations may be transferrable to similar populations in similar settings. This is consistent with current gamification research which is explorat ory in nature. It is important to note that there we re numerous other variables that may have impact ed employee performance metrics. In addition, this intervention may have appealed to this target population specifically, since they are between the age of 18 and 26 and the intervention may have had different results with other age groups. Findings from the survey represent a subset of the target population and may not be reflective of the entire staff. This research attempt ed to compare Spring semester data from previous semesters to the Spring 2017 semester data for staff performance, in an effort to compare the most similar data

PAGE 103

103 sets. Fall semester data was excluded, as it is often inflated due to the number of technology issues at the start of an academic year. As a proponent of many of the motivational benefits of video game elements, my own bias needed to be situated within the context of the study. Recognition of the Hawthorne Effect and the potential that participants would want to please me as a cowo r ker, w as also important (Gilbert, 2007) Survey responses were anonymous in an effort to mitigate this effect. Ethical Considerations and Bias As the Director of the Techworks Program, I am typically responsible for evaluating participant performance each semester. Techworks student employees are aware that their performance is tracked within the web portal at all times and the data used in this study is the same performance data that is nor mally tracked and reviewed with employees on a monthly basis As stated previously, p articipation with the gamified elements of the web portal was entirely voluntary and any feedback was anonymous. The intent of this implementation wa s to create a personal experience for staff to track their progress and see their achievements as they complete d job requirements. For the semester in which the study t ook place, performance evaluations were conducted by another full time employee, without knowledge of whether employees participated in the gamification implementation or the content of their feedback. The survey used within this study was anonymous and voluntary All data used in the study, both retrospective and current, was stripped of any identifying features and only summative totals for the entire staff were used Summary Chapter 4 outlined the research methods and approach used to investigate a gamification incentive implementation in an online web portal. The context is unique in

PAGE 104

104 that student staff wer e college students living in residence halls and completing work autonomously, remotely and with minimal supervision. The intent of this research wa s to design a gamification intervention in accordance with researched best practices from the literature that align ed with SDT and the MDA Game Design Framework analyze the fidelity of the design through a vetted SDT questionnaire explore student employee perceptions of the gamification design, and compare retrospective data concerning performance indicators for Techworks student staff to performance indicators for current staff who experienced the gamification intervention to see if there was any impact on performance.

PAGE 105

105 CHAPTER 5 RESULTS Chapter 5 looks at the results of the research in the context of the overarching research questions guiding the study. Each research question is examined through a discussion of corresponding results from each of the three sources of data analyzed as part of this r esearch. Likert results fr om the IMI survey are presented and mapped to the four factors of interest/enjoyment, perceived competence, perceived choice, and pressure/tension. O pen ended feedbac k from participants are organized according to thematic elements that emerged during the open coding process. P erformance data from several key performance metrics is compared for the entire sta ff from comparable three month w indows in the prior two years Descriptive statistics for each of the quantitative measures ar e provided Finally, a brief summary of the results is presented prior to the discussion that follows in Chapter 6 Data Analysis for Research Question 1: How Does a Gamification Implementation Designed U sing th e Self Determined Gamification Framework Ali gn With the Motivation Continuum W ithin Self Determination Theory? IMI Survey Results The Self Determination Theory (SDT) Intrinsic Motivation Indicator ( IMI ) S ur vey was made available to staff on March 27, 2017 and closed on April 1, 2017 after three months of using the achievements in the web portal (Deci & Ryan, n.d.) During the week the survey was available, 15 individuals submitted feedback. The IMI Survey consisted of 22 statements and asked participants to indicate how true they fel t each statement was on a 7 point Likert scale with 1 indicating not at all true, 4 indicating somewhat true, and 7 indicating very true. Means and standard deviations for each que stion are provided in Appendix G Likert data for items 2, 9, 11, 14, 19, an d 21 were reverse scored as indicated in the instructions for issuing the

PAGE 106

106 survey to ensure that scores could be accurately compared across the f our factors. An average Likert rating between 4 and 7 on any factor would indicate that participants felt that the factor was present when considering achievements. An average Likert rating between 1 and 4 on any factor would indicate that participants felt that the factor was not present when considering achievements. The closer the average Likert response across a factor is to 7, the stronger the factor is represented and the closer the average Likert response across a factor is to 1, the weaker the factor is represented. Table 5 1 provides the means and standard deviations for each of the four factors. Table 5 1 I MI factor alignment Factor Mean St d d eviation I tem c ount Interest/Enjoyment 5.429 1.400 7 0.92 Perceived Competence 5.347 1.289 5 0.83 Perceived Choice 5.787 1.553 5 0.70 Pressure/Tension 1.520 0.795 5 0.85 Each of the four factors aligns with the autonomy continuum within Self Determination Theory. Low levels of interest and enjoyment, perceived competence, perceived choice, and high levels of pressure and tension in a work environment are associated with a lack of employee motivat ion, employee apathy and control on the part of the employer. High levels of interest and enjoyment, perceived competence, perceived choice and low levels of pressure and tension are associated with internalizing organizational and individual goals, equit y in the workplace, and autonomous extrinsic motivation. Survey responses indicated a score of greater than 5 for each of the first three factors On the Likert scale, this reflects feelings of high interest and enjoyment, high levels of perceived competen ce and a high level of choice whe n engaging with the achievements At the same time, the Likert score of 1.520 for

PAGE 107

107 the fourth factor suggest ed a lack of pressure or tension when engaging with the achievements. When looking at responses across the four fac tors, participant responses indicated that the design of the achievements in the web portal aligns with more autonomous forms of extrinsic motivation on the autonomy continuum. Data Analysis for Research Question 2 : How Does the Presence of a Gamification Layer in an Online Web Portal Impact College Student Employee P erceptio ns of the Online Workplace E nvironment? Open E nded Feedback From the 15 survey responses, participants provided open ended feedback about their perceptions of the achievements while completing work in the portal. Specifically, participants responded to the following three questions. Please provide any feedback regarding the achievements added to the Techworks web portal this semester. If any of the achievements impacted the work you completed in the Techworks web portal this semester, please describe how. If possible, please provide examples. What changes would you make to the achievements to improve this feature in future semesters? Responses were grouped by question and coded using an open coding process Here, words, concepts and ideas emerged from the data that represented codes across the responses. Specific suggestions as they related to the research question around employee perceptions and changes in performance were also catego rized into codes In total, seven codes emerged grouped into t hree overarching t hemes from the open ended responses that are outlined below based on the coded data set. Theme 1: Positive Experience The primary theme throughout the responses was the positive experience participants expressed in interacting with the achievements. Responses included words

PAGE 108

108 like fun (7), cool (3), interesting (2), enjoyed (1) and neat ( 1) and expressed the sentiment of li king the addition of achievements. to see this at work. It made work more fun and gave me specific things to target when responses and will be explored in greater depth, but multiple responses shared this feeling of enjoyment. Another particip ant referred am working to for the semester. The positive experience was consistent in all but one response where a participant expressed a mbivalence, having not engaged with the achievements. Here, the No participant indicated a negative experience or expressed disliking the achieve ments. Theme 2: Motivation and Impact The second overarching theme that emerged from the open ended responses deals with motivation and specific performance impact. Coded responses pointed to staff engaging in goal setting behavior through interacting wit h the achievements and in several cases suggested that they targeted specific achievements. Additionally, staff indicated specific changes to work habits as they navigated the new achievement structure Staff also expressed increased motivation and noted t he incentivizing nature of the achievements on their work. In some cases staff felt that the achievements gave recognition for the work they completed within the web portal.

PAGE 109

109 Setting g oals and t argeting a chievements Several participants made reference to targeting specific achievements or using the achievements to set specific goals. Three individuals referenced goal setting using the achievements and seven referenced targeting achievements specifically Three additional responses described the practice of targeting or goal setting without using that exact language. In total, thirteen of the responses dealing with general feedback or practice. vements gave me something to target with my work. Another participant referenced looking through all the achievements that were available to see what the requirements were in order to focus work efforts on th ose requirements. References to specific job related tasks that participants targeted are discussed in the next section, but the general practice of setting goals or targeting was referenced consistently. While one participant indicated they would be gradu ating in a few months, they also stated that if the hope to work here for the next three Job specific impact Beyond the general practice of targeting achievements, p articipants indicated very specific areas where their work was impacted by the addition of achievements. Two individuals mentioned notic ing that there were achievements for logging into the portal and began trying to make sure they logged in each day. I saw there were achievements for logging in each day so I started making that part of my routine, to log in and see if there was anything Another participant

PAGE 110

110 One participant mentioned leaving more performance feedback about other staff due to the achi them gave me targets to shoot for with my work like leaving performance feedback about someone when I might not have otherwise now I had a reason to where before I n all these examples, the participant indicated the achievements impacting their work by changing how often they engaged in a specific job related task. In another example of this pattern, a participant discussed the impact of achievements on performing p roactive door to door technology checks, One participant referenced visiting the achievements page to see the number of tickets he or she had worked on and using this to se t goals for the semester while another referenced signing up for extra events. Two responses also indicated that the participant went above and beyond in certain job requirements because of the associated achievements, without referencing the specific are as where work was impacted In total, ten of the fifteen responses dealing with how the achievements impacted work referenced specific areas where a participant refocused efforts through interacting with the achievement system. No participant indicated tha t the achievements had a negative impact on work, with one participant stating that they felt their performance stayed the same and was not impacted by achievements. Incentivizing w ork and f eelings of l egitimacy The practice of targeting and goal setting, along with specific areas where participants changed work habits due to the achievements were accompanied in some cases by comments around incentivizing work and motivating participants to com plete

PAGE 111

111 work. Two responses specifically referenced an increase i n motivation to perform work tasks and two responses mentioned the achievements as incentivizing their work. One Another indicated appreciating the positive reinforcement the achievements provided. The concept of the achievements to go above and beyond in Theme 3: Design Considerations The third theme that emerged in the open ended responses focused on design considerations. Several responses suggested a need to adjust the r equirements for attaining specific achievements. Additionally, responses pointed to specific suggestions t o improve or change the design of the achievements. These suggestions ranged from specific interface adjustments to adding additional game elements. Attainability and r equirements One of the most consistent codes from the open ended responses involved participants describ ing certain achievements as being too difficult to attain or the inability to obtain certain achievements due to graduating or leaving the position. One Along those lines, multiple participants indicated they may have approached work differently had the achievements been in place l onger. Another participant he achievement

PAGE 112

112 Similarly, some participants made suggestions to increase the number of realistically achievable achievements or to add more achievements that could be earned by staff who only work two to four semesters. It was apparent from some responses that the difficult achievements to earn would require working for six to eight semesters, but others expressed a desire to be able to collect all the achievements in a shorter window of time. Regardless, eight of the responses dealing with e ither general feedback or suggest ing changes to achievements recommend ed lowering the requirements for difficult achievements or increasing the number of easier achievements. Design s uggestions Many of the responses made suggestions to improve the design of the system or add additional functionality. Two responses mentioned that the achievements are in the background, making them an optional interface to interact with, but that this design decision also served to deemphasize the achievements. in the interface. first mo nth or so, I kind of forgot the achievements were there. Maybe have a way to put them more front One participant suggested making the achievements pop up in the interface when earned and several expressed a desire to have the achievem ents emphasized from the outset of working in Techworks covering the achievements in initial training materials and working with staff to use them to set goals. Three responses suggested leveraging the achievements to foster healthy competition amongst th e staff. In all three cases, the participant specifically referenced incorporating

PAGE 113

113 leaderboards, though the approach was different. In one case, the suggestion included all time records with historical data, while another focused on current staff. Two of t he leaderboard suggestions referenced making the leaderboards anonymous in some way, but allowing staff to see how many people had earned certain achievements. One response also indicated the color scheme was too bright and suggested more minimalist icons for the achievements. Data Analysis for Research Question 3: Does College Student Employee Performance Change After the I mplementation o f a Gamification Layer Within an Online Web Portal Used to Complete Job R equirements? Performance Indicator s In total, 18 performance indicators were tracked across three different time periods during the months of January, February and March in 2015, 2016 and 2017. With the achievements in place for the 2017 months, retrospective performance totals for each performance me tric were compared to totals after the gamification layer was in place. Because the total number of staff differed from year to year, both the summative totals for each performance indicator and the average per staff member are reported. Descriptive statis tics for each performance indicator across the three year period are presented below. It is important to note that the performance indicators represent all the job responsibilities a student employee is asked to complete. Achievements were mapped to each p erformance in dicator in an effort to emphasize the importance of all job requirements and not de emphasize any job requirements through omission While some job responsibilities afford an employee the opportunity to go above and beyond, others have a finit e or relative expectation based on the volume of work. At the same These

PAGE 114

114 distinctions will be expla ined in greater detail in the sections below. The performance indicators are grouped into four categories to make this distinction clear. While it was unlikely achievements could impact performance for job responsibilities outside an and where an employee could not go above and beyond the expectation it was worth tracking all aspects of the work completed to determine if any changes in performance were seen. Figure 5 1 shows the performance indicators grouped into four quadrants base those that allow an employee to go above and beyond the expectation The numbered quadrants will be referenced in the sections that follow. Figure 5 1 Performance i ndicators grouped by employee contro l and ability to go above and b eyond

PAGE 115

115 Performance Indicators Quadrant 1 Can Go Above and Beyond Expectations Performance indicators quadrant 1 performance feedback submissions Techworks staff can leave performance feedback about other employees through the Techworks Portal. This feedback can be either anonymous or identified and can be either viewable by the employee receiving the feedback or only viewable to management. Performance feedback provides qualitative da ta in evaluating employee performance and staff are encouraged to leave feedback when they have any interactions with other staff they would like to highlight This performance indicator is a numeric total of all performance feedbacks the employee posts. I n Table 5 2, the total number of performance feedback submissions is reported for each time period along with the staff mean and standard deviation. Table 5 2 Performance indicators performance feedback s ubmissions Year PI t otal Total s taff Staff m ean Std. d eviation 2017 47 43 1.09 4.879 2016 5 46 0.11 0.526 2015 14 48 0.29 0.683 The mean for performance feedback entered for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 1.09 (4.879), 0.11 (0.526), and 0.29 (0.623), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 2). Performance feedback submissions increased a great deal during the 2017 time frame when compared to 2016 and 2015. With less sta ff, the average number of performance feedback posts per staff member also increased a great deal in 2017. This performance metric is entirely within a staff time. This perfo rmance metric is also an area where staff can go above and beyond

PAGE 116

116 management expectations since there is no limit to the number of performance feedbacks that can be entered. Figure 5 2 Bar g raph a verage performance feedback s ubmissions p er s taff m ember Performance indicators quadrant 1 proactive equipment checklists Techworks staff complete proactive checklists on all technology equipment permanently installed in the residence halls. This equipment includes touch screen digital signs, HDTVs, ga me systems, desktops, communal printers, and collaborative workstations. This performance indicator is a numeric total of all proactive checklists an employee completes. In Table 5 3, the total number of proactive equipment checklists completed is reported for each time period along with the staff mean and standard deviation. Table 5 3. Performance indicators proactive equipment c hecklists Year PI t otal Total s taff Staff m ean Std. d eviation 2017 2172 43 50.51 11.196 2016 1296 46 43.20 15.175 2015 2159 48 37.48 13.756 1.09 0.11 0.29 0.00 0.20 0.40 0.60 0.80 1.00 1.20 2017 2016 2015 Average performance feedback submissions per staff member

PAGE 117

117 The mean for proactive equipment checklists completed for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 50.51 (11.196), 43.20 (15.175), and 37.48 (13.756), respectively. Staff averages ar e also represented in a bar graph to visually see these distinctions (Figure 5 3) Completion of proactive equipment checklists increased in 2017 when compared to comparable time frames in 2016 and 2015. The average number of equipment checklists completed by a staff member also increased in 2017. This performance indicator is within a control to complete since proactive checklists are available in the web portal at the start of each semester. Additionally, staff have the ability to go above and beyond expectations by taking on additional proactive checklists or volunteer ing to complete checklists throughout the semester. Figure 5 3 Bar g raph a verage proactive equipment checklists c ompleted per staff m ember Performance i ndicators q uadrant 1 p roactive h all sweeps Each semester, Techworks staff visit each room on c ampus to inform residents about the service and verify there are no technical issues. Staff are encouraged to 50.51 43.20 37.48 0.00 10.00 20.00 30.00 40.00 50.00 60.00 2017 2016 2015 Average proactive technology checklists completed per staff member

PAGE 118

118 complete hall sweeps as early in a semester as possible so that the interactions have a higher impact. This performance indicator is a numeric tot al of all proactive hall sweeps an employee completes. In Table 5 4, the total number of proactive hall sweeps completed is reported for each time period along with the staff mean and standard deviation Table 5 4 Performance indicators proactive hall s weeps Year PI t otal Total s taff Staff m ean Std. d eviation 2017 4971 43 115.60 77.051 2016 1296 46 28.17 16.928 2015 2159 48 44.98 21.486 The mean for proactive hall sweeps completed for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 115.60 (77.051), 28.17 (16.928), and 44.98 (21.486), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 4) The number of proactive hall sweeps completed in 2017 exceeded those completed for comparable time frames in 2016 and 2015, as did the average number of proactive hall sweeps completed by staff. sweeps are available i n the web portal at the start of each semester. In addition, staff can go above and beyond expectations by taking on extra hall sweeps. Staff are encouraged to complete hall sweeps during the first two months of a semester so they have a greater impact.

PAGE 119

119 Figure 5 4 Bar g raph a verage proactive hall sweeps c ompleted per staff m ember Performance indicators quadrant 1 office shifts completed Each semester, Techworks student staff are required to complete a minimum of six office shifts at the Techworks office. This performance indicator is a numeric total of the office shifts comple ted by an employee. In Table 5 5 the total number of office shifts completed is reported for each time period along with the staff staff mean and standard deviation. Table 5 5 Performance i nd icators office shifts c ompleted Year PI t otal Staff t otal Staff m ean Std. d eviation 2017 176 43 4.09 2.255 2016 174 46 3.78 1.965 2015 182 48 3.79 2.361 The mean for the number of office shifts completed for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 4.09 (2.255), 3.78 (1.965), and 3.79 (2.361), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 5). The total number of offi ce shifts completed by staff in 2017 was slightly less than in the comparable time periods in 2016 115.60 28.17 44.98 0.00 20.00 40.00 60.00 80.00 100.00 120.00 140.00 2017 2016 2015 Average proactive hall sweeps completed per staff member

PAGE 120

120 and 2015. Due to fewer staff in 2017, the average number of office shifts completed was slightly higher than in 2016 or 2015. This performance indicator is w ithin a staff semester within the web portal. Additionally, staff can go above and beyond expectations by signing up for extra office shifts. Figure 5 5 Bar g raph a v erage office s hifts c ompleted per staff m ember Performance indicators quadrant 1 late notices received For all Techworks job responsibilities tied to a due date, employees receive a proactive reminder. If the employee is late in completing a job requirement that can still be completed after the due date, they receive a late notice. It is important to note that when an employee receives a late notice, she would still have the opportunity to complete the job requirement. This performance indicator i s a numeric total of all late notices an employee receives. In Table 5 6 the total number of late notices received by staff is reported for each time period along with the staff mean and standard deviation. 4.09 3.78 3.79 0.00 1.00 2.00 3.00 4.00 5.00 2017 2016 2015 Average office shifts completed per staff member

PAGE 121

121 Table 5 6 Performance indicators late notic es r eceived Year PI t otal Total s taff Staff m ean Std. d eviation 2017 17 43 0.40 0.583 2016 30 46 0.65 0.795 2015 74 48 1.54 2.405 The mean for late notices received for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 0.40 (0.583), 0.65 (0.795), and 1.54 (2.405), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 6). The number of late notices, along with the average number of late notices per staff m ember was lower in 2017 when compared to 2016 or 2015. With each late notice representing an incident of a staff member who was late in completing a job requirement, this performance metric is entirely within a staff e able to go above and beyond expectations by receiving no late notices during a semester, with the expectation being set that one to three late notices over an entire semester is acceptable. Figure 5 6 Bar g raph a verage late notices r eceived per staff m ember 0.40 0.65 1.54 0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2017 2016 2015 Average late notices received per staff member

PAGE 122

122 Performance indicators quadrant 1 critical notices received For all Techworks job responsibilities tied to a due date, employees receive a proactive reminder. If the employee is late in completing a job requirement that cannot be comp leted after the fact, they receive a critical notice. It is important to note that when an employee receives a critical notice, she would no longer be able to complete the job requirement. This performance indicator is a numeric total of all critical notic es an employee receives. In Table 5 7 the total number of critical notices received by staff is reported for each time period along with the staff mean and standard deviation. Table 5 7 Performance indicators critical notices r eceived Year PI t otal Total s taff Staff m ean Std. d eviation 2017 21 43 0.49 0.736 2016 19 46 0.41 0.777 2015 29 48 0.60 0.962 The mean for critical notices received for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 0.49 (0.736), 0.41 (0.777), and 0.60 (0.962), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 7). The total number of critical notices remained fairly consistent in 2017, 2016 and 2015. The average n umber of critical notices per staff was slightly higher in 2017 when compared to 2016 but was lower when compared to 2015, however in all three years, the average remained less than one critical notice per staff member. There was very little change in this performance metric across the three time periods tracked. Since each critical notice represents an incident of a staff member missing a job responsibility, this performance metric is entirely within a staff ions by receiving no critical notices in a semester with expectations set that one critical notice is acceptable.

PAGE 123

123 Figure 5 7 Bar g raph a verage critical notices r eceived per staff m ember Performance indicators quadrant 1 mobile support site events attended Techworks staff put on mobile technology support sites rotated between the different residence halls each week. This performance indicator is a numeric total of all support sites an employee attends. Attendance is tracked by supervisors at the eve nt and is only recorded with active participation. In Table 5 8 the total number of mobile support site events attended by staff is reported for each time period along with the staff mean and standard deviation. Table 5 8 Performance indicators mobile support site events a ttended Year PI t otal Total s taff Staff m ean Std. d eviation 2017 213 43 4.95 1.573 2016 199 46 4.33 1.564 2015 141 48 2.94 1.508 The mean for the number of mobile support sites attended for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 4.95 (1.573), 4.33 (1.564), and 2.94 (1.508), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 8). Staff attendance at mobile 0.49 0.41 0.60 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 2017 2016 2015 Average critical notices received per staff member

PAGE 124

124 support site s increased in 2017, as did the average number of support sites attended by staff when compared to 2016 and 2015. Staff are expected to attend seven support sites in a semester, but they can sign up for as many support sites as they would like. With the ab ility to sign up for extra events and the availability of events at the outset of meeting or going above and beyond the requirement. Figure 5 8 Bar g raph a verage mobile support site events a ttended per s taff m ember Performance indicators quadrant 1 portal logs by day While Techworks staff are not required to log into the portal every day, they are encouraged to log in periodically to keep track of the work assig ned within the portal. This performance indicator is a numeric total of all distinct days that an employee logs into the online portal. Logging in is automatically track ed in the portal and repeat log ins during the same day do not count towards the total. In Table 5 9 the total number of distinct portal logs by day is reported for each time period along with the staff mean and standard deviation. 4.95 4.33 2.94 0.00 1.00 2.00 3.00 4.00 5.00 6.00 2017 2016 2015 Average mobile support site events attended per staff member

PAGE 125

125 Table 5 9. Performance indicators portal logs by d ay Year PI t otal Total s taff Staff m ean Std. d eviation 2017 1450 43 33.72 16.087 2016 1346 46 29.26 8.609 2015 1095 48 22.81 8.401 The mean for the number of distinct portal log ins by day for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 33.72 (16.087), 29.26 (8.609), and 22.81 (8.401), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 9). The total number of distinct portal logs increased during 2017 when compared to 2016 and 2015. The average number of distinct portal logs per staff member also increased during 2017. This indicates that staff are logging into the portal more frequently in 2017 than in the prior two years. Because staff can log into the portal any day, this performance metric is entire ly within their control. Staff are also able to go above and beyond the expectation, since they are only expected to log into the portal on days when they have a job requirement that needs to be recorded or completed in the portal. Figure 5 9 Bar g raph a verage portal logs by d ay per staff m ember 33.72 29.26 22.81 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 40.00 2017 2016 2015 Average portal logs by day per staff member

PAGE 126

126 Performance Indicators Quadrant 2 Can Go Above and Beyond Expectations Performance indicators quadrant 2 positive customer service surveys Each reactive support ti cket generates a customer service survey to clients. When a survey is submitted, it is reviewed along with the ticket worklog by a supervisor to ensure the content is in reference to the employee in question. This performance indicator is a numeric total o f all positive customer service surveys an employee receives. Positive surveys are defined as surveys that receive a rating of Good, Very Good or Outstanding on the five point Likert scale. In Table 5 10 the total number of positive customer service surve ys received by staff is reported for each time period along with the staff mean and standard deviation. Table 5 10 Performance indicators positive customer service s urveys Year PI t otal Total s taff Staff m ean Std. d eviation 2017 91 43 2.12 3.033 2016 57 46 1.24 1.754 2015 83 48 1.73 1.932 The mean for positive customer surveys received for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 2.12 (3.033), 1.24 (1.754), and 1.73 (1.932), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 10). While the total number of positive customer service surveys varied from year to year, the average number of positive customer service surveys received by each staff member was higher in 2017 at 2.12 per employee than the prior two years. This is a performance indicator that staff have little control over since a client initiates the feedback, however staff are asked to encourage clients to leave feedback du ring training. The increase in the average positive customer service surveys per staff member could be indicative of staff making an effort to

PAGE 127

127 encourage clients to fill out the survey Staff are able to go above and beyond expectations with this performanc e metric by receiving more surveys than the expected amount Figure 5 10 Bar g raph a verage positive customer service s urveys received per staff m ember Performance indicators quadrant 2 negative customer service surveys This performance indicator is a numeric total of all negative customer service surveys an employee receives. Negative surveys are defined as surveys that receive a rating of Below Good or Unsatisfactory on the five point Likert scale. In Table 5 11 the total number of negative custom er service surveys received by staff is reported for each time period along with the staff mean and standard deviation. Table 5 11 Performance indicators negative customer service s urveys Year PI t otal Total s taff Staff m ean Std. d eviation 2017 0 43 0.00 0.000 2016 1 46 0.02 0.147 2015 2 48 0.04 0.202 2.12 1.24 1.73 0.00 0.50 1.00 1.50 2.00 2.50 2017 2016 2015 Average positive customer service surveys received per staff member

PAGE 128

128 The mean for negative customer surveys received for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 0 (0), 0.02 (0.147), and 0.04 (0.202), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 11). Very few negative customer service surveys were submitted by clients during any of the time periods tracked here. That said, there were no negative survey s submitted during the 2017 time period compared to 1 and 2 respectively during the previous two years. When looking at staff averages, 2017 saw the fewest per staff member and the fewest per ticket when compared to the previous two years. The difference b etween years is marginal however. While staff have little control over whether a client submits a customer service survey, they can minimize the likelihood through providing quality customer service. Staff can also go above and beyond the expectations set at the outset of the semester by receiving no negative customer service surveys. Figure 5 11 Bar g raph a verage negative customer service s urveys received per staff m ember 0.00 0.02 0.04 0.00 0.01 0.01 0.02 0.02 0.03 0.03 0.04 0.04 0.05 2017 2016 2015 Average negative customer service surveys received per staff member

PAGE 129

129 Performance indicators quadrant 2 event assists completed Each semester, Tech works student staff provide technology assistance at events in the residence halls. This performance indicator is a numeric total of all events an employee assists. In Table 5 12, the total number of event assists completed is reported for each time period along with the staff staff mean and standard deviation. Table 5 12. Performance indicators event assists c ompleted Year PI t otal Staff t otal Staff m ean Std. d eviation 2017 41 43 0.95 0.844 2016 21 46 0.46 0.504 2015 19 48 0.40 0.574 The mean for event assists complet ed for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 0.95 (0.844), 0.46 (0.504), and 0.40 (0.574), respectively. Staff averages are also represented in a bar graph to visually see these dis tinctions (Figure 5 12). There were twice as many event assists during the 2017 time period when compared to either 2016 or 2015. The average number of event assists per staff increased greatly during this time period as well. While this performance metric is somewhat outside the control of a staff member, since clients request assistance at events, staff are encouraged to reach out to Housing staff to try to foster partnerships and seek out events where they could provide assistance. Staff are able to go a bove and beyond the expectation in assisting events by completing one or more event assists.

PAGE 130

130 Figure 5 12 Bar g raph a verage event assists c ompleted per staff m ember Performance Indicators Quadrant 3 Cannot Go A bove and Beyond Expectations Performance indicators quadrant 3 monthly evaluations completed Each month, Techworks staff engage in a 360 evaluation, where they evaluate their own performance, the performance of their supervisor, the performance of any supervisees, and the overall Techworks program. Results of this feedback are viewable to individual employees and supervisors on their performance snapshot page, along with information about all other performance indicators detailed herein, accessible from evaluations completed by an employee. In Table 5 13, the total number of monthly evaluations completed is reported for each time period along with the staff mean and stand ard deviation. Table 5 13 Performance indicators m ont hly evaluations c ompleted Year PI t otal Total s taff Staff m ean Std. d eviation 2017 127 43 2.95 0.213 2016 131 46 2.85 0.363 2015 130 48 2.71 0.459 0.95 0.46 0.40 0.00 0.20 0.40 0.60 0.80 1.00 1.20 2017 2016 2015 Average event assists completed per staff member

PAGE 131

131 The mean for monthly evaluations completed for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 2.95 (0.213), 2.85 (0.363), and 2.71 (0.459), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 13). Th e total number of monthly evaluations submitted was relatively consistent during 2017, 2016 and 2015. The average number of monthly reports submitted increased slightly each subsequent year with 2017 having the highest average but this difference was margi nal. Each staff member is asked to submit a report each month, making three the maximum the staff average could be during the given time period. 2017 saw staff come the closest to the average of 3 per staff member. Submitting the monthly report is a perfor mance metric entirely within the control of the staff member, but they cannot go above and beyond the expectation. Figure 5 13 Bar g raph a verage monthly evaluations c ompleted per staff m ember 2.95 2.85 2.71 0.00 0.50 1.00 1.50 2.00 2.50 3.00 2017 2016 2015 Average monthly evaluations completed per staff member

PAGE 132

132 Performance indicators quadrant 3 area meetings attended Techworks staff attend regional meetings with their immediate supervisor and other staff in their region. This performance indicator is a numeric total of all area meetings an employee attends. Attendance is tracked by supervisors at the event and is onl y recorded with active participation. In Table 5 14 the total number of area meetings attended is reported for each time period along with the staff mean and standard deviation. Table 5 14 Performance indicators area meetings a ttended Year PI t otal Total s taff Staff m ean Std. d eviation 2017 109 43 2.53 0.702 2016 97 46 2.11 0.971 2015 108 48 2.25 1.021 The mean for area meetings attended for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 2.53 (0.702), 2.11 (0.971), and 2.25 (1.021), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 14). Area meeting attendance, along with the average number of area meetings attended per staff member remained r elatively consistent across all three years tracked. With a finite number of area meetings, staff are unable to go above and beyond with this performance metric. That said, there were 3 area meetings each staff member could have attended, assuming the meet ings fit their class schedule. 2017 saw the staff average come closer to three than in 2016 or 2015, however the difference was marginal.

PAGE 133

133 Figure 5 1 4 Bar g raph a verage area meetings a ttended per staff m ember Performance i ndicators q uadrant 3 s taff m eetings a ttended Techworks staff attend periodic staff meetings with the entire staff. This performance indicator is a numeric total of all staff meetings an employee attends. Attendance is tracked by supervisors at the event and is only recorded wi th active participation. In Table 5 15 the total number of staff meetings attended by staff is reported for each time period along with the staff mean and standard deviation. Table 5 15 Performance indicators staff meetings a ttended Year PI t otal Total s taff Staff m ean Std. d eviation 2017 67 43 1.56 0.734 2016 72 46 1.57 0.750 2015 78 48 1.63 0.733 The mean for the number of staff meetings attended for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 1.56 (0.734), 1.57 (0.750), and 1.63 (0.733), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 15). Staff meeting attendance was lower in 2017 than in 2016 or 2015, however the average number of staff meetings 2.53 2.11 2.25 0.00 0.50 1.00 1.50 2.00 2.50 3.00 2017 2016 2015 Average area meetings attended per staff member

PAGE 134

134 attended per staff member remained relatively consistent across the three time frames. There were two staff meetings a staff member could have attended, assuming the meeting time fit their class schedule. 2017 had the lowest attendance aver age between the three time frames, but the difference was marginal. Although a staff member could not go above and beyond in this performance metric, attendance at the two staff F igure 5 15 Bar g raph a verage staff meetings a ttended per staff m ember Performance Indicators Quadrant 4 Cannot Go Above and Beyond Expectations Performance indicators quadrant 4 technical forum posts Techworks staff utilize a techni cal forum to ask questions and offer support to one another as they address client issues in the field. This allows for asynchronous communication and documentation of the resolution method for a variety of technical problems. This performance indicator is a numeric total of all posts made by an employee that were flagged as quality posts by a supervisor. Quality posts are defined as work related, relevant posts that contributed to the discussion. In Table 5 16 the 1.56 1.57 1.63 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 2017 2016 2015 Average staff meeting attendance per staff member

PAGE 135

135 total number of technical forum posts is reported for each time period along with the staff mean and standard deviation. Table 5 16 Performance indicators technical forum p osts Year PI t otal Total s taff Staff m ean Std. d eviation 2017 87 43 2.02 2.739 2016 90 46 1.96 4.120 2015 301 48 6.27 8.338 The mean for technical forum posts for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 2.02 (2.739), 1.96 (4.120), and 6.27 (8.338), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 16). The number of technical forum posts decreased in 2017, when compared to the previous two years. The average number of posts per staff member was lower in 2017 than in 2015, but remained relatively constant when compared to 2016. This is a trend that extends beyond the three years under review, with technical forum posts decreasing each year for the past four years. The number of reactive support tickets has also decreased consistently for the past four years and may contribute to the decline in forum posts. With less reactive issues, there are less questions about support and less need to post to the forum. Additionally, as issues are documented in the forum there is less need to make new posts, since there is a repository of technical solutions for staff to browse. Staff have little control over this metric and little opportunity to go above and beyond since participation is limited by issues being reported Posting to the forum is only encouraged if you have actual questions or need assistance, with a focus on substantive posts.

PAGE 136

136 Figure 5 16 Bar g raph a verage technical forum p osts per staff m ember Performance indicators quadrant 4 assigned reactive support tickets Techworks staff are assigned reactive support tickets as clients report issues to the university Service Desk. This performance indicator is a numeric total of all reactive support tickets that an employee is assigned. In Table 5 17 the total number of reactive support tickets is reported for each time period along with the staff mean and standard deviation. Tabl e 5 17 Performance indicators assigned reactive support t ickets Year PI t otal Total s taff Staff m ean Std. d eviation 2017 482 43 11.21 17.277 2016 463 46 10.07 12.797 2015 1045 48 21.77 18.556 The mean for assigned reactive support tickets for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 11.21 (17.277), 10.07 (12.797), and 21.77 (18.556), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 17). The number of assigned reactive support tickets in 2017 increased slightly from the corresponding time period in 2.02 1.96 6.27 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 2017 2016 2015 Average technical forum posts per staff member

PAGE 137

137 2016. Th at said, both totals have decreased when compared to 2015 and the annual totals have been lower each year for the past four years. With less staff in 2017, the average number of tickets completed by a staff member increased slightly when compared to 2016, but both decreased when compared to 2015. This performance metric is outside the control of staff, as assigned reactive support tickets occur when clients report issues and they have little opportunity to go above and beyond in this category as a result. Figure 5 17 Bar g raph a verage assigned reactive support t ickets per staff m ember Performance indicators quadrant 4 ad hoc reactive support tickets Techworks staff create reactive support tickets when clients come to them directly. We track these tickets separately from those that are assigned to an employee, as they indicate availability to residents in the community. This performance indicator is a numeric total of all ad hoc support tickets that a n employee creates. In Table 5 18 the total numb er of ad hoc support tickets is reported for each time period along with the staff mean and standard deviation. 11.21 10.07 21.77 0.00 5.00 10.00 15.00 20.00 25.00 2017 2016 2015 Average assigned reactive support tickets per staff member

PAGE 138

138 Table 5 18 Performan ce indicators ad hoc support t ickets Year PI t otal Total s taff Staff m ean Std. d eviation 2017 194 43 4.51 5.091 2016 233 46 5.07 4.587 2015 526 48 10.96 9.607 The mean for ad hoc reactive support tickets for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 4.51 (5.091), 5.07 (4.587), and 10.96 (9.607), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 18). Ad hoc ticket totals during the 2017 window were lower than the prior two time frames in 2016 and 2015. Like technical forum posts and other forms of reac tive support, annual totals reflect this trend as well, with reactive totals decreasing each year over the past four years. The average number of reactive ad hoc tickets per staff member was also less in 2017 than in 2016 or 2015. Staff have little control over this performance metric, as a reactive measure of support, since tickets are only created if clients report issues. Consequently, staff have little opportunity to go above and beyond in this category. Figure 5 18 Bar g raph a verage ad hoc reactive support t ickets per staff m ember 4.51 5.07 10.96 0.00 2.00 4.00 6.00 8.00 10.00 12.00 2017 2016 2015 Average ad hoc reactive support tickets per staff member

PAGE 139

139 Performance indicators quadrant 4 mobi l e support site reactive support tickets Techworks staff complete reactive support tickets at mobile support sites. This performance indicator is a numeric total of all mob ile support site tickets that an employee works on at the Techworks s upport site events. In Table 5 19 the total number of mobile support support tickets is reported for each time period along with the staff staff mean and standard deviation. Table 5 19 Performance indicators mobile support site t ickets Year PI t otal Total s taff Staff m ean Std. d eviation 2017 56 43 1.30 1.846 2016 74 46 1.61 1.437 2015 162 48 3.38 2.922 The mean for reactive support site tickets for each group of staff (with standard deviations in parenthesis) for 2017, 2016, and 2015 were 1.3 (1.846), 1.61 (1.437), and 3.38 (2.922), respectively. Staff averages are also represented in a bar graph to visually see these distinctions (Figure 5 19). Totals for reactive mobile sup port site tickets were lower in 2017 than in 2016 or 2015. The average per staff was also lower, even with less staff. This is consistent with other forms of reactive support over the past four years, where totals for all forms of reactive support have dec reased year to year. Like other forms of reactive support, this performance metric is outside the control of staff and they have little opportunity to go above and beyond, since tickets are only created when a client reports an issue.

PAGE 140

140 Figure 5 19 Bar g raph a verage mobile support site reactive support t ickets per staff m ember Performance Indicators Summary To better understand the changes in performance that occurred, it was useful to examine each of the three comparative time frames in more detail. I n the sections that follow, each of the three time periods is examined independently, prior to an overall summative discussion. First 2017 performance indicators are compared to 2016 in more depth followed by 2017 indicators compared to 2015. 2016 perform ance indicators are also compared to 2015 to provide additional context A table for each time frame is also provided summarizing areas where performance improvement occurred. Performance i ndicators s ummary 2017 compared to 2016 The 2017 time frame saw performance improvement in 14 of the 18 performance indicators when compared to 2016. These improvement areas include performance feedback submissions, proactive equipment checklists, proactive hall sweeps, office shifts, late notices, mobile support event attendance, portal logs by day, positive customer service surveys, negative customer service surveys, event assistance, 1.30 1.61 3.38 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 2017 2016 2015 Average mobile support site reactive tickets per staff member

PAGE 141

141 monthly evaluation completion, area meeting attendance, technical forum posts and assigned reactive support tickets. Where performance improvement occurred, it was frequently in areas where an employee had either control or the ability to go above and beyond expectations, characteristics of indicators in quadrants 1, 2, and 3. Table 5 20 summarizes the areas of improvement below. The two areas in quadrants 1, 2 and 3 where performance improvement did not occur saw performa nce remain relatively constant across, with staff receiving a similar number of critical notices and attending a similar number of staff meetings. Table 5 20 Performance i ndicators s ummary 2017 compared to 2016 Performance m etric Quadrant Improvement in 2017 Performance f eedback 1 Yes Equipment c hecklists 1 Yes Hallsweeps 1 Yes Office s hifts 1 Yes Late n otices 1 Yes Critical n otices 1 No Mobile support e vents 1 Yes Portal logs by d ay 1 Yes Positive customer s urveys 2 Yes Negative customer s urveys 2 Yes Event a ssists 2 Yes Monthly e valuations 3 Yes Area m eetings 3 Yes Staff m eetings 3 No Forum p osts 4 Yes Assigned t ickets 4 Yes Ad hoc t ickets 4 No Support site t ickets 4 No Performance i ndicators s ummary 2017 compared to 2015 The 2017 time frame saw performance improvement in 13 of the 18 performance indicators when compared to 2015. These improvement areas include performance feedback submissions, proactive equipment checklists, proactive hall sweeps, office

PAGE 142

142 shifts, late notices, critical notices, mobile support event attendance, portal logs by day, positive customer service surveys, negative customer service surveys, event assistance, monthly evaluation completion, and area meeting attendance. All performance improvement took plac e in areas where an employee had either control or the ability to go above and beyond expectations The one area in quadrant 3 that did not see improvement saw performance remain relatively constant with staff attending a similar number of staff meetings a cross the two time periods. Table 5 21 summarizes the areas of improvement below. Table 5 21 Performance indicators s ummary 2017 compared to 2015 Performance m etric Quadrant Improvement in 2017 Performance f eedback 1 Yes Equipment c hecklists 1 Yes Hallsweeps 1 Yes Office s hifts 1 Yes Late n otices 1 Yes Critical n otices 1 Yes Mobile support e vents 1 Yes Portal logs by d ay 1 Yes Positive customer s urveys 2 Yes Negative customer s urveys 2 Yes Event a ssists 2 Yes Monthly e valuations 3 Yes Area m eetings 3 Yes Staff m eetings 3 No Forum p osts 4 No Assigned t ickets 4 No Ad hoc t ickets 4 No Support site t ickets 4 No Performance i ndicators s ummary 2016 compared to 2015 While comparing performance between 2016 and 2015 was not a major focus of this research it was useful to explore the changes in performance to get a clearer picture of the changes in 2017. The 201 6 time frame saw performance improvement in

PAGE 143

143 8 of the 18 performance indicators when compared to 201 5 These improvement areas include proactive equipment checklists, late notices, critical notices, mobile support event attendance, portal logs by day, negative customer service surveys, event assistanc e, and monthly evaluation completion. All performance indicators where improvement occurred were in areas where an employee had either control or the ability to go above and beyond expectations however there were more performance indicators where no impro vement occurred in q uadrants 1, 2, and 3 when looking at the other comparative time periods Table 5 22 summarizes the areas of improvement below. Table 5 22. Performance indicators s ummary 2016 compared to 2015 Performance m etric Quadrant Improvement in 2016 Performance f eedback 1 No Equipment c hecklists 1 Yes Hallsweeps 1 No Office s hifts 1 No Late n otices 1 Yes Critical n otices 1 Yes Mobile support e vents 1 Yes Portal logs by d ay 1 Yes Positive customer s urveys 2 No Negative customer s urveys 2 Yes Event a ssists 2 Yes Monthly e valuations 3 Yes Area m eetings 3 No Staff m eetings 3 No Forum p osts 4 No Assigned t ickets 4 No Ad hoc t ickets 4 No Support site t ickets 4 No Performance indicators summary key findings The 2017 time frame saw performance improvement across quadrants 1, 2 and 3 when compared to 2016 and 2015, where employ ee s had control over completing a requirement and where employ ee s had the ability to go above and beyond

PAGE 144

144 expectations. When both those conditions were present, performance improvement was more likely with 7 of the 12 areas of improvement between 2017 and either 2016 or 2015 occurring in quadrant 1. Those 7 areas include performance feedback submissions, completion of equipment checklists, completion of hall sweeps, office shifts completed, receipt of late notices, attendance at mobile support sites, and portal logs by day. A comparison of 2016 with 2015 saw less consistent improvement across the 18 performance indicators, suggesting that 2017 was unique in that respect. As expected, performance indicators where employ ee s had little control or no ability to exceed expectations saw little to no improvement in 2017. Summary Chapter 5 outlined the data collected and analyzed as part of this research study. S urvey results from the IMI Survey within Self Determination Theory indicate that the achievements within the web portal align with more autonomous extrinsic motivation. Participants reported high levels of interest and enjoyment with the system, a high deg ree of competence in using the system and a high degree of choice when engaging with the achievements. Participants also reported very little pressure or tension in using the achievements. Open ended responses suggested staff had a primarily positive exper ience in using the achievements and that the achievements served to motivate and incentivize certain aspects of their work. Performance data reflects this as well in several of the metrics tracked during the time period achievements were in use particular ly when staff had the ability to go above and beyond and when they had control over the performance indicator in question In total, twelve of the eighteen performance indicators showed improvement in 2017 when compared to both 2016 and 2015 These finding s suggest that achievements have great potential to incentivize work

PAGE 145

145 when designed with SDT and the MDA Game Design Framework in mind. Several additional d esign considerations emerged from the data around accurately mapping achievement requirements to reas onable performance expectations. T hese results are discussed in greater depth in the context of this research in Chapter 6.

PAGE 146

146 CHAPTER 6 DISCUSSION Overview This study sought to address a problem of practice by incentivizing job requirements through the addition of achievements in an online web portal. To do this, a workplace analysis was conducted, along with a thorough review of relevant literature. The result was the creation of a framework for designing gamified systems entitled Self Determined Gamifi cation. Using this framework as a guide, an attempt was made to design achievements with Self Determination Theory (SDT) in mind, paying particular attention to organizational goals, equity in attaining achievements, employee autonomy in interacting and en gaging with the achievements, and individual goals. Design best practices were incorporated through the use of the MDA Design Framework in addition to drawing upon design considerations from the SAPS Reward System (Zichermann & Cunningham, 2011) Player Types (Bartle, 1996; Zichermann & Cunningham, 2011) (Lazzaro, 2004; Zichermann & Cunningham, 2011) (Dreyfus & Dreyfus, 1980; Zichermann & Cunningham, 2011) eory (Czikszentmihalyi, 1989; Zichermann & Cunningham, 2011) With the design of the system as a major focus of this research, determining the success of the design in aligning with SDT considerations was important. Additionally, looking at performance ind icators along with employee feedback about the achievements provide d insight into the overall success of the design and future considerations. Chapter 6 begins by discussing the major findings that emerged from the data along with several additional findings A discussion of design considerations follows that

PAGE 147

147 takes into account entries from the research journal, along with employee feedback and results from the IMI survey. Employee performance is also discussed in the co ntext of specific performance indicators and employee self reported feedback. Implications of these results are discussed in the context of the specific research context along with the field as a whole. A discussion of both implications and significance of the research follows Chapter 6 concludes with suggestions for future research and a n overall summary of the research Discussion of Findings Two major findings emerged through an analysis of the results discussed in Chapter 5 The intentional design of the system using the Self Determined Gamification framework, which incorporated design best practices and relevant theory, was successful in creating a system that aligns with more autonomous forms of motivation on the autonomy continuum within Self Determ ination Theory. In addition, e mployees indicated specific job requirements where their performance was impacted positively by the achievements P erformance metrics indicated improvements in several areas after achievements were implemented, particularly in performance areas where employees could go above and beyond and where they had control in completing the requirement Several additional findin gs are also worth mentioning E mployees responded positively to the achievements and expressed enjoying the new feature. They expressed high levels of interest and enjoyment, perceived competence and perceived choice, and low levels of pressure and tension regarding interacting with the achievements. Employees also engaged in targeting and goal setting when interacting with the achievements and highlighted the incentivizing nature of the achievements. These findings are encouraging, but it is also important to note that d esigning a system

PAGE 148

148 of this kind is an iterative process and additional adjustments are needed around achievement benchmarking in particular. Along those lines, c hanges to an organization and changes to specific job requirements need to be add ressed and reflected in the achievement system over time through the iterative nature of the design process. Gamification Design Gamification Design Overview The gamification within the Techworks Portal was designed very intentionally using the Self Dete rmined Gamification framework. Here, an approach rooted in Self Determination theory incorporated considerations for an achievement system that would organizational goals with the hope that staff would internalize these goals. The framework also sought to incorporate design best practices in selecting game elements and designing specific achievements that addressed the needs of different video game player archetypes, different levels of skill in the job, and different types of fun. Underlying this design was the hope that the difficulty of any task would maintain a balance with the degree of interest an individual might have. The overall goal was to create a personaliz ed experience that facilitated a more enjoyable work environment. Gamification Design Staff Perceptions and SDT Alignment Results from the open ended responses indicated that staff enjoyed the achievements and found them both fun and interesting. This i s consistent with early gamification research where self reported feedback is frequently positive about gamification implementations (Hamari et al., 2014). Staff pointed to specific areas of their work that changed as a result of interacting with the achie vements and indicated that these were positive changes. At the same time, staff referenced the goal setting

PAGE 149

149 behavior that is characteristic of more autonomous forms of extrinsic and intrinsic motivation on the autonomy continuum within Self Determination T heory. Here, SDT identifies two forms of more autonomous extrinsic motivation, identified regulation and integrated regulation ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). Integrated regulation sees organizational and personal goals as the motivation contingen cy, where identified regulation sees the internalization of those organizational goals on the part of the individual as the motivation contingency ( Gagn & Deci, 2005 ; Ryan & Deci, 2000). Staff also self reported feelings of motivation and feeling that the achievements legitimized some of the work they perform. Numerous studies found similar results, in non work contexts, with participants indicating that badges or achievements incentivized or motivated a specific behavior ( Anderson et al., 2013; Cheong et al., 2013; Conaway & Garay, 2014; Cruz & Penley, 2014; De Schutter & Abeele, 2014; Denny, 2013; Domng uez et al., 2013; Grant & Betts, 2013 ). These findings support other research that postulated gamification in the workplace might increase employee mo tivation, particularly when aligned with SDT considerations (Cardador et al., 2017; Perryer et al., 2016). The open ended responses were encouraging regarding the design of the overall achievements and they were supported by the Likert responses on the IMI survey. Here, participants reported high levels of interest and enjoyment with the achievements, along with high levels of competence in using the achievements. High levels of interest and enjoyment in a workplace setting characterize a work environment m ore conducive to autonomous extrinsic motivation, according to SDT ( Gagn & Deci, 2005 ; Ryan & Deci, 2000 ). Similarly, high levels of competence in a workplace setting is an indicator

PAGE 150

150 that employees are displaying more autonomous motivation on the autonomy continuum ( Gagn & Deci, 2005 ; Ryan & Deci, 2000 ). Staff also indicated a high degree of perceived choice when interacting with the achievements, implying that the decision to do so did not feel forced. Participants indicated low levels of pressure or con trol in interacting with the achievements. The presence of high pressure or controlling characteristics in a work environment is associated with less motivating workplace settings and is an indicator of a motivation at work ( Gagn & Deci, 2005 ; Ryan & Deci 2000). The lack of pressure or control along with the perceived degree of choice in interacting with the achievements suggests that staff were motivated but did not feel forced to do so. These findings are encouraging and point to the importance of inten tionally designing systems of this kind. On all four factors tracked by the IMI Survey, participant responses indicate alignment with the autonomous side of the autonomy continuum. SDT considerations during the design process were incredibly important in d esigning a system that staff were comfortable with. Gamification Design Additional Theory Alignment Considerations I nformal interactions with staff concerning the achievements were rare, though the research journal pointed to two interactions that indic ated staff were interested in attaining achievements and curious about the impact of certain work as they related to achievement attainment. Staff appeared to be continuing to work as normal, with most interactions recorded in the journal focusing on work related tasks or questions around standard end of year considerations such as graduation, move out procedures, and reference requests. Journal entries noted this lack of discussion on multiple occasions throughout the notes on implementation, with only two notable exceptions. In the first, a staff member mentioned the achievements during an office shift. Here the staff member

PAGE 151

151 asked if there were any hidden or secret achievements other than the ones visible in the portal and indicated this would be a fun add ition to the system. This behavior is characterized in the literature by the explorer player type, where an individual is interested in exploring a game or system and finding aspects of a game that others might not discover (Bartle, 1996; Zichermann & Cunn ingham, 2011). In the other exception, a staff member asked after a staff meeting if being late to a staff meeting still counted as attending and if that would impact earning related achievements. Here, the staff member displayed characteristics of the ach iever player type, where an individual is interested in meeting goals within a game and completing tasks (Bartle, 1996; Zichermann & Cunningham, 2011). Other open ended responses through the survey also pointed to characteristics from each of the four play er types. While evidence of the four player types were apparent through the survey responses and research journal, other theoretical considerations in the Self Determined Gamification framework were less apparent. References to the achievements being fun, for example, were difficult to map to the four types of fun. Gamification Design Iterative and Continuous Improvement S ome participants mentioned the potential need to adjust requirements for some of the more difficult achievements to attain. This was also reflected in the analysis of performance metrics where achievement requirements for reactive support may need to be lo wered. This includes technical forum posts, assigned reactive support tickets, ad hoc reactive support tickets and reactive mobile support site tickets. Adjustments of this kind are expected and reflect the iterative nature of a design process of this kind Research also points to the need to make additional changes to the design of a system after a gamification implementation ( Attali & Arieli Attali, 2014; Cruz & Penley, 2014; De

PAGE 152

152 Schutter & Abeele, 2014; Farzan & Brusilovsky, 2011; Fitz Walter et al., 2011 ; Hakulinen et al., 2013; Hanus & Fox, 2015; Osipov, Nikulchev, et al., 2015 ). The research journal highlighted these considerations as well The initial design of the system set the highest achievement benchmarks based on historical examples of exemplary performance. Additional achievements were scaled up to that benchmark with gradually increasing levels of difficulty that could be attained over time. Throughout the journal, during the design of the system, the difficulty in appropriately aligning the ach ievement benchmarks with a reasonable performance expectation arose as a concern. The primary issue arose around outliers with inflated examples of performance expectations. This occurred when an employee worked for a significantly longer period of time, s uch as an individual who worked for the organization for all eight semesters during their undergraduate career, and then continued to work during graduate school. Other mitigating circumstances that impacted exemplary examples of employee performance inclu ded major events, such as a software rollout or virus outbreak, or contextual changes, such as an increased need for event assistance prior to the permanent installation of technology in the halls that addressed the same need. Ultimately, a decision to rem ove outliers and focus on reasonable performance expectations lowered the required benchmarks and leveled the achievement requirement conditions in a number of categories, such as reactive support tickets and event assists. Though not directly related to the research, the journal entries frequently included thoughts around the changing nature of the work environment. Where early work centered on reactive support, 2014 saw the addition of more proactive measures in

PAGE 153

153 supporting the technology infrastructure i n the halls. Practices like door to door hall sweeps and performing routine technology checklists saw the need for reactive support decrease. This trend, and the need to consider the maturing nature of the IT environment in designing the achievements was a consistent theme expressed as a concern throughout the research journal. Both during the design and the implementation, this theme emerged as a consideration, along with the expressed need for an iterative process of reflection and redesign as an organiza tion changes. Along those lines, the reflection on additional services that might be offered by the organization, along with services that were retired in the past came up as a consideration. These entries reflected the need to be able to add achievements that correspond to new job requirements, along with retiring legacy achievements when a service is no longer offered and is no longer a job requirement for employees. Other design suggestions from the open ended responses may be motivating for some, but o verly competitive and potentially de motivating for others. Particularly the suggestion of incorporating a leaderboard could be viewed as adding pressure to an otherwise low pressure work environment This is consistent with findings from other studies whi ch suggested some individuals found leaderboards to be demotivating or controlling while others were highly motivated by the competition (Hanus & Fox, 2015 ; Witt et al., 2011 ) Gamification Design Summary The design process in this research benefitted greatly from considerations outlined by Self Determination Theory and the MDA Game Design framework. The Self Determined Gamification framework used in this research led to a system that staff enjoyed Staff also pointed to specific areas of their work tha t were influenced by the

PAGE 154

154 achievements and the performance data reflects these changes. At the same time, additional design considerations point to the need for a cyclical and iterative design process that builds on prior findings and that can adapt to a ch anging work environment. Continuing to refine the Self Determined Gamification framework will be essential, but initial results suggest it outlines key considerations for designing gamification in a workplace setting. Employee Performance With a gamification layer in place in January of 2017, 18 performance metrics were tracked over a corresponding time period in 2017, 2016 and 2015. In total, twelve of the fourteen performance indicators where an employee had control in completing the job requirement or where the employee had the opportunity to go above and beyond expectations saw some improvement in 2017 when compared to both 2016 and 2015. Those twelve improvement areas include performance feedback submissions, proactive equipment checkli sts, proactive hall sweeps, office shifts, late notices, mobile support event attendance, portal logs by day, positive customer service surveys, negative customer service surveys, event assistance, monthly evaluation completion, and area meeting attendance These performance improvements point to the potential of achievements in incentivizing completion of job requirements in a workplace setting when employees have control and the ability to exceed performance expectations. Interestingly, the four performan ce indicators where an employee could not go above compared to 2015. This could be due to the general trend of an increase in proactive support and a decrease in reactive s upport. It could also be the case that aspects of an

PAGE 155

155 above and beyond expectations will naturally show less improvement. Self reported feedback from the open ended responses pertaining to the achievements indicated that some staff logged into the portal to see job responsibilities more frequently and that some staff targeted specific achievements by completing hall sweeps early or signing up for extra events. The per formance metric data reflects this in that the 2017 time frame saw more portal log ins, fewer late notices on average per staff member, and an increase in both average service event sign ups and average event assist sign ups. The reduction in late notices is noteworthy in particular, as it reflects an overall improvement in performance by completing work within the expected timeframe. Performance issues with staff often coincide with a high number of late notices. For performance metrics within a staff memb That said, certain areas remained fairly consistent over time, such as attendance at staff meetings and the number of critical notices received on average. This may be an indicator that those metrics are less l ikely to change from semester to semester. For and their subsequent ability to attend. Open ended responses pointed to a positive experience when interacting with achievemen ts within the web portal. Participants described the achievements as fun, cool, interesting, and neat. Participants indicated being impressed by the new feature and even suggested it made aspects of the job more enjoyable. Participants also indicated chang ing their work practices through the achievement interface, both referencing setting goals and targeting specific achievements. Responses indicated that

PAGE 156

156 the achievements had a motivating effect in some cases, or in one response, no impact on performance. R esponses also indicated that the achievements helped recognize the work being completed and encouraged staff to log into the web portal to check requirements. P erceptions of the online work environment after the incorporation of a gamification layer were primarily positive in nature. In one case, a participant indicated not interacting with the achievements, but suggested this had no impact on performance or comp leting work requirements. No responses indicated a negative perception of the online work environment or the achievements, outside of proposed adjustments to color schemes or iconography. In the end, results point to improved performance in several key ar eas after the achievement implementation. In particular, when a performance indicator was within an expectations, performance improvement was frequently present when looking at staff averages. Anonymous, self reported feedback indicated that some staff were more motivated or targeted specific aspects of their work through interacting with the achievement interface The performance metrics indicated that this was true in some cases as well. This sparks several suggestions for future research, discussed later in Chapter 6 Implications and Significance Context Specific Implications and Significance This study sought to address a problem of practice in a work environment where employees c omplete job responsibilities remotely with minimal supervision. Results from this research are encouraging in this regard and the achievements worked particularly well in giving staff autonomy in selecting areas where they wanted to focus

PAGE 157

157 efforts The desi gn of the gamification layer using the Self Determined Gamification framework resulted in a positive work environment and seemed to have a perceived motivating effect on staff in addition to improved performance in several areas The achievement system add resses a problem of practice in the Techworks work environment by providing an incentive for staff to complete job responsibilities. The decrease in the number of late notices is particularly encouraging and suggests that staff are completing work in a mor e timely manner on average resulting in improved service to the client population In addition to addressing a problem of practice, staff within Techworks now have consistent access to performance data and can check their status in relation to the achieve ments at any time. Going forward, achievement benchmarks in certain areas will be adjusted based on staff feedback and an assessment of earned achievements. Results from this study indicate that designing gamification intentionally, utilizing a framework o f this kind can have a positive impact within a work setting and Self Determination Theory is a strong contender to guide this process. As the iterative nature of the design framework is followed, improvements to the system will be realized in future semesters within the Techworks portal The cyclical nature of a student staffing model, along with the improvements to design will create opportunities for both follow up studies and longitudinal studies in this environment. As the achievement structure is adjusted and a process that addresses entropy within the work environment is defined in relation to the achievements there is potential to expand the web portal and achievements to other areas of the university. Particularly in areas where s tudent staff are employed, the web portal could be adjusted to accommodate different work environments with achievements integrated and mapped

PAGE 158

158 to specific job requirements in those contexts. Beyond a focus on student employees, the portal could be adjusted and tested at the university with full time staff or in non work contexts. Broader Implications and Significance To date many examples of gamification in the literature do not specify a formal design approach, design best practices, or theoretical consi derations. In conducting this research, the Self Determined Gamification framework resulted in a standardized approach that incorporates both design best practices and relevant theory. This framework represents a significant contribution to the field by pr oviding both a model for designing a system of this kind and a method for evaluating that design. Here, researchers of performance improvement, gamification, or workplace motivation, have the opportunity to use a tested framework that moves beyond theoreti cal considerations and offers a practical example of those considerations in practice. The success of this framework in designing a gamified system highlights the importance of SDT considerations within a workplace setting such as autonomy, equity and both organization and individual goals. Continuing to use this approach to design other gamification implementations will help refine and improve the model At the same time, several studies have suggested the potential of gamification in a workplace setting, but few studies exist that actually test that potential. This study addresses a gap in the literature by laying the groundwork for both gamification design in a workplace setting and the specific application of game elements as they relate to completion of job responsibilities. Findings from this study suggest that gamification can improve employee motivation and incentivize work. From a performance improvement standpoint, this

PAGE 159

159 intervention represents an additional environmental support that employers can leverage to help motivate staff to set individual goals and potentially internalize organizational goals. This study also suggests that employee s having control over the c ompletion of job requirements and the opportunity to exceed expectations in completing job requirements are both important considerations for anyone designing an achievement structure. Where possible, adjusting job requirements to accommodate these two con siderations, in conjunction with the design of gamification can yield improved performance. In a work setting achievements offer a non monetary incentive structure that can grow and adjust over time based on the changing needs of an organization Employer s have an opportunity to realize real performance improvement through this incentive structure, which is particularly relevant when monetary incentives are not available as a means of recognizing the work employees complete Suggestions for Future Researc h It is important to note that the impact and positive reception of gamification in a workplace setting may be unique to college age student employees. The positive interactions expressed by participants may also be a reflection of the novelty effect or a predisposition for gamification in general Additionally, there are several variables outside of the gamification implementation that could have impacted performance metrics. With these considerations in mind, more research in this area is needed. While th e results of this research are encouraging, there are several potential areas for additional research that could build off this approach. To begin with, this study took place over a three month time period. Although retrospective data allowed for compariso ns to be made over time, a more longitudinal study is needed within the field. With any study in a workplace setting, particularly one that changes frequently, it is

PAGE 160

160 difficult to sufficiently isolate a variable like a gamification implementation. A longitu dinal study would help control for this by examining the potential impact over a longer period of time. The Self Determined Gamification framework helped organize the design process and ground the approach in relevant theory. The results were primarily pos itive, but the use of this framework in other contexts, with other populations, and with other game elements would help evaluate the framework further. Additional studies that make use of the framework will help improve and adjust the overall approach and the need to focus on specific aspects of the framework. For example, while incorporating the SAPS reward system and the Four Types of Fun may have helped create a more balanced system overall, the open ended responses made no reference to status when discu ssing the achievements and references to fun were difficult to link to a specific categorization of fun. These design considerations may be present, but unnoticed by staff, or they may be less important than some of the other design considerations. Buildin g out an assessment to track specific design considerations used in the framework could help determine the relative need for each consideration. Designing a gamification study utilizing the Self Determined Gamification framework that allows for inferentia l statistics would also be beneficial to the field. Here, either tracking individual performance changes over time, instead of staff averages, or comparing groups without any staff overlap would allow for inferential statistics to be run. This would help i n determining if any performance improvements were also statistically significant.

PAGE 161

161 Follow up studies represent another area of future research in the field that would be incredibly beneficial. The Self Determined Gamification framework offers an iterative design process for designing gamification implementations. Though beyond the scope of this study, conducting research using a similar method to the one described here, along with a follow up study that incorporates feedback and improvements to the system would help mature the field and identify any potential impacts of gamified systems. Summary This study sought to address a problem of practice in a specific work context. At the same time, it attempted to identify a gap in the literature and address that gap through the incorporation of a formalized design process for gamification grounded in app ropriate theory in a workplace setting Within the context of the problem of practice, the gamification implementation had positive results with staff self reporting improved work practices and performance metrics reflecting this improvement. The study als o provided a meaningful contribution to the field through the development of the Self Determined Gamification framework Here, it was demonstrated that through utilizing design best practices and relevant theory, gamification can be designed intentionally and lead to both a positive user experience and real performance improvement. The framework not only offered a model for designing a gamification solution, but also offered a method of evaluating that model. Through a mixed methods approach, data from the IMI survey, open ended responses, and performance metrics over time combined to provide a comprehensive assessment of the gamification implementation which can help guide future assessments of this kind. SDT considerations in the work environment pertainin g to autonomy, equity, organizational goals and individual goals,

PAGE 162

162 in addition to the need for employees to have control in completing job requirements and the opportunity to exceed expectations all emerged as areas of focus when designing gamification in a work environment. Several opportunities to expand on this research also emerged where the Self Determined Gamification framework can be refined and improved through follow up studies or gamification implementations in other contexts. While additional rese arch might help determine if all aspects of the framework are needed and the extent to which there is a causal relationship between the introduction of achievements and changes to performance the results of this study are encouraging and build upon prior efforts in the field

PAGE 163

163 APPENDIX A ORGANIZATIONAL GOALS Table A 1. Organizational g oals Goal a bbreviation Goal d etail A To enhance the educational environment within the residence halls through technology B To enhance the entertainment opportunities within the residence halls through technology C To provide quality customer service to every client through timely and reliable on site technology support D To support fellow Techworks staff through timely communication, clear expectations, and mentoring

PAGE 164

164 APPENDIX B MAPPING ORGANIZATIONAL GOALS TO JOB REQUIREMENTS Table B 1. Mapping organizational goals to job r equirements Job r equirement Job c ategory Organizational g oal customer service surveys received customer service C technical forum posts tech forum A, B, C, D assigned reactive support tickets support tickets A, B, C ad hoc reactive support tickets support tickets A, B, C mobile support site support tickets support tickets A, B, C performance feedback submissions general D proactive equipment checklists other support A, B proactive hallsweeps other support A, B, C office shifts other support C, D event assists other support A, B, C monthly evaluations evaluations D late notices received general C critical notices received general C mobile support site events attended events A, B, C, D area meetings attended events D staff meetings attended events D portal logs by day general C, D

PAGE 165

165 APPENDIX C ACHIEVEMENT LIST AND DYNAMICS MAPPING Table C 1. Achievement list and dynamics mapping Achievement t itle Threshold ML PT F Tech Guru Receive a Tech Guru End of Semester Award E K, A, S H Community Builder Receive a Community Builder End of Semester Award E K, A, S H Customer Service FISH Receive a Customer Service FISH end of semester award E K, A H Team Player Receive a Team Player end of semester award E K, A, S H I'll Be There Receive an I'll Be There end of semester award E K, A H Tech of the Semester Receive a Tech of the Semester award M K, A H Rookie of the Semester Receive a Rookie of the Semester Award M K, A H Senior Tech of the Semester Receive a Senior Tech of the Semester award M K, A H Awarded! Receive a monthly award PS K, A H Awarded Again! Receive more than one monthly award in a semester E K, A H Notable Recognition Receive 5 or more monthly awards while working for Techworks M K, A H Retire My Jersey Receive any combination of Tech Senior Tech or Rookie of the Semester (or multiple of one) V K, A H Well Rounded Receive any combination of semester awards during multiple semestesrs (e.g. Tech Guru and Community Builder) M K, A H Consistently Consistent Receive any single semester award during multiple semesters (e.g. Tech Guru multiple times) M K, A H Helping Hand Receive your first positive customer service survey N K, S E Helping Hand x5 Receive 5 positive customer service surveys PS K, S E Helping Hand x10 Receive 10 positive customer service surveys PS K, S E Helping Hand x25 Receive 25 positive customer service surveys PS K, S S Helping Hand x50 Receive 50 positive customer service surveys E K, S S

PAGE 166

166 Table C 1. Continued Achievement t itle Threshold ML PT F Helping Hand x100 Receive 100 positive customer service surveys E K, S S Helping Hand x175 Receive 175 positive customer service surveys M K, S S Helping Hand x250 Receive 250 positive customer service surveys V K, S S Customer Driven Receive 20 positive customer service surveys in a single semester M K, S S Customer Focused Receive 40 or more positive customer service surveys in a single semester V K, S S Your first review! Attend your first midterm review N A, S, E P More Midterms x 2! Attend 2 midterm reviews PS A, S P More Midterms x 4! Attend 4 midterm reviews E A, S P More Midterms x 6! Attend 6 midterm reviews M A, S P More Midterms x 8! Attend 8 midterm reviews V A, S P Your First Report! Complete your first monthly report N K, A, E E Reporting in x 6 Complete 6 monthly reports PS K, A E Reporting in x 12 Complete 12 monthly reports E K, A E Reporting in x 18 Complete 18 monthly reports M K, A E Reporting in x 24 Complete 24 monthly reports V K, A E Full Report Complete all monthly reports for a single semester E K, A E An Outstanding Month! Receive an overall evaluation of Outstanding on a Monthly Report M K, A H An Outstanding Semester! Receive an overall evaluation of Outstanding for every month in a semester M K, A H You're Outstanding! Receive an overall evaluation of Outstanding on a Semester Evaluation M K, A H You've Been Great x 2! Receive 2 overall evaluations of Very Good or Outstanding on a Semester Evaluation M K, A H You've Been Great x 4! Receive 4 overall evaluations of Very Good or Outstanding on a Semester Evaluation M K, A H You've Been Great x 6! Receive 6 overall evaluations of Very Good or Outstanding on a Semester Evaluation V K, A H You've Been Great x 8! Receive 8 overall evaluations of Very Good or Outstanding on a Semester Evaluation V K, A H

PAGE 167

167 Table C 1. Continued Achievement t itle Threshold ML PT F Your first evaluation! Complete your first semester evaluation N K, A, E E Thanks for the feedback x 2 Complete 2 semester evaluations PS K, A E Thanks for the feedback x 4 Complete 4 semester evaluations E K, A E Thanks for the feedback x 6 Complete 6 semester evaluations M K, A E Thanks for the feedback x 8 Complete 8 semester evaluations V K, A E Meeting Master Attend 24 staff meetings and 24 area meetings V K, A, S P Your First Area Meeting Attend you first area meeting N A, E, S P Meet and Greet Attend all the area meetings for your region in a semester N K, A, S P Assembling the Area x 4 Attend 4 area meetings N K, A, S P Assembling the Area x 8 Attend 8 area meetings PS K, A, S P Assembling the Area x 16 Attend 16 area meetings E K, A, S P Assembling the Area x 24 Attend 24 area meetings M K, A, S P Your first support site! Attend your first move in support site N E, S P So Much Support! Attend more than 2 move in support sites in a single semester M K, A, S P Move in Support x2! Attend 2 move in support sites PS A, S P Move in Support x 4! Attend 4 move in support sites E A, S P Move in Support x 6! Attend 6 move in support sites M A, S P Move in Support x 8! Attend 8 move in support sites V A, S P A Life of Service Attend your first service event N A, E, S P A Life of Service x 7 Attend 7 service events PS A, E, S P A Life of Service x 14 Attend 14 service events PS A, E, S P A Life of Service x 28 Attend 28 service events E A, E, S P

PAGE 168

168 Table C 1. Continued Achievement t itle Threshold ML PT F A Life of Service x 42 Attend 42 service events M A, E, S P A Life of Service x 56 Attend 56 service events V A, E, S P You've Been Served! Attend 10 service events in a single semester M K, A, E, S P You've Been Served x 14! Attend 14 service events in a single semester V K, A, E, S P Your First Tech Gathering Attend your first staff meeting N A, E, S P A Meetingful Semester Attend all the staff meetings in a semester E K, A, S P Tech Gathering x 6 Attend 6 staff meetings N A, S P Tech Gathering x 12 Attend 12 staff meetings PS A, S P Tech Gathering x 18 Attend 18 staff meetings E A, S P Tech Gathering x 24 Attend 24 staff meetings M A, S P Welcome to Techworks Attend your first training session N A, E, S P Trained Attend all required training sessions during your first semester PS K, A, S P Around the Block Live in three Techworks Areas while working for Techworks E A, E E Housed Live in four Techworks Areas while working for Techworks M A, E E Getting to Know You Fill out all optional fields in your profile PS A, E E Dependable Receive no critical notices for a semester E K, A H Steady as a Rock Receive no critical notices for four or more semesters M K, A H Steadfast Receive no late notices or critical notices for a semester M K, A H Unswerving Receive no late or critical notices for four or more semesters V K, A H Reliable Receive no late notices for a semester E K, A H Unfailing Receive no late notices for four or more semesters M K, A H Welcome Log into the portal for the first time N E E Logged in x 5 Log into the portal 5 times N K, A E Logged in x 10 Log into the portal 10 times N K, A E

PAGE 169

169 Table C 1. Continued Achievement t itle Threshold ML PT F Logged in x 25 Log into the portal 25 times N K, A E Logged in x 50 Log into the portal 50 times N K, A E Logged in x 100 Log into the portal 100 times PS K, A E Logged in x 200 Log into the portal 200 times PS K, A E Logged in x 300 Log into the portal 300 times PS K, A E Logged in x 400 Log into the portal 400 times PS K, A E Logged in x 500 Log into the portal 500 times E K, A E Logged in x 600 Log into the portal 600 times E K, A E Logged in x 700 Log into the portal 700 times E K, A E Logged in x 800 Log into the portal 800 times E K, A E Logged in x 900 Log into the portal 900 times M K, A E Logged in x 1000 Log into the portal 1000 times M K, A E Logged in x 1250 Log into the portal 1250 times M K, A E Logged in x 1500 Log into the portal 1500 times V K, A E Shout Out! Leave a performance feedback about another staff member N A, E, S E Shout Out x 5! Leave 5 performance feedbacks about other staff members N A, S E Shout Out x 10! Leave 10 performance feedbacks about other staff members PS A, S E Shout Out x 25! Leave 25 performance feedbacks about other staff members PS A, S E Shout Out x 50! Leave 50 performance feedbacks about other staff members E A, S E Shout Out x 75! Leave 75 performance feedbacks about other staff members E A, S E Shout Out x 100! Leave 100 performance feedbacks about other staff members M A, S E Shout Out x 200! Leave 200 performance feedbacks about other staff members M A, S E Shout Out x 300! Leave 300 performance feedbacks about other staff members V A, S E Thorough Feedback x 10 Leave 10 performance feedbacks about other staff members in a single semester E A, S E Thorough Feedback x 25 Leave 25 performance feedbacks about other staff members in a single semester M A, S E Thorough Feedback x 50 Leave 50 performance feedbacks about other staff members in a single semester V A, S E

PAGE 170

170 Table C 1. Continued Achievement t itle Threshold ML PT F Profile Updated Update your profile N K, A, E E Change of Scenery Live in two or more residence halls while working for Techworks PS A, E E New Recruit Start your first semester N A E Returner Start your third semester working for Techworks PS A E Returner Again Start your fifth semester working for Techworks M A E Veteran Start your seventh semester working for Techworks V A E Your First Sweep! Complete your first hallsweep N A, E, S P Swept Away x 25 Complete 25 hallsweeps N A, E, S P Swept Away x 50 Complete 50 hallsweeps N A, E, S P Swept Away x 100 Complete 100 hallsweeps PS A, E, S P Swept Away x 250 Complete 250 hallsweeps E A, E, S P Swept Away x 500 Complete 500 hallsweeps M A, E, S P Swept Away x 1000 Complete 1000 hallsweeps M A, E, S P Swept Away x 2000 Complete 2000 hallsweeps V A, E, S H Sweeping Up the Halls Complete all hallsweeps assigned to you in a semester M K, A, E, S P The Grim Sweeper x 50 Complete 50 hallsweeps in a single semester PS K, A, E, S P The Grim Sweeper x 100 Complete 100 hallsweeps in a single semester E K, A, E, S P The Grim Sweeper x 200 Complete 200 hallsweeps in a single semester M K, A, E, S H The Grim Sweeper x 400 Complete 400 hallsweeps in a single semester V K, A, E, S H Community Presence Complete your first housing assist N A, E, S P Community Presence x3 Complete 3 housing assists PS A, E, S P Community Presence x6 Complete 6 housing assists E A, E, S P Community Presence x12 Complete 12 housing assists M K, A, E, S H Community Presence x25 Complete 25 housing assists V K, A, E, S H Friend of Housing Complete more than one housing assist in a single semester E K, A, E, S H Your First Shift! Complete your first office shift N A, E E Shifting Gear x 6 Complete 6 office shifts PS A E Shifting Gear x 12 Complete 12 office shifts PS A E

PAGE 171

171 Table C 1. Continued Achievement t itle Threshold ML PT F Shifting Gear x 24 Complete 24 office shifts E A E Shifting Gear x 36 Complete 36 office shifts M A E Shifting Gear x 40 Complete 40 office shifts V A E Overshifting x 10 Complete 10 shifts in one semester M K, A E Overshifting x 20 Complete 20 shifts in one semester V K, A S Your First Rounds! Complete your first set of rounds N A, E E Rounding x 3 Complete 3 sets of rounds PS A, E E Rounding x 5 Complete 5 sets of rounds PS A, E E Rounding x 10 Complete 10 sets of rounds PS A, E E Rounding x 15 Complete 15 sets of rounds PS A, E E Rounding x 25 Complete 25 sets of rounds E A, E E Rounding x 35 Complete 35 sets of rounds E A, E E Rounding x 45 Complete 45 sets of rounds M A, E E Rounding x 55 Complete 55 sets of rounds M A, E E Rounding x 75 Complete 75 sets of rounds V A, E E In a Round About Way Complete all rounds assigned to you for a semester M K, A, E E Round House Kick Complete 10 sets of rounds in a single semester M K, A, E E Round House Kick x 15 Complete 15 sets of rounds in a single semester V K, A, E S Task Completionist Complete a task in task central other than an office shift or weekly rounds N K, A, E E Task Completionist x 5 Complete 5 tasks in task central other than an office shift or weekly rounds E K, A, E S Task Completionist x 10 Complete 10 tasks in task central other than an office shift or weekly rounds M K, A, E S Your first ad hoc! Create your first ad hoc ticket N A, E P 10 ad hocs! Work on 10 ad hoc tickets N K, A, E P 20 ad hocs! Work on 20 ad hoc tickets N K, A, E P 30 ad hocs! Work on 30 ad hoc tickets PS K, A, E H 40 ad hocs! Work on 40 ad hoc tickets PS K, A, E H 50 ad hocs! Work on 50 ad hoc tickets PS K, A, E H 75 ad hocs! Work on 75 ad hoc tickets E K, A, E H 100 ad hocs! Work on 100 ad hoc tickets E K, A, E H 125 ad hocs! Work on 125 ad hoc tickets E K, A, E H 150 ad hocs! Work on 150 ad hoc tickets E K, A, E H 175 ad hocs! Work on 175 ad hoc tickets E K, A, E H 200 ad hocs! Work on 200 ad hoc tickets E K, A, E H 225 ad hocs! Work on 225 ad hoc tickets E K, A, E H

PAGE 172

172 Table C 1. Continued Achievement t itle Threshold ML PT F 250 ad hocs! Work on 250 ad hoc tickets E K, A, E H 275 ad hocs! Work on 275 ad hoc tickets E K, A, E H 300 ad hocs! Work on 300 ad hoc tickets E K, A, E H 325 ad hocs! Work on 325 ad hoc tickets M K, A, E H 350 ad hocs! Work on 350 ad hoc tickets M K, A, E H 400 ad hocs! Work on 400 ad hoc tickets M K, A, E H 450 ad hocs! Work on 450 ad hoc tickets M K, A, E H 500 ad hocs! Work on 500 ad hoc tickets M K, A, E H 25 ad hocs in a semester! Work on 25 ad hoc tickets in a single semester N K, A, E P 50 ad hocs in a semester! Work on 50 ad hoc tickets in a single semester PS K, A, E P 75 ad hocs in a semester! Work on 75 ad hoc tickets in a single semester PS K, A, E H 100 ad hocs in a semester! Work on 100 ad hoc tickets in a single semester E K, A, E H 125 ad hocs in a semester! Work on 125 ad hoc tickets in a single semester E K, A, E H 150 ad hocs in a semester! Work on 150 ad hoc tickets in a single semester M K, A, E H 175 ad hocs in a semester! Work on 175 ad hoc tickets in a single semester M K, A, E H 200 ad hocs in a semester! Work on 200 ad hoc tickets in a single semester V K, A, E H Ticket Master x 10 Work on 10 tickets in one semester N A, E H Ticket Master x 25 Work on 25 tickets in one semester N A, E H Ticket Master x 50 Work on 50 tickets in a single semester PS A, E H Ticket Master x 100 Work on 75 tickets in a single semester PS K, A, E H Ticket Master x 150 Work on 100 tickets in a single semester E K, A, E H Ticket Master x 200 Work on 125 tickets in a single semester E K, A, E H Ticket Master x 250 Work on 150 tickets in a single semester M K, A, E H Ticket Master x 300 Work on 175 tickets in a single semester M K, A, E H Ticket Master x 400 Work on 200 tickets in a single semester V K, A, E H First Client! Work on your first ticket N A, E P 25 Ticket Tech Work on 25 Techworks tickets N A, E P 50 Ticket Tech Work on 50 Techworks tickets N A, E P

PAGE 173

173 Table C 1. Continued Achievement t itle Threshold ML PT F 75 Ticket Tech Work on 75 Techworks tickets PS K, A, E H 100 Ticket Tech Work on 100 Techworks tickets PS K, A, E H 125 Ticket Tech Work on 125 Techworks tickets PS K, A, E H 150 Ticket Tech Work on 150 Techworks tickets E K, A, E H 175 Ticket Tech Work on 175 Techworks tickets E K, A, E H 200 Ticket Tech Work on 200 Techworks tickets E K, A, E H 225 Ticket Tech Work on 225 Techworks tickets E K, A, E H 250 Ticket Tech Work on 250 Techworks tickets E K, A, E H 275 Ticket Tech Work on 275 Techworks tickets E K, A, E H 300 Ticket Tech Work on 300 Techworks tickets E K, A, E H 350 Ticket Tech Work on 350 Techworks tickets E K, A, E H 400 Ticket Tech Work on 400 Techworks tickets E K, A, E H 450 Ticket Tech Work on 450 Techworks tickets E K, A, E H 500 Ticket Tech Work on 500 Techworks tickets M K, A, E H 600 Ticket Tech Work on 600 Techworks tickets M K, A, E H 700 Ticket Tech Work on 700 Techworks tickets M K, A, E H 800 Ticket Tech Work on 800 Techworks tickets M K, A, E H 900 Ticket Tech Work on 900 Techworks tickets M K, A, E H 1000 Ticket Tech Work on 1000 Techworks tickets V K, A, E H Poster Child x5 Complete 5 tech forum posts in a single semester N A, S E Poster Child x10 Complete 10 tech forum posts in a single semester N A, S E Poster Child x20 Complete 20 tech forum posts in a single semester PS A, S E Poster Child x30 Complete 30 tech forum posts in a single semester E K, A, S S Poster Child x40 Complete 40 tech forum posts in a single semester M K, A, S S Poster Child x50 Complete 50 tech forum posts in a single semester V K, A, S S First Post! Complete your first tech forum post N A, E, S E 25 posts Complete 25 tech forum posts PS A, S E 50 posts Complete 50 tech forum posts E K, A, S S 75 posts Complete 75 tech forum posts E K, A, S S 100 posts Complete 100 tech forum posts M K, A, S S 125 posts Complete 125 tech forum posts M K, A, S S 150 posts Complete 150 tech forum posts M K, A, S S 175 posts Complete 175 tech forum posts M K, A, S S 200 posts Complete 200 tech forum posts M K, A, S S 250 posts Complete 250 tech forum posts M K, A, S S 300 posts Complete 300 tech forum posts V K, A, S S

PAGE 174

174 Table C 1. Continued Achievemen t t itle Threshold ML PT F Tech Guru Receive a Tech Guru Award E K, A, S H Community Builder Receive a Community Builder End of Semester Award E K, A, S H Customer Service FISH Receive a Customer Service FISH end of semester award E K, A H Team Player Receive a Team Player end of semester award E K, A, S H I'll Be There Receive an I'll Be There end of semester award E K, A H Tech of the Semester Receive a Tech of the Semester award M K, A H Rookie of the Semester Receive a Rookie of the Semester Award M K, A H Senior Tech of the Semester Receive a Senior Tech of the Semester award M K, A H Awarded! Receive a monthly award PS K, A H Awarded Again! Receive more than one monthly award in a semester E K, A H Notable Recognition Receive 5 or more monthly awards while working for Techworks M K, A H Retire My Jersey Receive any combination of Tech, Senior Tech or Rookie of the Semester (or multiple of one) V K, A H Well Rounded Receive any combination of semester awards during multiple semestesrs (e.g. Tech Guru and Community Builder) M K, A H Consistently Consistent Receive any single semester award during multiple semesters (e.g. Tech Guru multiple times) M K, A H Helping Hand Receive your first positive customer service survey N K, S E Helping Hand x5 Receive 5 positive customer service surveys PS K, S E Helping Hand x10 Receive 10 positive customer service surveys PS K, S E Helping Hand x25 Receive 25 positive customer service surveys PS K, S S Helping Hand x50 Receive 50 positive customer service surveys E K, S S Helping Hand x100 Receive 100 positive customer service surveys E K, S S Helping Hand x175 Receive 175 positive customer service surveys M K, S S

PAGE 175

175 Table C 1. Continued Achievement t itle Threshold ML PT F Helping Hand x250 Receive 250 positive customer service surveys V K, S S Customer Driven Receive 20 positive customer service surveys in a single semester M K, S S Customer Focused Receive 40 or more positive customer service surveys in a single semester V K, S S Your first review! Attend your first midterm review N A, S, E P More Midterms x 2! Attend 2 midterm reviews PS A, S P More Midterms x 4! Attend 4 midterm reviews E A, S P More Midterms x 6! Attend 6 midterm reviews M A, S P More Midterms x 8! Attend 8 midterm reviews V A, S P Your First Report! Complete your first monthly report N K, A, E E Reporting in x 6 Complete 6 monthly reports PS K, A E

PAGE 176

176 APPENDIX D ABBREVIATION KEY FOR ACHIEVEMENT LIST AND DYNAMICS MAPPING Table D 1. Abbreviation key for achievement list and dynamics mapping Abbreviation Detail ML Mastery Levels this abbreviation is a title header referencing the five levels of mastery (Dreyfus & Dreyfus, 1980). -N Novice one of the five levels of mastery associated with being new to a system. -PS Problem Solver one of the five levels of mastery associated with minimal experience within a system. -E Expert one of the five levels of mastery associated with beginning to understand a system. -M Master one of the five levels of mastery associated with understanding the system and spending significant time in the system. -V Visionary one of the five levels of mastery associated with identifying new ways to improve the system due to high level of understanding. PT Player Types this abbreviation is a title header referencing the four player types (Bartle, 1996). -K Killers one of the four player types with motivation stemming from engaging with other players and competition. -A Achievers one of the four player types with motivation stemming from completing tasks and goals. -S Socializers one of the four player types with motivation stemming from collaborating with others. -E Explorers one of the four player types with m otivation stemming from exploring different aspects of an environment. F Types of Fun this abbreviation is a title header referencing the four types of fun (Lazzaro, 2004). -E Easy Fun fun associated with open ended curiosity. -S Serious Fun fun associated with open ended relaxation and excitement. -P People Fun fun associated with goal oriented social amusement. -H Hard Fun fun associated with goal oriented triumph over challenges.

PAGE 177

177 APPENDIX E IMI SURVEY (Deci & Ryan, n.d.) TASK EVALUATION QUESTIONNAIRE For each of the following statements, please indicate how true it is for you, using the following scale: 1 2 3 4 5 6 7 not somewhat very at all true true true 1. While I was working on the task I was thinking about how much I enjoyed it. 2. I did not feel at all nervous about doing the task. 3. I felt that it was my choice to do the task. 4. I think I am pretty good at this task. 5. I found the task very interesting. 6. I felt tense while doing the task. 7. I think I did pretty well at this activity, compared to other students. 8. Doing the task was fun. 9. I felt relaxed while doing the task. 10. I enjoyed doing the task very much. 12. I am satisfied with my perform ance at this task. 13. I was anxious while doing the task. 14. I thought the task was very boring. 15. I felt like I was doing what I wanted to do while I was working on the task. 16. I felt pretty skilled at this task. 17. I thought the task was very inte resting. 18. I felt pressured while doing the task. 19. I felt like I had to do the task. 20. I would describe the task as very enjoyable. 21. I did the task because I had no choice. 22. After working at this task for a while, I felt pretty competent.

PAGE 178

178 AP PEN DIX F MODIFIED IMI SURVEY Techworks Portal Achievement Questionnaire For each of the following statements, please indicate how true it is for you, using the following scale: 1 2 3 4 5 6 7 not somewhat very at all true true true 1. While I was working on earning achievements in the Techworks Portal I was thinking about how much I enjoyed it. 2. I did not feel at all nervous about earning achievements in the Techworks Portal 3. I felt that it was my choice to earn achievements in t he Techworks Portal 4. I think I am pretty good at earning achievements in the Techworks Portal 5. I found earning achievements in the Techworks Portal very interesting. 6. I felt tense while earning achievements in the Techworks Portal 7. I think I did pretty well at earning achievements in the Techworks Portal, compared to other employees 8. Earning achievements in the Techworks Portal was fun. 9. I f elt relaxed while earning achievements in the Techworks Portal 10. I enjoyed earning achievemen ts in the Techworks Portal very much. earning achievements in the Techworks Portal 12. I am satisfied with my performance at earning achievements in the Techworks Portal 13. I was anxious while earning achievements in the Techworks Portal 14. I thought earning achievements in the Techworks Portal was very boring. 15. I felt like I was doing what I wanted to do while I was working on earning achievements in the Techworks Portal 16. I felt pretty skilled at earning achievements in the Techworks Portal 17. I thought earning achievements in the Techworks Portal was very interesting. 18. I felt pressured while earning achievements in the Techworks Portal 19. I felt like I had to earn achievements in the Techwo rks Portal 20. I would describe earning achievements in the Techworks Portal as very enjoyable. 21. I earned achievements in the Techworks Portal because I had no choice. 22. After working at earning achievements in the Techworks Portal for a while, I felt pretty competent.

PAGE 179

179 APPENDIX G IMI SURVEY RESPONSES MEANS AND STANDARD DEVIATIONS Table G 1. IMI survey responses means and standard d eviations Survey s tatement Factor Mean Standard d eviation While I was working on earning achievements in the Techworks Portal I was thinking about how much I enjoyed it. Interest and Enjoyment 4.733 1.751 I did not feel at all nervous about earning achievements in the Techworks Portal. Pressure and Tension 6.533 0.743 I felt that it was my choice to earn achievements in the Techworks Portal. Perceived Choice 6.400 0.828 I think I am pretty good at earning achievements in the Techworks Portal. Perceived Competence 5.467 1.187 I found earning achievements in the Techworks Portal very interes ting. Interest and Enjoyment 5.333 1.496 I felt tense while earning achievements in the Techworks Portal. Pressure and Tension 1.467 0.834 I think I did pretty well at earning achievements in the Techworks Portal, compared to other employees. Perceived Competence 4.800 1.474 Earning achievements in the Techworks Portal was fun. Interest and Enjoyment 5.600 1.404 I felt relaxed while earning achievements in the Techworks Portal. Pressure and Tension 6.333 0.816 I enjoyed earning achievements in the Techworks Portal very much. Interest and Enjoyment 5.800 1.373 achievements in the Techworks Portal. Perceived Choice 2.267 1.944 I am satisfied with my performance at earning achievements in the Techworks Portal. Perceived Competence 5.733 1.033 I was anxious while earning achievements in the Techworks Portal. Pressure and Tension 1.533 0.915 I thought earning achievements in the Techworks Portal was very boring. Interest and Enjoyment 2.333 1.291 I felt like I was doing what I wanted to do while I was working on earning achievements in the Techworks Portal. Perceived Choice 5.400 1.242

PAGE 180

180 Table G 1. Continued Survey s tatement Factor Mean Standard d eviation I felt pretty skilled at earning achievements in the Techworks Portal. Perceived Competence 5.133 1.506 I thought earning achievements in the Techworks Portal was very interesting. Interest and Enjoyment 5.333 1.234 I felt pressured while earning achievements in the Techworks Portal. Pressure and Tension 1.467 0.743 I felt like I had to earn achievements in the Techworks Portal. Perceived Choice 2.267 1.534 I would describe earning achievements in the Techworks Portal as very enjoyable. Interest and Enjoyment 5.533 1.187 I earned achievements in the Techworks Portal because I had no choice. Perceived Choice 2.333 1.952 After working at earning achievements in the Techworks Portal for a while, I felt pretty competent. Perceived Competence 5.600 1.121

PAGE 181

181 LIST OF REFERENCES Anderson, A., Huttenlocher, D., Kleinberg, J., & Leskovec, J. (2013). Steering user behavior with badges. In Proceedings of the 22nd international conference on World Wide Web (pp. 95 106). Rio de Janeiro, Brazil Ang, C. S., Zaphiris, P., & Mahmood, S. (2 007). A model of cognitive loads in massively multiplayer online role playing games Interacting with Computers 19(2), 167 179. Attali, Y., & Arieli Attali, M. (2014). Gamification in assessment: Do points affect test performance?. Computers & Education 83, 57 63. Ayres, P., & Paas, F. (2007). Making instructional animations more effective: A cognitive load approach. Applied Cognitive Psychology 21, 695 700. Baard, P. P., Deci, E. L., & Ryan, R. M. (2004). The relation of intrinsic need satisfaction to p erformance and wellbeing in two work settings Journal of Applied Social Psychology 34, 2045 2068. Bajdor, P., & Dragolea, L. (2011). The gamification as a tool to improve risk management in the enterprise. Annales Universitatis Apulensis : Series Oeconomica 13(2), 574 583. Barab, S., Gresalfi, M. & Arici, A. (2009). Why educators should care about games. Educational Leadership 67(1), 76 80. Barab, S., Gresalfi M., & Ingramp Noble, A. (2010). Transformational play: Using games to position perso n, content, and context. Educational Researcher 39 (7), 525 536. Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal of MUD research 1(1), 19. Bartle, R. (2004). Designing Virtual Worlds Indianapolis, IN: New Riders. Bedar d, A. T. (2015). What candy crush saga teaches us about motivating employees. Performance Improvement 54(4), 43 46. Blohm, I., & Leimeister, J. M. (2013). Gamification: Design of IT based enhancing services for motivational support and behavioral change. Business & Information Systems Engineering 5(4), 275 278. Brownhill, I. (2013). Does gaming have a role to play in employee engagement? Strategic HR Review 12(1), 34 35.

PAGE 182

182 g an ICT course: Influences on engagement and academic performance. Computers in Human Behavior 69, 98 107. Cardador, M. T., Northcraft, G. B., & Whicker, J. (2017). A theory of work gamification: Something old, something new, somet hing borrowed, somethin g cool? Human Resource Management Review 27(2), 353 365. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction 8, 293 332. Chapman, J., & Rich, P (2017 ). Identifying motivational styles in educational g amification. In Proceedings of the 50th Hawaii International Conference on System Sciences (pp. 1318 1327 ) Cheong, C., Cheong, F. & Filippou, J. (2013). Quick Quiz: A Gamified Approach for Enhancing Learning. In Proceedings of Pacific Asia Co nference on Information Systems Jeju Island, Korea. Chorney, A. I. (2013). Taking the game out of gamification. Dalhousie Journal of Interdisciplinary Management 8(1). Conaway, R., & Garay, M. C. (2014). Gamification and service marketing. SpringerPlus 3, 653. Creswell, J. W. (2013). Qualitative inquiry and research design Los Angeles, CA: Sage. Creswell, J. W. (2014). Research design. qualitative, quantitative and mixed methods a pproaches 4th edition. Sage Publications. Creswell, J. W., & Clark, V. L P. (2007). Designing and conducting mixed methods research 1st edition. Sage Publications. Cruz, L., & Penley, J. (2014). Too cool for school?: The effects of gamification in an advanced interdisciplinary course. Journal of Teaching and Learning with Te chnology 3(2), 1 11. Csikszentmihalyi, M. (1989). Optimal Experience in Work and Leisure. Journal of Personality and Social Psychology 56 (5), 815 22. Dale, S. (2014). Gamification : Making work fun, or making fun of work?. Business Information Review 31 (2), 82 90. De Schutter, B., & Abeele, V. V. (2014). Gradequest Evaluating the impact of using game design techniques in an undergraduate course. Foundations of Digital Games

PAGE 183

183 Deci, E. L., Connell, J. P., & Ryan, R. M. (1989). Self determination in a work organization. Journal of Applied Psychology 74, 580 590. Deci, E. L., & Ryan, R. M. (n.d.). Self Determination Theory. Retrieved May 01, 2016, from http://selfdeterminationthe ory.org/ Deci, E. L., Ryan, R. M., Gagne, M., Leone, D. R., Usunov, J., & Kornazheva, B. P. (2001). Need satisfaction, motivation, and well being in the work organizations of a former Eastern Bloc country. Personality and Social Psychology Bulletin 27, 930 942. Deckop, J. R., & Cirka, C. C. (2000). The risk and reward of a double edged sword: effects of merit pay programs on intrinsic motivation. Nonprofit and Voluntary Sector Quarterly 29, 400 418. Denny, P. (2013). The Effect of Virtual Achievements o n Student Engagement. In Proceedings of CHI 2013: Changing Perspectives (pp. 763 772). Paris, France Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game d esign elements to gamefulness: D efining gamification. In Proceedings of the 15th Inte rnational Academic MindTrek Conference (pp. 9 15 ) Dickey, M. D. (2007). Game design and learning: A conjectural analysis of how massively multiple online role playing games (MMORPGs) foster intrinsic motivation. Educational Technology Research and Develop ment 55(3), 253 273. Domnguez, A., Saenz de Navarrete, J., de Marcos, L., Fernndez Sanz, L., Pags, C., & Martnez Herriz, J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education 63, 380 392. Dreyfus, S. E ., & Dreyfus, H. L. (1980). A five stage model of the mental activities involved in directed skill acquisition (No. ORC 80 2). California Univ Berkeley Operations Research Center Eden, D. (1975). Intrinsic and extrinsic rewards and motives: replication an d extension with Kibbutz workers. Journal of Applied Social Psychology 5, 348 361. Farzan, R., & Brusilovsky, P. (2011). Encouraging user participation in a course recommender system: An impact on user behavior. Computers in Human Behavior 27(1), 276 284 Farzan, R., DiMicco, J. M., Millen, D. R., Brownholtz, B., Geyer, W., & Dugan, C. (2008a). Results from deploying a participation incentive mechanism within the enterprise. In Proceedings of the twenty sixth annual SIGCHI conference on Human factors in c omputing systems (pp. 563 572). ACM, Florence, Italy

PAGE 184

184 Farzan, R., DiMicco, J. M., Millen, D. R., Brownholtz, B., Geyer, W., & Dugan, C. (2008b). When the experiment is over: Deploying an incentive system to all the users. In Symposium on Persuasive Technol ogy Ferrara, J. (2013). Games for persuasion argumentation, procedurality, and the lie of gamification. Games and Culture 8(4), 289 304. Fitz Walter, Z., Tjondronegoro, D., & Wyeth, P. (2011). Orientation passport: using gamification to engage university students. In Proceedings of the 23rd Australian Computer Human Interaction Conference (pp. 122 125). ACM, Canberra, Australia Frith, J. (2013). Turning life into a game: Foursquare, gamification, and personal mobility. Mobile Media & Communication 1(2), 248 262. Gagn, M., & Deci, E. L. (2005). Self determination theory and work motivation. Journal of Organizational behavior 26 (4), 331 362. Giannetto, D., Chao, J., & Fontana, A. (2013). Gamification in a social learning environment. Issues in Informing Science & Information Technology Education 10, 195 207. Gilbert, T. F. (2007). Human competence: Engineering worthy performance. San Francisco, CA: Pfeiffer. Goehle, G. (2013). Gamification and web based homework. Primus : Problems, Resources, and Issues in Mathematics Undergraduate Studies 23(3), 234 246. Gooch, D., Vasalou, A., Benton, L., & Khaled, R. (2016 ). Using gamification to motivate students with dyslexia. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 969 9 80 ) Grant, S. & Betts, B. (2013). Encouraging user behaviour with achievements: An empirical a tudy. In Proceedings of the 10th Working Conference on Mining Software Repositories (pp. 65 68). San Francisco, CA, USA Gustafsson, A. & Bng, M. (2008). Evalua tion of a pervasive game for domestic energy engagement among teenagers. In Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology (pp. 232 239). ACM, Yokohama, Japan Hakulinen, L., Auvinen, T., & Korhonen, A. (2 013). Empirical study on the effect of achievement badges in TRAKLA2 online learning e nvironment. In Proceedings of Learning and Teaching in Computing and Engineering (LaTiCE) conference (pp. 47 54). Macau Hamari, J. (2015). Do badges increase user activi ty? A field experiment on the effects of gamification. Computers in Human Behavior 1 10.

PAGE 185

185 Hamari, J., & Eranti, V. (2011). Framework for designing and evaluating game achievements. In Proc. DiGRA 2011: Think Design Play (pp. 122 134 ) Hamari, J., & Koivisto, J. (2013). Social mo tivations to use gamification: A n empirical study of gamifying exercise. In Proceedings of the European Conference on Information Systems Utrecht, The Netherlands. Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamificati on work? A literature review of empirical studies on gamification. In Proceedings of the 47th Hawaii international conference on system sciences Hawaii, USA Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitu dinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education 80, 152 161. Huang, W. (2011). Evaluating learners' motivational and cognitive processing in an online game based learning enviro nment. Computers in Human Behavior 27(2), 694 704. Ilardi, B. C., Leone, D., Kasser, T., & Ryan, R. M. (1993). Employee and sup ervisor ratings of motivation: M ain effects and discrepancies associated with job satisfaction and adjustment in a factory setti ng. Journal of Applied Social Psychology 23, 1789 1805. Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field methods 18 (1), 3 20. Jakobsson, M. (2 011). The achievement machine: U nderstanding Xbox 360 achievements in gaming practices. Game Studies 11(1), 1 22. Jung, J. H., Schneider, C., & Valacich, J. (2010). Enhancing the motivational affordance of information systems: The effects of real time performance feedback and goal setting in group collaboration environments. Management Science 56(4), 724 742. Kalinauskas, M. (2014). Gamification in fostering creativity. Socialines Technologijos 4(1), 62 75. Kasser, T., Davey, J., & Ryan, R. M. (1992). Motivation and employee supervisor discrepancies in a psychiatric vocational rehabilitation setting. Rehabilitation Psychology 37, 175 187. Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of instructional development 10 (3), 2 10. Kim, B. (2015b). Gamification. Library Technology Reports 51(2), 10 17.

PAGE 186

186 Kim, J., & Lee, W. (2012). Dynamical model for gamification: Optimization of four primary factors of learning games for educational effectiveness. In Computer Applications for Graphic s, Grid Computing, and Industrial Environment (pp. 24 32). Springer Berlin Heidelberg. Kim, J., & Lee, W. (2013). Dynamical model for gamification of learning (DMGL). Multimedia Tools and Applications 1 11. Kirkley, S. E., Tomblin, S., & Kirkley, J. (200 5). Instructional design authoring support for the development of serious games and mixed reality training. In Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) Koivisto, J., & Hamari, J. (2014). Demographic differences in perc eived benefits from gamification. Computers in Human Behavior 35, 179 188. Landers, R. N. (2015). Dev eloping a theory of gamified learning linking serious games and gamification of l earning. Simulation & Gaming 45(6), 752 768. Lazzaro, N. (2004). Why we play games: Four keys to more emotion without story. LeCompte, M.D. & Schensul, J. (2013). Analysis and interpretation of ethnographic data New York: Altimira Press. Li, W., Grossman, T., & Fi tzmaurice, G. (2012). GamiCAD: A gamified tutorial system for f irst time autocad users. In Proceedings of the 25th annual ACM symposium on User interface software and technology (pp. 103 112). ACM, Cambrid ge, Massachusetts, USA Lieberoth, A. (2014). Shallow gamification: Testing psychological effects of framing an ac tivity as a game. Games and Culture 10(3), 229 248. Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation. American Psychologist 57(9), 705 717. Maan, J. (2013). Social business transformation thro ugh gamification International Journal of Managing Information Technology 5(3), 9 16. McAuley, E., Duncan, T., & Tammen, V. V. (1989). Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: A confirmatory factor ana lysis. Research Quarterly for Exercise and Sport, 60 48 58. McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York, NY: Penguin Press. Montola, M., Nummenmaa, T., Lucero, A., Boberg, M., & Korhonen, H. (2009). Applying game achievement systems to enhance user experience in a photo sharing service. In Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitou s Era (pp. 94 97). ACM, Tampere, Finland

PAGE 187

187 Muntean, C. I. (2011, October). Raising engagement in e learning through gamification. In Proc. 6th International Conference on Virtual Learning ICVL (pp. 323 329 ) Nacke, L. E., & Deterding, S. (2017). The maturin g of gamification research. Computers in Human Behavior 71, 450 454. Nacke, L. E., Drachen, A., & Goebel, S. (2010). Methods for evaluating gameplay experience in a serious gaming c ontext. International Journal of Computer Science in Sport 9(2). Nicholson, S. (2015). A recipe for meaningful gamification. In Gamification in Education and Business (pp. 1 20). Springer International Publishing. Onwuegbuzie, A. J., & Johnson, R. B. (2006). The validity issue in mixed research. Research in the Schools 13 (1), 48 63. Oprescu, F., Jones, C., & Katsikitis, M. (2014). I play at work ten principles for transforming work processes through gamification. Frontiers in Psychology 5, 1 5. Osipov, I. V., Nikulchev, E., Volinsky, A. A., & Prasikova, A. Y. (2015). Study of gamification effectiveness in online e learning systems. International Journal of Advanced Computer Science and Applications (IJACSA), 6(2). Osipov, I., Volinsky, A., & Grishin, V. (2015). Gamification, virality and retention in educational online platform International Journal of Advanced Computer Science and Applications 6(4), 11 18. Perryer, C., Celestine, N., Scott Ladd, B., & Leighton, C. (2016). Enhancing workplace motivation through gamification: Transferrable lessons from pedagogy. The International Journal of Management Education 14, 327 335. Rapp, A. (2017). From games to gamification: A classification of rewards in World of Warcraft for the design of gamified systems. Simulation & Gaming 1 21. Ryan, R. M., & Connell, J. P. (1989 ). Perceived locus of causality and internalization: E xamining reasons for acting in two domains. Journal of Personality and Social Psychology 57, 749 761. Ryan, R. M., & Deci, E. L. (2000). Self determination theory and the facilitation of intrinsic moti vation, social development, and well being. American psychologist 55 (1), 68. Sailer, M., Hense, J., Heinz, M., & Klevers, M. (2013). Psychological perspectives on motivation through gamification. Interaction Design and Architecture(s) Journal 19, 28 37.

PAGE 188

188 Seaborn, K., & Fels, D. (2015). Gamification in theory and action: A survey. International journal of human computer studies 74, 14 31. Shirom, A.,Westman, M., & Melamed, S. (1999). The effects of pay systems on blue collar T he mediating effects of objective and subjective work monotony. Human Relations 52, 1077 1097. Squire, K. (2006). From content to context: Videogames as designed experiences. Educational Researcher 35 (8), 19 29. Su, C. H., & Cheng, C. H. (2014). A mobi le gamification learning system for improving the learning motivation and achievements. Journal of Computer Assisted Learning 31, 268 286. Tashakkori, A., & Teddlie, C. (2006, April). Validity issues in mixed methods research: Calling for an integrative f ramework. In annual meeting of the American Educational Research Association, San Francisco Thom, J., Millen, D., & DiMicco, J. (2012). Removing gamification from an enterprise SNS. In Proceedings of the ACM 2012 conference on Computer Supported Cooperati ve Work (pp. 1067 1070). ACM, Se attle, Washington, USA Tobias, S., Fletcher, J.D., & Wind, A.P. (2014). Game Based Learning. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of Research on Educational Communications and Technology Fourth Edition (pp. 485 503). New York, NY: Springer Science + Business Media. Tulloch, R. R. (2014). Reconceptualising gamification: Play and pedagogy. Digital Culture & Education 6(4), 317 333. Turan, Z., Avinc, Z., Kara, K., & Goktas, Y. (2016). Gamif ication and education: Achievements, cognitive loads, and views of students. International Journal of Emerging Technologies in L earning 11(7), 64 69. van Manen, M. (1990). Researching lived experience New York: State University of New York Press. Van Tie m, D., Moseley, J. L., & Dessinger, J. C. (2012). Fundamentals of performance improvement: Optimizing results through people, process, and organizations : Interventions, performance support tools, case studies San Francisco, CA: Pfeiffer. Witt, M., Scheiner, C., & Robra Bissantz, S. (2011) Gamification of online idea competitions: Insights from an explorative c ase INFORMATIK 2011 Informatik schafft Communities 41. Jahrestagung der Gesellschaft fr Informatik Berlin, Germany.

PAGE 189

189 Zichermann, G. & Cun ningham, C. (2011). Ga mification by design: Implementing game mechanics in web and mobile a pps Zuckerman, O., & Gal Oz, A. (2014) Deconstructing gamification: E valuating the effectiveness of continuous measurement, virtual rewards, and social comparison for promoting physical activity Personal and Ubiquitous Computing 18(7), 1705 1719.

PAGE 190

190 BIOGRAPHICAL SKETCH Paul Wolff is an information technology professional and leader in education at a major research university He holds an Ed.D from the University of Florida (2017) in C urriculum and I nstruction, with a concentratio n in Educational T echnology, a M aste r of S cience from the University of North Carolina at Chapel Hill in Information Science (2010), a Master of A rts from the University of North Carolina at Chapel Hill in Teaching (2003), and a Bachelor of Arts from the University of North Carolina at Chapel Hill in H istory (2002) His research interests and professional experience include designing technology tools and applications, gamification implementations, the use of mobile technology, and performance improvement in workplace settings.