<%BANNER%>

UFIR



Mind the GAP: The assessment cycle for a game
CITATION PDF VIEWER
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/IR00001062/00001
 Material Information
Title: Mind the GAP: The assessment cycle for a game
Physical Description: Conference Papers
Creator: American Library Association (ALA) ( Conference )
Johnson, Margeaux ( Author, Primary )
Leonard, Michelle ( Author, Primary )
Publication Date: June 23, 2012
Copyright Date: 2012
 Subjects
Subjects / Keywords: Plagiarism
Games-Based Learning
Usability Testing
Assessment
Genre:
 Notes
Abstract: Formative assessment is an invaluable tool for creating learner-centered environments. This presentation will describe the process of creating an online, interactive game to engage STEM graduate students with research ethics. In 2010, the science librarians at the University of Florida received a $298,000 NSF Ethics Education in Science & Engineering (EESE) grant to develop a game-based learning environment for Responsible Conduct of Research (RCR). In partnership with the Digital Worlds Institute, scientists, and academic integrity experts, the librarians lead the team and focus on content design, usability testing, and evaluation. During game development, the team used an iterative design process that allowed for direct feedback from learners that could be integrated into game design. In this session, participants will learn about the process of designing usability tests as a way to formatively assess instructional design and adapt the user interface to optimize playability. This process is ideal for game development, online tutorial development, and online course development.
Acquisition: Collected for University of Florida's Institutional Repository by the UFIR Self-Submittal tool. Submitted by Margeaux Johnson.
Publication Status: Published
 Record Information
Source Institution: University of Florida Institutional Repository
Holding Location: University of Florida
Rights Management: All rights reserved by the submitter.
System ID: IR00001062:00001

Downloads

This item is only available as the following downloads:

ALA_Assessment_GAP ( PDF )


Full Text

PAGE 1

MIND THE GAP The assessment cycle for a game Michelle Leonard, mleonard@uflib.ufl.edu Margeaux Johnson, margeaux@ufl.edu University of Florida Funded by NSF ISS EESE Grant #1033002

PAGE 2

NSF Grant Ethics Education in Science & Engineering Vision: Online, self directed interactive game will provide a role adopting environment in which Science,Technology Engineering, and Mathematics (STEM) graduate students will learn to recognize and avoid plagiarism, data falsification, and data fabrication. Intellectual Merit: Training STEM graduate students in U.S. institutions to function effectively and ethically as authors within multi national research teams Broader Impacts: Adaptability and scalability Open Source

PAGE 3

Why a game? Goals: Create a transferable training environment that aids U.S. institutions in complying with Sec.7009 of the America COMPETES Act Develop a tool reflective of the future ethical considerations faced by U.S. global researchers Incorporate game design strengths identified by the NSF Assure scalability and robustness of design to permit future content enhancements

PAGE 4

Objectives Students completing the game will: Identify major types of contemporary plagiarism, including unique aspects of STEM publishing List the basic rules to avoid plagiarism in research activities. Demonstrate ability to apply the rules in increasingly complex scenarios. Explain the potential consequences of plagiarism academically and professionally.

PAGE 5

Grant Plan Evaluation Usability Testing Game Development Content Development Implementation SEPT. 2010 FEB. 2011 DEC. 2010 DEC. 2011 APR. 2011 DEC. 2011 DEC. 2011 APR. 2012 APR. 2012 AUG. 2012

PAGE 6

Grant Plan Highlighting Assessment Usability Testing Game Development Content Development Implementation Evaluation Formative Evaluation Pre/Post Tests Summative Evaluation

PAGE 7

Assessment Cycle

PAGE 8

Assessment Cycle Analysis Environmental Scan Focus Groups Formative Usability Tests Internal Review Summative Learning gains Pre/Post Tests

PAGE 9

Analysis: Campus Partners I 3 College of Engineeri ng Inter national Center Dean of Students Academic Integrity Taskforce Graduate School

PAGE 10

Analysis: The Environmental Scan: Perceptions of Plagiarism Survey 650 participants: graduate students in the STEM disciplines Format types of questions: Multiple choice questions Likert Scale Open ended KSA (Knowledge Skills, Attitudes )* K=understanding of ideas, A=willingness to use the evidence, to change ideas, to engage in critical reflection in the Inquiry Wynne Harlen Foundations. Vol 2, ch 11. http://www.nsf.gov/pubs/2000/nsf99148/lcd/ch_11.htm

PAGE 11

Analysis: Focus Groups Pre game Assessment Focus groups to assess: What kinds of games do STEM graduate students play? What kinds of game scenarios are interesting? Results were not what we expected

PAGE 12

No Robots

PAGE 13

Formative Assessment: Usability Usability testing throughout the game development phase Dialog between programmers, content team, and end users Rapid proto typing Determine the playability and content understanding Refine game mechanics and content

PAGE 14

Usability Test Cycle

PAGE 15

Usability Target Schedule Original target dates: Prototype Delivery Date Module 1 Paper Prototype 4/1/2011 Module 2 Paper Prototype 5/6/2011 Module 3 Paper Prototype 4/1/2011 Meta game Paper Prototype 9/2/2011 Module 1 6/24/2011 Module 2 9/30/2011 Module 3 8/5/2011 Meta game 10/28/2011

PAGE 16

Usability Target Schedule Reality: Prototype Usability Starts Module 1 Paper Prototype 4/1/2011 Module 2 Paper Prototype 5/6/2011 Module 3 Paper Prototype 4/1/2011 Meta game Paper Prototype 9/2/2011 Module 1 6/24/2011 Module 2 9/30/2011 Module 3 8/5/2011 Meta game 10/28/2011

PAGE 17

Usability Recruitment Campus Partners: Departments especially College of Engineering I cubed (NSF grant) Partners recruited and scheduled the same day the game prototype was received

PAGE 18

Usability Protocols Pre session checklist for facilitator and observer Think aloud test Introduce test, sign release forms, acceptable prompts 8 open ended questions: What was this game about? Did you understand how to play the game? If you had to change one thing about the game, what would it be and why?

PAGE 19

Usability Tests

PAGE 22

Overall Usability Results Caught glitches, discovered what worked Improved playability Refined content

PAGE 23

Summative Assessment: Evaluation Plan Traditional summative assessment that will evaluate the overall game experience using data from the pre test, playing the game (activity), and the post test. The Plan: Internal distribution: UF STEM graduate students External distribution: partner institutions, colleagues

PAGE 24

Evaluation Process Pre Test Game 1 Game 2 Game 3 Post Test Collect data and feedback Analyze results

PAGE 25

Evaluation Results

PAGE 26

Future Research Comparison study Learning gains with game v. f2f class v. tutorial Team based learning in the game Qualitative discussions/ lesson plans Transgressive

PAGE 27

MORE INFO ON GAP Margeaux Johnson margeaux@ufl.edu http://blogs.uflib.ufl.edu/GAP Michelle Leonard mleonard@uflib.ufl.edu