An Investigation into Using a Model to Teach Systems Thinking Skills

MISSING IMAGE

Material Information

Title:
An Investigation into Using a Model to Teach Systems Thinking Skills
Physical Description:
1 online resource (77 p.)
Language:
english
Creator:
Colley,Lara Michelle
Publisher:
University of Florida
Place of Publication:
Gainesville, Fla.
Publication Date:

Thesis/Dissertation Information

Degree:
Master's ( M.S.)
Degree Grantor:
University of Florida
Degree Disciplines:
Forest Resources and Conservation
Committee Chair:
Monroe, Martha C
Committee Members:
Cohen, Matthew
Thiele, Leslie P

Subjects

Subjects / Keywords:
balancing -- concepts -- delayed -- feedback -- indirect -- interventions -- mechanisms -- reinforcing -- strategies -- systems -- teaching -- thinking
Forest Resources and Conservation -- Dissertations, Academic -- UF
Genre:
Forest Resources and Conservation thesis, M.S.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract:
There is a need to develop effective means to teach systems thinking to a variety of audiences, some of whom will not have copious amounts of time to invest in systems education. Though there is a large body of research dedicated to teaching systems thinking to students over a period of weeks or months there has been little research on effectively teaching the basic components of systems thinking in a two-hour period. In this study we developed a short systems thinking intervention for undergraduates. We used multiple and relevant examples, function-centered Structure-Behavior-Function as a framework to develop the intervention, and a computer model to test the effectiveness of that strategy. We used a pre/posttest design with focus groups to measure the use of thinking skills and focused on three specific concepts: feedback, delay, and indirect causality. The results of the study indicate that a short, well-conceived systems thinking lecture can relay concepts of indirect and delayed causality and feedback mechanisms. After receiving a 50-minute lecture on systems thinking, posttest scores were significantly greater than pretest scores (p<. 001). Two groups of students received an additional systems presentation/discussion, one using a computer model to relay concepts, the other using static graphs to convey the same information. These students? posttests were also significantly increased over pretest scores (p< 0.001), but were not significantly different from each other. The results indicate that computer models are not necessary to teach systems thinking concepts in short interventions. Engaging presentations using multiple examples that stress relationships between components are useful strategies to teach these systems thinking concepts. The information obtained in this study can be used to help college and non-formal educators develop interventions to help adult learners understand the complexity of the world around them.
General Note:
In the series University of Florida Digital Collections.
General Note:
Includes vita.
Bibliography:
Includes bibliographical references.
Source of Description:
Description based on online resource; title from PDF title page.
Source of Description:
This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility:
by Lara Michelle Colley.
Thesis:
Thesis (M.S.)--University of Florida, 2011.
Local:
Adviser: Monroe, Martha C.
Electronic Access:
RESTRICTED TO UF STUDENTS, STAFF, FACULTY, AND ON-CAMPUS USE UNTIL 2012-08-31

Record Information

Source Institution:
UFRGP
Rights Management:
Applicable rights reserved.
Classification:
lcc - LD1780 2011
System ID:
UFE0043488:00001


This item is only available as the following downloads:


Full Text

PAGE 1

AN INVESTIGATION INTO USING A MODEL TO TEACH SYSTEMS THINKING SKILLS By LARA COLLEY A THESIS PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTE R OF SCIENCE UNIVERSITY OF FLORIDA 2011

PAGE 2

2 2011 Lara Colley

PAGE 3

3 To my d ad, who always gives me perspective and encouragement. I love you dearly.

PAGE 4

4 ACKNOWLEDGMENTS I would like to thank my committee for their guidance and insightful advice. I hav e learned a great deal from them and this project was a success largely due to their extensive knowledge and collaborative ideas. I would like to extend a special thank you to Dr. Martha Monroe, my committee chair, whose whole hearted support, counseling, and passion are unparalleled. She inspires her students to achieve greatness and her faith in her students seems to have no boundary Thanks also to Matthew Cohen for conducting the interventions an d sacrificing his time and energy to ensure the project w as as successful as it could be I thank my family and friends who have never lost sight, even when my direction was unclear. The support of those close to me has allowed me to believe in myself and pursue goals that seemed unattainable. I am eternally gr ateful for my graduate student cohort, whose friendship, advice, and knowledge ha ve been vital to this process. I thank them for having the time to support my endeavors with notepads and video cameras in hand. I especially want to thank Geetha Iyer, Lindse y McConnell, Sarah Hicks, Annie Oxarart, Deb Wojcik, and Annelena Porto Delgado who will always be part of my family. I want to thank the UF Water Institute and College of Agriculture and Life Sciences for providing funding to pursue a graduate degree tha t would have never been possible otherwise. I thank Dina Leibowitz and Anna Cathey for allowing me to use their data. Their model was integral to this project and provided us with rich insights into the usefulness of models. I thank Dina for her friendshi p, kind words, time, moral support, and conversation.

PAGE 5

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS ................................ ................................ ................................ .. 4 LIST OF TABLES ................................ ................................ ................................ ............ 7 LIST OF FIGURES ................................ ................................ ................................ .......... 8 ABSTRACT ................................ ................................ ................................ ..................... 9 CHAPTER 1 INTRODUCTION ................................ ................................ ................................ .... 11 2 METHODS ................................ ................................ ................................ .............. 19 Study Design ................................ ................................ ................................ .......... 19 The Intervention ................................ ................................ ................................ ...... 19 Diagramming t he System ................................ ................................ ........................ 21 Testing the Hypotheses ................................ ................................ .......................... 22 Data Collection Tools ................................ ................................ .............................. 23 Foc us Groups ................................ ................................ ................................ ......... 24 Final Exam ................................ ................................ ................................ .............. 24 Pilot Testing ................................ ................................ ................................ ............ 25 Analysis ................................ ................................ ................................ .................. 26 3 RESULTS ................................ ................................ ................................ ............... 29 Pre and Posttests ................................ ................................ ................................ ... 29 Focus Groups ................................ ................................ ................................ ......... 30 Final Exam Question ................................ ................................ ............................... 35 Limitations ................................ ................................ ................................ ............... 36 4 DISCUSSION ................................ ................................ ................................ ......... 41 Hypothesis 1 ................................ ................................ ................................ ........... 41 Hypothesis 2 ................................ ................................ ................................ ........... 41 Focus Groups ................................ ................................ ................................ ......... 41 Final Exam ................................ ................................ ................................ .............. 42 Synthesis ................................ ................................ ................................ ................ 42 5 CONCLUSION ................................ ................................ ................................ ........ 44 APPENDIX A PRETEST ................................ ................................ ................................ ............... 49

PAGE 6

6 B POSTTEST ................................ ................................ ................................ ............. 52 C SCORING RUBRIC ................................ ................................ ................................ 54 D FOCUS GROUP QUESTIONS ................................ ................................ ............... 57 E IRB FOR PRE AND POSTTEST DATA ................................ ................................ .. 58 F IRB FOR FOCUS GROUPS ................................ ................................ ................... 60 G CODE SHEE TS FOR FOCUS GROUP ANALYSIS ................................ ............... 62 LIST OF REFERENCES ................................ ................................ ............................... 74 BIOGRAPHICAL SKETCH ................................ ................................ ............................ 77

PAGE 7

7 LIST OF TABLES T able page 3 1 Mean scores for pre and posttest for each treatment ................................ ........ 38 3 2 Paired t tests for pre and posttest means for e ach of the treatments ................. 38 3 3 One way ANOVA on the posttests of Treatments 2 and 3 ................................ 38 3 4 Correlation statistics for student demogr aphics for year in school, prior systems experience and pre/posttest scores ................................ ...................... 39 3 5 Correlation statistics for student demographics for course grades, majors, GPA, and pretest scores ................................ ................................ .................... 39 3 6 Responses to the final exam question about algae and springs ......................... 40

PAGE 8

8 LIST OF FIGURES Figure page 2 1 Diagram of the springs system from the presentation/discussion ....................... 28

PAGE 9

9 Abstract of Thesis Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for t he Master of Science Degree AN INVESTIGATION INTO USING A MODEL TO TEACH SYSTEMS THINKING SKILLS By Lara Colley August 2011 Chair : Martha C. Monroe Major: Forest Resources and Conservation There is a need to develop effective means to teach systems th inking to a variety of audiences, some of whom will not have copious amounts of time to invest in systems education. Though there is a large body of research dedicated to teaching systems thinking to students ov er a period of weeks or months there has been little research on effectively teaching the basic components of systems thinking in a two hour period In this study we developed a short systems thinking intervention for undergraduates. We used multiple and relevant examples, function centered Structur e Behavior Function as a framework to develop the intervention, and a computer model to test the effectiveness of th at strategy. We used a pre/post test design with focus groups to measure the use of thinking skills and focused on three specific concepts: feedback, delay, and indirect causality. The results of the study indicate that a short, well conceived systems thinking lecture can relay concepts of indirect and delayed causality and feedback mechanisms. After receiving a 50 minute lecture on systems t hinking, posttest scores were significantly greater than pretest scores (p<. 001). Two groups of students received an additional systems presentation/discussion, one using a computer model to relay

PAGE 10

10 concepts, the other using static graphs to convey the same posttests were also significantly increased over pretest scores (p< 0.001), but were not significantly different from each other. The results indicate that computer models are not necessary to teach systems thinking concepts i n short interventions. Engaging presentations using multiple examples that stress relationships between components are useful strategies to teach these systems thinking concepts. The information obtained in this study can be used to help college and non fo rmal educators develop interventions to help adult learners understand the complexity of the world around them.

PAGE 11

11 CHAPTER 1 INTRODUCTION When we try to pick something up by itself, we find it hitche d to everything in the universe. -John Muir, 1869 The world abounds in complexity T he self organizing patterns of the ridge and slough landscape in the Everglades of south Florida are a prime example of a visible indicator of underlying complexity T his system is characterized by a complex set of feedback mechanisms between the sediments, hydrology, and vegetation that operate in such a way as to create elevated ridges of sawgrass surrounded by deeper open water (Watts et al. 2010). The world hosts ma ny other complex systems which scientists seek to understand Socially c onstructed systems such as p ublic transportation, global financial markets, and generational poverty along with e nvironmental problems such as groundwater depletion, climate change, and deforestation are complex and t herefore are difficult to remedy. Understanding the dynamic and complex nature of systems allows us to approach solutions to environmental and social problems and better understand the world in which we live Riess and Mi scho (2010 pg 707 ) describe systems thinking as the ability to recognize, describe and model (e.g., to structure, to organize) complex aspects of reality as systems Systems thinking is a cognitive process that allows a person to identify and understand the interactions occurring within a defined area by constructing a mental or visual model of the system The world systems that are dynamic, self organizing, and continually adapt Orion

PAGE 12

12 2010 pg. 1254) th erefore having the ability to conceptualize systems is a vital step in the process of understanding them. While i t may be inferred that systems thinking is integral for researchers and problem solvers of complex systems and less im portant for the general public, such is not the case. The last few decades have seen a wave of support for increasing the making -a n important nerations past (Chittenden 2011; Davies 2009; Rowe and Frewer 2000 ; Stave 2000 ). People need to be part of the decision making process in order to understand and play a role in solving complex environmental problems For the public to be effectively involved, they need to be able to make informed decisions a bout complex issues (Fr ewer, Howard and Shepherd 1998) and be scientifically literate (Durant, Evans and Thomas 1992;Feinstein 2010 ; Hawthorne and Alabaster 1999; Stave 2002 ; ). informed public pol Miller 1998 pg. 204 ) Systems thinking provide s an analytical framework to promote scientific literacy among the public. Thinking systemically requires that people dev elop a mental image of a system, yet it is challenging for most people to conceptu alize a functioning system with all of its interconnections, due to our general into an under 2010 pg. 116) While systems thinking may not be intuitive ( Liu and Hmelo Silve r, 2009 ; Verhoeff, Waarlo and Boersma 2008) once mastered, it can provide a means to simplify a complex world of many different systems. There is a broad recognition of the need to teach systems t hinking to the public (Johnson 2010; Liu and Hmelo Silver 2009; Verhoeff, Waarlo and Boersma 2008) and

PAGE 13

13 to explore methods of teaching systems thinking, especially in non formal education forums. Computer models are conventionally used as a means to teach about systems and systems thinking, howev er they are diff icult and time consuming to master because they are often not intuitive (Deaton 2000 pg 13; Liu and Hmelo Silver 2009; Thompson and Reimann 2010). Outside of universities and high schools with courses dedicated to exploring systems, however, there is not l ikely to be much demand for 12 week systems thinking courses Further, the public tends not to invest major time commitments unless the issue is perceived as threatening (Monroe et al. 2009 ). If we expect to captivate learners for systems thinking interven tions, it will likely be for brief periods of time T herefore it is important to develop effective strat egies for short interventions exploring both how to enhance understanding and what the essential concepts might be. Simple computer models that allow learners to explore the intricacies of a system without being required to have a deep understanding of how the model was designed might provide a useful tool to teach systems (Thompson and Reimann 2010). While the nomenclature varies from field to field, the concepts of systems thinking are the same. Ev ery system must have stocks, flows, drivers, causal relationships and feedback lo ops (Deaton and Winebrake 2000; Meadows 2008 (Meadows 2008 pg.1 7) Meadows 2008 pg 189). Flows can be described as the movement of stocks in and out of the system (Meadows 2008). A causal relationship is one where one variable or system component has some e ffect on another within the system (Deaton and Winebrake 2000). Feedback mechanisms are signals that accelerate (reinforcing or positive) or decelerate (balancing or negative) a process within a system ( Dea t on and

PAGE 14

14 Winebrake 2000 s of causal connections from a stock, through a set of decisions, rules, phys ical laws, or actions that are dependent on the level of stock and back again through a flow dows 2008 pg 189). Delays or lags are inherent to many systems and can be described as an effect that is not immediately evident within the system due to a delay in the feedback signal on the stock. A clear understanding of stocks, flows, causal relations hips, feedback mechanisms, and lag or delay is vital to developing a systems pers pective (Deaton and Winebrake 2000 ; Hmelo, Holton and Kolodner 2000; Jacobson and Wilensky 2006; Meadows 2008; Reiss and Mischo 2010), as these are fundamental to a variety of complex systems negative feedback loops] in a real life environmental system can lead to significant and Winebrake 2000 pg 16 ). Being able to r ecognize and identify these functions enab les people to think in systems. Some of these concepts, however, should be familiar to undergraduate students if they have some initial understanding of the system, such as stocks and flows in an ecosystem. Conveyi ng the concept of indirect causality is important because many causal relationships are not directly related to the outcome and cannot be explained in terms of linear causality In addition, linear causality is intuitive and most frequently emphasized in t raditional education systems, ( Hmelo, Holton and Kolodner 2000 ; Hmelo Silver, Marathe and Liu 2007). For example, for a person to comprehend the numerous causes of climate change they must understand that burning fossil fuels is not the sole perpetrator, b ut instead there are a number of seemingly unrelated factors that also play a role.

PAGE 15

15 Feedback mechanisms are also important to understand because most causal relationships in the world are dynamic and loop like instead of linear and chain like. Lags and delays or delayed causality are important to understand because often the effects of some causes do not become evident immediately. For example, the effects of DDT on bird populations could not be predicted until long after it was used as a pesticide. Many systems experts suggest that these concepts are among the most basic and most important for the public to better understand ( Deaton and Winebrak e 2000; Hmelo, Holton and Kolodner 2000; Jacobson and Wilensky 2006; Meadows 2008; Reiss and Mischo 2010) To h elp educators develop the skills needed to enable students to understand systems Behavior B F) theory of complex systems was developed as a way to teach systems thinking (Prabhakar and Goel 1998). The fundamental approach of S B F is that students begin by learning the structure or systems components and their respective relationships ( Hmelo, Holton and Kolodne r 2000 ; Liu and Hmelo Silver 2009). They then learn about the behavior or mechanism of the structures and finally the fu nction or role each structure has within the system ( Liu and Hmelo Silver 2009) Encouraging learners to conceptualize beyond isolated structures is necessary for understanding the system. This can be accomplished by focusing on the function (Function cen tered S B F [fS B F]) and behavior of the structures and this is an easier task if the learners are already familiar with the structures. An fS B F approach provides learners with a context for understanding how the structures interact, which allows them t o build on their knowledge, connect concepts and then apply them (Hmelo, Holton and Kolodner 2000; Liu and Hmelo Silver 2009; Meadows 2008).

PAGE 16

16 A few studies have explored the use of short interventions using computer models and S B F to convey systems think ing skills. ( Assaraf and Orion 2005, 2010; Ossimitz 2000 ) In one noteworthy example, Liu and Hmelo Silver (2009) developed two hour computer modules on the respiratory system for pre service teachers and middle school stud ents to teach systems concepts. S tudents viewed the same information organized in two different modules ; some looked at structures of the respiratory system first and others were introduced to the functional aspects of those structures first. The pre service teacher posttest s revealed t hat participants who received the function (p < .0 5 ), and functions (p <.05 ) at the non salient or micro level, which included concepts such as gas exchange and transport. The m iddle school students posttest s indicated that students who received the function centered module were able to identify more behaviors (p<.05), trended toward being able to identify more functions (p <. 0 5 ), and had no difference in ability to identify struc tures. Another demonstration of the useful difference between traditional S B F and fS B F can be seen in the degree to which experts and novices hold a systems perspective ( Jacobson and Wilensky 2006 ; Liu and Hmelo Silver 2009 ). When experts talk about s ystems, they describe the function and behavior where novices, whose understanding is often confined to structure, do not ( Hmelo, Holton and Kolodner 2000 ; Liu and Hmelo Silver 2009) For example, a non expert might describe an Intensive Care Unit (ICU) b y focusing on the patients, doctors, nurses, medical equipment, and medicines parts of the details, elements, and structures of an ICU. An expert is more likely to explain patient life support and critical care. These are functions of the unit, which requi re the same structural elements, but the focus is on how those structures operate as a unit

PAGE 17

17 rather than merely defining them as separate entities. An expert would also include complex, emergent behaviors that arise within an ICU; such as how the types of p atients in the unit, such as intoxicated drivers who have killed someone or drug dealers who have been shot, can have negative impacts on the staff morale and attitudes. The un derstanding of the system is heightened. To effectively teach systems thinking skills in a shorter period of time, a number of strategies may be useful. T he most effective methods of teaching complex concepts and skills include multiple examples and engagi ng learners in physically or mentally working wi th concepts (DeYoung and Monroe 1996 ; Eysink et al 2009 ; Krajcik and Sutherland 2010 ). Engaging learners while teaching, instead of relying solely on passive instruction, has been shown to increase cognitive involvement, which also factors into a situations (DeYoung and Monroe 1996 ; Dunleavy and Milton 2008; Lepper and Malone 1987 ). Simplified systems diagrams are also tools that can be employed to conve y systems thi nking concepts. For this project, I will measure the effectiveness of conveying the concepts of indirect causality, feedback mechanisms, and delays to the learners using multiple examples and function centered S B F (fS B F ), an effective strategy to teac h complex systemic behavior in a two hour unit. I will also explore whether seeing a computer model of a system enables students to understand indirect causality and feedback mechanisms I hypothesize that 1) a short lecture that provides examples of indi rect causality, feedback loops, and delays using function centered S B F explanations and multiple, concrete examples can effectively convey systems to learners and 2) technology, as

PAGE 18

18 represented by a computer model, coupled with a discussion and presentati on will be a more useful component to help learners understand indirect causality, feedback mechanisms, and delay than the discussion and presentation alone.

PAGE 19

19 CHAPTER 2 METHODS This chapter will describe the study, three data collection tools, pilot testi ng of the intervention and tools, sample population, and types of analyses used for this project. Study Design An experimental pre/post design was developed to test the two hypotheses. Prior to the intervention 50 undergraduate students registered for th e course Society and Natural Resources in the spring of 2011 were g iven a pretest. One week after the pre test, all students received a 50 minute lecture on systems thinking using multiple examples of systems to explain concepts. The class was then divide d into three groups based on their course section. The first group received the post test shortly after the lecture (Treatment 1). This group of students then received a presentation/discussion without a computer model The second group received the lectur e followed by the presentation/discussion without a computer model and finally the posttest (Treatment 2). The third group received the lecture then a presentation/discussion that included computer model, and then a posttest (Treatment 3). Follow up focu s groups helped explore how students remembered the exercise and discussion. One question on the term memory. The data gathered from students in Treatment 1 were grouped with the data from Treatment 2 for the focus groups and final exam question because they received the educational strategy. The Intervention All of the material for this intervention was prepared by a faculty member who is an expert in systems ecology. The first part of the intervention was a 50 minute lecture which detailed five systems, including a home rat infestation, an arctic fox and island

PAGE 20

20 seabird interaction, and overfishing in the Philippines. Students read the article, Environmental Tipping Points: A New Slant on Strategic E nvironmen talism (Marten, Brooks and Suutari 2005), which details three case studies with a focus on indirect causality, feedback mechanisms and delays. Information from this article was used to reinforce these concepts. The examples referenced components as part o f a larger system where behaviors emerged as a result of the interactions and feedback mechanisms within the sys tem. The presentation a nd reading included multiple examples that described the functional aspects of thes e system components. The next phase o f the intervention were 5 0 minute presentation /discussions, which conveyed nearly identical concepts, but differed in the delivery. Th is presentation used one local example -fr eshwater springs -and was based on data collected by PhD studen ts at the Univers ity of Florida The treatments were designed to give learners a context for evaluating the drivers and feedback mechanisms that might be responsible for nuisance algae growth in this system. The beginning of both presentations were identical and provided s tudents with information on the springs ecosystem, including the functions of the system elements: nutrients, snails, dissolved oxygen, flow rate, and vegetation (i.e., an fS B F strategy). Students were shown a systems diagram (Figure 2 1) of the relation ships between the components and asked to evaluate what might be causing the algae to grow to nuisance levels. Treatments 1 and 2 used static graphs to illustrate the relationships between snails, aquatic vegetation, dissolved oxygen, and algae, while Trea tment 3 used a computer model generated in Microsoft Excel to convey the relationships between the same organisms. FS B F provided a framework for the systems presentation and treatments. As such, functional components and their relationships within the systems were the primary

PAGE 21

21 focus of the information presented. Because most students were from wildlife, forestry, and natural resources, we assumed they were somewhat familiar with the structural components of ecosystems (i.e. foxes, rats, cats, dogs, shor e birds, algae, snails, and nutrients) as well as their functions in the scenarios used in the lecture and treatments. Diagramming the System Diagramming systems can be a useful tool to vis ualize conceptual ideas (Luckie 2011). Further, engaging learners i s preferable to passive transmission of information because it increases cognitive involvement, which aids in the learn ing process (DeYoung and Monroe 1996). Using conceptual mapping exercises to help teach systems thinking concepts allows learners to gras p the relationships between components This enables learners to construct a conceptual image and reveal the complexi ty of the system (Luckie 2011). During the treatment, students were presented with a skeleton diagram that consisted of words related to th e springs ecosystem printed across the page: algae, nitrates, recreation, grazers, water flow, aquatic vegetation, manatees, human population and dissolved oxygen. Before receiving information about the springs, they were asked to draw arrows between compo nents they believed to be related. They were also asked to write positive and negative signs to explain the types of interactions that occur between components. The students were reminded that a plus sign would indicate a positive relationship and a negati ve sign would indicate a n inverse relationship For example, if they believed that more manatees would cause algae t o s point. The stud ents were asked to continue add ing to the diagram as they listened to the information on springs. Diagramming was

PAGE 22

22 used as a method to engage learners, and ensured that students were listening to lecture, as each handed in a completed diagram. Testing the H ypotheses H1 : A short presentation that provides examples of indirect causality, feedback loops, and delays using fS B F explanations and multiple, concrete examples can effectively convey systems concepts to learners. Treatment 1: T hese students t ook a p retest, received the systems lecture t ook a posttest and then received Treatment 2, the non model discussion This group was used to determine if systems thinking concepts can b e taught in a one hour lecture. This group is also used as the control group f or H2. H2 : T echnology, as represented by a computer model, coupled with a discussion and presentation will be a more useful component to help learners understand indirect causality, feedback mechanisms, and delays and is more effective than the discussio n and presentation alone. Treatment 2: T hese students t ook a pretest, r eceived the systems lecture received a presentation/ discussion on freshwater springs health that stressed the relationships between components and drivers in the springs ecosystem th en took a posttest This group was presented with a static graph that showed little or no correlation between algae and nitrate concentration. Treatment 3 : T his group t ook a pretest, r eceived the systems lecture received a similar presentation/discussion on springs but instead of graphs viewed a model created in Microsoft Excel of the springs ecosystem then took a posttest The instructor made manipulations to the model and ran it while students observed. The manipulations included increasing nitrate concentration, reducing populations of organisms that graze

PAGE 23

23 upon algae and decreasing concentration of submerged aquatic vegetation to show their individual e ffects on algae growth. Each of these changes to the model produced a graphical output. Data Collection Tools A pretest of 18 open ended questions and 6 demographic questions (Appendix A) was designed to measure familiarity with systems concepts. The same instrument was used for the posttest without demographic questions (Appendix B ). The se tools of feedback loops, indirect causality, and delays, which literature ( Deaton and Winebrake 2000; Hmelo, Holton and Kolodner 2000 ; Jaco bson and Wilensky 2006; Meado ws 2008; Reiss and Mischo 2010) suggests are paramount to understanding systems. The pretest demographic questions included items about years at the university, GPA name, and major, as well as questions to determine prior systems knowledge Questions abou t systems concepts were conveyed through seven scenarios. The scenario questions asked participants to read a short paragraph ( Appendix A ) and then provide an explanation of the behavior The scenarios were followed by open ended questions that asked parti cipants to describe another scenario where the same phenomenon occurred. A rubric was developed (Appendix C) to score all responses to the pre and posttest questions. Answers were scored between 0 3, depending on the depth of the explanation provided. A no response received a score of 0, a response that indicated little understanding of the concept received a 1, a score of 2 was given to responses that revealed some understanding of the concept, but lacked

PAGE 24

24 mention of the pro cess, and a score of 3 was given to answers that adequately or better described the process. All scores were summed to give a final score for each student. Focus Group s Focus groups were organized 3 months after the initial interventions to determine if i nformation from the presentation understanding of systems and what students remembered about the systems intervention Information from the focus groups was also used to explain the results from the students posttest as well as those who scored high on both tests were invited to participate in a focus group. These students were chosen because they understood the concepts and would be likely to provide the most relevant information about what was most helpful to their learning the concepts Four focus groups were held with a total of 12 students in April 2011. Students who participated in the focus groups were from each of the treatments A ser ies of questions (Appendix D) were asked of students to determine if the presentations affected the way they look at complex systems, both related a nd unrelated to springs health. These focus groups were audio recorded, transcribed in full, and analyzed. All of the participants signed consent forms ( IRB Protocol # 2010 U 0763, Appendix E) Final Exam At the end of the semester, after incorporating systems thinking and simplified systems diagrams into four case studies covered in lecture we asked students a question on the final exam to determine if they retained a systemic perspective in terms

PAGE 25

25 of freshwater springs The question was based on information presented to the students during the presentation /discussion Pilot T esting The lecture presentation/ discussion, model, and pre/posttest were pilot tested with 71 students from September 2010 and significant improvements were made to the treatments and tests The pilot pre and posttest s were iden tical and largely multiple choice questions asking students to identify the type of systems conc ept exhibited in each scenario The students were randoml y assigned to one of two groups: Treatment 2 (n=33) and another received Treatment 3 (n=38). Observers were present and the treatments were video recorded. Several flaws were identified and remedied from the pilot version of this project. The issues that would have a substantive impact on the study were the delivery of the treatments, the survey tool, and t he model. The treatments during the pilot study were given to the participants by different instructors whose teaching styles were undoubtedly different One instructor provided all of the material in the final execution of this project The pilot study us ed multiple knowledge of terminology rather than knowledge of the concepts The pre and posttest s were changed to open ended questions where students were provided with scenarios and asked to explain the processes involved. This allowed an assessment of their systems knowledge or ability to think in systems notwithstanding any prior formal instruction in systems. The original computer model contained several errors that required the model to be recalibra ted before using it in the final version of the project.

PAGE 26

26 The improved version of the pre and posttest was pilot tested on another group of undergraduates registered for a Fire Ecology course in the spring 2011. None of these students were enrolled in Socie ty and Natural Resources. Several additional changes were made based on the information gathered Some questions were eliminated for the sake of time and others were re worded for clarity Analysis The data were analyzed using both quantitative and quali tative methodologies. The quantitative analysis was performed using IBM SPSS Statistics version 19. A one were significant differences between way ANOVA was performed to determine if there the groups based on pretest scores. Paired t t ests were conducted to determine if t he differences between the means of the pre and posttest s scores were statistically significant A one way ANOVA was used to determine if differences between the posttest scores minus the pretest scores from Treatments 2 and 3 were statis tically significant. Calculations were performed to determine if there were correlations between demographic information and scores. Qualitative analyses of the data included rubric creation, coding, and use of the Constant Comparison Method (Dye et al. 2 000). A rubric (Appendix C) was developed to score the pre and posttest questions and to ensure internal validity, an unaffiliated individual scored the tests in addition to the Principle Investigator. The scoring was consistent between scorers. The conte nt of the focus group interviews was also analyzed qualitatively. The recordings were transcribed and then coded to establish themes. The Constant Comparison Method was used to develop 10 codes (Appendix G). The transcriptions were read numerous times and codes were continuously revised to accurately reflect

PAGE 27

27 the theme revealed within the data. Themes emerged out of the codes and rel ated ideas were grouped (Glesne 2006 pg 147 172). The answers to the final exam question were scored 0 3 ( Appendix C ).

PAGE 28

28 Fig ure 2 1 Diagram of the springs system from the presentation/discussion

PAGE 29

29 CHAPTER 3 RESULTS Fifty undergraduate students received the initial systems lecture but t en students were removed from the sample population. One student spoke English as a second language and was unable to comprehend the questions. The others were absent from the presentation /discussion or did not return their posttest The sample size of the ent ire population was 40 students with 9 students receiving Treatment 1 18 students rece iving Treatment 2 and 13 receiving Treatment 3 These students were a fairly diverse group of undergraduat es though most were majoring in Forestry /Natural Resource Conservation (n=12) Wildlife Ecology (n=14) with the remaining students majoring in Marke ting, Environmental Sciences, Biology, Agricultural Education, and Religion Nearly all were either juniors (n=18) or seniors (n=19) reported GPAs ranged from 2.02 to 4.0. There were 22 females and 18 males. All students gave their info rmed consent to allow us to use their data ( approved IRB Protocol # 2010 U 0763, Appendix F). When asked an open ended question about their prior experience with systems (Appendix C), about a third (n=13) reported to have heard of systems briefly either on television, from someone, in a course/courses, or in journals, but the majority (n=26) had never heard of systems. Only one student reported to have taken a course dedicated to systems Pre and Posttest s The one way ANOVA of the pretest differences betwe en gro ups was not significant suggesting the groups were similar at the beginning of the intervention. All of the mean scores improved from pre and posttest (Table 3 1 ) and a paired t test of the

PAGE 30

30 means showed significant differences (p<0.01) (Table 3 2 ). A one way ANOVA was also performed to determine if there were significant differences between posttest score improvement by concept. There were no significant differences, which indicates that uggest that systems thinking can be taught in a one hour presentation to undergraduate students conveying the concepts of indirect causality, feedback mechanisms, and delay. A one way ANOVA was performed on the posttest minus pretest scores of Treatments 2 and 3 (Table 3 3 ) and there are not significant differences between scores (p > .05 ). This suggests that immediately after the presentation/discussion students who received the model treatment were no more knowledgeable in systems thinking concepts than those who received the static graphs, Correlation analyses were p er (Table s 3 4 and 3 5 ). The analysis revealed significantly positive correlation s between student classification (i.e. freshman, sophomore, junior, senior) and pretest scores (p< .01). This indicates that students who had more years in school had higher scores This did not impact the study, however, because upper class students were evenly distributed across treatment groups. There were also signifi cant correlations between scores on the pretest and scores on the posttest (p< .001 ) which is not unexpected There were not significant correlations between other demographic information such as final c ourse grade and posttest scores. Focus Groups The focus group interviews were conducted with 12 students representing each of the treatment groups There were two students from Treatment 1 and five each from Treatments 2 and 3. The information from the students who were in Treatments 1 was

PAGE 31

31 comb ined into Treatment 2 (non model treatment) for the purpose of discussing the use of the model and learning about nutrients and algae The interviews yielded 10 useful themes and well as provided rich data that helped to explain what factors helped these students pe rform well on the pre and posttest s. Their responses revealed some of what was confusing and easy to learn, what they already believed, and what notions were hard to change in this short intervention The y explained the concept of delay was well understood and retained multiple and relevant examples helped them understand the material, and systems thinking ha d been useful after the presentation and treatments An analysis of which students provided comments on the causes of algae in the springs ecosystem s uggested notable differences in their perceptions. It appears that delay was one of the easier concepts for these students to comprehend. There was an example from the pre and posttest s as well as the lecture that used sunburn as a means to describe the c oncept of delay Students referenced this example more than any other when asked to recall systems thinking principles from the pre and posttest s : Participant 1.3.3: (1/1/6) There was the delay one the sunburn [example]. Participant 1.4.3: (1/2/1) The sunburn one was delay. Participant 2.3.3: (2/1/15 17) The sunscreen one was time lag, the time differences between two variables. The sunburn example was easy for students to relate to the concept of delayed c familiarity with sunburn. The use of multiple relevant, real world examples made systems thinking less abstract for the students who pa rticipated in the focus groups :

PAGE 32

32 Participant 1.3.3: (1/5/28) I think the biggest help was all the examples of everything Participant 1.2.2: (1/7/2) The examples w ere really helpful. (1/6/1 2) Like the cats and dogs, there were all these terms and then you give the reasons and examples it made perfect sense Participant 2.3.3: (2/14/7 9) I think the real world examples and the simplicity of it initially helped to understand the general concepts. It jus t made it easier to understand. Students indicated that the many examples of systems they were familiar with helped them to understand the common dimensions of systems we were conveying Students also commented on the usefulness of the diagramming exercise and system diagrams used in the lecture and presentation/discussion : Participant 3.3.1: (3/22/11 15) I liked the discussion period, I thought it was fun, because we had following along, trying to draw our lines. I wanted to keep drawing my lines and try to figure out h ow everything had a connection. (3/23/15 26) everything when you are looking at it. But then to have all of the different factors written out, you think, and how you could draw everything together help ed me to visually understand it. Participant 4.3.2: (4/32/8 9 ) I learned that you could diagram it and have i t manifested on p aper to talk to people about it. According to the respondents, d iagramming exercises and simplified systems diagrams can help learners develop a conceptual mental image of the connections.

PAGE 33

33 Some of the students reported visualizing other s cenarios through a systems perspective as well as using it as a framework to think about the interconnectedness of situations in general : Participant 1.4.3: (1/4/27 ) It helped us see how it applied to everything. Participant 1.3.3: (1/10/1 5) So I think having this really opened it up so I could broaden it to more things than just what we talked about in those classes Participant 2.2.3: (2/15 16/39 4) system where as the number of trees increase the reflective surface fro m the snow and the sunlight is reflected back up but if there are trees not as much gets reflected so its absorbed so it gets warmer, so more tre e s grow and it s this big cycle Students applied the concepts they learned in the treatments to think about a variety of other situations both in their courses and in their everyday lives. springs for a moment, I woul d like to know from each of you to what degree do you think an abundan ce of nutrients is c ausing excessive algal growth, o n a scale of 0 They were asked to explain their answers. The answers ranged from 5 10 Students from Treatment 2 tended to give higher scores indicating they were more certain that nutrients cause al gae and less certain of other possible causes. Participant 1.2.2: (1/3/32 34) then obviously th e input is creating an output.

PAGE 34

34 Participant 2.5.2: (2/1 2 /16 22) I think 10 .. I studies and personal research that links them heavily and beyond that, what else is there that we were really provided with that could show algal blooms. Participant 2 .4.2: (2/12/23 24) I think it (nitrates) is a primary cause but then I think that there are other ca not educated on. Students from Treatment 3 responded with lower scores and indicated they were open to alternative causes of algae. Parti cipant 1.3.3: (1/4/1 opportunity for the e utrophication stuff to happen. Participant 1.4.3: (1/4/ 9 10 ) the amount of oxygen, the nutrients that we were talking about Participant 2.2.3: (2/12/25 28 ) extent that the nutrients are going to cause algal factors that once it reaches t really grow anymor e if something else limiting it. (2/12/30 34) In the lecture it was like a feedback loop where the algae was blooming a lot and taking a lot of the oxygen out of the water which was killing the snails or making th e snails not eat as much which made the algae bloom more, so it sarily just all the nutrients. (2/12 13/35 5) 9 or 10 but after the lecture I feel really surprised that when we did that modeling thing I thought that it seemed to have a lot less of a direct effect. I would probably say maybe like a 4 to a 6. It seemed like we moved the amt of nutrients up a lot and it barely had any effect and I think it had more having to do with sunlight exposure. I think it definitely has an effect but not as much as I

PAGE 35

35 tho ught before. Because its depende nt on a lot of different variables as well. The students who received Treatment 3 still attributed nutrients as the primary cause of nuisance algae growth, but were more noticeably more open to other less direct causes. This may be an indication that the presentation of the model was more effective at relaying the concept of indirect causality Final Exam Q uestion A question was asked on the final exam to determine if students had retained information they learned in the treatments. The question was: the best response to the question of excess algae in springs? a. Increased nutrients have been pro ven to be the cause b. Increased nutrients are the most likely cause c. Increased nutrients may be one cause d. There is little evidence that increased nutrients are the cause Four of the students from the initial study were not included in this data set -two stu dents dropped the course and two others left the an swer blank on their final exams, the sample size of this population was 36. The responses to this question indicated that perhaps most students were at least able to question the importance of nutrients i n the springs systems (Table 3 6 ). Out of the students who participated in the study, half (n=23) indicated that nutrients may be only one cause and three students indicated that there was little evidence that nutrients cause increased algae a point that w as made in all discussion sections. This was remarkable, since on the pretests, only three from the entire population of students listed any causes other than nutrients as being responsible for excess algae.

PAGE 36

36 A number of Treatment 2 students (n= 9 ) respon ded that nutrients were most likely the cause, compared to one from Treatment 3 who g ave this respon se Although all students received information that should have caused them to consider additional factors, a large minority failed to remember them. Limit ations As with any research study, there are limitations. For this project the small sample size, non random assignment of treatment groups, timing of discussion periods, length of pre/post tests compared to time allotted to take them, and quality of instr ument were some of the potential limitations. The population was small and determined by the number of students who registered for the course. The uneven distribution of participants in the treatment groups was determined by the timing of the discussion s ections. Having a larger sample population, with an equal number of participants in each treatment would have given the results greater statistical power and made them more generalizable to the population of undergraduate students The students were assig ned to the treatment groups based on the course section in which they were registered. This did not allow for non r andom assignment, which can adversely impact the external validity of the results. Each of the treatments did, however, contain a relatively diverse mixture of s tudents based on the number of majors student classification and self reported GPAs Therefore the potential effects of the validity of the study should have been minimized. Another limitation was the timing of the discussion sections ; the Treatment 2 group received the treatment directly after the systems thinking presentation, while the other two groups received treatments one day after later. This may have impacted the

PAGE 37

37 results, as the subjects in Treatment 3 had slightly higher post test scores (mean score= 30.92) than the Treatment 2 (mean score= 29.28). Treatment 3 participants may have benefited from the additional time between presentation and treatment, as it may have served to allow them to absorb and then apply the information. Another limitation was the length of the pre and posttest s and the amount of time allotted to take them. The pretest s were administered at the beginning of lecture a week prior to the intervention. Students were allowed 20 minutes to finish the test, but several were given extra time. All students completed the pretest s within the lecture period. However, the posttest s were given after the interventions, which took up most of the discussion period. In order to prevent students from being late to other cla sses, they were given the option to take the post tests home and return them the following day during lecture. The open ended nature of the responses made it possible to allow students to complete the post test at home; there were no outside resources that could have assisted them. Students were also assured that there were no wrong answers, in an attempt to mitigate any desire the students had to work together to get perfect scores on the test. The fact that the data collection tool was lengthy and student s had a limited amount of time could have impacted the quality of the answers they provided but shortening the tool would have removed potentially helpful questions However, the short period of time provided the opportunity to capture their initial think ing, which was more useful than provid ing lengthy answers that students were less sure of. Another potential limitation is the quality of the instrument used to test the effectiveness of the teaching strategies. Although expert review help ed increase cont ent validity, it is possible that the questions could have been improved to better a scertain knowledge of systems.

PAGE 38

38 Table 3 1 Mean scores for pre and posttest for each treatment Paired Samples Statistics for Mean Scores Mean N Std. Deviation Std. Error Mean Treatment 1 mean Pre Treatment 1 m ean Post 20.22 9 8.614 2.871 27.89 9 10.517 3.506 Treatment 2 m ean Pre Treatment 2 m ean Post 21.06 18 4.556 1.074 29.28 18 4.944 1.165 Treatment 3 m ean Pre Treatment 3 mean P ost 20.92 13 5.299 1.470 30.92 13 7.522 2.086 T able 3 2 Paired t tests for pre and posttest means for each of the treatments Paired Differences for Mean Scores t df Sig. (2 tailed) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Treatment 1 Pre Treatment 1 Post 7.667 3.808 1.269 10.594 4.740 6.040 8 .000 Treatment 2 Pre Treatment 2 Post 8.222 5.128 1.209 10.773 5.672 6.802 17 .000 Treatment 3 Pre Treatment 3 Post 10.000 6.683 1.854 14.039 5.961 5 .395 12 .000 Table 3 3 One way ANOVA on the posttests of Treatments 2 and 3 Sum of Squares df Mean Square F Sig. Between Groups 20.434 1 20.434 .541 .468 Within Groups 1094.534 29 37.743 Total 1114.968 30

PAGE 39

39 Table 3 4 Correlation statistics for student demographics for year in school, prior systems experience and pre/posttest scores Prior ST exp Total Pre Total Post Final exam Classification Pearson Correlation .116 .439 ** .035 .173 Sig. (2 tailed) .483 .005 .833 .319 N 39 39 39 35 P rior ST exp Pearson Correlation 1 .007 .113 .000 Sig. (2 tailed) .968 .486 1.000 N 40 40 40 36 Total Pre Pearson Correlation .007 1 .675 ** .054 Sig. (2 tailed) .968 .000 .754 N 40 40 40 36 Total Post Pearson Correlation .113 .675 ** 1 .022 Si g. (2 tailed) .486 .000 .901 N 40 40 40 36 Table 3 5 Correlation statistics for student demographics for course grades, majors, GPA, and pretest scores Class Grade Major GPA Classificatio n Class Grade Pearson Correlation 1 .343 .242 .158 Sig. (2 tailed) .035 .161 .351 N 38 38 35 37 Major Pearson Correlation .343 1 .080 .125 Sig. (2 tailed) .035 .636 .449 N 38 40 37 39 GPA Pearson Correlation .242 .080 1 .374 Sig. (2 tailed) .161 .636 .025 N 35 37 37 36 Total Pre Pearson Corr elation .242 .181 .183 .439 ** Sig. (2 tailed) .142 .265 .279 .005 N 38 40 37 39

PAGE 40

40 Table 3 6 Responses to the final exam question about algae and springs Group Nutrients are the cause Nutrients are most likely the cause Nutrients may be one cause The re is little evidence that nutrients are the cause Treatment 2 5% (n=2) 19% (n=7) 36% (n=13) 5% (n=2) Treatment 3 0 5% (n=1) 27% (n=10) 3% (n=1) More students who received Treatment 2 were le ss open to additional factors

PAGE 41

CHAPTER 4 DISCUSSION Hypothesis 1 The data clearly suggest that key systems thinking concepts can be taught in a one hour lecture By using multiple and relevant examples and focusing on an fS B F format, undergraduate students were able to apply these concepts to different scenarios. Tr eatment 1 posttest scores were significantly higher than the pretest s, so we must reject null hypothesis that a short presentation using examples of indirect causality, feedback loops, and delays using function centered S B F explanations and multiple, con crete examples will not effectively convey systems concepts to learners. Hypothesis 2 U s ing the computer model did not yield any significant difference in posttest scores between Treatments 2 and 3 however t he mean posttest scores for these treatments wer e increased (p >.0 5 ) (see Table 3 2 ). So we cannot reject the null hypothesis A computer model coupled with a presentation/discussion was not a more useful component to help learners understand indirect causality, feedback mechanisms, and delay than the discussion and presentation alone. Focus Groups The information gathered in the focus groups helped to provide insight into what students believe are the most effective strategies to help them understand the systems concepts we emphasized in this interven tion. Many of the students commented on the helpfulness of seeing multiple examples of systems that they could relate to and were familiar with. They also said the diagramming exercise helped them to visualize the

PAGE 42

connections and to see how the components interacted within the system. Only one student mentioned that the computer model was useful. Final Exam The data from the final exam question reveal 19% of students from Treatment 2 believe that nutrients are most likely the cause of excessive algae compa red 5% from Treatment 3. Students received identical information related to this topic, except that Treatment 3 saw a computer model to show the complexity of the springs system. Assuming the students in the two groups are simila r and the presentation/disc ussion was effective, e ither the students from Treatment 2 were not paying attention, rejected the new information, or did not believe the new information because they did not see it presented in the form of the computer model. Synthesis There is literatu re to suggest that using computer models to teach about systems can be quite successful in terms of participant learning and retention ( Jacobson and Wilensky 2006 ; Liu and Hmelo Silver 2009 ; Reiss and Mischo, 20 10 ), however most of those studies were cond ucted over a period of several weeks to a semester and students manipulated the model themselves There is a lengthy time commitment associated with learning how models operate The intention of this study was to determine if using the model as a visual ai d could help learners conceptualize the processes of this complex system. The focus groups and final exam question indicate that the model was more useful in helping learners have a more holistic perception of springs health and causes of excessive algae, though the posttest scores do not corroborate that finding Follow up focus groups with all of the students from the study

PAGE 43

and a direct question about the computer model might have enabled us to better understand the usefulness of the model Systems think ing concepts can be taught in short interventions if characteristics of the intervention include multiple familiar, relevant examples, as well as exercises that engage learners, such as diagramming exercises. The information gathered during the focus grou ps indicate the students considered those factors to be integral to their understandin g of systems thinking concepts. This information can be used to develop effective interventions to teach systems thinking concepts to a variety of audiences, from college classrooms to non formal education settings. This study shed some light on the characteristics that a successful systems thinking intervention could contain: relevant and multiple examples and a diagramming exercise. These results could be useful to educa tors, systems thinking experts or not, who can develop units or lectures to conv ey systems thinking skills to their students.

PAGE 44

CHAPTER 5 CONCLUSION Much of the world can be described in terms of complex and dynamic systems consisting of feedback mechanis ms and non linear causal relationships A systems thinking framew ork can help people understand complex interactions It is becoming increasingly more necessary for the public to understand concepts such as indirect causality, feedback mechanisms and dela y ed causality, especially a s environmental degradation, water scarcity, and threats of a changing climate promise to affect humans on a global scale. The public needs to be involved in resolving science based issues that affect socio ecologic systems (Stave 2002) In order for citizens to be involved in the decision making process, they must be present and have a presence among scientists In order for this to occur, the public need s to have a working knowledge of systems and have the ability to apply system s thinking to a variety of situations. Therefore it is important that systems thinking be developed as a thinking paradigm for all. Understanding what the audience knows about a topic is integral to developing successful interventions. An audience assessm ent would be ideal, as it can reveal what the intended audience knows, what they do no t and whether or not they hold any misconceptions about the information at ha nd (Jacobson, McDuff and Monroe 2007). In prior knowledge probably hi ndered their ability to consider alternate explanations about the causes of excessive algae Several focus group participants exposed their firmly embedded notions about the causal relationships between nitrates and algae and had a difficult time consideri ng other possibilities. This is not surprising, as literature states that when people are provided with information that

PAGE 45

is counter to their mental model, they disregard the new, unfamiliar information ( Festinger 195 7 ; Kearney and Kaplan 1997). Despite pr esenting information that would cause attentive students to recognize some level of cognitive dissonance and learn new information, many students did not. The focus group data suggest that prior knowledge about nutrients, eutrophication, and algae are part Only one student expressed the impact of the surprising revelation of the computer model and this student was a very high achiever throughout the course It would likely require more than one presentation for most students to call prior knowledge into question Even though the students were provided with information that should have caused them to consider things other than nitrates and fertilizers as culprits in promoting excessive algae growth, less than half (n=12) of the 31 students in the Treatments 2 and 3 included additional factors in their posttest answers. This is after the students received a presentation on the topic and diagrammed many of the possible causes of increased algae. During the focus groups, respondent s from Treatment 2 stated there was a strong connection between algae and nitrates, while Treatment 3 students were less certain Interestingly, on the final exam, most (n=26) of the 36 students from the Treatments 2 and 3 considered things other than nitr ates as causing nuisance algae. This could be a result of the multiple iterations of systems throughout the course or perhaps students learned that most systems often surprise us and therefore were not willing to provide a response that indicated certainty Systems thinking skills can be taught in a variety of contexts to a diverse array of audiences. The results of this study suggest that lengthy courses dedicated to systems thinking are not the only way to convey systems concepts. Undergraduate students

PAGE 46

c ould benefit from presentations introducing systems concepts in a variety of courses Brief units that teach systems thinking concepts can be useful tools to aid educators in relaying concepts of delay, feedback, and indirect causality to students and incr easing science literacy. Undergraduate students in a variety of majors can use these concepts to explain market fluctuations, ecosystem function, and international negotiations. These u ndergraduates may be similar to other audiences, as well. The types of adults who are likely to be interested in working on a local issue and voicing their ideas about solutions may be similar to the undergraduates who took this course as an elective. If so, then interested adults could probably learn about systems c oncepts i n a short lecture and gain enough understanding to apply this information to the issue of interest There are particular contexts in which these adults gather that may be most conducive to learning about systems Groups where participants meet repeatedly a nd could understand that having a background in systems as a worthy investment could apply systems thinking to problems they want to solve. Participatory processes, such as community forums, fo cus groups, working groups, faith based study circles, and citi zen advisory boards, especially those that seek to make sense of complex environmental problems could benefit from teaching strategies such as th e s e However, if prior knowledge and common perceptions interfere with understanding the relationships between elements, more effort might be needed to help learners assimilate the new information. This study demonstrates that it is possible to teach systems thinking skills using short interventions. Students in this study gained a different understanding of how to

PAGE 47

view ecological problems. Using multip le examples (DeYoung and Monroe 1996; Eysink et al. 2009) with scenarios that were relevant and familiar, enabled students to direct their cognitive attention t oward learning the skills associated with systems thin king. Because systems can be confusing, non intuitive and difficult for even highly educated individuals to understand (Sweeney and Sterman 2007), it is important that learners involved in systems thinking programs are not overloaded by attempting to lear n new information as well as a new way of thinking Using familiar content could allow presentations to be developed using function centered S B F as the framework. While computer models have proved to be an important tool to teach systems thinking skills to learners (Deaton and Winebrake 2000 pg. 13; Liu and Hmelo Silver 2009 ; Thompson and Reimann 2010), in this context and at least in the short term the model was not significantly more helpful than no model Students improved scores in both treatments, with and without the computer model. Perhaps this implies that systems thinking presentations can be effectively simplified and that purveyors of systems thinking interventions should not be intimidated by believing that a computer model is requi si te for r elaying the concepts. However, the differences in the focus group data and final exam question do indicate that the model may have been useful in allowing students to consider the roles of the system components if the treatment groups were sufficiently eq uivalent which they appear to be These differences may also be attributed to the continued reinforcement of the concepts throughout the course that somehow amplified the minor differences between treatment groups Relaying the concepts of indirect and de layed causality and feedback mechanisms to learners is foundational to developing a systems perspective. Having an

PAGE 48

understanding of these skills enables a person to make sense of myriad systems. Fostering a systems perspective in learners can provide them with a valuable framework for thinking as well as enhanc ing science literacy In response to the need to promote science literacy among the public, a N ational S cience F oundation panel concluded, One of the most compelling challenges of our time is to enha nce the al. 2003 pg 41 ) Developing s trategies for effectively teaching systems thinking concepts in short interventions deserves further exploration This study elucid ated some important characteristics of successful methods to teach systems thinking skills. However more work could be done determining the best methods for teaching each concept and how the concepts are best mastered Further research is also needed to u nderstand long term effects of learning systems thinking skills, how often these skills should be reinforced, and how easily these concepts are transferred from one type of system to others.

PAGE 49

49 APPENDIX A PRETEST What is your Major? ________________ _______________________ What is your Minor, if you have one? _________________________________________________ What is your GPA? ____________________ According to the number of credits you have earned, are you classified as a: A. Freshman B. Sophomore C Junior D. Senior E. Graduate student How long have you been a student at UF? A. Less than a year B. 1 2 years C. 2 3 years D. 3 4 years E. More than 4 years If you have heard of systems or systems thinking, such as Systems Ecology, Mechanical Systems etc. before today, where did you hear about it? Circle the answer that best describes your understanding of systems thinking. A. Not at all B. A little understanding C. Moderate understanding D. Pretty good understanding E. Understand it very well Pl ease answer each question with a specific sentence or two at most. Please print if you is most appropriate. 1. An older woman broke her left wrist as a child. It hea led, but not perfectly, and she shied away from using it throughout life. She recently developed a pain in her right wrist,

PAGE 50

50 not surprised about pain in the right wr ist. What is the best explanation why the right wrist is bothering the woman? 1a. Can you give another example from your own experiences where this sort of phenomenon occurs? 2. Kelp forests are found in marine environments and are formed by a variety of giant algae that is anchored to the ocean floor. One of the main herbivores of kelp are sea allowing the kelp to float away. Sea otters eat urchins. Historical hunting of otters altered the entire ecosystem. How would hunting otters affect the kelp? 2a. Can you think of another example where altering one part of the system unexpectedly affected some other part? 3. What are some ways that reduced snowfall affects the econom y? 4. There is a device that measures the soil moisture content of lawns and controls when sprinklers come on. The purpose is to save water by eliminating unnecessary watering. Please explain process that occurs for this technology to conserve water. 4 a. Can you give another example from your own experiences where this sort of phenomenon occurs? 5. Imagine two people sharing a bed in Minnesota with a dual electric blanket that has two controls, one for each half of the blanket. When one person becomes chilly, she turns the dial to heat up her half of the blanket. The controls on this particular blanket side of the blanket and vice versa. Can you explain what happens? 5a. Can you give another example from your own experiences where this sort of phenomenon occurs? 6. Pine forests are fire adapted in that pine needles burn easily and pine bark is highly protective of the tree. Frequent fire (say every 2 3 years) reduce s competition from other trees that could grow faster and taller than pines when fire is absent, but are likely to die when fire is present. Moreover, these competitors have leaves that burn much less easily, so as they outcompete pines, fire also becomes much less likely. What do you think the effect would be of a decade or more of fire exclusion? 6a. Can you think of another example from your own experience where a small change was self perpetuating? 7. Growing up, you were taught to apply waterproof sunscreen throughout the day to prevent sunburn. On a trip to the beach, you diligently check your skin for signs of pink and reapply consistently. When you get home, however, you find that you are sunburnt. Why?

PAGE 51

51 7a. Can you give another example from y our own experiences where this sort of phenomenon occurs? check your balance each time you make a purchase to be sure you have enough money. A week after Christmas you receive 7 bounced transaction notices in the mail. Why were you not aware that this was going to happen? 8a. Can you give another example from your own experiences where this sort of phenomenon occurs? 9. Have you been to any of the springs in nort h Florida? 9a. How familiar are you with the causes of increased algae in Florida Springs ecosystem? A. Not at all B. Slightly familiar C. Somewhat familiar D. Fairly familiar E. Very familiar 9b. Can you explain what might be the causes of the algae in the springs and spring runs?

PAGE 52

52 APPENDIX B POSTTEST Circle the answer that best describes your understanding of systems thinking. A. Not at all B. A little understanding C. Moderate understanding D. Pretty good understanding E. Understand it very well Please answer each question with a specific sentence or two at most. Please print if you is most appropriate. 1. An older woman broke her left wrist as a child. It healed, but not perfectly, and she shied away from using it throughout life. She recently developed a pain in her right wrist, not surprised about pain in the right wrist. What is the best explanation why the right wrist is bothering the woman? 1a. Can you give another example from your own experiences where this sort of phenomenon occurs? 2. Kelp forests are found in marine environments and are formed by a vari ety of giant algae that is anchored to the ocean floor. One of the main herbivores of kelp are sea allowing the kelp to float away. Sea otters eat urchins. Historical hunting of otters altered the entire ecosystem. How would hunting otters affect the kelp? 2a. Can you think of another example where altering one part of the system unexpectedly affected some other part? 3. What are some ways that reduced snowfall affects the eco nomy? 4. There is a device that measures the soil moisture content of lawns and controls when sprinklers come on. The purpose is to save water by eliminating unnecessary watering. Please explain process that occurs for this technology to conserve water. 4a. Can you give another example from your own experiences where this sort of phenomenon occurs? 5. Imagine two people sharing a bed in Minnesota with a dual electric blanket that has two controls, one for each half of the blanket. When one person becom es chilly, she turns the dial to heat up her half of the blanket. The controls on this particular blanket side of the blanket and vice versa. Can you explain what happens ? 5a. Can you give another example from your own experiences where this sort of phenomenon occurs?

PAGE 53

53 6. Pine forests are fire adapted in that pine needles burn easily and pine bark is highly protective of the tree. Frequent fire (say every 2 3 years) redu ces competition from other trees that could grow faster and taller than pines when fire is absent, but are likely to die when fire is present. Moreover, these competitors have leaves that burn much less easily, so as they outcompete pines, fire also becom es much less likely. What do you think the effect would be of a decade or more of fire exclusion? 6a. Can you think of another example from your own experience where a small change was self perpetuating? 7. Growing up, you were taught to apply waterproo f sunscreen throughout the day to prevent sunburn. On a trip to the beach, you diligently check your skin for signs of pink and reapply consistently. When you get home, however, you find that you are sunburnt. Why? 7a. Can you give another example from your own experiences where this sort of phenomenon occurs? check your balance each time you make a purchase to be sure you have enough money. A week after Christma s you receive 7 bounced transaction notices in the mail. Why were you not aware that this was going to happen? 8a. Can you give another example from your own experiences where this sort of phenomenon occurs? 9. Have you been to any of the springs in no rth Florida? [Yes] [No] 9a. How familiar are you with the causes of increased algae in Florida Springs ecosystem? A. Not at all B. Slightly familiar C. Somewhat familiar D. Fairly familiar E. Very familiar 9b. Can you explain what might be the causes of the algae in the springs and spring runs?

PAGE 54

54 APPENDIX C SCORING RUBRIC Prior systems experience 0=Not heard of systems 1= Heard of it briefly (mentioned, TV, etc) 2= Heard it in course/courses or read about it in journals 3=Took at least one class rel ated to systems Systems Understanding 1=A 2=B 3=C 4=D 5=E 1. Wrist Indirect causality 1= No understanding of indirect causality or incorrect/incomplete explanation 2= Overuse due to neglect of Left wrist 3= Great explanat ion of indirect causality 1a. Indirect causality 1= No understanding of indirect causality or incorrect/incomplete explanation 2= Adequate explanation of indirect causality 3= Great example of indirect causality 2. Kelp forests indirect causality 1= No understanding of indirect causality or incorrect /incomplete explanation 2= Fewer otters to eat urchins leads to increased grazing 3= Great explanation of indirect causality 2a. Indirect causality 1= No understanding of indirect causality or incorrect/incomplete explanation 2= Adequate explanation of indirect causality 3= Great example of indirect causality 3. Sprinklers Balancing feedback 1= No understanding of balancing feedback or incorrect /incomplete explanation 2= The sprinklers come on when the soil is dry, thus conserving water 3= When the soil becomes dry, the sprinklers turn on and stay on until the soil reaches a certain wetness, then they turn off

PAGE 55

55 3a Balancing feedback 1= No understanding of balancing feedback or incorrect/incomplete explanation 2= Adequate explanation of balancing feedback 3= Great example of balancing feedback 4. Heating blanket Reinforcing feedback 1= No understanding of reinforcing feedback or incorrect/incomplete explanation 2= Adequate explanation of reinforcing feedback One gets hot and turns down the dial 3= Great example of reinforcing feedback As one gets colder they turn up the dial, he ating up the other, who turns down the dial. One continually gets hotter, as the other gets colder 4a. Reinforcing feedback 1= No understanding of reinforcing feedback or incorrect/incomplete explanation 2= Adequate explanation of reinfo rcing feedback 3= Great example of reinforcing feedback 5. Fire exclusion Reinforcing feedback 1= No understanding of reinforcing feedback or incorrect/incomplete explanation 2= Explanation of the process Pines would be replaced by ha rdwoods 3= Adequate explanation of reinforcing feedback Pines would be replaced by hardwoods, which would then exclude fire, allowing hardwood succession to continue 5a. Reinforcing feedback 1= No understanding of reinforcing feedback or incorrect/incomplete explanation 2= Explanation of process without connection of the cycle 3= Adequate explanation of reinforcing feedback 6. Sunburn Delay or lag 1= No understanding of delays/lags or incorrect/incomplete explanation 2= Adequate explanation of delay/lag 3= Great example of delay/lag 6a. Delay or lag 1= No understanding of delays/lags or incorrect/incomplete explanation 2= Adequate explanation of delay/lag 3= Great example of delay/lag

PAGE 56

56 7. Bounced tr ansactions Delay or lag 1= No understanding of delays/lags or incorrect/incomplete explanation 2= Adequate explanation of delay/lag 3= Great example of delay/lag 7a. Delay or lag 1= No understanding of delays/lags or in correct/incomplete explanation 2= Adequate explanation of delay/lag 3= Great example of delay/lag 8. Snowfall affects on economy rely on snow suffer (snow plows, ski resorts, salt companies) 2= Lower heating bills, leaving people with more money 3= Additional indirect effects mentioned snowmelt, river flows, albedo and climate, affects on crops 9. Visited springs 0= No 1=Yes 9a. Familiar with causes of al gae 0= Not at all 1= Slightly familiar 2= Somewhat familiar 3= Fairly familiar 4= Very familiar 9b. Explain causes of algae 2= Fewer than 3 reduced grazers, DO, recreation, invasives, flow, nitrates, reduced consumption of algae 3= 3 or more reduced grazers, DO, recreation, invasives, flow, nitrates, and reduced consumption of algae Final Exam Question 0= Increased nutrients have been proven to be the cause 1= Increased nut rients are the most likely cause 2= Increased nutrients may be one cause 3= There is little evidence that increased nutrients are the cause

PAGE 57

57 APPENDIX D FOCUS GROUP QUESTION S 1. Do you remember any of the scenarios from the quizzes at the beginning of the sem ester? a. What was the principle involved in the scenario? 2. Since the systems thinking lecture, have you seen that principle in other situations? 3. Thinking abo ut north central Florida springs for a moment, I would like to know from each of you, to what degree do you think an abundance of nutrients is causing excessive algal growth. On a scale of 0 10. a. Can you explain why you feel that way? 4. Lets take a m example. a. Did they cause you to think about other systems? 6. Have you thought about systems in your day to day life since Matt a. Can you think of examples of systems you have seen? 7. Can you think of some different examples of systems that demonstrate basic features that we were explaining? 8. You were selected because you had great improvement from the pre to posttest. What do you think helped you to understand systems? a. And what about those things were most useful? 9. Can you think of anything that we could have done or explained differently t hat would have been more helpful? 10. For those of you who heard of ST in course, which course was it? a. What kinds of things did they say about systems? b. What was the context that they used to talk about systems? c. Did you find yourself thinking a bout systems differently after learning about systems?

PAGE 58

58 APPENDIX E IRB FOR PRE AND POST TEST DATA

PAGE 59

59

PAGE 60

60 APPENDIX F IRB FOR FOCUS GROUPS

PAGE 61

61

PAGE 62

62 APPENDIX G CODE SHEETS FOR FOCU S GROUP ANALYSIS Code Definition: Students were able to recall the correct systems thinking concept and as it related to the example used in the lecture Coder: Lara Colley Gerund: Remembering correct concept from example Protocol Belongs to: Lara Colley Systems Thinking Focus Group Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/1/6 There was the delay one the sunburn 1/2/1 The sunburn one was delay. 2/1/15 17 The sunscreen one was like time lag, the time differences between two variables and I think that was also about the checking acct. 3/ 20 21/31 2 I guess indirect causality is obvious. 1 : Do you remember which scenario that was related to? 2: I guess otters and algae and kelp. 3/23/27 28 I remember the foxes and rabbits with the different time cycles. I guess the slow and fast effects. 4/25/13 15 Time lags and delays, there was an example about going out in the sun. Wait is this systems thinking? Yeah, the sunburn

PAGE 63

63 Coder: Lara Colley Gerund: Applying systems thinking to other contexts Protocol Belong s to: Lara Colley Systems Thinking Focus Group Code Definition: Students gave examples describing instances where they have used systems thinking other than on pre and posttest Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/ 2/18 20 thinking and stuff like that 1/4/27 It helped us see how it applied to everything 1/10/1 5 After this lecture I was made aware that I have had ST examples in previo us classes, think having this really opened it up so I could broaden it to more things than just what we talked about in those classes. 2/15/10 13 I actually noticed I could apply it to one of my other clas ses where I had to make a site plan for a development in an urban planning class and just seeing the impacts on the environment and the effects and I applied the principles like mentally. 2/15 16/39 4 as the number of trees increase the reflective surface from the snow and the sunlight is reflected back up but if there are trees not as much gets reflected so its absorbed so it gets warmer, so more tress grow and its this big cycle. 2/16/18 25 Cayman Islands for a week cause my dad teaches a coral reef ecology experience with that but I would imagine that coral reefs is a huge system so I think all the the pollut ion and it effecting the global climate change and the heat which is effecting the acidity in the water, which is effecting the reef, which is effecting the reef so you could apply it to that.

PAGE 64

64 2/16/5 11 Kind of off the general theme of ecology in one of my classes its kind of agricultural theory and different ideas about how to communicate in agriculture and politically speaking I think systems thinking could be local scale in their own communities is effectin interacting, the farm bureau and things along those lines. 2/16/20 25 that but I would imagine that coral reefs is a putting in the environment and the pollution and it effect ing the global climate change and the heat which is effecting the acidity in the water, which is effecting the reef, which is effecting the reef so you could apply it to that. 2/19/1 3 and it was so focused and only related to that marshland ecosystem and just the fiddler crab that it was difficult to apply it to greater things but I thought back to that because he chose intricate food webs that went on for pages and pages went on for pages and pages but it tied back into ST cause he was able to relate the model with sound science 2/19/6 7 I agree in a biology standpoint just in the human body with the feedback loops, like with the brain, like with homeostasis. 2/19/21 22 When they provided it as thinking methodology that could be applied to a wider varie ty of things, I think it was really helpful. 4/26/30 31 delay in the fact that they are eating an insane amount of baby birds and they are finally starting to quantify that. 4/27/1 3 In my research I kind of hinted on that (Lags). There are immediate i mpacts and there are long term impacts of climate change. The immediate impacts are on leaf chemistry and long term on biomass. 4/30/28 30 There will be delay (in decreasing traffic), just mean everyone will rid e. 4/31/1 2 How there are different drivers for things you see in nature. It has to do with lots of factors, not just temp. its precipitation, interactions with other species, other organisms. My head is in ecological systems.

PAGE 65

65 Coder: Lara Colley G erund: Using systems thinking as a framework Protocol Belongs to: Lara Colley Systems Thinking Focus Group Code Definition: Students describe how systems thinking can be used as a framework to think about a number of topics or issues Protocol Code: (P rotocol #/page #/line #) Answers Recorded in Protocol 1/3/9/12 As for putting terms on things cause it was always there but after the lecture, like I said, it was like OHHHH. Kinda helped me out with the class kinda helped me do better on the test. 1/5/ 3 35 yea that and it helped you realize that you could almost think of anything and it would become a system if you associated it with something else. 1/7/3 6 and you just never had a way to define it and now you have something definite not just something in your head that you just made up. Something that actually happened. 1/9/30 33 Yea its really useful. It opens your mind up to how something becomes something. For criminology and psychology and showing what causes a person to be a certain way. I think its useful. 1/9/34 35 I agree it can be taught and applied to 2/12/1 4 Same with me in some of my other classes thinking and having that exposure kind of helped me understand what systems thinking was and then apply it to the problems and already have a jump start 2/15/5 9 I think it helped me a lot just seeing systems mentioned earlier, in my cl doing things that involve systems and my systems thinking but that how I see it and it helps me work out the problems.

PAGE 66

66 2/15/14 22 I think one of the reasons I liked it so much is because sometimes between what you do in a classroom and then how you can actually apply that to real life and I feel like for systems thinking its not think, so I think I have been able to apply it t o whatever and just think this is systems thinking, is just the way you think about it. 2/16/28 3 I think is provi des an analytical framework to look at things and prior to that you just analyze on there own and its just the thinking allows you analyze things as a whole and I knowledge of a biomass plant in Gainesville? To be honest when you get into the real world or when you get into whatever industry you rest example in Guatemala, but its much more likely that almost guaranteed at this point because 2/17/1 5 I definitely think you can turn everything into a system so I fe struggling trying to come up with specific examples cause I think you could almost say think about this and talk about it as a system so it just depends on what you want to talk about. 2/19/10 15 I had exposed to it but not the term sy stems thinking. I understood like feedback loops and cycles but never really the term systems but I never thought to apply that ST in other terms. I think having the term ST really of thinking and you can apply it anywhere rather than just saying this is A system. 2/19/19 22 I always associated it with just ecological interactions on a biological level. When they provided it as thinking methodology that could be applied to a wider variety of things, I think it w as really helpful

PAGE 67

67 4/32/10 13 That it could be applied outside of an before. It kind of clicked. It made more sense in terms of an ecosystem, because we have been learning about that for so long. 4/32/14 21 You c an use systems thinking to describe any policy or decision. People always ask me what do you want to do this or that and if you know the whole system, you can give them affects on other people, and climate change is a big one where you are trying to connect everything. I think Systems is the only way you can connect in terms of climate change. People have to learn about it. Coder: Lara Colley Gerund: Diagramming useful Protocol Belongs to: Lara Colley Sys tems Thinking Focus Group Code Definition: Students indicate that the diagramming exercise was a useful tool Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/6 7/34 2 I am a visual learne r, so the hands on stuff helps. The be fore and after pictures and the loops always helped me just so I could see it. Better than just words. Showing this has a negative impact and then seeing where the arrows go from there. The examples were really helpful. 3/22/11 15 I liked the discussion period, I thought it was fun, because we had that paper and we were following along and trying to draw our lines. But I also was a little confused on the concepts. But I wanted to keep drawing my lines and try to figure out how everything had a connection. I did find it interesting, it was a little confusing as well.

PAGE 68

68 3/23/15 26 I feel like seeing it visually was helpful. Because especially with the algae example, when you are looking at it. But then to have all of the different factors written out, you think How can you connect them? I think the exercise really helped me, Wow so the manatees and this. I think seeing that..the pluses and minuses still get me. But seeing the web and how you could draw everythin g together helped me to visually understand it. Just hearing about it and seeing one arrow between one thing is an example, but seeing even all of the factors you could add in, but it nk about how boats might affect algae or whatever was on there. 4/31/24 28 Do you think more instruction on the diagramming part would have been helpful? Like is that a useful tool to helping you understand systems? Yeah, the visual element is very usefu l. 4/31/24 28 Do you think more instruction on the diagramming part would have been helpful? Like is that a useful tool to helping you understand systems? Yeah, diagramming is useful Coder: Lara Colley Gerund: Seeing diagrams was helpful Protocol Belo ngs to: Lara Colley Systems Thinking Focus Group Code Definition: Students indicate that seeing the systems diagrams during the lecture and treatments were helpful Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/8/16 20 the re indeer one was the one I remember the most. You could see how the reindeer were effecting this which in turned affected the nesting bird leaving then led to a big deal with fish and wildlife service. the graph with the arrows helped cause otherwise it wou ld be like a paragraph. 1/8/29 31 I agree that having it right in front of you helped and being able to pick out one thing to look out and can see how it effects others.

PAGE 69

69 1/9/1 3 a good way to explore all the different inputs and outputs cause its not going to cause something to change you can 1/9/11 20 I think that if I read a paper about the springs and it had a graph that showed all of those things that we diagrammed in class, I thi nk that would be the thing I would focus on, instead trying to read through the discussion and results, and kind of gather from that. If I found the diagram interesting and wanted to learn more about a certain input, then I would go back through the paper to find out about it. The diagram would peak my interest and then cause me to go back and read about that complex issue. It would help me understand it instead of trying to read through a paper. 4/32/8 9 I learned that you could diagram it and have it ma nifested on paper to talk to people about it Coder: Lara Colley Gerund: Learning from relevant examples Protocol Belongs to: Lara Colley Systems Thinking Focus Group Code Definition: The use of relevant examples made salient the abstractness of sys tems thinking Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/6/1 2 Like the cats and dogs, there were all these terms and then you give the reasons and examples it made perfect sense 1/6/6 10 ke a real something personal that happened in his life imagine. 2/14/3 6 The graphi cal representation helps; I think The fact that he boiled it down to examples like cats and rats everyone was able to first grasp it on that level and then move to an environmental level. 2/14/7 9 I agree with that. I think the real world examples and the simplicity of it initially

PAGE 70

70 helped to understand the general concepts. It just made it easier to understand. 3/22/16 19 I enjoyed the original lecture. I like when he explained it with the system of the dogs an d cats and neighborhood. I thought that helped explain the concepts in a simple way and then he expanded it with bigger examples and more subtle interactions. 4/29/11 16 I thought it was interesting, we did it in another class but it was all very abstrac t with was talking about cats, dogs, and rats, I know what those are, I know what happens in that kind of scenario. It cleared a lot of things up, it was interesting to finally learn about it. Coder: Lara Colley Gerund: Learning from multiple examples Protocol Belongs to: Lara Colley Systems Thinking Focus Group Code Definition: The use of multiple examples helped students understand concepts Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/5/28 I think the biggest help was all the examples of everything. 1/5/36 The examples are the most effective 1/7/2 The examples were really helpful. 3/23/14 Having multiple examples

PAGE 71

71 Coder: Lara Colley Gerund: Remembering terms was d ifficult Protocol Belongs to: Lara Colley Systems Thinking Focus Group Code Definition: The students had difficulty remembering the correct terminology Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/2/13 not in the specific terms 1/8/9 11 until you said it, but I would be able to recognize that when this keeps happening, this will happen more that sort of thing. 2/21/15 I cant remember the terms. 4/25/27 28 Positive feedba Coder: Lara Colley Gerund: Using signs appropriately is confusing Protocol Belongs to: Lara Colley Systems Thinking Focus Group Code Definition: Using positive and negative symbols were challenging Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/9/8 10 The plus signs and minus signs got confusing. I knew what I wanted to say but the way I put it, well that is not really what you saying. 1/9/24 26 the plus signs and minus signs are not intuitive and confusing. You think a positive sign means something increases and that is confusing 3/23/20 the pluses and minuses still get me

PAGE 72

72 Coder: Lara Colley Gerund: Recognizing factors that cause algae Protocol Be longs to: Lara Colley Systems Thinking Focus Group Code Definition: Response to their thoughts on how strongly they felt nitrates cause algae Protocol Code: (Protocol #/page #/line #) Answers Recorded in Protocol 1/3/32 34 nutrients then obviously the input is creating an output 1/3/35 37 as directly as they assumed. After they did all they did all the charts and graphs in the class it was that but it was thru other sources. 1/4/1 5 erosion of the natural grasses is gonna have a impact on that too and that kinda provides more oppo rtunity for the eutrophication stuff to happen. 1/4/9 10 the amount of oxygen, the nutrients that we were talking about 1/4/11 there was something about manatees 1/4/12 14 I remember there was never just one thing, it was everything coming together. M ore people, more nutrients and everything all at once 2/11/16 22 between the two more than just the cursory seen other studies and personal research that links them heavily and beyond that, what else is there that we were really provided with that could show algal blooms besides I guess even interaction but its all interrelated back to the systems thinking methodology. 2/12/23 24 I think it (nitrates) is a primary c ause but then educated on 2/12/25 28 that the nutrients are going to cause algal factors that once it reaches t hat threshold it limiting it. 2/12/30 34 In the lecture it was like a feedback loop

PAGE 73

73 where the algae was blooming a lot and taking a lot of the oxygen out of the water which was killing the snails or making the snails not eat as much which made the algae the nutrients. 2/12 13/35 5 but after the lecture I feel really surprised that when we did that modeling thing I t hought that it seemed to have a lot less of a direct effect. I would probably say maybe like a 4 to a 6. It seemed like we moved the amt of nutrients up a lot and it barely had any effect and I think it had more having to do with sunlight exposure. I thin k it definitely has an effect but not as much as I thought before. Because its dependant on a lot of different variables as well. 2/21/25 29 8, I think a lot of fertilizer run off and stuff is a major cause of algae blooms in water. It may not be the onl y cause, but to grow out of control its either that or sunlight, its not physical things. 3/21/30 34 I would probably say 6, because I remember as well, so I am not as confident. It brought back memories, I can kind of remember there being other factor s involved, I just cant remember. 4/27/21 25 8 or a 9. I think its really high. I have only been here for a few years and going to Ichetucknee then and now, there is a definite correlation. On top of that, if you drive around the area, you can see that Ag operations are way up in the area. Even people who work there will tell you that the farmers runoff is causing algae 4/27/26 27 10. That is just something I learned early on in school. Some Env. Class in elementary or middle school 4/27/28 32 At the s pring head, its probably an interaction between human or structural damage to eelgrass and then the algae being able to take over. But its probably mostly driven by runoff. 7 or 8 maybe.

PAGE 74

74 LIST OF REFERENCES Assaraf, B.Z.O. and N. Orion ent of System Thinking Skills in the Journal of Research in Science Teaching. 42 ( 5 ) : 518 560. Assaraf, B.Z.O and Four Case Studies, Six Years Later: Developing System Thinking Skills in Junior High Sc hool Journal of Research in Science Teaching. 47 (10) : 1253 1280. Commentary: Roles, Opportunities, and Challenges Science Museums Engaging the Public in E Journal of Na noparticle Research. 13 (4): 1549 1556. Doing Dialogue: Genre and Flexibility in Public Engag ement with Science as Culture 18 ( 4 ) : 397 416 Deaton, M., and J. Winebrake. 2000. Dynamic Modeling of Environmental Systems New Yo rk: Springer Verlag. De Young, R. and M.C. Monroe, Environmental Education Research 2 (2): 171 187. Dunleav Student Engagement for Effec tive Teaching and Educa tion Canada 48 (5) : 4 8. Dura Public Under standing of Science in Public Understanding of Science 1 : 161 182. Dye, J. F., Scha Constant Comparison Met hod: The Qualitative Report 4(1/2) Eysin Learner Performance in Multimedia Learning Arrangements: An Analysis A cross Instructional American Educational Research Jour nal 46 (4): 1107 1150. Science Education 95 (1) : 168 185 Festinger, L. 1957. A theory of cognitive dissonance Stanford, CA: Stanford University Press. Frewer, L.J., Howard, C., and R. Shepherd, R.19 he Influence of Initial Attitudes on Responses to C ommunication Agriculture and Human Values 15 (1) : 15 30. Glesne, C. 2006 Becoming Qualitative Researchers Boston: Pearson Education 147 172.

PAGE 75

75 Hawthor ne, M. and T. Alabster. 1999 Citizen 2000: D evelopment of a M odel of E nvironmental Global Environmental Change 9: 25 43. Designing t o Learn About Complex Journal of the Learni ng Sciences 9 (3): 247 298. Hmelo Silver C. E, Marathe S and B reathe: Expert novice un Journal of the Learning Sciences 16: 307 331. Jacobson, M. and U. Wilensky. 2006. Compl ex Systems in Education: Scientific and Educational Importance and Implica The Journal of the Learning Sciences 15 (1): 11 34. Jacobson, S. K.,McDuff,M.D and M. C. Monroe. 2007. hrough the Arts: Outreach fo Conservation Biology 21 (1): 7 10. The Future of the Social Sciences and Humanities in the Science of Innovation: The European Journal of Social Science Research 23 (2) : 115 134. Kearney, A. Toward a Methodology for the Measurement of Knowledge Structures of Ordinary People: Th Environment and Behavior 29 (5) 579 617. Krajcik J.S. and L.M. Sutherland. 2010 Supporting Students in Devel oping Literacy in Science. 328 (5977): 456 459. Lepper, M.R. and T.W. Malone. 1987 Intrinsic Motivation and Instructional Effectiveness in Computer based Education in: R.E. SNOW & M.J. FARR (Eds) Aptitude, Learning, and Instruction, Vol. 3 (Hil lsdale, NJ, Lawrence Erlbaum Associates) Pg 260. Liu, L. and C. Hmelo Promoting Complex Systems Learning through the Use of Conceptual Journal of Research in Science Teaching 46 (9): 1023 1040. Lucki e, D. Harrison, S. and D. Ebert Model based Reasoning: Using Visual Tools to Reveal S tudent Advances in Physiology Education 35 (1) : 59 67. Slant ON srt World Watch 18 (6): 9 14. Chelsea Green Publishing Company. VT.

PAGE 76

76 Public Understanding of Science. 7: 203. Mo nroe, M. C., L. McDonell, A. Oxarart, and R. Plate. 2009. Using Community Forums to Enhance Public Engagement in E Journal of Education for Sustainable Development 3 (2): 171 182. Pfirman, S. and the AC ERE. 2003 Complex Environm ental Systems: Synthesis for Earth, Life, and Society in the 21st Century, A report summarizing a 10 year outlook in environmental research and education for the National Science Foundation Wash ington D.C.: NSF, pgs 41. Functional Modeling for Enabling Adaptive Design of Artificial Intelligence in Engineering 12 (4) 417 444. Promoting Systems Thinking through Biolog International Journal of Scien ce Education 32 (6) : 705 725. Public Participation Metho ds: A Framework for Science, Technology, & Human Values 25 (1) : 3 29 Stave, Using System Dynamics to Improve Public Participation in Envir System Dynamics Review 18 (2) : 139 167. Sweeney, L.B and J.D. Sterman. 2 System Dynamics Review 23 ( 2 3 ) : 285 311. Thompson, K. and P. Reimann. 2010 Patterns of Use of an Agent based Model and a System Dynamics Model: The A pplicat ion of Patterns of Use and the Impacts on Computers and Education 54 (2) : 392 403 Verhoeff, W., Boersma R. P., Waarlo, A. J., and K.T. B Systems modeling and the development of coherent understanding of ce International Journal of Science Education 30 (4) 543 568. Watts, D.L., Cohen, M.J., Heffernan, J.B., and T.Z. Osborne 2010. Hydrologic Modification and the Loss of Self organized Patterning in the Ridge S lough Ecosystems 13: 813 827.

PAGE 77

77 BIOGRAPHICAL SKETCH Lara Colley is a 4 th generation Floridian. She was born in Leesburg, Florida where she attended public school s Lara has been a passionate environmentalist since the age of 11, which caused her to stand out among her peers. She took some time off after high school and worked various jobs, but after becoming a Unit Clerk at Shands in Gainesville in 2002, she decided to return to co llege and pursue a degree in Environmental Education. Lara received her degree from the University of Florida in the School of Forest Resources and Conservation in 200 9 and Master of Science from the School of Forest Resources and Conservation i n 2011 She plans to teach middle school s cience, where she first developed a passion for protecting the environment. She plans to integrate the knowledge she gained from this study to instill the same passion she has for the environment into her students.