<%BANNER%>

An Assessment of Knowledge Construction in an Online Discussion Forum

Permanent Link: http://ufdc.ufl.edu/UFE0042037/00001

Material Information

Title: An Assessment of Knowledge Construction in an Online Discussion Forum The Relationship Between Content Analysis and Social Network Analysis
Physical Description: 1 online resource (151 p.)
Language: english
Creator: Buraphadeja, Vasa
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: discussion, forum, knowledge, learning, network, online
Teaching and Learning -- Dissertations, Academic -- UF
Genre: Curriculum and Instruction (ISC) thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: This study examines the relationship between two ways of measuring critical thinking in online discussion forums. The study proposes that the results from content analysis, a well-developed technique that is used to measure level of thinking and knowledge construction in online environment, would relate to the results from social network analysis, a proposed technique that may give us insight into the process of knowledge construction. This study helps online educators provide better support to nurture learners critical thinking skills and provide them with simple methods for monitoring online classrooms. It also helps instructional designers develop and maintain online learning environments that foster critical thinking skills and increase levels of knowledge construction. Finally, it adds to the body of research in critical thinking in relation to validation of existing measurements and finding novel yet simple methods for assessing critical thinking in online learning environments.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Vasa Buraphadeja.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Dawson, Kara M.
Local: Co-adviser: Cavanaugh, Catherine S.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0042037:00001

Permanent Link: http://ufdc.ufl.edu/UFE0042037/00001

Material Information

Title: An Assessment of Knowledge Construction in an Online Discussion Forum The Relationship Between Content Analysis and Social Network Analysis
Physical Description: 1 online resource (151 p.)
Language: english
Creator: Buraphadeja, Vasa
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2010

Subjects

Subjects / Keywords: discussion, forum, knowledge, learning, network, online
Teaching and Learning -- Dissertations, Academic -- UF
Genre: Curriculum and Instruction (ISC) thesis, Ph.D.
bibliography   ( marcgt )
theses   ( marcgt )
government publication (state, provincial, terriorial, dependent)   ( marcgt )
born-digital   ( sobekcm )
Electronic Thesis or Dissertation

Notes

Abstract: This study examines the relationship between two ways of measuring critical thinking in online discussion forums. The study proposes that the results from content analysis, a well-developed technique that is used to measure level of thinking and knowledge construction in online environment, would relate to the results from social network analysis, a proposed technique that may give us insight into the process of knowledge construction. This study helps online educators provide better support to nurture learners critical thinking skills and provide them with simple methods for monitoring online classrooms. It also helps instructional designers develop and maintain online learning environments that foster critical thinking skills and increase levels of knowledge construction. Finally, it adds to the body of research in critical thinking in relation to validation of existing measurements and finding novel yet simple methods for assessing critical thinking in online learning environments.
General Note: In the series University of Florida Digital Collections.
General Note: Includes vita.
Bibliography: Includes bibliographical references.
Source of Description: Description based on online resource; title from PDF title page.
Source of Description: This bibliographic record is available under the Creative Commons CC0 public domain dedication. The University of Florida Libraries, as creator of this bibliographic record, has waived all rights to it worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.
Statement of Responsibility: by Vasa Buraphadeja.
Thesis: Thesis (Ph.D.)--University of Florida, 2010.
Local: Adviser: Dawson, Kara M.
Local: Co-adviser: Cavanaugh, Catherine S.

Record Information

Source Institution: UFRGP
Rights Management: Applicable rights reserved.
Classification: lcc - LD1780 2010
System ID: UFE0042037:00001


This item has the following downloads:


Full Text

PAGE 1

1 AN ASSESSMENT OF KNOWLEDGE CONSTRUCTION IN AN ONLINE DISCUSSION FORUM : THE RELATIONSHIP BETWEEN CONTENT ANALYSIS AND SOCIAL NETWORK ANALYSIS By VASA BURAPHADEJA A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVE RSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2010

PAGE 2

2 2010 Vasa Buraphadeja

PAGE 3

3 To my f amily

PAGE 4

4 ACKNOWLEDGMENTS This dissertation wo uld not have been completed without the support and help of people around me. My sincere gratitude goes first to my family. I wish to thank my father, Savang, who has modeled himself as a lifelong learner and a scholar, and my mother, Vichitra, for her lov e, understanding, and support. I also wish to thank my brother, Vip, who is indeed a very important person in my life. He introduced me to literature and set me o n the path to writing professially. In addition to my family in Thailand, my committee members deserve my deepest gratitude. First, I wish to thank my academic advisor, Dr. Kara Dawson, who has been a role model in my profession and life. Her unflagging support for my research, writing, and rewriting has helped me successfully publish my first acad emic journal article, which subsequently has become a crucial part of this dissertation. She has helped me navigate through the academic journey with freedom to explore my interests and fully supported my pursuit of learning and research with an open mind and keenness. Her mentor style and dedication to research will undoubtedly benefit my career for years to come. Dr. Dawson provides an exemplary model that one can could lead a happy and fulfilled life by maintaining a balance between an academic career an d family. For the knowledge of instructional design and an excellent role model in the field of educational technology, I am indebted to Dr. Cathy Cavanaugh, who served as my co -chair. She has also been an inspiring mentor. Dr. Cavanaughs first rate and prompt critiques are greatly appreciated among our cohort of doctoral students. Her thoughtful observations, recommendations, and insights are embedded in this dissertation. I owe thanks to committee methodologist, Dr. Cynthia Garvan, for her time, knowled ge and insights in statistics. I am thankful not only for her professional opinions but also for her encouragement, willingness to help, and caring. Dr. Erik Black also has provided exceptional

PAGE 5

5 support as a member of my dissertation committee. As a former classmate, I consider him a good friend and a role model in our cohort. In all my contacts with Dr. Black, he has consistently been helpful and approachable. Furthermore, I am thankful for Dr. Swapna Kumar who invited me to work on the content analysis pr oject with her d uring the early stages of my dissertation. The experience from the collaboration has become invaluable in my dissertation work. Throughout my journey toward finishing my doctoral degree I have shared many a good meal, enjoyed many an inspiring and pleasant conversation, and traveled with many on a business trip and a mini break s I am thankful to them all. My time in Gainesville would have been extremely difficult without Dr. Atul Bali, my former roommate and my best friend, who has kept me company during my roughest time away from home. I must mention the colleagues and unforgettable friends I have in Education Technology, including Be n Campbell, Chris Sessums, Irvi k a Francois, Jeff Boyer, Joe DiPietro, Leslie Merryman, Mary Edwards, Nate Poling, and Nicola Wayer. My special thanks go to Wendy Drexler and Kathryn Kennedy who helped me with my dissertation work, especially Kathryn who provided editorial assistance with a fast -track turnaround. Special thank s also go to Wendy Thornton for her editorial support I am grateful to each member of our unpretentious support group, including Feng Liu, He Huang, Ji Young Kim, Joanne Laframenta, Shih-Fen Yeh, and Zhuo Li. During the arduous trek of the dissertation journey, we have shared joys and sor rows, and given each other academic and emotional support whenever needed without hesitation. I am certain the camaraderie of our group will last beyond the dissertation years and I wish everyone a successful career after graduation. I also wish to warmly thank the last member of the group, Chu -Chuan Chiu, for her friendship, morale -boosting, and nourishment during the later stages of my dissertation.

PAGE 6

6 Finally, I want to acknowledge Assumption University of Thailand for their financial support and the Office of Distance Education at the University of Flora for providing access to the data for this study. Their help has made possible the completion of this dissertation.

PAGE 7

7 TABLE OF CONTENTS ACKNOWLEDGMENTS .................................................................................................................... 4 page LIST OF TABLES ................................................................................................................................ 9 LIST OF FIGURES ............................................................................................................................ 11 ABSTRACT ........................................................................................................................................ 12 CHAPTER 1 INTRODUCTION ....................................................................................................................... 13 Purpose of the Study ................................................................................................................... 15 Research Question ....................................................................................................................... 17 Significance of the Study ............................................................................................................ 17 Summary ...................................................................................................................................... 18 Operational Definitions ............................................................................................................... 1 8 2 THEORETICAL AND LITERATURE REVIEW ................................................................... 21 Introduction ................................................................................................................................. 21 Critical Thinking and Knowle dge Construction ....................................................................... 21 Demands for Distance Education in Higher Education ............................................................ 24 Critical Thinking in Asynchronous Learning Environment s ................................................... 26 Using Content Analysis to Assess Critical Thinking ................................................................ 28 Henris Analytical Model .................................................................................................... 29 Indicators of Critical Thinking for Content Analysis ........................................................ 33 Practical Inquiry Model in Community of Inquiry Framework ....................................... 35 Five -Phase Interaction Analysis Model (IAM) .................................................................. 42 Content Analysis: In Search of a Partner ................................................................................... 51 Social Network Analysis ............................................................................................................ 52 Using Social Network Analysis to Assess Critical Thinking ................................................... 61 Summary ...................................................................................................................................... 62 3 METHODOLOGY ...................................................................................................................... 72 Theoretical Framework ............................................................................................................... 72 Research Design .......................................................................................................................... 74 Context of the Study ................................................................................................................... 74 Materials Selection ...................................................................................................................... 75 Data Collection ............................................................................................................................ 79 Content Analysis Data Collection ...................................................................................... 79 Social Network Analysis Data Collection .......................................................................... 79 Data Analysis ............................................................................................................................... 80

PAGE 8

8 Content Analysis .................................................................................................................. 80 Social Network Analysis ..................................................................................................... 81 Degree centrality .......................................................................................................... 82 Closeness centrality ...................................................................................................... 82 Spearmans Correlation Coefficient ................................................................................... 85 Limitation of the Stu dy ............................................................................................................... 85 4 RESULTS .................................................................................................................................... 92 Introduction ................................................................................................................................. 92 Results of IAM ............................................................................................................................ 92 Results of SNA ............................................................................................................................ 96 Research Question ....................................................................................................................... 98 Summary of Findings .................................................................................................................. 99 5 DISCUSSION AND IMPLICATIONS ................................................................................... 107 Introduction ............................................................................................................................... 107 Summary of the Study .............................................................................................................. 107 Findings of IAM related to the Literature ................................................................................ 109 Implications Related to Outcomes of IAM ...................................................................... 116 Implications Related to Instructional Design and Organization ..................................... 116 Implications Related to Facilitating Discourse ................................................................ 120 Findings of SNA related to the Literature ............................................................................... 121 Findings Related to the Relationship between IAM and SNA ............................................... 124 Implica tions Related to Online Teaching and Learning .................................................. 126 Recommendations for LMS Developers .......................................................................... 128 Recommendations for Research ....................................................................................... 128 Conclusion ................................................................................................................................. 130 APPENDIX A SOCIAL NETWORK ANALYSIS .......................................................................................... 135 B ENTITY RELATIO NSHIP DIAGRAM ................................................................................. 141 C INSTRUCTION TO THE EME 6458 DISCUSSION PARTICIPATION ........................... 142 LIST OF REFERENCES ................................................................................................................. 144 BIOGRAPHICAL SKETCH ........................................................................................................... 151

PAGE 9

9 LIST OF TABLES Table page 1 1 Lipmans concepts of critical think and corresponding IAM ph ase(s) ............................... 20 2 1 Lipmans examples of behaviors associated with concepts of critical thinking ................ 63 2 2 Henris analytical model: cogn itive skills ............................................................................ 65 2 3 Henris analytical model: processing information (examples only) ................................... 65 2 4 Newman et al. indicators of critical (+) and uncritical ( -) thinking (examples only) ........ 65 2 5 Elements and Categories in Community of Inquiry ............................................................. 66 2 6 Guidelines for Coding Cognitive Presence in the Practical Inquiry Model ....................... 68 2 7 The Transcript Analysis Tool (TAT) .................................................................................... 68 2 8 Results of three alignme nts and the practical inquiry .......................................................... 68 2 9 Gunawardena et al. five -phase interaction analysis model .................................................. 69 2 10 IAM coding results after i nter rater checks .......................................................................... 70 2 11 Comparison of Veerman et al. model and Gunawardena et al. model ............................... 70 2 12 Overview of the research condi tions of De Wever (2009) .................................................. 70 2 13 Lipmans concepts of critical thinking and associated behaviors ....................................... 71 3 1 Descriptive information of the selected online courses ....................................................... 87 3 2 Forum details of the selected online courses ........................................................................ 88 3 3 Jonassen et al.s criteria for constru ctivist learning environments ..................................... 89 3 4 Rubric for constructivist learning environments .................................................................. 89 3 5 Rating configuration and results ........................................................................................... 90 4 2 Means and Standard Deviations of messages in the forum EME 645805 ...................... 101 4 3 Spearmans rho correlations between IAM and lengt h of messages ................................ 101 4 4 Mean level of knowledge construction, normalized degree and betweenness centrality measures of EME 6458 discussion 4 .................................................................. 102

PAGE 10

10 4 5 Spearmans rho correlations between IAM and SNA measures ....................................... 106 4 6 Spearmans rho correlations between IAM and SNA measures (without instructor) ..... 106 4 7 IAM coding results from previous studies .......................................................................... 106 5 1 Components in teaching presence and IAM coding results from previous studies and the study ................................................................................................................................ 132 5 2 Rubric for individual performance on a team from Palloff and Pratt (2005) ................... 133

PAGE 11

11 LIST OF FIGURES Figure page 1 1 Conceptual framework of the present study ......................................................................... 20 2 1 Development of content analysis frameworks in CMC ....................................................... 64 2 2 Practical inquiry model of Garrison et al. (2001) ................................................................. 67 3 1 Network topology. A) Star network. B) Line network. C) Circle network ........................ 91 4 1 Sociogram of EME 6458 discussion 4, where node size is based on out -degrees centrality ............................................................................................................................... 103 4 2 Sociogram of EME 6458 discussion 4, where node size is based on in-degrees centrality ............................................................................................................................... 104 4 3 Sociogram of EME 6458 discussion 4, where node size is based on betweenness centrality ............................................................................................................................... 105 5 1 Potential paths to higher l evels of knowledge construction forum ................................... 134 B1 Entity relationship diagram of forum discussion tables used in this study ...................... 141

PAGE 12

12 Abstract of Dissertation Presente d to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy AN ASSESSMENT OF CRITICAL THINKING IN AN ONLINE DISCUSSION FORUM: THE USE OF CONTENT ANALYSIS AND SOCIA L NETWORK ANALYSIS By Vasa Buraphadeja August 2010 Chair: Kara Dawson Co -chair: Catherine Cavanaugh Major: Curriculum and Instruction This study examines the relationship between two ways of measuring critical thinking in online discussion forums. The study proposes that the results from content analysis, a well developed technique that is used to measure level of thinking and knowledge construction in online environment, would relate to the results from social network analysis, a proposed technique tha t may give us insight into the process of knowledge construction. This study helps online educators provide better support to nurture learners critical thinking skills and provide them with simple methods for monitoring online classrooms. It also helps instructional designers develop and maintain online learning environments that foster critical thinking skills and increase levels of knowledge construction Finally, it add s to the body of research in critical thinking in relation to validation of existing measurements and find ing novel yet simple methods for assessing critical thinking in online learning environments.

PAGE 13

13 CHAPTER 1 INTRODUCTION Critical thinking has always been recognized as one of the most important skills in education as well as in everyd ay life. Critical thinking skills are self -correcting and context sensitive (Lipman, 2003) and have been linked to higher levels of Blooms taxonomy including analysis, synthesis, and evaluation (Gokhale, 1995) and used to facilitate judgment based on criteria (Lipman, 2003) Most educators would agree that learning these skill s is one of the most valuable goals in schooling and is beneficial for facing social, political, and ethical challenges (Abrami et al., 2008) In higher education, critical thinking is often presented as an ostensible goal (Garrison, Anderson, & Archer, 2000) Despite the fact that the importance of such thinking skills has been rec ognized since the early twentieth century, there has not been much research in this area (Clark & Mayer, 2008) partly due to the difficulty involved in assessing such skills (Ennis, 1993) At the tu rn of the twenty-first century, the burgeoning demand for online learning radically changed the way we discern evidence of critical thinking. Many scholars put forward various theories reg arding how learning within asynchronous web -based discussions can bolster critical thinking (Buraphadeja & Dawson, 2008; Garrison et al., 2000; Havard, Du, & Olinzock, 2005; Yang, Newby, & Bill, 2005) Asynchronous modes of learning are believed to promote critical thinking and deeper learning by providing a learner -centered environment and allowing time for learners to reflect on and respond to the discussion; the asynchronism in online learning may lead to better understanding and retention of information (Havard et al., 2005) The benefit of asynchronous online courses seem s to be congruent with the growing interest in offerin g such alternative s The National Center for Education Statistics ( NCES) reported that distance education in general is growi ng steadily and that asy nchronous online

PAGE 14

14 courses are becoming more and more popular as opposed to synchronous online courses (Parsad & Lewis, 2008; Waits & Lewis, 2003) One of the techniques used to measure the level s of knowledge construction in asynchronous online courses is content analysis a technique used to extract desired informa tion from materials (Smith, 2000) One of the well known content analytic mod els used in measuring levels of knowledge construction is the interaction analysis model (IAM) of Gunawardena, Lowe, and Anderson (1997) IAM is typically used to analyze conversational transcripts from learning activities such as discussion forums or chat transcripts. Based on the social development theory of Vygotsky (1978) and social constructivism, the IAM consists of five phases that demonstrate levels of knowledge construction in a constructivist learning environment: (a) sharing and co mparing information, (b) dissonance, (c) knowledge co -construction, (d) testing and modification, and (e) agreement and application. These phases can be aligned with Lipmans definition of critical thinking (2003) as shown in Table 1 1. Phase II (dissonance) to phase IV (testing and modification) serve as a foundation for supportive behaviors while phase V (agreement and application) is equivalent to judgment, the goal of Lipmans concept of critical thinking. More specifically, operations within each phase in IAM can be related to behaviors that support the concepts of critical thinking. Detailed information related to this alignment can be found in Table 2 13. The alignment of critical thinking and IAM helps us connect concepts of criti cal thinking skills to the levels of knowledge construction; that is, it explains that IAM is a proxy for finding evidence of critical thinking skills. Ultimately, the alignment helps us develop the conceptual framework of this study (Figure 1 1). More det ails of this framework are discussed in Chapter 2.

PAGE 15

15 Although content analysis in CMC has been proven to be a fairly successful method during the past two decades, scholars raise concerns about the time-consuming process of this manuallyevaluated technique Incidentally t he growing popularity of online courses opens up new opportunities for educators to garner information without interfering with learners. Researchers have begun to tap into large data repositories that many learning management systems (LMS ) automatically store arguing that meaningful reports may be uncover ed (Black, Dawson, & Priem, 2008) One of the techniques being proposed is social network analysis (SNA) a theory and method that seeks to uncover social structure and its consequences (Freeman, 2004) Rooted in sociology, SNA is versatile and can be applied to many contexts from explain ing concrete social circumstances such as how Americans form their confid ant network s (Knoke & Yang, 2008) to far more abstract social phenomena such as the spread of happiness (Fowler & Christakis, 2009) Scholars also begin to use SNA to help uncover social structure s of students in online learning environment s (Harrer, Zeini, & Pinkwart, 2006; Reffay & Chanier, 2002; Shen, Nuankhieo, Huang, Amelung, & Laffey, 2008; Zhu, 2006) particularly because it can be applied to na turally occurring data repositories in an unobtrusive fashion as opposed to contrived structures (e.g., self report ing questionnaire interview s ) (Knoke & Yang, 2008; Lewis, Kaufman, Gonzalez, Wimmer, & Christakis, 2008) Nurmela, Lehtinen, and Palonen (1999) specifically proposed that such an approach may help us g ain insigh t into the process of knowledge -building and acquisition. With this proposition, it seems plausible that social network analyses may help better understand how critical thinking is carried out in online learning environments. Purpose of the Study The purpose of this study was to discern the nature of the relationship between two different ways of measuring critical thinking skills in online learning environment s using

PAGE 16

16 discussion forums The two methods employed in this study we re SNA and IAM Base d on the notion of social constructivism and Lipman s (2003) critical thinking this study investigated whether centrality measures in social network analysis relate to mean levels of knowledge construction resulting from the IAM C entra lity measures are common relationship measures that seek to quantify the notion of an actors prominence within the network (Knoke & Yang, 2008) Network centralities concern three different types of s tructural location s of actors in a network: (a) Degree : a ctor s who ha ve more connections or degrees may have more power in exchanging resource s (b) Closeness: a ctors who can reach other actors via shorter path lengths have advantages over others, and (c) Betweenness: actors who lie betwee n other pairs of actors have more capacity to broker contacts among others. Network r esou rces, as previously mentioned, could be happiness access to confidants or as in this study, knowledge Th ese network measures reveal certain patterns of relationship among actors in the network and different kinds of social power that they hold (Hanneman & Riddle, 2005) Targeting online courses that utilized discussion forums t his study suggested that centrality m easures from network analysis may give us insights into phenomena in online learning environments Based on t he suggestion that individuals create knowledge by interacting with me mbers in their community (Gunawa rdena et al., 1997) and that more interactions would result in higher levels of knowledge construction (Schellens & Valcke, 2005) t he study postulated that resulting social network measures of discussion activities d esigned for students to demonstrate their critical thinking skills relate to mean levels of knowledge construction resulting from I AM The study used data available in LMS systems to discern the nature of the relationship between IAM and SNA These automatically -stored data in the LMS include recorded transcripts

PAGE 17

17 (for IAM content analysis) and relations of the dialog (for SNA). Studi es have shown that SNA is useful for early identification of students who may be lagging in contributions and has potential to help us better understand how members in a certain community participate in knowledge construction (Nurmela et al., 1999; Shen et al., 2008) This study intend ed to confirm the latter argument. To discern the relationship between the results of the interaction analysis coding and the measures in SNA, Spearmans correlation s were performed. Research Question The following research question was used to guide this study: Do levels of knowledge construction in online learning environment s relate to the centrality measures of socia l networks? Specifically, the study focused on results of interaction analysis model of Gunawardena et al. (1997) and its relationship with the resulting c entrality measures from social network analysis. Significance of the Study Many researchers raise concerns about time-consuming processes of manuallyassessed methods of content analysis techniques; some specifically argue that alternative methods that rely on unobtrusive data collection such as data mining, text mining, and SNA would support or even substitute for the se tedious analysis process es (Bratitsis & Dimitracopoulou, 2008; Dringus & Ellis, 2005; Huang & Chuang, 2008; Shen et al., 2008) In defiance of this promising suggestion, there have been only a handful of attempts to merge SNA with content analysis to confirm fruitful results of such a proposal. These s tudies found that instru ctors were influential in control ling the flow of communi ca tion (Nurmela et al., 1999) and that topics chosen by instructors might regulate students interactional behaviors (Zhu, 2006) This study contributed to the field of online education that has become todays fastest growing educational setting. Specifically, the potential implications of this stu dy were (a ) to help online educators provide better support to nurture learners critical thinking skills and provide

PAGE 18

18 them with simpl e methods for monitoring online classrooms ( b ) to help instructional designers develop and maintain online learning enviro nments that foster critical thinking sk ills, and ( c ) to add to the body of research in critical thinking in relation to validation of existing measurement s and (d) to fi nd novel yet simple method s for assessing critical thinking in online learning environm ents The study also provided empirical results for further studies especially for future research design and methodology in terms of content analysis and S NA in online learning environment s Summary The introduction sets the framework for the importance of this study as it pertains to growing demand in online education. The present study supports the dialogue among educators and researchers in the field of online education regarding the need for promoting and maintaining critical thinking skills in onlin e learning environments. The study was anticipated to extend this dialogue to additional methods to measure such learning skills. The remainder of this man uscript is organized as follows: After a literature review, the research design including the model of content analysis and SNA techniques is explained. D ata analysis and findings are discussed next. Finally, discussion and implications as well as conclusions and recommendations complete the study. Operational Definitions Critical Thinking This is th inking that facilitate s j udgment based on criteria, self correction, and in recognition of the sensitivity of context. Such thinking is evident in higher levels of knowledge construction Knowledge Construction Consisting of five phases (outlined at the beginning of this chapter) knowledge is a process individuals use to create new insight or understanding in a learning community

PAGE 19

19 Online Learning This is a learning environment that utilizes Internet technology without face -to -face interaction between l earners and instructor. Prominent Actors Actors that are visible to other actors in the network; two classes of prominence are centrality and prestige. Actor Centrality Actors that are extensively involved in relationships with other actors. The most w idely used centrality measures are degr ee, closeness, and betweenness.

PAGE 20

20 Table 1 1. Lipmans concepts of critical think and corresponding IAM phase(s) Concepts Corresponding phase(s) from IAM Self correction II dissonance III knowledge coconstructi on Acquiring sensitivity to context II dissonance III knowledge coconstruction Being guided (and goaded) by criteria II dissonance IV testing and modification Judgment V agreement and application Figure 1 1 Conceptual framework of the present study

PAGE 21

21 CHAPTER 2 THEORETICAL AND LITE RATURE REVIEW Introduction This chapter offers theory and literature to frame the present study. The first three sections discuss historical perspectives and significance of critical thinking in higher ed ucation and its connection to distance learning. These sections include the definiti on of critical thinking and opportunities for capturing evidence of critical thinking skills from online learning environments The middle section discuss es methods of ass essing critical thinking in computer -mediated communication and novel methods of social network analysis used for fostering and assessing critical thinking in online education. The last section discusses how the two methods may strengthen the present study and proposes to merge the two methods together. Critical Thinking and Knowledge Construction One of the early works that contribute d to the development of critical thinking was John Deweys How We Think (1910) In this work, Dewey made a distinction between ordinary thinking and reflective thinking, by which he meant an exploration of additiona l evidence, of new data, that will develop a suggestion. Reflective thinking involves (a) a state of uncertainty, perplexity, hesitation and doubt, and (b) an act of searching for further facts or testing that validates or nullifies the suggested belief. O ver time, the result of this thinking becomes a prescribed belief that helps us glide smoothly from one idea to another and arrests the condition of mental uneasiness by accepting any plausible suggestion. Reflective thinking is a kind of thinking that is always troublesome because it requires a suspension of judgment during further inquiry and involves a willingness to maintain a state of doubt and endure a condition of mental disturbance. Lipman (2003) argued that Deweys observation of an age old algorithm for everyday problem solving reflects the dawn of the critical thinking movement As he explains

PAGE 22

22 it was Deweys emphasis on reflective thinking that was the true harbinger of critical thinking in this century (p.35). Benjamin Bloom and the taxonomy group (1956) also contributed to the field with their taxonomy by ordering edu cational behaviors and outcomes. The taxonomy contains six major classes: knowledge, comprehension, application, analysis, synthesis, and evaluation. They argued that this taxonomy is a hierarchical order of the different classes of objectives, and the objectives in one class are likely to build on the behaviors in the preceding classes. One of the guiding principles in developing this taxonomy was for it t o be used in existing educational units and programs; that is, the taxonomy would help educators define their educational objectives, instructional materials, and instructional methods. According to Lipman (2003) this effort appeared to be another milestone in the critical thinking movement as Bloom et al. (1956) may have intended to upgrade evaluation of thinking and downgrade knowledge. T he upper three levels of Blooms taxonomy of educational objectives are often referred to as critical thinking (Ennis, 1993) Robert Ennis (1993) speculated that levels in Blooms taxonomy are in fact interdependent rather than hierarchical as suggested b y the theory. Moreover, these concepts are too vague to help develop and judge critical thinking assessment. Ennis version of critical thinking was reasonable reflective thinking focused on deciding what to believe or do (p.180). In order to develop cri tical thinking skills a person needs to do the following interdependently (Ennis, 1993) : Judge the credibility of sources Identify conclusions, reasons, and assumptions Judge the quality of an argument, in cluding the acceptability of its reasons, assumptions, and evidence

PAGE 23

23 Develop and defend a position on an issue Ask appropriate clarifying questions Plan experiments and judge experimental designs Define terms in a way that is appropriate for the context Be open -minded Try to be well informed Draw conclusions when warranted, but with caution Ennis (1993) explained that this list of abilities and dispositions, with creative aspects of critical thinking, can serve as a set of goals for an entire curriculum or other instructional sequence and provide sufficient elaboration for the development of critical thinking assessments. However, Lipman (2003) criticized Ennis definition of critical thinking fo r its lack of precision and interconnectedness among characteristics. T he aim of Ennis critical thinking, to believe and to do, is cause for confusion for education and schooling While schooling attempts to indoctrinate and create conformity of thought a nd behavior, education insists otherwise (Lipman, 2003) Ennis definition also disregards social aspects of the cog nitive process. For Lipman, critical thinking takes a defensive role to protect us from being coerced or brainwas hed into believing what others want us to believe without our having an opportunity to inquire for ourselves (p.47). Lipman (2003) describes critical thinking as thinking that (1) facilitates judgment because it (2) relies on criteria, (3) is self-correcting, and (4) is sensitive to context ( p. 212). Criteria are reliable kinds of reason, which may include standards, principles, factual evidence, and procedures (Lipman, 2003) Self -correction is a method of inquiry among members of the group that enables them to evaluate and adjust their procedures. Further, critical thinking is sensitive to particularities and may not be transferable between different domains. Lipman also provide b ehavior s that are associa ted with each of these concepts (Table 2 1)

PAGE 24

24 Lipman maintains that pursuing a degree in higher education is a process of becoming a specialist in a certain branch of knowledge and usually involves becoming a member of a community of experts in the field. To im prove historical judgment, Lipman explains, one must be exposed to historical content. Thus, the content or domain is as important as the process and methods in any disciplinary inquiry. In other words, there is no way around teaching specific disciplinar y contents (Lipman, 2003, p. 48) Although the idea of developing critical thinking skills has been widespread throughout the world of education during the last decade of the twentieth century (Lipman, 2003) and many publications about teaching thinking skills are available valid evidence supporting the implementation and outcomes of educating for the development of thinking skills remains scarce (Clark & Mayer, 2008) This is in part because measu ring such skills are complicated (Ennis, 1993; Lipman, 2003) Ennis (1993) argue d that comprehensive critical thinking assessments (i.e., short answer, essay, and performance assessment) require resources such as assessment budgets and grading time. While multiple -choice tests are less expensive on a large scale, they are arguably les s valid than other rigorous forms of tests. Although these more rigorous test formats are feasible on a smaller scale, grading them still takes more time than multiple -choice tests. Demands for Distance Education in Higher Education At the turn of the twen ty -first century, the burgeoning demand for online and blended learning has radically changed the way we discern evidence of critical thinking. The National Center for Education Statistics (NCES) published two studies about the growing nature of distance e ducation over the last decade. The NCESs 200001 report (Waits & Lewis, 2003) defined distance education as education or training courses delivered both synchronously and asynchronously to remote sites via audio, video, or computer technologies. The 200607 report

PAGE 25

25 (Parsad & Lewis, 2008) cast a wider n et to include correspondence course s both written and using various forms of technology (e.g., CD ROM). The 200001 academic year report revealed that 56% of all 2 year and 4 year degree granting institutions offered distance education (DE) courses and 1 2% planned to offer such courses in the next three years (Waits & Lewis, 2003) Ninety percent of institutions offering DE courses chose to offer asynchronous Internet courses, while 43% offered synchronous Internet courses. The follow up report during the 200607 academic year found 66% of such institutions offered DE courses. Of these institutions, 92% made use of asynchronous Internet -based technologies, while only 33% used synchronous Internet -based technologies (Parsad & Lewis, 2008) Although Parsad and Lewis (2008) suggested that findings from the two reports may not be co mparable, data revealed that while distance education in general is gaining popularity and course content is commonly delivered asynchronously, the use of synchronous learning is declining This trend has been attributed to the appealing factors of anywhe re, anytime learning where learners can combin e education with other commitments in life (Hrasti nski, 2008) as well as to the direct benefits of dealing with content effectively (Schwier & Balbar, 2002) There is also a solid association between strength of interactions and achievement for asynchronous courses (Bernard et al., 2009) In any case, there is reason to believe that this delivery mode could foster critical thinking. Therefore, this study attempted to further investigate for evidence of such thinking skills usi ng different methods to measure levels of knowledge construction in online discussion forums. The next section discusses previous studies that examine d the connection between critical thinking skills and asyn chronous learning environments

PAGE 26

26 Critical Thinking in Asynchronous Learning Environments During the past years, many scholars put forth the idea of facilitating and assessing critical thinking in asynchronous learning environments (Celani & Co llins, 2005; Garrison et al., 2000; Havard et al., 2005; Henri, 1991; Yang et al., 2005) Garrison et al. (2000) argued that interactions in traditional education mostly rely on oral communications which tend to be fast pa ced, spontaneous, and short lived. In the face to -face context, oral communication also provides multiple nonverbal cues such as facial expression s and tone of voice and is therefore considered a rich medium. Alternatively written communication restricts participation to text based medium without any non -verbal cues. But this medium offers some advantages as it provides time for reflection and encourages discipline and rigor in thinking and communicating. Having the potential to promote thinking about com plex issue s and deep, meaningful learning, the lean medium, as opposed to oral communication, has a stronger connection with the achievement of higher -order learning objectives (Garrison et al., 2000) Henri (1991) argued that asynchronous interaction is free from constraints imposed by time and space. Participants can express themselves freely and contribute to the richness of the content, instead of the richness of the interactive process as in the face-to -face counterpart. Hara, Bonk, and Angeli (2000) suggested that delayed capabilities of interaction tools increase wait time and opportunities for reflective learning and information processing. Celani and Collins (2005) concurred with the importance of asynchronicity which allows participants to elaborate on their thinking, plan carefully, structure and review their contribution before they publish to the community. The 24/7 presence of the content also makes it possible for participants to revisit their own and others messages and raise awareness about how they can define or understand the content at hand. Likewise, Havard, Du, and Olin zock (2005) argued that asynchronous online discussio n promotes critical thinking and deeper learning by providing a learner -centered environment and

PAGE 27

27 ample time for learners to reflect on and respond to the discussion. These unique characteristics of asynchronous learning environments will lead to better understanding and retention of infor mation, thus helping students demonstrate a higher level of thinking. However, they cautioned against an assumption that this medium automatically promote s deeper learning without instructor facilitation Yang, Newby, and B ill (2005) c oncurred with this recommendation and added that Socratic questioning a teaching technique for stimulating learners minds by continuously probing into the subject with thought -provoking questions can enhance students critical thinking skills in universit y level distance learning courses. An asynchronous text based online discussion is considered a form of computer -mediated communication (CMC ). CMC refers to interaction occurring between spatially separated learners using networked computers such as e -mail chat and discussion forum s (Jonassen, Davidson, Collins, Campbell, & Haag, 1995) Although CMC supports synchronous and a synchronous group communication, it is the asynchronous (time -delayed) feature of the CMC that makes anywhere, any time collabor ative work possible (Gunawardena & McIsaac, 2004) Based on findings and recommendations from previous studies, I believe that a synchronous CMC is a viable platform to enhance higher levels of thinking and promote meaningful learning. More over, most of the online courses offered through the Distance Education office are grounded i n social constructivist learning theory so they allow learners to see multiple perspectives, as well as provide them with more time for information processing and reflectio n on complex issues. Therefore, t ranscripts from online discussion forums have become promising sources that many resea rchers spend hours scouring for evidence of critical thinking One of the well known techniques that researchers use to examine transcripts is know n as content analysis. The next section discusses this technique with so me notable analytic models

PAGE 28

28 being used to evaluate levels of critical thinking and other related cognitive skills in the body of literature. Using Content Analysis to Assess Critical Thinking According to Smith (2000) content analysis is a technique used to extract desired information from a body of material (usually verbal) by systemat ically and objectively identifying specified characteristics of the material (p.314). Over the past two decades, t his technique has been used to examine evidence of critical thinking in CMC Similar to the traditional method, c ontent analysts in the CMC i nfer meaning from text using a set of procedures or a model to discern and define a target variable, to collect samples of representative text, and to devise reliable and valid rules to categorize segments of the text (Anderson, Rourke, Garrison, & Archer, 2001) Many models of content analysis have been put forward to measure levels of critical thinking in CMC, particularly in asynchronous communication (P. J. Fahy et al., 2000; Garrison et al., 2000; Gunawardena et al., 1997; Henri, 1991; Newman, Webb, & Cochrane, 1995) Figure 2 1 illustrates notable models in analyzing content in CMC France Henri (1991) spearheaded the movement with a study using an analytical model to measure five dimensions of the learning process characterized by the content of CMC: participation, interaction, social, cognitive, and metacognitive Arguing that there is no distinction between cogniti on and metacognitive di mensions and that the better way is to look for signs of critical thinking, Newman et al. (1995) piggybacked on Henris study by adding a further dimension of five -stage critical think ing process (Garrison, 1992) which include s elementary clarification, in -depth clarification, inference, applicability, and integration Gunawardena et al. (1997) and Garrison et al. (2000) also proposed their own models, details of which will be presente d in the next sections. Regardless of any specific defini tions of critical thinking these analytic models were developed

PAGE 29

29 by the notion s of critical thinking and related to cognitive skills. In other words, t hese models were essentially designed to capture critical thinking in action The following sub-sections review notable models of content analysis in CMC during the past two decades. Henris Analytical Model Henri (1991) proposed a method of content analysis in CMC which includes a framework that defines the dimensions of analysis, an analytical model corresponding to the dimensions, and a technique for message analysis. Five dimensions of the framework are listed below: Participat ive: compilation of the number of messages or statements transmitted by one person or group Social: statement or part of a statement not related to formal content of subject matter Interactive: chain of connected messages Cognitive: statement exhibiting kn owledge and skills related to the learning process Meta cognitive: statement related to general knowledge and skills and showing awareness, self -control, and self regulation of learning Of these five dimensions, the last two dimensions are related to thinking and learning processes. Cognitive dimension is closely connected with understanding, reasoning, the development of critical thinking and problem solving skills (Henri, 1991) Henri (1991) reconstructed the analytical model for cognitive skill s by grouping skills in the taxonomy of critical thinking (Ennis, 1986) into five categories (Table 2 2 ). Henri (1991) commented that the results of the model, which help identify the presence and frequency of use of cognitive skills, were rather superficial. She then de veloped another model based on the notion of surface and deep i nformation processing (Table 2 3 ). Henri (1991) suggested that the processing information model should be employed after the cognitive skills in content have been identified by the preceding model. She reasoned that the purpose of these results was not to establish an exhaustive de scription of cognitive activity but

PAGE 30

30 to offer valuable information regarding limitations of the learners and to help educators prepare for appropriate cognitive support. Examining meta -cognition of learners in message con tent is beyond the scope of the present study, partly due to the subject ivity of the process. Traditionally, researchers have observe d learners meta -cognitive processes by inviting them to participate in think aloud sessions or to describe the operations they would accomplish in the given task. Similarly, researchers may also observe the meta -cognitive processes in a CMC environment by asking learners to describe their thinking process in writing (Henri, 1991) Nevertheless, many research ers agree that measuring meta -cognitive skills is problematic. Hara, Bonk, and Angeli (2000) said that such skills are extremely difficult to capture and rely on learners preparedness and willingness to share. Henri (1991) acknowledged that even if evidence of meta -cognitive activity was not found in the message content, one should not conclude that learners are weak in this area. Researchers agreed that Henris seminal framework (1991) is often used as a starting point in CMC studies (De Wever, Schellens, Valcke, & Van Keer, 2006; L. Rourke, Anderson, Garrison, & Archer, 2001) The framework reappeared, sometimes as a variation, in many studies over the past years after its inception (Aviv, Erlich, Ravid, & Geva, 2003; Bullen, 1998; Hara et al., 2000; Newman et al., 1995; Pena -Shaff, Martin, & Gay, 200 1; Pena -Shaff & Nicholls, 2004) In reviewing the analysis models in CMC, Gunawardena, Lowe, and Anderson (1997) argued that one of the shortcomings of Henris model (1991) was the fact that it was built on a teacher -centered instructional paradigm, which is inappropri ate for analyzing communication in adult learning settings. Further, Henris s uggestion (1991) for using meaning units (also known

PAGE 31

31 as thematic units) to analyze levels of cognitive skills (the first analytical model) would generate superficial results that show only the presence and frequency of using the skills. For the level of information processing (the second analytical model), the analysis could not reveal learning processes among participants who we re engaged in negotiation of meaning and collaborative construction of knowledge. Pena Shaff and Nicholls (2004) agreed with many scholars that Henris framework was vaguely defined making it difficult to categorize messages, and was not empirically tested. Finally, a number of scholars criticized the framework for appear ing to be based on a teacher -centered model (Gunawardena et al., 1997) with a strong teacher presence, and is not applicable to a student -centered conferencing s etting (McLoughlin & Luca, 1999) Nonetheless, Rourke et al. (2001) argued that although Henris coding scheme (1991) has bee n widely criticized and either modified or discarded, the fact that her model was the only model in the field that has been replicated, and thus has drawn a lot of criticism, posed a serious problem in the field of content analysis in CMC. They reasoned that replicating a research study improves the validity of the method and supports its efficacy. Hara, Bonk, and Angeli (2000) built an analytical model based upon Henris framework (1991) in order to study twelve weeks of electronic collaboration activity which accounted for 10% of a students final grades. They added several categories, specific criteria, and included indicators from additional analytical models to the framework to match their needs and to reduce ambiguity. The rese arch team also noted that they decided to drop an analysis of learners strategic knowledge (part of the meta -cognition ca tegory) due to its subjectivity. Instead, they added reflection, and included self -questioning into the regulation category. In t his discussion activity, each student was to sign up at least once for the role of starter to initiate the discussion and wrapper to summarize the discussion of the readings for the

PAGE 32

32 week. Each student was required to participate only once per week. The researchers randomly chose four weeks of discussion on which to conduct qualitative content analysis on a paragraph as a unit of analysis. Social, cognitive, and meta cognitive categories were of interest. Inter rater reliability of the cognitive category, as well as the aggregate inter rater reliability across categories, was 75%. They found that of the chosen four -week discussions, 33% of the student messages were at the surface level, 55% were at an in -depth level, and 12% involved both levels of proces sing. The team concluded that although their model was able to capture the richness of the discussion and the online conference did provide opportunity for students to exercise more cognitive skills, no evidence of meaning negotiation was found. They sugge sted that strategies beyond course requirements are necessary to motivate students to participate in the discussion and to generate deeper reflection. In their study examining social, response, and reasoning processes in an asynchronous learning network (A LN), Aviv et al. (2003) adopted Henris five level cognitive skills analytical model (1991) to study a seventeen -week blended course with three one -week asynchronous discussions that accounted for 30% of the final grade. The course required students to read and summarize discussion papers two weeks prior to the discussion. For each week, three students were assigned as panel members to initiate, moderate, and summarize the discussions. Of the five levels of cognitive skills, inference and judgment were found to be highest among other processes. In other words, Aviv et al. argu ed that it was possible for students in a student centered, group activity to reach the highest levels of reasoning. They concluded that structuring an ALN in a cooperative fashion with clearly defined roles for learners yielded high levels of reasoning as it extended thinking time.

PAGE 33

33 Indicators of Critical Thinking for Content Analysis Newman, Webb, and Cochrane (1995) set out to find a content analysis model based on the notion of surface and deep learning ; while surface learning refers to skimming, memorizing, and regurgitating for examinations, deep learning requires a thorough understanding of material. They also suggested that social context is essential for learners to develop higher level, critical th inking skills through interaction. That is, there is an apparent relationship between deep learning, critical thinking, and group learning. Newman et al.s model (1995) was based on Henris critical reason ing skills (1991) and Garrisons theory of critical thinking (1992) They claimed that the indicators contain both declarative knowledge (Henris cognitive category) regarding the person, task, and learning strategy and procedural knowl edge relevant to evaluation, planning, regulation, and self awareness (Henris meta -cognitive category). The model contains extensive sets of paired indicators that measure ten categories of critical thinking in discussions, whether the fragment of transcr ipts is critical (deep learning, signified by a plus sign) or uncritical (surface learning, signified by a minus sign). These categories cover content in terms of relevance, importance, novelty, outside knowledge, clarification, linking of ideas, justifica tion, critical assessment, practical utility, and width of understanding. For example, linking ideas and interpretation has four indicators as shown in Table 2 4 : Once the scripts are marked, the raters tally each (+) and ( ) indicator and calculate depth of critical thinking ratio, x ratio = ( x+ x)/( x+ + x). The ratio, they argued, is independent of the quantity of participation and reflects the quality of the messages. Using the indicators of critical thinking, Newman et al. (1995) conducted controlled experiments where half of the course seminar was done face-to -face and the other half over an asynchronous discussion forum Irrespective of the type of setting for discussion the seminars were desi gned to encourage critical thinking among students about controversial issues in

PAGE 34

34 information technology and society. The research team found evidence for critical thinking in both conference settings. While the face to -face setting demonstrated a tendency to generate new ideas, the discussion resulted in important, justified, linking of ideas. Newman et al. (1995) speculated that asynchronous learning environments might discourage students from contributing novel, creative ideas, as opposed to dynamic conversation (e.g., brainstorming) that occ ur in face -to -face discussions. In a subsequent paper, Newman, Johnson, Web & Cochrane (1997) presented a detailed analysis that demonstrated that the overall depth of the critical thinking ratio in asynchronous discussion was significantly higher than the fac e to -face seminar counterparts. Wickersham and Dooley (2006) chose the analytical model of Newman et al. (1995) as a theoretical framework to study the quality of online discussions of small group communities within an online course. They examined whether there were discrepancies among the six groups of graduate students that were never exposed to the whole class discussion. Students were assigned to read a book chapter and additional research articles prior to the online discussion and were informed that grading was based on quality of posting. In order to analyze the data sources the researchers employed read aloud protocol with consensus -building -measure and a color coding system to distinguish categories of critical thinking. They reported that the majority of the students integrated several categories of critical thinking, and that adult learners brought prior experiences and knowledge into the discussion. W ickersham and Dooley (2006) concluded that learners experiences affect the level of learner -learner interaction within an online community and that Newman et al.s (1995) critical thinking model served as an excellent framework for content analys is of small group, online discussions. In their comparative study of analytical models between Newman et al. (1995) and the five -phase interaction analysis model (IAM) (Gunawardena et al., 1997) Marra, Moore, and

PAGE 35

35 Klimczak (2004) argued that Newman et al.s (1995) definitions of the coding scheme are detailed and straightforward. However, the exhaustive list of indicators, along with mixed units of analysis (e.g., sentence, phrase, paragraph, or the e ntire posting), made it difficult to apply without referring back to the code list, impractical to calculate inter rater reliability, and difficult to interpret the resulting data. Practical Inquiry Model in Community of Inquiry Framework Another notable c ontent analysis model in the realm of critical thinking is the practical inquiry model proposed by Garrison, Anderson, and Archer (2000; 2001) The Community of Inquiry (CoI) framework (Garrison et al., 2000) consists of three elements: cognitive presence, social pr esence, and teaching presence (Table 2 5 ). According to Garrison et al. (2000) cognitive presence is a vital element in critical thinking and is fundamental to success in higher education. The term refers to the ability of participants in any particular configuration of a CoI to construct meaning through sustained communication. Social presence is the ability of participants to project their personal characteristics into the community, to present themselves to others as re al people. Social presence directly contributes to the success of the learning experience as well as to supporting cognitive presence. Teaching presence has two general functions: design of the educational experience and facilitation of learning. Design o f the educational experience, usually performed by teachers, includes selection, organization, and presentation of the course content as well as the design of learning activities and assessment. Facilitation is a responsibility usually shared among teacher s and sometimes other participants or students. This sharing duty is appropriate in higher education and is typical in CMC The element of teaching presence is seen as a means to an end, to support and enhance social and cognitive presence to fulfill learning outcomes.

PAGE 36

36 Garrison and colleagues (2000) argued that other analytical models faced methodological challenge s in creating and applying valid indicators specific enough to be meaningful and broad enough to be usable in the actual analysis of transcripts. Critical thinking, argued Garrison et al., is a holistic multi -phased process associated with a triggering event, exploration, integration and resolution. It is domain -specific and context -dependent. Further, it is an ite rative and reciprocal relationship in the CoI rather than a reflective process internal to one mind. They proposed a model called practical inquiry as a generic structure in cognitive presence (Figure 2 1). Based on Deweys concept of practical reflection (1910) the model represents the dynamic relationship between personal meaning and shared u nderstanding. Deweys practical form of inquiry had three stages pre reflection, reflection, and post reflection. Reflection is the vital part of the thinking process, which is framed by an initial puzzlement and a resolution at the close. Practical inqu iry is based on experience. It emerges through practice, shapes practice, and results in resolution. According to the model, the axes represent the shared and personal worlds, while the quadrants reflect the logical sequence of practical inquiry. The verti cal axis (action / deliberation) is a reflection on practice, while the horizontal axis (perception / conception) is an assimilation of information and a construction of meaning. The first category of cognitive presence, the l ower left quadrant of Figure 2 2 is a state of dissonance challenged by an experience. The second category, an exploration, is a search for clarification and an attempt to orient ones attention. The third category is an integration of knowledge and information into a coherent idea or concept by looking for insights and gaining understanding of the acquired knowledge and information. The fourth category, resolution, is an application of an idea or

PAGE 37

37 hypothesis to resolve an issue or a problem. Successful application and a validated idea will determine sustainability of the process of inquiry. In order to assign data into the categories of the cognitive presence, Garrison et al. (2001) created a list of indicators, or symptoms, for each of the phases of critical thinking. They also added descriptors that specify events in particular pha ses. These descriptors reflect general attitude of the phases (Table 2 6 ). After experimenting with several types of units, Garrison et al. (2001) found that message was the most suitable unit of analysis for the goals of their study. Unlike breaking up a message into smaller units that cannot be readil y identified, messages are clearly demarcated in the transcripts, and, thus, multiple coders can easily make decisions about coding. A complete message offers sufficient information to infer underlying cognitive processes. Further, using the message as a u nit is also appealing because the length and content of such unit is decided by its author rather than by the coder. When the message reflects multiple phases of cognitive presence, they recommend two approaches: code down if it is not clear which phase is evident, and code up if clear evidence of multiple phases is reflected Based on the initial practical inquiry model, the study analyzed three, one -week transcripts from two computer -conference graduate courses. The first transcript was a thirteen -week co urse led by two students acting as moderators. They facilitated, stimulated, and summarized the discussion while the instructor pass ively supervised the conference and only interjected to summarize messages with reinforcement and expert advice. The second and third transcripts were taken from the same graduate -level course led by an instructor who actively guided the discussions.

PAGE 38

38 The research team hired two coders and provided three training sessions of coding. The coders were encouraged to refine the proto col as they successively coded the first and second transcripts. After each round of coding, c oding results were then evaluated for inter rater reliability. Based on the revised coding scheme, the third transcript was coded and reported as final results. T he inter -rater reliability of t he final results using Cohens k appa reached k = .74. Although this reliability level was somewhat below the acceptable range (.80 to .90), Garrison et al. (2001) argued that it was caused by the fact that the coding system is breaking new ground with concepts that are rich in analytical value and has not been extensively tested. Garrison and colleagues (2001) found that the first phase, triggering event, had an 8% response. The low percentage of this phase resulted from the fact that the problem or issue was likely to be well -framed by the teacher. The second phase, exp loration, had 42%, which was the highest frequency of coded responses. They asserted that this is also expected because participants are likely to interact during a brainstorming phase as they share insights and contribute information. The frequency of the responses of the last two phases, integration and resolution, dropped to 13% and 4%, respectively. Garrison et al. (2001) suspected that this was due to an instructional goal that did not focus on advanced inquiry or an insufficient facilitation from the instructor in terms of guiding the discourse toward higher levels of critical thinking. They added that the model may not be appropriate for the learning context in these transcripts as it was based on Deweys work which has a pragmatic focus, respecting l ived experiences and application of knowledge. They concluded that practical inquiry is best for education where applied knowledge is valued, particularly in adult, continuing, and higher education. Fahy (2 002) compared the practical inquiry model with a model that he and his colleagues have developed, the Transcript Analysis Tool (TAT). Using a sentence as a unit of analysis, Fahy

PAGE 39

39 argued it might reveal interactions that are not necessarily evident in a me ssage -level analysis. Further, Fahy claimed that this finer granularity of sentence level analysis has the ability to detect and describe the nature of various social interactions, and differences in networking patterns in online communities. The TAT reco gnizes that messages in CMC contain both social and task related materials, reflecting individual differences regarding social and content outcomes. The model classifies sentence s into one of eight categories with five primary types (Table 27 ). Fahy (2002) argued that the TAT was capable of being aligned with the practical inquiry phases in several combinations to reflect different assumptions about the linguistic and social behavior with the phases. He set up three possible alignments of the practical inquiry phases and the TAT (Table 2 8 ). Each alignment represented different themes of education settings. For example, alignment 1 fits a typical education as it considered only questions as triggers or prompts from teachers, while the other alignments had regard to referential statements (2B). Non -referential statements (2A) were likely to reflect the nature of the problem in the exploration phase, but scaffolding/engagement statements (4) could also r eflect interaction between the private and shared worlds. While integration could be reflected in many combinations, scaffolding/engagement (4), referential statements (2B), and quotations/citations (5A/5B) may all indicate integration as in alignments 1 and 2. A more personal form of integration could also be found in reflections (3) as in alignment 3. Interestingly, Fahy claimed that resolution may be related to reflection (in alignment 1 and 2) or published resourc es (5A, and 5B in alignment 3). Fahy (2001) used the TAT to analyze transcripts of 2,550 sentences in 356 messages and applied it to three alignments of the practical inquiry phases (Table 2 9 ). The results of both

PAGE 40

40 mod els revealed that exploration was the most common type of sentence, followed by integration. The third alignment of the TAT was virtually identical to the practical inquiry of Garrison et al. (2001) However, the results of the terminal phase of the practical inquiry, resolution, varied widely among the three alignments. Fahy speculated that differences among the three alignments and the practical inquiry brought to light the potential variation in the proportion of transcript content, and, therefore, the final results could be presented as an expected r ange of each of the phases For example, triggering event statements were in between 3% and 13%. Fahy (2001) concluded that using sentences as units of analysis offered greater preci sion and variation of interaction types within the postings; that is, the cognitive presence model (Garrison et al., 2001) can be detected in a smaller unit. I also suggest that another interesting implication i s the degree of autonomy among the three alignments. Fahy (2001) argued that alignment 1 reflects the traditional approach, teacher led discussion, and the results of alignment 3 were similar to those of the practical inquiry in the CoI framework, which give s a flav or of social constructivism. It is possible to assume that these two alignments represent opposite ends of the pedagogical continuum where alignment 1 indicates teacher led instruction and alignment 3 indicates a student -centered approach. In Brazils con tinuing teacher education, Celani & Collins (2005) believed that the development of critical thinking and collaborative practice are major challenge s The y adopted the practical inquiry as a framework to promote critical discourse in both face to -face and online conferences. Participants were encouraged to interact for group work, discuss issues happening in their professional lives, and go beyond the excha nge of information and exploration of ideas to gain critical thinking by integrating ideas and solving problems. Twenty -seven students and teachers from the distance cohort produced 79 messages from the first discussion forum, and

PAGE 41

41 twenty three students and teachers produced 60 messages from the second session. Four -hour session transcripts from three sessions of face to -face discussions were selected; one at the beginning, one in the middle, and one at the end of the module. Approximately twenty student tea chers completed this module. Using practical inquiry along with the models of social and teaching presence, they found that as the online sessions progressed, interaction increased and social presence became stronger. Similar results were found in the disc ussion from the face to -face counterpart where 40% of the conversation from the first module was held by the instructor, in the form of mini -lectures. A lthough this pattern continued to a certain extent in the second module, dialogical interaction becomes more apparent in the last module. Regarding the cognitive presence, transcripts from the first forum of the online cohort revealed that phase one and two, expression of problem and dilemmas, was the most frequent. The same theme continued to occur in the s econd forum. Phase two, exploration and exchange of information, was also common in both sessions of the online cohort. Furthermore, the expressions found in both forums reflected strong social presence as well as cognitive presence, acknowledging others a nd encouraging collaboration to establish group cohesion. Transcripts from all three face -to -face sessions revealed a similar picture where problem recognition and sense of puzzlement were most prevalent. However, phase two brainstorming, suggestions or di scussion of ambiguities occurred more frequently from the first session of the face to -face cohort. Celani & Collins commented that the interaction in the online course was geared toward mutual recognition of its members as a group, rather than toward an a ttempt to conceptualize the discussion topic. On the other hand, the face -to -face cohort has been in the program for over two

PAGE 42

42 semesters and therefore they reached a higher order of cognitive presence but never reached the creation of solutions. Celani & Collins (2005) concluded that the existence of cognitive presence does not necessarily result in new knowledge construction, but interaction in class does strengthen learners social relations; thus, the community feels at ease and willing to share problems and experiences. However, I suspect that another reason that led Celani & Collins (2005) to find no evidence of cognitive presence may have been the fac t that participation requirements were never imposed upon the learners. Although the practical inquiry model in the CoI framework has proven useful for measuring cognitive presence in CMC, the model has recently evolved into survey instruments. In an attem pt to gain legitimacy for their theory of online learning and to tackle large inter disciplinary samples, Garrison & Arbaugh (2007) shifted the focus of the model to developing and employing psychometrically -sound instruments. Five -Phase Interaction Analysis Model (IAM) Gunawardena, Lowe, and Anderson (1997) developed an interaction anal ysis model that examined social construction of knowledge in CMC The interaction analysis model (IAM) is firmly grounded in Vygotskys (1978) social development theory. The theory emphasizes that cultural development appears on the social level and individual level, also kn own as the process of internalization Gunawardena et al. (1997) argued that the five phases of knowledge co-construction (Table 2 9 ) would occur when learners are engaged in the social construction of knowledge in a const ructivist learning environment. Gunawardena et al. (1997) applied the model to analyze interactions that occurred in a virtual conference discussion conducted through CMC The virtual discussion aimed to

PAGE 43

43 demonstrate and de velop effective learning activities which support quality discussion The research team chose an online debate as a learning activity and invited 554 subscribers, mostly professionals in the field of distance education, to participate in either the affirm ative or opposing side of the statement given by the debate leaders. The topic No Interaction, No Education was chosen to maximize the difference in opinion of both sides. The debate was scheduled for a week with the following details: Monday: Leader and members from the affirmative team began posting their statements; the leader summarized the arguments by midnight. Tuesday: The negative team leader and members posted their comments on Tuesday, and the leader summarized their statements. Wednesday: The a ffirmative team rebutted statements made by the negative team, and the leader posted a summary. Thursday: The negative team argued against statements made on Monday and Wednesday by the affirmative team; the leader recapitulated the argument. Friday: The affirmative team answered the arguments raised the previous day by the negative team and restated the case; the leader concluded the day with a summary statement. Saturday: Once again the negative team answered arguments made on Friday and restated the case; the leader summed up their arguments. Sunday: Volunteer judges discussed the outcome of the debate. Gunawardena et al. (1997) claimed that the debate served as an exemplary use of CMC in the co construction of knowle dge, largely due to the sharp focus of the discussion which was suitable for the participants. In analyzing the transcripts, they argued that using units of meaning as a unit of analysis, as suggested by Henri (1991) made it very difficult to code since the unit whether it be a statement, a paragraph, or two paragraphs did not capture the essence of meaning of the message. As a result, they decided to use a message as a unit of analysis and code each message according to the phases and operations from the IAM. Analysis of the debate

PAGE 44

44 transcripts indicated that the majority of postings f e ll into phases II and III, proving quality of the discussion as participants were engaged in discovering dissonance or inconsistency among ideas and negotiating meaning and co -constructing knowledge. McLoughlin and Luca (1999) set out to investigate the quality of interactions that occur in online discussion forums in a blended undergraduate course. Students were required to take part in the discussion, which accounted for 30% of the total score, by assuming specific roles (e.g., leader, questioner, and summarizer). Grounded in social constructivis t theory, the forums were truly student centered and allowed only minimal intervention from tutors. McLoughlin and Luca (1999) chose the IAM as an analytic model and a questionnaire designed for students to rate the value of the forum discussion and whethe r it supported group work, collaboration, feedback, and collective goals. The authors coded messages from forum transcripts, discussed, and concluded coding results in order to reduce inconsistency. Most of the messages were in the first phase of the IAM w ith little evidence of new knowledge construction. McLoughlin et al. concluded that although the forum did not foster higher phases of knowledge co -construction, it helped students consolidate existing knowledge schemas and therefore fulfilled the learning experience. Although these higher levels of knowledge co-construction were not observed in the content analysis sessions, questionnaire results were positive in almost every aspect including the value of interaction, knowledge construction, and fostering of new ideas. They speculated that the interactions remaining at the lower level caused by the lack of teacher intervention and a team -based approach to discussion replicated a teacher -center ed approach to instruction. They suggested that constructivist i nstructions, such as tutor modeling or role -based activity, should be explicitly integrated into the discussion to help students reach higher levels of knowledge co-construction

PAGE 45

45 Marra, Moore, and Klimczak (2004) conducted a study of an online graduate course and used the IAM and the model of Ne wman et al. (1995) as analytical protocols. The one -week, student -led forum on case studies was worth 4% of the final grade. Students were asked to post substantive analysis as well as to integrate course reading s and personal experiences in their contributions. Three researchers began coding multiple sentences or a paragraph or two with a single phase in the IAM, then checked inter rater reliability by using the most advanced phase from each posting as a b asis (Table 2 10). Therefore, the unit of meaning became the entire posting, or the message. The inter rater reliability coefficient was 93% after inter rater reliability discussion sessions. The coding results revealed that approximately 60% of contributi ons were at phases II and III, suggesting that students were discovering and exploring dissonance or inconsistency, or engaging in the negotiation of meaning or knowledge co-construction. They concluded that a large unit of analysis in IAM (a) makes it eas y to remember and discuss during the inter rater check sessions, (b) makes it possible to calculate inter rater reliability, and (c) makes it easy to interpret the results. They also claimed that the IAM and the model of Newman et al. (1995) are not comparable as they explained that the IAM reveals the process of knowledge co -construction while the model of Newman et al. produces individual indicators of critical thinking. In other words, IAM is a hierarchi cal model focusing on the holistic view of discussion flow and level of knowledge co -construction while the model of Newman et al. is a categorical model focusing on specific indicators that show evidence of critical thinking. Grounded in information proce ssing and social constructivist theories, Schellens and Valcke (2005) analyzed cognitive processing in online discussions from freshmen courses in education sciences during the 199899 academic year using the IAM and a modified analysis

PAGE 46

46 model of Veerman and Veldhuis Diermanse (2001) as coding schemes (Table 2 1 1 ). Although Veermans model is similar to IAM, indicators are classified into two types: non task related and task related messages. Using two different coding schemes to code the mess ages, Schellens and Valcke (2005) argued that the research hypotheses can be based on two data sets, and the results would help validate against each other. Four sessions of the discussions were based on real -world situations. Participation in the discussions was mandatory and accounted for 25% of the final grade. Students were required to post, reply, and cite learning materials at least once per case. The moderator gave scaffolding feedback once a week. The transcripts of eight groups were randomly chosen from 23 discussion groups involving 230 freshmen. The coding scheme of Veerman and Veldhuis Diermanse (2001) yielded 5.8% non task oriented, and 94.2% task oriented communication from all four discussion sessions. The IAM of Gunawardena et al. (1997) resulted in very high percentages of interaction in phases 1 and 3, while phases 4 and 5 were virtually absent (Table 2 1 1 results from the team members to assess the inter rater reliability. After negotiations, the coefficient alpha varied between 0.88 0.99. Schellens and Valcke (2005) categorized the number of messages into three levels lower than 160, between 160 and 195, and greater than 195 and used Chi -square analysis to report that there were clear significant difference between groups from both model of Veerman et al. (2001) 2(4) 117.524, p (1997) 2(4) 192.662, p<0.01). They concluded that the more discussion activities occur in groups, the more phases of higher knowledge construction will be seen. They further speculated that this might be related to social cohesion and that more interactions among students will lead to better learning results.

PAGE 47

47 However, Schellens and Valcke (2005) found mixed results regarding the same question: can a CMC environment foster higher phases of knowledge construction? The model of Veerman et al. (2001) showed that most of the messages fell into phases 3 (New i dea: theory) and 5 (Evaluation) probably due to the fact that students were asked to ground their arguments and to critically respond to other students. The results built on the IAM of Gunawardena et al. (1997) A very low proportion of messages was found in phases 4 and 5, the higher phases of knowledge construction. Schellens et al. criticized the model for lacking in discriminant capability; that is, the model failed to discriminate adequately among the types of statements in the transcripts and resulted in large portions of the transcripts ending up in very few cate gories. They concluded that the preset task structure (requirements for the discussion) affects outcomes of the discussions and suggested that adding roles to the discussion may yield even better outcomes. Schellens, Van Keer, and Valcke (2005) employed a similar method to study students in the same course in the 19992000 acade mic year with the same set of requirement to participate in asynchronous discussion groups from the previous year. However, students were assigned specific roles moderator, theoretician, summarizer, and source searcher as the researchers wanted to furt her investigate the significance of role assignment in discussion groups. Further, the Approaches and Study Skills Inventory for Students (ASSIST) was used to gather information about students learning styles (Entwistle, Tait, & McCune, 2000) Using an entire message as a unit of analysis, three researchers carried out coding tasks independently, conducted negotiation sessions, and the measured quality of coding by determining percent age of agreement, which was .91 after negotiations. Analysis results revealed that student and task characteristics sign ificantly influenced students mean level of knowledge

PAGE 48

48 construction; that is, higher individual numbers of postings and a positive attitude toward the learning environment resulted in a higher mean level of knowledge construction. Although no significant e ffects were found in group characteristics (i.e., intensity of group interaction) and role assignment, students who were assigned the roles of discussion summarizers often obtained significantly higher levels of knowledge constructi on compared to the other roles. Using transcripts from the same first-year undergraduate, blended course, De Wever, Van Keer, Schellens, & Valcke (2009) investigated the impact of role assignment and self assignment on students level of knowledge construction. They argued that that empirical evidence r evealed that self assessment affects cognition, encourages deep approaches to learning, and helps students monitor and improve their learning. Students were required to participate in four three week asynchronous discussion sessions and to contribute at l east four times per theme. Participation in the online discussion accounted for 25% of the final grade. Three research conditions were set up with a combination of role assignments and self assessment for the last condition (Table 2 1 2 ). Since each discus sion group was comprised of ten students, roles were assigned to the first five students for the earlier session and another five for the later session. These roles were a starter who initiated the discussion and motivated the group; a moderator who obse rved the discussion, asked critical questions, and probed into others opinion; a theoretician who introduced theories and ensured all relevant theories and concepts were used; a source researcher who sought external information to stimulate learning bey ond course content; and a summarizer, who posted interim and final synopses. Selected groups of students in the third condition were asked to evaluate their process of knowledge construction. This self assessment did not affect the final grade.

PAGE 49

49 With this research design, De Wever and colleagues (2009) aimed to examine (a) whether students are capable of judging their own social knowledge construction processes, (b) whether role assignment and the moment of introduction of the role assignment have an impact on the knowledge construction process, and (c) whether self assessment has a surplus value to stimulate students knowledge construction through social negotiation. Using the IAM of Gunawardena et al. (1997) researchers coded independently and randomly selected coding results to calculate inter The resulting coefficient was situated between 0.40 0.80, which corresponds to fair to good agreement beyond chance. A difference score between coding scores (observed scores) and self assessment scores (self report scores) was calculated for each level of knowledge construction in the IAM. Results indicated that students underestimated themselves at the first level and overestimated themselves at the other four, higher levels of knowledge co nstruction. The team argued that it was not surprising that first -year students misjudge their own learning and suggested that instructors should explicitly help students develop assessment skills and provide comparative information and feedback. However, the researchers found that role assignment and the moment of introduction of the role assignment significantly impacted students learning, as students in conditions with roles during the two initial themes (conditions 2 and 3) outperformed students in conditions with roles during the two later themes (condition 1) with respect to the level of knowledge construction. They argued that initial role assignment helped students internalize role related activities, and students carried such skills over to the next activities even without role assignments. However, no significant differences were observed in the fourth theme. They speculated that absence of the trend implied that the instructor might need to gradually decrease the role assignment; in other

PAGE 50

50 words fading of support or scaffolding, as opposed to a sudden drop, would result in effective internalization of desired skills. Regarding the impact of self assessment on students knowledge construction, they reported that mixed results were observed. Hence De Wever et al. (2009) concluded that introduction of recurrent self assessment procedures did not have a significant value on the processes of knowledge construction in the discussions. Although IAM is not originally intended to be used for finding evidence of critical thinking, many operations in the model are similar to Lipmans (2003) examples of observed behaviors that exe mplify critical thinking skills. Critical thinking, as previously mentioned in this chapter, is thinking that (1) facilitates judgment because it (2) relies on criteria, (3) is self correcting, and (4) is sensitive to context (Lipman, 2003, p.212) The alignment in Table 2 1 3 shows how behaviors in Lipmans concepts of critical thinking o verlap with phases in IAM. Essentially, behaviors that represent criti cal thinking skills fall into phases II to phase IV and behaviors that reflect judgment an ultimate goal of critical thinking are in phase V The overlap between concept s of critical th inking and phases in IAM gives grounds for using this content analytic model for examining evidence of critical think ing in online discussion forum s Furthermore, t his study aligns with Lipmans (2003) definition of critical thinking with its emphasis on the social aspect. This definition is appropriate for higher education settings where members of each community are groups of novices and experts learning the body of knowledge and skills that involve various kinds of criteria. These crit eria are often context specific. T his definition also leans toward social constructivist theory, a pedagogical assumption that stresses the importance of culture and context in understanding phenomenon in community (Kim, 2001)

PAGE 51

51 Content A nalysis : In Search of a Partner S ince Henri (1991) proposed her framework, t he idea diversified into several models that took different pedagogical and theoretical slant s Several analytic models for analyzing discussion transcript s have been develop ed and tested. Although content analysis in CMC has been proven a fairly successful method during the past two decades, scholars raise concern s about the time -consuming process of this manually -evaluated technique Some have already shifted the direction of their research For example, Garrison and Arbaugh (2007) argue that more empirical studies to assess the explanatory power of the CoI are needed if the framework is to gain legitimacy as a theory of online learning. Therefore they decided to shift their focus to developi ng and employing psychometrically sound instruments to study larger population samples Interestingly, only IAM an analytic model based on the notion of social constructivism has gained in popularity and is steadily receiving more evidence to add to its foundation (De Wever et al., 2009; Marra et al., 2004; McLoughlin & Luca, 1999; Schellens & Valcke, 2005; Schellens et al., 2005; Yang et al., 2005) Some studies have branched out into research designs that include new variables to better explain the phenomena ; for instance, De Wever et al. (2009) integrated role assignment s to better understand the process of knowledge const ruction and social negotiation O th er s studies that employed the IAM coding system have strengthen ed the validity of the framework helped compare results with a growing nor mat ive data, and thus offered more meaningful interpretation of the data in a larger context (Liam Rourke & Anderson, 2004) T his study aims to add further evidence into the foundation of content analysis in CMC and supp ort critical thinking skills in online education Figure 1 1 illustrates the conceptual framework of this study. Similar to other studies that used IAM, this study chose social

PAGE 52

52 constructivism as a theoretical framework. The IAM itself is based on the socia l development theory of Vygotsky (1978) T he concept of critical thinking of Lipman (2003) was added to give a justification for connecting such thinking skills to the levels of knowledge construction in the IAM coding scheme Despite the potential of the IAM coding system to measure levels of knowledge and critical thinking skills a search continues for an alternative method that could expedite the measurement process es to support the growth in popularity of distance learning. One of the possibilities for measuring levels of knowledge construction is to look deeper into the technical side of the learning platform s or the LMSs. Relations between dialogs in discussion forum s are data with such potential One of the technique s being proposed is SNA, a theory and method rooted in sociology that has been applied to many contexts from explaining concrete social circumstances such as how Americans form their confidant network (Knoke & Yang, 2008) to far more abstract social phenomena such as the spread of happiness (Fowler & Christakis, 2009) Many researchers argue that this technique may well provide better understanding of lear ners and the learning community (Harrer et al., 2006; Lowes, Lin, & Wang, 2007; Nurmela et al., 1999; Reffay & Chanier, 2002; Shen et al., 2008; Zhu, 2006) The following sections discuss this novel research agenda and how it can help the assessme nt of critical thinking skills. Social Network Analysis N etwork study is an interdisciplinary method originating in psychology, anthropology, and sociology and is becoming one of the influential analytical techniques used in social sciences (Knoke & Yang, 2008) T echnique s employed by this study are rooted in sociology and based on the intuitive notion that individuals are e mbedded in patterns of social connections that have important consequences for them (Freeman, 2004) Freeman explains that network analysts seek

PAGE 53

53 to uncover social patter ns, to determine the conditions in which those patterns arise, and to discover the c onsequences of such conditions. According to Knoke and Yang (2008) n etwork study focuses on the (a) actor (ego ) and related members connected to the actor (alters) individual or collective members such as a pol itical party and (b) relation s (or tie s ), meaning connections between a pair of actors (dyad) Network analysts may specify the relation as binary (present or absent) or valued tie (i.e., types or fre quency of relations ). Network study can roughly be categorized into two levels: micro level (egocentric ) and complete network ( sociocentric ). O ne of the large scale social network studies by t he General Social Survey or GSS (www.norc.org/GSS+Website/) inst antiates the use of the most common data collection technique in egocentric study called name generators, survey questionnaires that collect information from ego respondent s about relationships among alters that the ego has direct contact with. The GSS con fidant network surveys were conducted in the U.S. during 1985 and 2004 asking respondents to report as many as six names of colleagues with whom they discussed important matters and to provide a dditional information including biographical data how long th ey had known each other, how often they talked, and types of relationships (e.g ., spouse, parent). The reports showed that average confidants had dropped from 2.94 to 2.08 in 2004. T ypes of relationships were homogenous in age and education during the 1985 study. Although homogeneity of the types of relationships continued to be very high, educational heterogeneity decreased while racial heterogeneity increased. T he survey analysts hypothesized that the rise in Americans social isolation may have caused th ese changes, The GSS studies illustrate how social network analysis can uncover confidant network patterns and explain the conditions wherein patterns arise. It should be noted that the GSS studies are considered egocentric regard less of the scale of the study. Although sociocentric is

PAGE 54

54 preferable, sometimes it is not possible or cost -effective to track down all members of the network because c ollecting data for directed measures would require all egos to report the occurrence of a relation by using name -ge nerator instrument s H owever, i n the era of Internet based communication, collecting data from a large network can be less expensive, less obtrusive, and more immune from measurement errors (e.g., interview effects, imperfections in recall, or self report questionnaire s ) (Knoke & Yang, 2008; Lewis et al., 2008) Some network ana lysts began to capitalize on large data on the Internet especially data from social networking sites (Hanneman & Riddle, 2005) For instance, Lewis et al. (2008) collected and analyzed longitudinal data from F acebook (facebook.com) claiming that such data are natural ly occurring as opposed to contrived (i.e., unobtrusive) and offer several aspects of relationship (also called multiplex). Although sociocentric analysis can easily be employed in such data set s some caveats remain. It is difficult to define friendship in F acebook because it means different things to different people. Generalizing beyond online interaction is also problematic (Lewis et al., 2008) Working with public data set s such as social networking sites is not likely to give insight into work related relation s (Knoke & Yang, 2008) F urther, t he major obstacle to using data from Internet based communication, such as CMC, seems to shift from traditional issues (e.g., self report survey) to choosing the r ight data set that will reflect desired relations. Email communication in an office network for instance, may reflect professional relations rather than social ones. Social Network Analysis in Online Learning Environments: Sim ilar to scholars in the network study community, educational researchers began to show interest in the technique of SNA Several studies reported using this technique, at least in part, to analyze transcripts in CMC (Cho, Gay, Davidson, & Ingraffea, 2007; Martinez, 2009; Stefanone & Gay, 2008) Given the

PAGE 55

55 fact that communication in a discussion forum in an online classroom would better illuminate the network structure of an academic community than a friend network, m any researchers put forth the idea of using a network analysis to uncover social structure in online learning communities (Harrer et al., 2006; Lowes et al., 2007; Nurmela et al., 1999; Reffay & Chanier, 2002; Shen et al., 2008; Zhu, 2006) Nurmela, Lehtinen, and Palonen (1999) em ployed SNA to evaluate social structures and processes of a group studying in an online environment. Eighteen participants from educational science, psychology, and teacher training worked in pairs in a blended course that had 18 case based assignments wit h requirements to make new documents (i.e., assignments) and to assign tags to their documents. They could also comment, attach files, add reference links, and mark parts of text as either for or against other documents. These actions were categorized into 26 types of log activities that were automatically collected from the system. Nurmela et al. (1999) later selected only four log types that they considered the best actions to describe knowledge building. These four meaningful log types w ere finished making a new document, finished editing a document, reading a document, and added a comment, question, link or keyword to a document. Of these four, document reading was the largest portion (85%). Although the results revealed several types of learning strategies, from many short edits to a few lengthy edits of the documents, they found no statistically significant correlation between the measured action types and learning outcome scores earned in each of the assignments. Further, the r esults showed no significant links between student pairs in the same discipline, despite the authors assumption that these students would bette r communicate with one another.

PAGE 56

56 Nurmela et al. (1999) then used these log data to create sociomatrices an actor -by actor matrix that represent relations among them (Hanneman & Riddle, 2005) and calculate s measures of centrality measures that seek to quantify the notion of the actor s prominence (e.g., quantity of relations that an actor has ) within a complete network (Knoke & Yang, 2008) including the degree of use of asymmetric values and betweenness using symmetri c values The results were then used to create sociogram s of each of the log types with and without instructors. The sociogram s with instructors were almost perfect star -shaped with instructors as a hub indicating that instructors were prominent actors as they controlled the flow of communication in the network Althou gh sociogram s without instructors were also relatively star -shaped indicating that some students were central to the network none of them were as centralized as instructors in the previous sociogram s with instructors. Nurmela et al. (1999) we nt on to examine document content and found that eight selected documents revealed rather shallow content. They suggested that deeper analysis into the document content should be conducted. Nurmela et al. concluded that their methods can quickly organize l arge amounts of information and illustrate the structure of a learning community, including a search for central actors. This could be a starting point for an analysis of knowledge bui lding and acquisition processes Nurmela et al believe that such insigh t can be fulfilled w ith better tools to follow the elaboration of documents and interrelated groups of documents Grounding their study in structural theory, Reffay and Chanier (2002) argued that each individual acts according to the group in a discursive rather than seclu ded fashion. They employed S NA to study email network graphs for four groups of students using GraphViz, an open source graphing application. They generated a graph representing all email communication and subsequently took the tutor node out of the graph. The graph without a tutor s howed how the

PAGE 57

57 groups actually collaborated. Similar to the conclusions of other network studies in online learning environments t he authors claim that these graphs helped the coordinator and tutors detect communication problems. Reffay and Chanier conclud ed that distance learning educators should focus more on monitoring rather than guiding. Arguing that pure quantitative methods or applied methods might not be sufficient for understanding the depth of social structure Harrer, Zeini, and Pinkwart (2006) proposed the triangulation research design using qu alitative methods, statistical analysis, and social network analysis to study learning activities situated in an undergraduate computer science blended lecture. Data were drawn from small group online forum discussion s (for internal usage, and with assigne d customers), a class wiki, and the data from the Concurrent Versioning System (CVS) server. Qualitative methods were employed to examine patterns of these communication tools. Four categories emerge d from the data in the classs w iki: project management, clarification of terms, reference list, and coding conventions. Of the 20 project groups, ten used the wiki extensively while the rest used it very little or not at all. They also distinguished four differ ent strategies in forum usage for each group: struc tured and short threads, few topics but long threads, many topics and long threads, and use only for meeting organization. Harrer, Zeini, and Pinkwart then applied social network analysis to reconstruct social structure (e.g., communication paths) using the results of the overall forum posting and the three combinations of wiki, CVS, and forum usage (e.g., no wiki use, highest CVS, differentiated topics with short threads). On one hand, the analysis of a complete network of forum postings yielded a low degr ee centrality, and no significant actors in the class. On the othe r hand, t he sociograms graphs represent ing social network data, based on the three combinations yielded fairly rich results as they were able to elaborate on the relationship of each socio gram and the

PAGE 58

58 flow of communication. For example, a group with a low degr ee centrality, both out -degree and in -degree ( prestige ), tended to have internal communication problem s Although the results of both qualitative methods and social network analysis showed no signi ficant difference when compared with the final scores of the group projects, the combined usage of these communication tools was shown to produce be tter results. Harrer Zeini, and Pinkwart (2006) also commented that the sociograms and network measures provided additional information regardi ng communication structure. Shen, Nuankhieo, Huang, Amelung, and Laffey, (2008) examined the sense of community, a feeling of belonging to the group, and interaction in online learning. They argued that although these two factors are closely related, little empirical research has examined how the interaction shapes and sustains the sense of community in online learning environments. They employed Rovais Classroom Community Scale (2002b) and SNA to study individual perspec tives and interaction patterns among individuals in an online community. Shen et al. (2008) used three SNA measures in the study networ k density, degree centrality, and network centralization. Network density represents the number of ties in a particular network as a ratio of the total maximum ties that are possible with all the nodes of the network. The density D of an undirected graph w ith N nodes and M ties is defined as D = 2M/N( 1). Degree centrality is measured by in degree, the amount of people who interact with a specific student, and out degree, the amount of interaction that a student initiates with others. Network centralization is the degree of variance in the targe ted network and a fictitious per fect star network of the same size. This fictitious star network represent s the most centralized, unequal possible network because every actor has a degree of one except one actor, the star, has the degree of all number of other actors less one

PAGE 59

59 (Hanneman & Riddle, 2005) In other words, network centralization indicates how closely the graph is organized around its center point. Shen et al. (2008) chose to study two graduate courses about the design of educational technology. Ten students in the first course chose to participate while fifteen students in the second course chose to do so. The instructor randomly assigned the student s dyads and asked them to review and give feedback to their colleagues. While d yad assignments and individual projects were assigned to stude nts, all students were graded individually. Students were also asked to complete the Classroom Community Scale (Rovai, 2002b) survey electronically. Using two -way ANOVA, they found that students in the second course had a significantly higher level of perceived sense of community, F (1, 62) = 5.134, p < .05, than the students in the fi rst course. NetDraw 2.0 was used to generate diagrams to vis ualize three types of interactive patterns for course activities including content read, forum read, and forum reply. The three SNA measures, network density, degree centrality, and network centra lization, were computed using UCINET 6.0. The second course had significantly higher levels of out -degree than the first course. In addition, peer activity had significantly higher levels of out degree than individual activity. The study found that student s in the second course had more frequent interaction and more information exchange s which aligned with the results of the Classroom Community Scale where students in the second course perceived higher levels of sense of community. Shen et al. (2008) concluded that (a) the SNA shown that the instructor played an important role in both classes, (b) patterns of interaction were influenced by the task types as peer review activity yielded more interaction than individual activity, and (c) the network centralization showed that the networks of both classes were not perfect stars, meaning that interactions were not equally distributed. Finally, students who initiated interactions in the first

PAGE 60

60 course achieved high interaction scores, while students who were initially approached by others to interact in the second course achieved high interaction scores. They hypothesized that students with high interaction scores might have played different roles in their class; that is, students in the second course might have contributed fewer but higher quality postings so other students chose to interact with them. Shen et al. suggested that SNA is useful for early identification of students who are lagging in their contribution. In the study using multiple methodological approaches including SNA, content analysis, and survey questions about satisfaction, Lowes, Lin, and Wang (2007) found that measures in SNA density, and network centralization are highly correlate d with each other and with students satisfaction ratings. They argued that satisfaction ratings depend on dominance of the facilitator and the dispersion of the conversion among participants; that is, forums that had high satisfaction ratings had high fac ilitator involvement. They further explain that by having the facilitator questioned and challenged, participants were more likely to offer new information rather than only affirming or offering praise. Lowes et al. (2007) conclude that the way that instructor facilitates the discussion is crucial in greater in teraction among participants as they explained although it might be expected that greater interaction among participants would also be associated with lower centralization on the facilitator, in fact this was not the case here: it was the content of the f acilitation th at was the key (p.195). Social network analysis is indeed useful for educators and researchers who need to gain an understanding of social structures in learning communities. It helps researchers quickly grasp social structure and knowledge building in the classroom before conducting further examination of students actual contribution s such as conducting content analysis of students forum postings

PAGE 61

61 (Nurmela et al., 1999) Results of SNA can also be used to correlate with traditional approaches, such as surveys, to better understand social structure and interaction patterns (Shen et al., 2008) At th e classroom level, SNA helps instructors keep track of interaction and communication among students (Reffay & Chanier, 2002; Shen et al., 2008) Using Social Network Analysis to Assess Critical Thinking Wh ile content analysis studies are gaining a foothold in CMC research, many researchers raise concerns about the time -consuming processes of such manuallyassessed methods. In fact, many researchers believed that other methods that rely on unobtrusive data such as data mining and SNA would support or substitute the tedious process of content analysis (Bratitsis & Dimitracopoulo u, 2008; Dringus & Ellis, 2005; Huang & Chuang, 2008; Shen et al., 2008) Unfortunately, there is no study that attempts to merge the two fields to confirm the fruitful results of such a proposal. Nurmela and colleagues (1999) probably came cl ose to this mixed method when they employed SNA technique to study social structures in an online learning environment where students shared their learning through documents. They followed up the study with a deeper evaluation of the content of the documents that they found most strongly related (i.e., had the most comments and links). This was not an actual content analysis study since it lacked any particular framework; however, the researchers did look for quality of the most strongly -related document an d found no significant quality in it. The document was not considered theoretically sound, and the comments were rather shallow. They suggested that better tools to examine content of the documents should be developed. The literature informs us that using SNA to gain insights in to the structure of online learning communities provides a snapshot of the structure of a learning community (Reffay & Chanier, 2002; Shen et al., 2008) and could potentially be a s tarting point for extensive analysis of knowledge construction and acquisition processes (Nurmela et al., 1999) ; likewise a study

PAGE 62

62 that employed content analysis suggested that the more discussion activities (i.e., interactions) occur i n groups, the more phases of higher knowledge construction will be seen (Schellens & Valcke, 2 005) S uggestions from both side s are consonant with Gunawardena et al. (1997) argument that individuals construct their own understanding by interacting with shared knowledge in the community, even in disagree ment. Given the proposition that content analysis and SNA could complement one another, i t is t i mely to bring SNA into the search of a new method for measuring critical thinking in CMC Summary During the past few years, asynchronous Internet -based technologies have gained popularity among institutions offering distance learning. These text -based learning environments give hope to many scholars searching for evidence of critical thinking in online learning. Many content analysis models have been proposed to measure le vels of knowledge construction in these lean media. One of the models that has been frequently used during the last decade is the IAM of Gunawardena et al. (1997) While the model has proven fruitful in examining social co nstruction of knowledge in CMC by analyzing tr anscripts of the discussion, it is a tedious and time -consuming process and this concerns many scholars. Educational researchers began to turn to SNA in order to better understand social structure and the proce ss of knowledge construction in distance learning environments. Many proposed that this technique gives a synopsis of the results of content analysis. Unfortunately, none have actually followed such advice. This chapter reviews the literature of the two me thods and reveals the need for the mixture of the two The next chapter discusses the research methods used in this study content analysis and social network analysis to measure lev els of knowledge construction.

PAGE 63

63 Table 2 1. Lipmans examples of behaviors associated with concepts of critical thinking Concepts Example of associated behaviors Self correction Students demand reasons and criteria where none have been provided Students identify inconsistencies in discussions Students point out fallac ious assumptions or invalid inferences in texts Acquiring sensitivity to context Students differentiate among nuances of meaning stemming from cultural differences, differences in personal perspectives or points of view Students search for differences between seemingly similar situations whose consequences are different Being guided by criteria Students invoke standards: criteria for determining the degree of satisfaction needed to satisfy a criterion Students invoke tests: probes or interventions f or the purpose of eliciting empirical findings Judgment Students seek settlements of deliberations Students seek solutions to actual or theoretical problems Students seek evaluations of performances, services, objects, products, etc.

PAGE 64

64 Figure 2 1. Development of content analysis frameworks in CMC

PAGE 65

65 Table 2 2. Henris analytical model: cognitive skills Reasoning Skills Definitions Elementary clarification Observing or studying a problem, identifying its elements, and observing their linkages in order to come to a basic understanding In depth clarification Analyzing and understanding a problem to come to an understanding which sheds light on the values, beliefs, and assumptions which underlie the statement of the problem Inference Induction and deduction, admitting or proposing an idea on the basis of its link with propositions already admitted as true Judgment Making decisions, statements, appreciations, evaluations and criticisms Strategies Proposing coordinated actions for the application of a solution, or [following] through on a choice or a decision Table 2 3. Henris analytical model: processing information (examples only) Surface Processing In Depth Processing Repeating what has been said without adding any new elements Offering new elements of information Stating that one shares the ideas or opinions stated, without taking these further or adding any personal comments Generating new data from information collected by the use of hypotheses and inferences Proposing solutions without offering explanations Proposing one or more solutions with short, medium or long term justification Making judgments without offering justification Providing proof or supporting examples Perceiving the situation in a fragmentary or short term manner P erceiving the problem within a larger perspective Table 2 4. Newman et al. indicators of critical (+) and uncritical ( ) thinking (examples only) L+ Linking ideas, interpretation L+ Linking facts, ideas and notions L+ Generating new data from inform ation collected L Repeating information without making inferences or offering an interpretation L Stating that one shares the ideas or opinions stated, without taking these further or adding any personal comments

PAGE 66

66 Table 2 5. Elements and Categories in Community of Inquiry (Garrison et al., 2000) Elements Categories Indicators (example only) Cognitive Presence Triggering Event Sense of puzzlement Exploration Information exchange Integration Connecting ide as Resolution Apply new ideas Social Presence Emotional Expression Emotions Open Communication Risk free expression Group Cohesion Encouraging collaboration Teaching Presence Instructional Management Defining and initiating discussion topics Buil ding Understanding Sharing personal meaning Direct Instruction Focusing discussion

PAGE 67

67 Figure 2 2. Practical inquiry model of Garrison et al. (2001)

PAGE 68

68 Table 2 6. Guidelines for Coding Cognitive Presence in the Practical Inquiry Model (Garrison et al., 2001) Categories Descriptor Indicato rs Triggering Event Evocative Recognizing the problem Sense of puzzlement Exploration Inquisitive Divergence within the online community Divergence within a single message Information exchange Suggestions for consideration Brainstorming Leaps to conclusions Integration Tentative Convergence among group members Convergence within a single message Connecting ideas, synthesis Creating solutions Resolution Committed Vicarious application to real world Testing solutions Defe nding solutions Table 2 7. The Transcript Analysis Tool (TAT) (P. J. Fahy, 2002) Type Sub categories Description Type 1: questions 1A vertical questions A correct answer exists 1B horizo ntal questions May not be one right answer Type 2: statements 2A non referential statements Self contained, does not invite response 2B referential statements Makes reference to preceding statements Type 3: reflections Expresses thoughts, judgme nts, opinions or personal information Type 4: scaffolding / engaging Interpersonal interaction that connect or agree with others, comments without substantive meaning Type 5: quotations / citations 5A quotation of other sources 5B citations or at tributions of quotations or paraphrases Table 2 8. Results of three alignments and the practical inquiry Trigger Events Exploration Integration Resolution Alignment 1 1A, 1B 2A, 4 2B, 5A, 5B 3 3% 62% 14% 20% Alignment 2 1A, 1B, 2B 2A 4, 5A, 5B 3 12% 52% 15% 21% Alignment 3 1A, 1B, 2B 2A, 4 3 5A, 5B 12% 62% 21% 5% Practical inquiry (Garrison et al., 2001) 13% 63% 19% 6%

PAGE 69

69 Table 2 9. Gunawardena et al. five-phase interaction analysis model Phase I Sharing/Comparing of information, specific operations which may occur include: A A statement of observation or opinion B A statement of agreement from one or more other participants C Corroborating examples provided by one or more participants D Asking and answering questions to clarify details of statements E Definition, descripti on, or identification of a problem Phase II The discovery and exploration of dissonance or inconsistency among ideas, concepts or statements A Identifying and stating areas of disagreement B Asking and answering questions to clarify the source and ex tent of disagreement C Restating the participant's position, and possibly advancing arguments or considerations in its support by references to the participant's experience, literature, formal data collected, or proposal of relevant metaphor or analogy to illustrate point of view Phase III Negotiation of meaning/co construction of knowledge A Negotiation or clarification of the meaning of terms B Negotiation of the relative weight to be assigned to types of argument C Identification of areas of agre ement or overlap among conflicting concepts D Proposal and negotiation of new statements embodying compromise, co construction E Proposal of integrating or accommodating metaphors or analogies Phase IV Testing and modification of proposed synthesis o r co construction A Testing the proposed synthesis against received fact as shared by the participants and/or their culture B Testing against existing cognitive schema C Testing against personal experience D Testing against formal data collected E T esting against contradictory testimony in the literature Phase V Agreement statement(s)/applications of newly constructed meaning A Summarization of agreement(s) B Applications of new knowledge C Meta cognitive statements by the participants illustr ating their understanding that their knowledge or ways of thinking (cognitive schema) have changed as a result of the conference interaction

PAGE 70

70 Table 2 10. IAM coding results after inter rater checks Phase I* Phase II* Phase III* Phase IV* Phase V* Total Agreed ratings 10(21%) 16(34%) 14(30%) 4(9%) 0 44(94%) percentages based on total of 47 coded postings, including 3 where inter rater agreement could not be reached. Table 2 11. Comparison of Veerman et al. model and Gunawardena et al. model Modifi ed model by Veerman et al. (2001) Model by Gunawardena et al. (1997) Non task related Plannin g (20.6%) Technical (9.2%) Social (52.1%) Irrelevant (18.1%) Task related 1 New idea: fact (0.1%) Phase I: Sharing and Comparing (51.7%) 2 New idea: experiences / opinion (14.3%) Phase I 3 New idea: theory (2 9.6%) Phase I 4 Explanation (refining or elaboration) (15%) Phase II: Dissonance or inconsistency (13.7%) 5 Evaluation (41%) Phase III: Negotiation / co construction (33.1%) Phase IV: Testing and modification (1.2%) Phase V: statement / app lications of newly constructed meaning (0.4%) Table 2 12. Overview of the research conditions of De Wever (2009) Theme Condition 1 Condition 2 Condition 3 1 No role assignment Role assignment Role assignment + SA* 2 No role assignment Role assignment Role assignment + SA 3 R ole assignment No role assignment No role assignment + SA 4 Role assignment No role assignment No role assignment + SA *SA = self assessment

PAGE 71

71 Table 2 13. Lipmans concepts of critical thinking and associated behaviors Concepts Examples of Associated B ehaviors Corresponding phase(s) from IAM Self correction Students demand reasons and criteria where none have been provided II B Students identify inconsistencies in discussions III C Students point out fallacious assumptions or invalid inferences in texts II A Acquiring sensitivity to context Students differentiate among nuances of meaning stemming from cultural differences, differences in personal perspectives or points of view II C, III Students search for differences between seemingly similar si tuations whose consequences are different III C Being guided by criteria Students invoke standards: criteria for determining the degree of satisfaction needed to satisfy a criterion II C Students invoke tests: probes or interventions for the purpose of eliciting empirical findings IV Judgment Students seek settlements of deliberations V Students seek solutions to actual or theoretical problems V Students seek e valuations of performances, services, objects, products, etc.; V

PAGE 72

72 CHAPTER 3 METHODOLOGY This chapter presents the methodology of the study including the theoretical framework research design, participants, data collection, data analysis, and limitations. The focus of this study was to contribute to the field of critical thinking by fin ding novel methods to assess levels of knowledge construction. Specifically, it sought to answer the question: Do levels of knowledge construction in online learning environments relate to the centrality measures of social networks? The study used various quantitative methods including : (a) the interaction analysis model (IAM) of Gunawardena et al. (1997) content analysis method which has been extensively used in an alyzing transcripts in the CMC (b) social network analysi s (SNA), the novel technique that has potential to measure levels of knowledge construction and critical thinking skills and (c) Spearmans c orrelation s to discern the nature of the relationship between the se variables. Theoretical Framework This study was gro unded in social constructivism. The constructivist theory suggests that learners do not merely absorb information but actively organize and make sense of it. S ocial constructivism focuses on how people work together to construct knowledge they perce ive from the learning content (Ormrod, 2008) This theory fits well for the chosen methods of the present study, quantitative content analysis using IAM and SNA The content analysis model known as interaction analysis mod el of Gunawardena et al (1997) is firmly grounded in Vygotskys (1978) social development theory. Vygotsky posits that cultural development occurs first on a social level an d later on an individual level. Although Vygotskys discussion focuses on child development, I believe that such development could appear at any stage of life given the fact that individuals encounter new culture throughout their

PAGE 73

73 liv e s including during the period of higher education where novices are immerse d in a certain professional community. As learners enter into a community of practice, they become involved in a population that embodies certain belief s and behaviors (Lave & Wenger, 1991) Lave and Wen ger posit that t he understanding and experience in a community are in constant interaction and are mutually constitutive among members as they negotiate and renegotiate the meaning of knowledge in the socially constituted world Similarly, Gunawardena et al. (1997) argue that there is an interdependency of individual and social knowledge creation ; that is, a communitycreated knowledge at the social level, and i ndividuals create their own understan ding by interacting with s hared knowledge thinking of each individual is inevitably influenced by the thinking of the other members taking part in discussion, even if it is only to disagree (p.409) The underlying assumptions of the social network study (Knok e & Yang, 2008) are congruent with that o f social constructivism. Social network analysts believe that (a) to understand observed behaviors, social relations are often more important than attributes such as age, gender, and ideology ; ( b ) structural mechan isms that are socially constructed by relations among entities affect pe rceptions, beliefs, and actions; and (c) such relations should be viewed as dynamic processes. Similar to the notion of interaction in social constructivism, t hese assumptions in SNA i nfer that individuals are inevitably affect ed by interaction with members in their community and exposure to the culture they live in Social scientists use SNA to uncover network patterns and make sense of the conditions in which patterns ar ise. Studies r ange from understanding social patterns of confidant network s in the US (Knoke & Yang, 2008) to abstract concept s such as the spread of happiness (Fowler & Christakis, 2009) It is reasonable to speculate that SNA may help us discover pattern s of knowledge construction or critical thinking skill s in an online learning community.

PAGE 74

74 The following section discusses how the study was designed in order to merge the two techniques and how two different data sets (transcripts a nd relations data) from the same type of learning activities discussion forums, were used in this study Research Design This study adopted three quantitative approaches by using quantitative content analysis, SNA and Spearmans correlation s to discern level s of knowledge construction and measures in SNA The primary data of the study were transcript s and the relations among posting s in Moodle LMS discussion forums The five -phase interaction analysis model (IAM) of Gunawardena el al. (1997) was used to quantitatively analyze the transcripts from the online discussion sessions. This method was selected because it is a means to examine social construction of knowledge based on the premise of social constructivism, which is the theoretical framework of the study. Given the fact that this study used data set s from an online course management systems database where the population can be defined and all ties among population can be retrieved a macro level of analysis was sele cted to measure centrality measures Detail s of these methods will be disc ussed in the Data Analysis section. Ultimately, indicators from the social network analyses w ere used together with results of the content analysis to illuminate the connection betwe en the two groups of variables On that account, c orrelation coefficient s were used to discern the nature of the relationship between the se variables. C ontext of the Study There were 7 eight -week graduate level onl ine courses in Educational Technology pr ogram offered through the Distance Education office within a college of education at a major, research intensive university in the southeastern United States. Participants in the study were

PAGE 75

75 enrolled in one or more online courses where online discussion for ums were one of the academic activities. Selected online courses included EME 5207 Designing Technology Rich Curricula, EME 6205 Digital Photography and Visual Literacy, EDG 6931 Distance Education Leadership and Management, EME 6458 Distance Teaching & Le arning, EME 5404 Instructional Computing 2, EDG 6931 Issues and Current Research in Educational Technology, and EME 5405 Using the Internet in K12 Instruction All of these courses were offered fully online in the course management system, Moodle. Table 3 1 reports overview information of these courses Forum details including the number of discussions, average replies, and other aspects shown in Table 3 2 Material s Selection According to Smith (2000) c ontent analysis study requires selection of material and sources of that material Smith explains that probability sampli ng is not comm on in content analysis research mainly b ecause it may not be possible to identify all the members of the total relevant population The materials in this study, the discussion forums, are considered naturally occurring materials (Smith, 2000) Grounded in social constructivism, the study searched for an online discussion activity t hat exemplified the constructivist learning model rather than looking for general online discussion activities. The forum was selected using purposive sampling where materials in the large pool were filtered through criteria (Huck, 2004) including the following: Courses were offered solely online ; Courses were offered during the spring 2010 academic semester ; Courses i ncluded discussion forum activities (e.g., reflective discussion, case studies ) with clear instruct ion ; The discussion forums were not merely introductory sessi on s or space for uploading assignment s ; and

PAGE 76

76 The discussion forum received high scores via a rubric that was based on criteria for constructivist learning environment s (Jonassen et al., 1995) The seven online courses with a total of 39 forums, were initially selected based on the first four criteria. In order to find the best example of a constructivist discussion forum, 29 out of 39 forums were selected for further consideration based on implementation of the fifth criteri on To ensure content vali dity of the discussion activities, three experienced online educators with at least two years of online or blended teaching experience reviewed objectives of the selected courses and discussions. The experts reviewed the pedagogical orientation of the obje ctives of the discussion activities to determine whether they were fit for constructivist learning using the rubric based on the model of Jonassen et al. (1995) in Table 3 3 While t he original model of Jonassen et al. consists of four criteria, this modified rubric with a three -point scale in Table 3 4 separates the last criterion, require negotiation and refl e ction, into two different items This is due to the fact that some discussion activities ask ed learners to reflect on the learning materials but did not necessarily require them to collaborate. This rubric was further modified during the negotiation session among the three experts To negotiate, the experts used four forums to ensure that they a g reed on the wording and meaning of each item of the rubric Each of the remaining 25 forums was allocated to two experts to as sess using the modified rubric. It should be noted that sometimes instructors provide additional instruction s regarding forum activity in syllabi. T hese instruction s were retrieved and cons idered when raters assessed the instructions. Table 3 5 shows the rating configuration and resulting scores in each item from the rubric. Inter -rater reliability of this configuration, using the percent age of agreement between two raters was 68.9% for rat er 1 vs. rater 2, 62.5% for rater 1 vs. rater 3, and 65.0 % for rater 2 vs. rater 3 T he aggregate inter rater reliability across these pairs was approximately 65% and was deemed satisfactory given the subjectivity of the rubric

PAGE 77

77 Although further discussio n among raters was typically required to obtain a higher percent age of agreement such procedure was not necessary in this study after further investigat ing the initial results specifically when examining the total scores of each forum Since this study w as grounded in social constructivism and asked a specific question about constructive feature s of the discussion forum, it focused on the individual instead of the group level. Thus the main focus was to find an exemplary forum rather than to look for at the overall picture of forums in online courses. The initial coding results given by raters were L for low M for medium, and H for high. These results were translated into numerical form ranging from L=1, M=2, and H=3 in order to find forums that have a c lear pedagogical slant on constructivism. Of the 39 Educational Technology forums in seven online courses offered in the college of education, three forums, EME 620502 (12/14) EME 645805 (13/13) and EME 540403 (14/14), were the best candidates Other forums that did not match the last criterion were dropped from the study. This process allowed for experts to establish the content validity of the materials and thus support a process of data collection by making certain that only postings from intended discussion activities were selected and analyzed. Instructions and discussion transcripts of the three finalists were further examined to find the best candidate. In light of the underlying a ssumption that more interaction lead s to higher levels of knowledge construction (Schellens & Valcke, 2005) forum EME 620502, titled Sharing Tips with your peers, was dropped because of the low interaction within the forum (average of 1.29 reply messages ). Although forum EME 540403, titled Teaching Online Identity Forum for groups to communicate/collaborate ha d the highest average of replies, thus the highest interaction the nature of the discussion posed a complication in content analysis because the forum was intended to be a group assignment planning space. Most of the interaction

PAGE 78

78 in this forum w as regarding the main material outside the forum, e.g ., Google Docs prepared for the group paper thus t he students tend ed to discuss changes made to the documents and the confirmation and sometimes approval of the changes Although the group paper may be evidence of group knowledge construction, raters do n ot have access to the documents. The forum was dropped for this reason. Forum EME 645805, titled Discussion 4. Economics and Education in the U.S. had an average of 3.60 replies which was higher than forum EME 620502. Further, it was a one -week, self -containe d forum ; that is, students completed their work within the forum space. M ore importantly, this forum was the only forum that two raters (rater 1 and rater 2) had 100% agreement in all categories Of the three finalists this forum seem ed to be the best candidate for constructivist discussion forum This forum was the last forum out of the four total forums in the course. It focused on broad issues in distance education and economy The instructor asked students to discuss educational policy at the national level from the book titled The Race between Education and Technology by Claudia Golden and Lawrence Katz (2008) Specifically, the students were given instruction as follows : We conclude the course by considering broad issues in distance education: change and context. In this discussion, we will reflect on the messages of two economists who are influencing discussions of educational policy at the national level, Claudia Golden and Lawrence Katz of Harvard. They have published a new book, The Race between Education and Technology. For the purposes of the course, we will focus on the broad premises of the book as they relate to global forces and national responses. An excerpt of the book appears at the Har vard Press website. The web version is located at http://www.hup.harvard.edu/catalog/GOLRAC.html. I have also attached a PDF to this forum. In this discussion, share your thoughts about the book's assertion that "if, in addition to technological progress, the quantity and possibly the quality of education increases, then inequality could decrease" and supporting statements about recent changes in American education and economy. How do these viewpoints relate to the importance of distance education in the near future? In response to the reading, what message would you send to legislators who craft national economic and education policy with regard to distance education? Please be sure to reply to the posting of another participant.

PAGE 79

79 Data Collection There were t wo sets of data for this study: one for content analysis and another for SNA The following subsections discuss how data were collected and prepared prior to the analyses. Content Analysis Data Collection T he entire discussion transcripts of the selected online forum was copied and pasted into a separate Word document to be analyzed. Transcripts were organized and imported into TAMS Analyzer 3.61b8, open source software for qualitative research data analysis Social Network Analysis Data Collection Discussion r ecords in the Moodle LMS which were stored in a MySQL open source database were retrieved and pre -proce ssed for SNA In dealing with large amounts of data from the database management system the basic technique of data mining was applied. Three im portant steps in data mining according to Liu (2007), include data pre -processing, data mining, and data post -processing. Liu explains that data pre -processing involves cleaning and removing noises from the raw data. Irrelevant attributes of the large data were removed through sampling and attribute selec tion. Processed data were fed to an algorithm which produce d patterns or knowledge. The final step wa s to identify useful discovered patterns for applications and making decisions. Example s of data post processing are evaluatio n and visualization techniqu es. In this particular case, data pre -processing involved performing SQL (structured Query Language ) queries to re trieve desired records from MySQL tables (greater detail of MySQL tables and e ntity relationship diagram is discussed in Appendix B). the f our -digit identification number of the forum EME 645805 was used as a condition to perform SQL statements to select records from the MySQL tables The resulting records call ed the result -set, were kept in relational database format to perform further cleani ng. PHP scripts, an open source general -

PAGE 80

80 purpose HTML -embedded scripting language (php.net, 2010) were developed to transfo rm result -sets into a sociomatric one of the formal methods in SNA used to summarize and present patter ns of social relations by creating an actor -by actor matrix to represent relations among them (Hanneman & Riddle, 2005) The PHP scripts were designed for directed relations, valued data matrices Direct ed relations refer to a situation where either member of a dyad may initiate a relation with the other member. Relations in this study were interaction s ( i.e., initial and reply messages ) between students participating in online discussion forums Valued d ata measurement refer s to network s that include nominal or ordinal scales. Since this study examined frequency of interaction between actors, the valued data were measured in ordinal scale s Sociom a trices were imported to UCINET 6.258 (Borgatti, Everett, & Freeman, 2002) to perform data analysis UCINET was employed to calculate centrality network measures Based on the sociomatrices, NetDraw 2.091 (Borgatti, 2002) was used to develop sociograms, another formal method in SNA used to present social network data using graphic display that consists of points or nodes to represent actors and lines or edges to represent relations (Hanneman & Riddle, 2005) Data Analysis The data analysis of this study consisted of three steps: quantitative content analysis of the online discussion transcripts, SNA of the relation s data and correlation coefficient s using the re sults of the first two methods Content A nalysis To determine the levels of knowledge c onstruction within the online discussions, the IAM (Gunawardena et al., 1997) was applied. This quantitative content analysis method focuses on coding a large amount of data on which statistical tests are then performed (De Wev er et al., 2009) According to Gunawardena et al. (1997) five different levels of knowledge construction

PAGE 81

81 include: (a) sharing and comparing information, (b) exploration of dissonance, (c) knowledge negotiation and co -con struction, (d) testing and modification of knowledge, and (e) agreement and application. A s suggested by Gunawardena et al. (1997) a message was selected as a unit of analysis because : (a) it is clearly demarcated in the transcripts; (b) it offers sufficient information to infer underlying meaning of the coding scheme; and (c) it is appealing because the length of such a unit is decided by its author rather than by the coder (Garrison et al., 2001) Nonetheless, messages can be broken into multiple units if the coder deems it necessary during the first round of coding. In the case where a message clearly represents more than one level, multiple codes were collapsed into the highest phase. This technique is known as code up (Garrison et al., 2001) A t otal of 19 threads in the selected discussion foru m were coded using TAMS Analyzer For purposes of reliability checking, a second coder independently coded a randomly chosen sample of 1 0 % of the threads. The percentage of coding agreement was 80% (Cohens kappa = 0.722) which is considered a substantial agreement (Fleiss, 1981) The re sults of IAM analysis that use a message as a unit of analysis then were grouped by mean levels of knowledge construction for each student to later correla t e with SNA centrality measures. Social Network Analysis The second method of data analysis was to an alyze the pre-processed data from the Moodle LMSs records. The measures in this study i ncluded centrality measures in directed, valued relations of sociocentric (complete) networks of students and instructors in online courses. Three approaches of central ity that describe the locations of actors in terms of h ow center ed they are in a network are degree, closeness and b etweenness (Hanneman & Riddle, 2005)

PAGE 82

82 Degree centrality : concerns immediate relations o f actors as opportunities of choices; higher degrees make them less dependent on others and hence more powerful. For the star network in Figure 3 1 A, actor A has four degrees and is considered to have more power in this context. On the other hand, all act ors in the circle network have the same degree of two ( Figure 3 1 C ). For the line network in Figure 3 1 B actors at the end are at structural disadvantages (actors A and E). Closeness centrality : considers the path l engths between a pair of actors as a f orm of power ; a ctors who could reach others in short er path lengths (i.e., closer to other actors ) have an advantage over their fellow actors in gaining resources. In the star network, the central actor has the shortest path lengths ; that is, actor A has o ne degree connecting to other actors. The length of the shortest part conne cting a pair of actors is known as the geodesic distance The other actors in the star network are at a geodesic distance of two. In the line network, actor C is closer to all other pair of actors and actors A and D are at a disadvantage. All actors in the circle network have identical distributions of closeness. However, the closeness index is only meaningful for a connected graph; that is; all nodes are reachable. This is because u nreachable node s (0 degree node) would make the sum of geodesics become infinite () Betweenness centrality : considers the advantages of actors who lie between pathways connecting other pairs of actors (dyad) Actor A in the star ne twork lies between every other pa ir of actors, thus acting as a broker among others. Actor A and E at the end of the line network have no brokering advantages while actors closer to or in the middle of the chain are in advantaged position s In the circle network, a ll the actors share the same structural power. The relations (ties) in network examples given in Figure 3 1 are cons i dered nondirectional relations, where there is no distinction between senders and receivers of relations (Kno ke &

PAGE 83

83 Yang, 2008) In a directed network, a ctors differ by the number of out -degree relations and in degree relations. Actor s with high out degree s are influential or centralized actors while actors with in degrees are prominent actors or actors who have high prestige (Hanneman & Riddle, 2005) The d egree centrality, a quantification of the actors prominence (Knoke & Yang, 2008) for directed, valued relations, is the following: 1()()g Di ij jCNxij (3 1) where ()DiCN denotes the degree centrality of an actor i The formula simply counts the number of direct relations (nodal degree s ) that actor i has to t he 1 g other j nodes except the i s relations to itself () ij (i.e., the diagonal values). However, a degree centrality score reflects network size g ; that is, the larger the size the higher the score. A n ormalizing process is applied to eliminate this effect : () () (*)(1)Di Di DCN CN Cng (3 2) where (*)DCn denotes the maximum observed relational value in the data. Because closeness centrality can only be used for a connected network, this index was not used in the present study since it was likely that actors (students) participating in the forum mi ght not post a message or never reply to a m essage thus becoming unreachable nodes (infinite distances) and making t he closeness index meaningless. The b etweenness centrality index on the other hand, could be computed even if the graph is not connected. According to Knoke and Yang (2008) b etweenness refers to how other actors control or mediate ( lie on a g eodesic path ) dya ds that are directly connected Betweenness centrality indicate s the actors control over information exchange or resource flows in a network.

PAGE 84

84 The more extensively an actor lies on the geodesic path of other dyads, the higher the potentia l of that actor to control network interactions. To quantify actor i s betweenness centrality as they lie on the geodesic path between j and k th e number of geodesics between j and k that contain node i or ()jkigN is divided by the num be r of geodesic paths between the two nodes or jkg to measure the proportion of geodesic paths connecting j and k in which node i is involved. The following {()}BiCN indices can be used for both directional and nondirectional relations : () ()jki Bi jk jkgN CN g (3 3) To standardize actor betweenness centrality in a nondirectional relation the index is divided by the maximum theoretical value of (1)(2)/2 gg : ()2 '() (1)(2)Bi BiCN CN gg (3 4) Th e standardized actor betweenness centrality is 0.0 when the original betweenness centrality is 0.0, and is 1.0 when node i falls on the geodesic paths of every dyad among the remaining nodes (1) g ; thus, the closer to the standardized actor betweenness 1.0, the more actor i controls relations in the network. W hen the {'()}BiCN indices is used to calculate directional relations the maximum theoretical value is (1)(2) gg as o pposed to (1)(2)/2 gg in a nondirectional relation In other words, the indices are multiplied by two in a directional graph (Gould, 1987) Similar to other network measures in valued graphs the standardized actor betweenness centrality measures of a valued graph are not restricted to zero and one

PAGE 85

85 Sociomatrices ge nerated by PHP scripts were imported in to UCINET to calculate valued centrality measures including actor -level out and in -degree centrality indices with normalized (standardized) results, and actor betweenness centrality indices To visualize the social structure s ociomatrices were imported to NetDraw to generate sociogram s of the students networks Spearman s Correlation Coefficient To discern the nature of the relationship between levels of knowledge construction and SNA measures, Spearmans correlati on coefficient s were c alculated with the results of the IAM coding and measures from the SNA. It was posited that the two centrality measures from SNA, degree and betweenness will have a relationship to levels of knowledge construction; that is, the score s on the SNA measures will relate to the mean levels of knowledge construction. The correlation coefficients were initially calculated with all 21 actors including the instructor. T he instructor was excluded from further analysis to discern the nature of t he relationship between mean levels of knowledge construction and measures in social network analysis that focus on social constructivism That is, the correlation coefficients would reflect truly student centered learning environments without intervention by the instructor. Limitation of the Study There were several limitations to this study worth noting. First, the focus on critical thinking in online, graduate -level course s hindered the generalizability since critical thinking skills are context -specific by nature (Lipman, 2003) T he delivery mode of the course also affected the generalizability of the study to other learning environments and courses delivered in face -to -face or blended modes. It should be noted that complete net work data do not represent larger populations as in statistic s. Rather, such observations are considered populations of interest (Hanneman & Riddle, 2005)

PAGE 86

86 Second, intervention by the instructor was not taken into consideration, mainly because the study and the content analysis method were grounded in social constructivism which focuses on knowledge co -construction of the learners in the community; it was expected that this particularity would shape outc omes of the study.

PAGE 87

87 Table 3 1. Descriptive information of the selected online courses Course Code Title # of participants # of forums EDG 6931 Issues and Current Research in Educational Technology 29 13 EME 5405 Using the Internet in K12 Instruction 35 3 EME 6205 Digital Photography and Visual Literacy 20 3 EME 6458 Distance Teaching & Learning 21 5 EDG 6931 Distance Education Leadership and Management 26 2 EME 5207 Designing Technology Rich Curricula 25 2 EME 5404 Instructional Computing 2 3 3 11 189 39

PAGE 88

88 Table 3 2. Forum details of the selected online courses Forum Code Forum Title # of discussion Average replies Group activity EDG 6931 01 Introductions a little different 30 9.73 EDG 6931 02 Research Definition Forum 28 6.54 ED G 6931 03 Freakonomics Chapter 1 Discussion Forum 26 5.65 EDG 6931 04 Pseudoscience? Bad research? 27 7.19 EDG 6931 05 Freakonomics Discussion Forum Chapter 2 26 5.92 EDG 6931 06 Freakonomics Discussion Forum 26 6.42 EDG 6931 07 Annotation Forum 27 6.00 EDG 6931 08 Article Discussion Forum 6 44.00 Y EDG 6931 09 Freakonomics Discussion Forum Chapter 4 27 6.56 EDG 6931 10 Contemporary issues forum 8 23.75 Y EDG 6931 11 Feakonomics Discussion Forum Chapter 5 25 6.08 EDG 6931 12 Freakonomics Dis cussion Forum Chapter 6 22 4.50 EDG 6931 13 Final Project Forum 27 5.04 EME 5405 01 Introduce Yourself 34 10.50 EME 5405 02 Introductory Readings & Forum 32 5.31 EME 5405 03 Culminating Discussion 30 6.03 EME 6205 01 Introduce Yourself 18 4.06 EME 6205 02 Sharing Tips with your Peers 21 1.29 EME 6205 03 Lesson Plan 1 21 1.19 EME 6458 01 Discussion 1. The role of distance education in educational system 19 4.58 EME 6458 02 Discussion 2. Forces shaping distance education 20 5.75 EME 6458 0 3 Discussion 3. Types of Interaction 23 5.00 EME 6458 04 Course Project Forum. Due February 28 20 0.90 EME 6458 05 Discussion 4. Economics and Education in the U.S. 20 3.60 EDG 6931 01 Forum: your leadership experience, due March 8 1 101.00 EDG 693 1 02 Forum: leader interview questions 22 6.05 EME 5207 01 Introduce Yourself 25 5.52 EME 5207 02 Introductory Readings & Forum: Framework for 21st Century Learning 24 4.13 EME 5404 01 Introduction 32 6.00 EME 5404 02 Social Networks 33 2.24 EME 5404 03 Teaching Online Identity Forum for groups to communicate/collaborate 11 27.91 Y EME 5404 04 SecondLife 34 2.12 EME 5404 05 The potential of games and virtual worlds 35 1.09 EME 5404 06 Mobile Computing and Curriculum 33 2.30 EME 5404 07 Us ing mobile computing devices 37 1.00 EME 5404 08 Journal links 13 1.23 EME 5404 09 Access to Online resources 38 1.03 EME 5404 10 Bridging the digital divide 31 1.19 EME 5404 11 Reflection 21 0.71

PAGE 89

89 Table 3 3. Jonassen et al.s criteria for cons tructivist learning environments Characteristic Description Authentic R eflects the ordinary practices of the culture Meaningful to learners P romote s articulation of meaningful purpose of learning S upport s self -directed exploration of information and promote linking information to the learner's own schema Problem based F ocus es on problems and depth of understanding, decentralized control, and a broader knowledge community Require negotiation and reflection P romotes deliberate collaboration and conver sation among the participants Table 3 4. Rubric for constructivist learning environments Characteristic Low High AUTHENTIC Tasks are simplified and designed for a certain subject Tasks are theme based across disciplines and related to real world si tuation. Example: Ask how overall course content impacted learners practices MEANINGFUL TO LEARNERS Tasks are required rather than an intrinsic interest; goals are pre defined Allow learners to engage in tasks based on a sincere curiosity; encour age learners to set their own goals Example: Let learners choose additional materials (e.g., article) and critique based on their interest PROBLEM BASED Tasks are well structured and tend to have right answers Tasks are complex, ill structured; l earners are encouraged to refine and solve problems Example: Present tasks, and resources that are relevant to solve problem s that emerge from real world context s REQUIRE REFLECTION Tasks are given and do not encourage learners to reflect on and share their learning process Learners have some opportunities to discuss their work with others; encourage learners to discuss the processes and strategies in learning (both successful and unsuccessful) Example: Ask learners to regularly contribute to the discussion (e.g., posting initial responses/reactions, sharing professional expertise and experiences) REQUIRE NEGOTIATION Learners primarily work alone; roles are not given or shifted infrequently Tasks give learners the opportunities to de velop shared understanding of the activities; responsibilities are evenly distributed among learners Example: Assign roles and responsibility to learners (e.g., starter, summarizer)

PAGE 90

90 Table 3 5. Rating configuration and results Forum Code Group Ra1 R a2 Ra3 Re1 Re2 Re3 Re4 Re5 Total EDG 6931 01 1 1 1/1 2/1 1/1 3/2 2/2 9/7 EDG 6931 02 ----EDG 6931 03 1 1 3/3 2/2 3/2 3/2 2/1 13/10 EDG 6931 04 1 1 3/3 3/3 3/3 1/3 1/1 11/13 EDG 6931 05 1 1 3/3 3/2 2/2 3/1 2/1 13/9 EDG 6931 06 1 3 1 2 2 3 11 EDG 6931 07 ----EDG 6931 08 1 1 2/2 2/2 2/2 3/3 3/3 12/12 EDG 6931 09 1 1 3/3 3/2 3/2 3/2 2/2 14/11 EDG 6931 10 ----EDG 6931 11 1 1 2/2 2/2 2/2 2/2 1/1 9/9 EDG 6931 12 1 1 3/3 1/2 2/2 2/2 1/1 9/10 EDG 6931 13 ----EME 5405 01 1 1 1/2 2/2 1/1 3/2 2/1 9/6 EME 5405 02 1 2 2 2 3 1 10 EME 5405 03 1 1 3/3 2/2 2/2 3/3 1/2 11/12 EME 6205 01 ----EME 6205 02 1 1 1/3 3/3 2/2 3/3 3/3 12/14 EME 6205 03 1 1 3/3 2/3 3/3 2/2 1/1 11/12 EME 6458 01 1 1 3/2 2/3 3/2 2/2 2/1 12/10 EME 6458 02 1 1 3/3 2/3 2/2 2/2 2/1 11/11 EME 6458 03 1 1 3/3 3/3 1/2 3/2 2/2 12/12 EME 6458 04 ----EME 6458 05 1 1 3/3 2/2 3/3 3/3 2/2 13/13 EDG 6931 01 1 1 3/3 2/2 2/2 2/2 1/2 10/11 EDG 6931 02 ----EME 5207 01 1 1 1/1 2/1 1/1 3/2 2/1 9/6 EME 5207 02 1 1 3/2 2/2 2/3 3/2 1/1 11/10 EME 5404 01 ----EME 5404 02 1 3 2 2 2 1 10 EME 5404 03 1 1 3/3 3/3 2/3 3/2 3/3 14/14 EME 5404 04 1 1 3/3 2/3 2/2 2/2 1/1 10/11 EME 5404 05 1 1 3/2 2/1 3/2 2/2 1/1 11/8 EME 5404 06 1 1 3/3 2/2 2/2 2/2 1/1 10/10 EME 5404 07 1 1 2/3 2/3 2/3 2/2 1/1 9/12 EME 5404 08 ----EME 5404 09 1 1 3/3 2/2 2/2 2/2 1/1 10/10 EME 5404 10 ---EME 5404 11 1 3 3 2 2 1 11 Total 4 17 17 16

PAGE 91

91 Figure 3 1. Network topology. A) Star network. B) Line network. C) Circle network

PAGE 92

92 CHAPTER 4 RESULTS Introduction This chapter discusses results in three parts according to the m ixed quantitative approach that was used to address the research question. The first section is related to the results of the content analysis using interaction analysis model ( IAM ) (Gunawardena et al., 1997) The second section covers results regarding social network analysis (SNA) The final section examines the relation ship between the results of SNA and content analysis by computing c orrelation coefficient s Results of IAM D ata from the online discussion for um titled: Discussion 4. Economics and Education in the U.S from the course EME 6458, Distance Teaching & Learning consisted of the transcripts of 20 threads One of the threads was a duplicate and thus replies from the two identical initial messages thr ead discussions were merged. Twenty students contributed 19 initial messages (threads) and 68 reply messages, and one instructor (participant ID #21) contributed four reply messages, for a total of 91 messages. The transcripts of these 19 threads were copi ed and pasted into a Word document. The Word document comprises 3,270 lines, 72 pages T he transcripts were loaded into qualitative research software, TAMS. A ccording to the IAM content analysis, a majority of the discus sion forum messages in the online co urse that lean ed toward constructivism occurred in phase II (3 4.07% ) and phase III (37.36%) Table 4.1 shows the distribution among each phase The mean level of knowledge construction of the discussion forum was 2. 7 5 ( SD = 0. 6 1 ), with a minimum mean level of knowledge construction of 1.25 and a maximum of 4 .33. The information about individual mean level of knowledge construction is presented in table 4 2 on the second column

PAGE 93

93 Three major themes were observed in this forum. One was that the majority of the messages that were coded at phase V were initial messages p recisely 12 out of 1 4 posts (85.71% ). The m ean level of knowledge construction of the 19 initial messages was 4.13 (SD = 1.08) The other was that these initial messages provide d evidence of more than one phase usually progressing from lower to higher le vels of knowledge construction. F or instance: In our market -driven capitalist society, that which has value is that which produces value. The value of a commodity is considered only as it relates to a contribution to the bottom line. People are marginalized by the margin, no longer people, but human capital. Exploitation of the surplus value of labor is not accidental, it is endemic. Distance education in particular, and education in general might not do as much as we might hope to end inequality. Our educational system is interwoven into the structure and function of our society, and works to reinforce the dominant values of society. This does not mean that education is the cause of inequality, but that education is structurally and functionally mandated to reinforce the oppressor consciousness which tends to transform everything surrounding it into an object of its domination. The earth, property, production, the creations of people, people themse lves, time everything is reduced to objects at its disposal. In their unrestrained eagerness to possess, the oppressors develop the conviction that it is possible for them to transform everything into objects of their purchasing power; hence their strict ly materialistic concept of existence. Money is the measure of all things, and profit the primary goal. For the oppressors, what is worthwhile is to have more always more even at the cost of the oppressed having less or having nothing. For them, to be is t o have and to be the class of the h aves (Freire, 2008, p. 58). [ II -C] Education that is driven by the oppressor consciousness creates a world -view wherein one looks at people as objects instead of people, commodities that exist to power the engine of ca pitalism, instead of individual humans with individual needs. Theoretically, social inequality could decrease given nothing more than increasing technological progress along with the quality and quantity of education attained. However, the structure of cap italist society functions to prevent such equality from being more than a theoretical possibility. In actuality, increasing the supply of education and technology in the workforce could increase inequality, as more workers drive down the price of labor, an d demands for productivity facilitated by technology drive up the e xpectations for productivity. [ II -A ] T he margin, or the bottom line is a measure of the capitalists success in exploiting the surplus value of a commodity. Society is inherently structured and set up in order to reward exploitation of capital. Formal school -based education enabled American youths to change occupations over their lifetimes, to garner skills different from those of their parents, and to respond rapidly to technological chang e (Golding & Katz, 2008, p. 29). Taken at face -value, this is a positive. However, alternatively this is also negative as dynamic workers are workers who are increasingly interchangeable, easily replaced at lower wages. Economic inequality has grown becaus e the worker has been commodified

PAGE 94

94 into capital. Increasing inequality means increasing profits, as one is exchanging the value of their productive labor for less than it is worth in terms of t he value of what it produces. [IV -A ] Legislators crafting national economic and education policy in regard to distance education should be aware that the fundamental nature of distance education is one of change and adaptability. Trying to structure and standardize distance education, and attempting to make distance ed ucation conform to the same methods and mediums as traditional institutional face to -face education would be disastrous on many l evels. [ V A ] First, such attempts would be doomed to failure and frustration, not to mention wasted effort on both the side of those seeking freedom and enlightenment, and those driven to subjugate the individual to the will of the market. Second, such policies would squander the opportunity that distance education presents to enact real and lasting educational and societal reform (posted by student ID # 1) The above message, which was cod ed in multiple phases in the first round and was subsequently collapsed into the highest phase exemplified two types of operations in phase II. The student identified area s of disagreement and pr ovided literature and metaph or s to illustrate his/ her point of view. The student then moved the argument toward testing the proposed synthesis [phase IV -A ] and eventually summarized the argument [phase V -A ]. The following initial message built up the arg um ent from phase III, negotiated meaning and co -constructed knowledge moving toward phase V This message below began by trying to elucidate the term inequality, proposing and elaborat ing on the newly defined term. The student summarized the newly construc ted meaning [phase V -A]. D istance education can allow many of those who would have been restricted by location or other factors to continue their studies throughout their lifetime, while acquiring new careers and updating their skill sets to keep pace with societal change. But, will it decrease inequality? I think that is difficult to say withou t a definition of inequality. [ III -A ] Fitzberg (2000) suggests that educational reform and funding that produces educational equity would be useful, but only i n a context of social reform which addresses issues of poverty and access. For many, an increase in the quality and quantity of education may be helpful, but if the technology is not accessible, or only exists in the classroom it would have limited long -t erm social impact. Even now, bright students can be disadvantaged by the realities of survival. [ III -D ]

PAGE 95

95 If we are to continue to be competitive, the economics of the educational system and the new technologies that support distance education must be consi dered as priorities. [ V -A ] (posted by student ID # 2 ) M ost of the reply messages were coded in lower phases, as the mean level of knowledge construction of the 72 replies was 2.33 (SD = 0.8 1 ). T here were only two reply messages that went beyond phase III O ne was coded at phase IV and another was coded at phase V. The following reply message was an immediate response to the previous message by student ID #2, exemplifying the typica l pattern of the reply messages. I agree that any discussion of education, pa rticularly distance education, has to address the issues of poverty and access. Incorporating technology into education is wonderful and seems to have endless possibilities. The real danger is that we may leave a segment of our population behind if we fail to acknowledge that poverty and race are even more of a problem in distance education than in traditional education. [I B] The state provides a school bus and a classroom and books that can be taken home. The state does not provide a computer and access t o the Internet at home. Those students that have access to the Internet at home have a competitive advantage over those who do not. Funding should be used to address this problem. There is a community college in Virginia that built a new center with 100 computers to give their students access to updated technology. What makes this interesting is that most colleges are phasing out their computer centers because most students have laptops. However, only 35% community college students own laptops. (Campus Computing Project). Community College students tend to be less affluent and less able to afford new, updated laptops. [II -C] (replied by student ID # 7 ) The purpose of the above message, which was coded at phase s I and II, was to further the initial message as s tudent ID #7 agreed with student ID #2 and compared access to the internet to access to other school supplies, and provided more information to support the argument. Carrying this strand of argument, student ID # 14 collaborated with student ID #2 by answe ring questions to extend the discussion In response to the question you posed about spending resources for technological change, I found some interesting information from the Department of Education discussing the initiatives that it is funding. [ II B ] ht tp://www2.ed.gov/about/offices/list/os/ technology/index.html While as educators we all have issues with the No Child Left Behind legislation and its implementation, a positive aspect is the acknowledgement of inequities in the areas of technology and edu cation access as well as the resources that are being invested the department of Educational Technology into schools and communities where these resources were not once available.

PAGE 96

96 One reply message reached phases IV and three of them reached phase V. The following reply message su mmarized the agreements from other previous reply messages which were all coded in phase I, by addressing the overlap of the term equality and various factors involved in decreasing inequity. I think it makes a difference, when considering this quote, whether you are talking about equality between individuals, or equality between nations. I don't disagree that education can help equalize individuals or nations. But, as one of the other posts mentioned, there are many factors to consider such as a student whose parent's are ambivalent about education, or the digital divide which could lead to increased inequality when technology is utilized for education if not specifically addressed and provided for. [ V -A ] Beside s the fact that i nitial messages were coded in higher phases, the length of the initial messages were often longer than the reply messages ; that is, s tudents initially posted nearly four times as many words as student re plies. The average words with in initial messages was 603.84 (SD = 300.79) whil e the average words with in reply messages was 153.56 (SD = 183.05). However, a closer look into the messages during the analysis found that reply message #44 (1573 words ) contain ed an article that the st udents copied from a newspa per (1450 words) When the article was removed from the message, the average of words in reply messages decreased to 133.42 (SD = 68.78). The Correlation coefficient between the mean levels of knowledge construction and length of messages (r= .705) also ind icated significant positive relationship Detailed information regarding the initial and reply messages and correlation between the mean levels of knowledge construction and length of messages can be found in Table 4 2 and Table 4 3 respectively Results o f SNA During the second part of the data analysis, t he pre -processed data from Moodle LMSs records were transformed into a sociomatric, an actor byactor matrix representing relations among them (Hanneman & Riddle, 2005) T he socio matric was imported into UCINET to

PAGE 97

97 calculate normalized degree and betweenness centrality measures of each of the discussion forum networks ( detailed in Table 4 4 ) According to Hanneman and Riddle (2005) d egree centrality concerns immediate relations of actors as opportunities for choices where higher degrees make them more independent to others and hence more powerful. Betweenness centrality concerns the advantages of actors who lie b etween pathways connecting other pairs of actors (dyad) UCINET also reported normalized results to eliminate the effect of the network size, where larger netw ork s tend to yield higher score s This study utilized normalized results in calculating S pe arman s correlation co efficient s to avoid this effect. In addition, the sociomatric was imported into NetDraw 2.091 to generate sociogram s. Figure 4 1 to Figure 4 4 illustrate sociogram s of the same forum where their nodes size is based on out -degree relations in -degree relations and betweenness centralities In these figures, line width is proportional to tie strength. The third and fou rth columns of Table 4 4 present normalized in and out degree counts as percentages of the number of actors less one ( 1 g ). In a directed network, actors with high out degrees or centralized actors in a directed network are influential. Likewise, actors with high in degrees are prominent actors or actors who have high prestige (Hanneman & Riddle, 2005) The results of this study indicated that student 5 has the greatest out -degrees and is regarded as the most influential while s tudents 6 and 19 have the greatest in degrees and are considered actors wi th high prestig e The last column present s normalized betweenness centrality indices indicating that student 6 is the most influential in this regard, followed by students 3 20, and 19. W e can see that there was relatively low variatio n in actor out degre e and in -degree centralities (SD of 2.525 relative to a mean out degree of 5.442 and SD of 2.962 relative to mean in -degree of 5.442). The variation in actor betweenness was slightly higher ( SD of 6.610 relative

PAGE 98

98 to a mean betweenness of 8.33 ). Despite this the overall network centralization is very low. For degree centrality (out -degree of 6.25% and in-degree of 8.00%) this mean s there was no substantial amount of concentration ( or degrees) in the network and the degree centrality i s almost evenly dispers ed (nodes had similar degree centrality) That is, actors in the network held a similar amount of power and no one ha d positional advantages. For betweenness centrality of 15.37%, this means that there was no structural constrain t or that no actors held t oo much brokerage power and actors could reach others without significant aid from any intermediary. T o find out whether the centrality measures of social network s related to levels of knowledge construction, Spearmans correlation coefficient s were calcu lated with the results of the interaction analysis coding and centrality measures from the SNA The next sect ion discusses the results of this analysis. Research Question Do levels of knowledge construction in online learning environments relate to the cen trality measures of social networks? The fundamental premise of this question was that individuals construct knowledge by interaction with others and thus more interactions would lead to higher levels of knowledge construction (Gunawardena et al., 1997; Schellens & Valcke, 2005) Social network analysis was utilized to uncover patterns o f interaction in forum activity in an online course and to quantify the power that actors possess regarding their location in the network Content analysis using IAM was used to analyze forum transcripts of the same forum to find mean l evel s of knowledge c onstruction Spearmans correlation s were utilized to analyze the resulting data from the two analytical techniques It should be noted that although the populatio n reported that results of online discussions are typically organized by the number of messag e s or postings according to the choice of the unit of analysis, this study rearranged the results by grouping mess ages by students, and

PAGE 99

99 calculating the mean level of knowledge construction by participants (i.e., students and an instructor ) in order to cor relate with centrality measures from SNA. Three centrality measures out deg ree, in -degree, and betweenness were used to measure the mean level of knowledge construction in a sample of 20 students and one instructor The analysis detailed in Table 4 5 revea ls no significant relation ship between the mean level of knowledge construction and each of the three centrality measures (p > .05) The result s of Spearmans correlations provide context for the assertion that there is no relationship between mean levels of knowledge construction and centrality measures. A summary of the results is presented in Table 4 5 Although the instructor in this study was not considered a prominent actor with average score s in all centrality measures and low mean level s of knowledge construction, Spearmans correlations without instructor were calculated and no statistically significant relationship was found between mean level of knowledge construction and the three network measures (Table 4 6 ). Summary of Finding s The selected material, discussion transcripts from the forum EME 645805, developed as one of the primary candidates to reflect a constructivist learning environment, per the rating required by a rubric modified from criteria for a constructivist learning environment (Jonassen et al., 1995) The coding results of IAM confirm this assertion where most of the observed interactions occurred in phase II : discovery and exploration of dissonance or inconsistency among ideas, concepts, or statement s (34.07%) and ph ase III: negotiation of meaning/ co construction of knowledge (37.36%) Some of the observed interactions also reached phase V: agreement statement(s)/applications of newly constructed meaning (1 5 38%) Based on the previous studies that utilized IAM (see Table 4 7 ), a proportion of the interactions that reached phase V were considered markedly high However when results of IAM were correlated with

PAGE 100

100 centrality measures of SNA, there was no evidence that the two sets of results were related Moreover, a low proporti on of interactions in phase IV indicates that the particular forum did not foster testing and modification of knowledge. Rather, students advanced to the conclusion in phase V.

PAGE 101

1 01 Table 4 1 IAM coding results # of post Phase I Sharing/Comparing of infor mation 11 ( 12.09% ) Phase II The discovery and exploration of dissonance or inconsistency among ideas, concepts or statements 31 ( 3 4 07 % ) Phase III Negotiation of meaning/co construction of knowledge 34 ( 37.36% ) Phase IV Testing and modification of propo sed synthesis or co construction 1 ( 1.10% ) Phase V Agreement statement(s)/applications of newly constructed meaning 14 ( 1 5 38 % ) Table 4 2 Means and Standard Deviations of messages in the forum EME 645805 Words Mean level of Knowledge Construction ( meanKC) Type of message n M SD M SD Post 19 603.84 300.79 4.13 1.08 Reply 72 153.56 183.05 2.33 0.81 Reply (after removing the newspaper article) 72 133.42 68.78 2.29 0.75 Table 4 3 Spearmans rho correlations between IAM and length of messages Cen trality measures Mean level of knowledge construction Length of message Correlation Coefficient .705** Sig. (2 tailed) 000 ** Correlation is significant at the 0.01 level (2 -tailed)

PAGE 102

102 Table 4 4 Mean level of knowledge construction, normalized degr ee and betweenness centrality measures of EME 6458 discussion 4 Participant meanKC nrmOut nrmIn nBetweenness 1 3.00 3.175 3.175 0.000 2 2.50 7.937 7.937 13.851 3 3.17 7.937 3.175 21.741 4 3.00 3.175 6.349 5.346 5 2.38 11.111 6.349 8.197 6 3.33 3.175 11.111 22.974 7 2.67 7.937 6.349 7.991 8 2.40 6.349 6.349 6.377 9 3.00 1.587 3.175 1.026 10 3.00 3.175 3.175 5.395 11 3.33 3.175 3.175 1.053 12 3.25 4.762 4.762 5.697 13 2.86 9.524 7.937 6.579 14 2.83 7.937 4.762 8.132 15 2.25 4.762 4.762 3.083 16 3.00 4.762 1.587 6.465 17 2.00 1.587 0.000 0.000 18 2.25 4.762 6.349 5.412 19 2.75 4.762 12.698 15.996 20 3.00 6.349 7.937 18.917 21 1.25 6.349 3.175 10.768 Mean 2.72 5.442 5.442 8.33 SD 0.51 2.525 2.962 6.610 Network Centralization (out degree) 6.25% Network Centralization (in degree) 8.00% Network Centralization (betweenness) 15.37%

PAGE 103

103 Figure 4 1. Sociogram of EME 6458 discussion 4, where node size is based on out degrees centrality

PAGE 104

104 Figure 4 2. Sociogram of EME 6458 discussion 4, where node size is based on in-degrees centrality

PAGE 105

105 Figure 4 3. Sociogram of EME 6458 discussion 4, where node size is based on betweenness centrality

PAGE 106

106 Table 4 5 Spearmans rho correlations between IAM and SNA measures Centrality measures Mean level of knowledge construction Normalized Out degree Correlation Coefficient .391 Sig. (2 tailed) .079 Normalized In degree Correlation Coefficient .041 Sig. (2 tailed) .861 Normalized Betweenness Correlation Coefficient .069 Sig. (2 tailed ) .765 Table 4 6 Spearmans rho correlations between IAM and SNA measures (without instructor) Centrality measures Mean level of knowledge construction Normalized Out degree Correlation Coefficient .378 Sig. (2 tailed) .101 Normalized In degree Correlation Coefficient .128 Sig. (2 tailed) .590 Normalized Betweenness Correlation Coefficient .005 Sig. (2 tailed) .983 Table 4 7 IAM coding results from previous studies Treatment / Group Phases (percentage) I II III IV V Total Gunawa rdena et al. (1997) 92.72 2.43 1.94 0.97 1.94 100.00 McLoughlin & Luca (1999) W eek4 67.02 21.28 6.38 5.32 0.00 100.00 W eek5 64.95 22.68 9.28 3.09 0.00 100.00 W eek6 66.07 19.64 8.93 3.57 1.79 100.00 Marra, R. M., Moore, J. L., & Klimczak, A. K. (2004) 22.73 36.36 31.82 9.09 0.00 100.00 Yang, Newby and Bill (2005) PartA1 75.44 10.09 2.63 0.00 11.84 100.00 PartB1 63.76 17.11 5.37 0.00 13.76 100.00 PartA2 63.89 15.74 6.48 0.46 13.43 100.00 PartB2 68.29 14.15 4.39 0.00 13.17 100.00 Schellens & V alcke 2005 51.70 13.70 33.10 1.20 0.40 100.10 Schellens, Van Keer, De Wever & Valcke (2007) N on script 51.70 13.70 33.10 1.20 0.40 100.10 S cript 52.90 6.10 29.80 2.80 6.50 98.10

PAGE 107

107 CHAPTER 5 DISCUSSION AND IMPLI CATIONS Introduction This chapter pre sents a summary of the study, and discusses the findings of the analyses by addressing the research question that is guiding this study. T he findings fro m each technique are discussed separately Specifically, the chapter first presents the findings from c ontent analysis technique using interaction analysis model ( IAM ), then the social network analyses ( SNA ) are discussed, and finally, the chapter concludes with discussion and implications of the nature of the relationship between the two methods, as well a s suggestions for future studies Summary of the Study The purpose of this study was to discern the nature of the relationship between two different methods for measuring critical thinking skills in online learning environments using discussion forums. The two methods were content analysis using IAM and SNA Specifically, this study investigated whether centrality measures in social network analysis relate to mean levels of knowledge construction resulting from the IAM. The IAM was aligned with the notion o f critical thinking (Lipman, 2003) and social development theory (Vygotsky, 1978) (see Figure 1 1 ) to connect concepts of critical thinking skills to the levels of knowledge construction and thus to critical thinking itself in the IAM wit hin a constructivist learning environment In other words, IAM is a proxy for finding evidence of critical thinking skills. Network centrality measures, common relationship measures that seek to quantify the notion of an actors prominence within the netwo rk (Knoke & Yang, 2008) were proposed as alternative measures that provide insights into the knowledge construction process in online learning environments. The study postulated that resulting centrality measures of discussion activit ies designed for student s to demonstrate their critical thinking skills would relate to mean

PAGE 108

108 levels of knowledge construction from IAM. This proposal was based on the assertion that individuals construct knowledge by interacting with others (Gunawardena et al., 1997) and more interactions would result in higher levels of knowledge construction (Schellens & Valcke, 2005) Finding alternative measures to assess levels of knowledge construction and evidence of critical thinking skills are of critical importance to online education today, as many research ers raise concerns about the time -consuming process of manuallyassessed methods using content analysis techniques. SNA is one of the proposed alternative methods (Bratitsis & Dimitracopoulou, 2008; Shen et al., 2008) A few studies have followed such proposal s Nurmela et al. (1999) found that instructors were influential in controlling the flow of communication. More recently, Zhu (2006) reported that topics chosen by instructors might regulate students interactional behaviors. Lowes, Lin, and Wang (2007) also found that measures in SNA density, and network centralization were highly correlated with each other and with students satisfaction ratings. As detailed in Chapter 2, the idea that c ontent analysis and SN A could complement one another is the underlying principle of this study. To find a forum that is ideal for fost ering a constructivist learning environment, a p urposive sampling technique was utilized to fi lter a large pool of 39 forums from seven, eight -week, graduate -level online courses in an e ducational t echnology program. Three online educators rev iewed forum i nstructions and provided rating s based on a r ubric modified from the Jonassen et al. model (1995) (see Table 3 4) With a percentage of agreement of approximately 65% between each pair of raters, three forums e merged as the best candidates. One of the forums was forum EME 645805, an online course titled Distance Teaching & Learning Two sets of data for this study were retrieved from this forum. Discussion transcripts were used in content analysis using IAM. The database tables f r o m

PAGE 109

109 the same forum were retrie ved and used to generate a sociomatric of the forum which was imported into UCINET 6.258 t o calculate centrality measures. Sociograms of each measure were also generated using NetDraw 2.091. Results of IAM indicated that a majority of the discussion forum messag es in the course occurred in phases II (34.07%) and III (37.36%). The mean level of knowledge construction of the forum was 2.75 (SD = 0.61) (Table 4 1). However, when messages were categorized into initial and reply messages, the mean level of know ledge construction of the 19 initial messages was 4.13 (SD = 1.08) and of the 72 replies was 2.33 (SD = 0.81) Differences were also found in the average words within initial messages (M = 603.84, SD = 300.79) and reply messages (M = 133.42, SD = 68.78) Results of SNA suggested low variation in actor out -degree and in-degree centralities (SD of 2.525 relative to a mean out -degree of 5.442 and SD of 2.962 relati ve to mean in -degree of 5.442) with a slightly higher variation in actor betweenness (SD of 6.610 relative to a mean betweenness of 8.33) However, the overall network centralization is very low with an out degree of 6.25%, in-degree of 8.00%, and betweenness of 15.37%. The results indicated that in general actors did not hold significant structural power in any centrality measures. Finally, Spearmans correlations reveal ed no significant relation ship between the mean level of knowledge construction and each of the three centrality measures (p > .05) Findings of IAM related to the Literature Many of the studies that have utilized IAM to analyze discussion transcripts reported their findings by presenting five -phase mean levels of knowledge construction. IAM is a hierarchical model focusing on the flow and levels of knowledge construction including (a ) sharing and comparing of information, (b) discovery and exploration of dissonance or inconsistency among ideas, concepts or statements, (c) negotiation of meaning/co -construction of knowledge, (d)

PAGE 110

110 testing and modification of proposed synthesis or co -cons truction, and (e) agreement statement(s)/applications of newly -constructed meaning. Results from these studies (Table 5 1) show two distinct patterns of levels of knowledge construction. The first pattern follows the line of research of Gunawardena et al. (1997) that interactions in online discussion generally fall into the first three lower phases. Studies that report such pattern s include Marra, Moore, and Klimczak (2004) McLoughlin and Luca (1999) Schellens and Valcke (2005) Schellens, Van Keer, and Valcke (2005) and Sing and Khine (2006) In this pattern, s ome studies repo rted more percentage in phase I, others found evenly distributed percentage s across three phases and some studies foun d a few interactions in phase IV. Although the distribution of the three lower phases varies in these findings, the common theme across these studies is that interactions in phase V were virtually absent. On the contrary, results from this study showed a pattern similar to those of Yang, Newby, and Bill (2005) That is, the majority of the interactions were in the first three phases and in phase V, while interactions in phase IV were virtually non -existent. Although most of the interactions were in the first three phases int eractions in Yang et al. s result s were coded toward the lower end with an average of 67% in phase I while interactions in this study were coded in phases II (35.2%) and III (37.4%) The two emerging patterns of IAM results included : (a) high occurrence of interaction within the three lower phases and low or non-existent interaction in phases IV and V, and (b) observed interaction in three lower phases and in the final phase but low or non-existent interaction in phase IV These results might be consequenc es of different focuses of online discussion and research design. E mphasis on ce rtain components of instruction is related to the teaching presence in the community of inquiry framework (Garrison & Arbaugh, 2007)

PAGE 111

111 According to Garrison and Arbaugh (2007) teaching presence consists of three components: (a) instructional design and organization; (b) facilitating discour se; and (c) direct instruction. Online discussion activity seems to relate to the first two components; that is, p rior to the beginning of the course, the instructor chooses discussion materials/topics and gives directions and expectation s for student participation. The tasks also include giving guidelines and setting up a schedule for the discussion activities. Once the course starts, the instructor decide s whether and how to be involve d in the discussion. Such decisions may depend upon the planned instruction and/or pedagogical approach of the discussion activity. The last component of teaching presence may be perfor med by the instructor if the participation i s mandatory and accounted for in the grade. To explain the resulting patterns of the IAM coding result s and to add further evidence to the foundation of content analysis in CMC, literature was revisited to find out whether the instructors and researchers designed and placed focus on different components of the online discussion activities The review specifically connect ed literature to the present study in term s of teachin g presence. Several studies in the first pattern found most interactions in phase I and low proportions in phases II and III. The setting s of the discussion ranged from a week-long debate where participants in affirmative and opposing sides took turns each day (Gunawardena et al., 1997) or a three phase, project -based discussion where learners had to discuss theories, develop and share their lesson plan and write reflection s about the experience of the discussion and the learned content (Sing & Khine, 2006) Ot her studies that utilized roles assignment all of which were grounded in social constructivist theory, were found to have results similar to the first pattern McLoughlin and

PAGE 112

112 Luca (1999) designed an online discussion activity by assigning specific roles t o students. The roles reflect ed some desirable operation s related to the IAM, such as questioner (phase II) and summarizer (phase V). Schellens and Valcke (2005) asked students to post, and reply at least once per case discussion In their subsequent study (Schellens, Keer, Wever, & Valcke, 20 07) design -based research was employed as they collected data from the second cohort of the same course; the only redesign feature of the course for the second cohort was the scripting of the discussion as four students in each group assumed roles of mod erator, theoretician, summarizer, and source searcher. They reported that students in the second cohort (with assigned roles) outperformed students in the first cohort in acquiring higher levels of knowledge construction during discussion and in final exam scores. Interestingly, a significantly negative effect was found for students who worked as source searcher, while students who worked only as summarizer reached a significantly higher mean level of knowledge construction when compared to students in the non -script cohort. Several reasons may explain the resulting pattern of the high occurrence in the lowest phases and the low occurrence in phases IV and V. Gunawardena et al. (1997) attributed the interactions in lower pha ses to the format of the discussion. T hat is, choosing debate hinder ed the efforts to reach higher phases of knowledge co-construction : they explained that the debate allowed participants to solicit agreement on propositi ons (phase I) and introduce d incons istencies between statements (phase II). But the debate hindered the desire of the participants to reach a compromise or a synthesis on the propositions at Phase III and above, as the debate leaders tried to keep the two sides apart (p. 417). Further, i t could be argued that the ability to reach higher phases was hampered by choosing minimal interventions as a strategy for facilitation, which was

PAGE 113

113 the case in studies explicitly grounded in social constructivism (McLoughlin & Luca, 1999; Schellens et al., 2007; Schellens & Valcke, 2005) The aforementioned stu dies, although reporting low mean l evel s of knowledge construction, seem to make sense with respect to the process of knowledge co -construction as the IAM implies hierarchical structure where the higher phase s are built upon the lower ones (Buraphadeja & Dawson, 2008) T he second emerging pattern where interactions reach phase V but pha se IV is virtually non -existent, ho wever, did not follow this logic. Such results were found in this study and in the study by Yang et al (2005) To foster students critical thinking skills, Yang, Newby and Bill (2005) employed the Socratic questioning technique in addition to typical online discussion where pa rticipants were required to post at least one argument during the discussion and respond to at least one of the other students posting to extend the dialogue. To conclude each discussion, students were asked to summarize the points made during the discuss ion or to write a short reflection. Two treatments were set up for the study; the Socratic questioning was introduced into treatment I during the first half of the semester, and during the second half of the semester for treatment II. Based on the results of pretest and post test evaluations using California Critical Thinking Skills Test (CCTST), results revealed that both groups showed significant gains. Interactions among students in the first treatment reached higher phases during the second half of the semester with statistically significant difference s The results for the first half of treatment II were also significantly higher tha n the first half of treatment I. This means that as a result of Socratic questioning students reach higher phases in IAM. Results from treatment II indicated that during the second half (without Socratic questioning from the instructor), students moved slightly back to lower phases. However, no significant difference was found between the first and the second

PAGE 114

114 half of the sem ester in treatment II. Yang et al. (2005) concluded that with appropriate course design and instructional interventions, critical thinking skills can be cultivated and maintained in online discussion. This study took a different approach from Yang et al. (2005) by searching for a n exemplary forum in terms of constructivist learning environments based on the expert reviews and appropriateness of the forum content detailed in Chapter 3 A one -week forum titled Discussion 4. Economics and Education in the U.S. was selected The st rategy for facilitation of this activity seemed to be minimal, since the individual mean levels of knowledge construction of the instructor was the lowest among participant s and the average centrality measures in SNA, implying that the instructor did not a ctively get involve d in the process of knowledge construction in the forum. Nonetheless, t he results of this study appeared to be similar to those of Yang et al. (2005) as substantial interactions were found in the three lower phases and in phase V of knowledge constr uction while interaction in phase IV was virtually absent. However, t he difference between the stud y of Yang et al. and this study was the distribution of the lower mean levels of knowledge construction. While messages in Yang et al.s study were largely coded in phase I (approximately 60% 70%), messages in the present study were mostly in phases II and III (35.2% and 37.4% respectively). Table 5 1 provides a summary of previous studies and the present study related to teaching presence, where IAM result s were available. Of these seven studies only Yang et al. (2005) stressed facilitating discourse by providing constant feedback using the Socratic technique. Yet the results of this study were similar to those of Yang et al.s, thus providing evidence of critical thinking skills. This seems counterintuitive with reference to the logic of IAM ; that is, with the absence of phase IV, testing and modification of propo sed synthesis or co -construction of

PAGE 115

115 knowledge, the participants should not have reached the highest phase o f knowledge construction, agreement statement(s) or applications of newly constructed meaning. In other words, students advanced toward the conclusion without testing or modifying against ideas in both studies. Some decision s during the course design and data analysis might have triggered this phenomenon. In Yang et al. (2005) s study where instructors took the leading role questioning and challenging students throughout, such testing and/or modifying against ideas might have been performed by the instructor. Although it was unclear whether the instructors contribution was included in the data analysis, it is generally accepted that the inclusion of an instructors messages would bias the results (Sing & Khine, 2006) If this was the case, the Socratic Method play s a huge role in group knowledge construction The second possibility which explain s why students advanced toward the conclusion maybe that the nature of instruction demands students build their own arguments and conclusion s before having a chance to interact with peers; in other words, the activity does not require students to collaborate, and instead provide s opportunities for them to craft their compositions as intended for their final work. The fact that the average number of words in initial posts (M = 603.84, SD = 300.79) was much higher than those in reply messages (M = 133.42, SD = 68.78) suggested that students did try to compose a complete message instead of dialogue with peers. A fter removing noises and an outlier, detailed in chapter 4, the higher mean level of knowledge construction of the initial posts (N = 19, M = 4.21, SD = 1.08) compared to that of the reply messages ( N = 72, M = 2.31, SD = 0.80) also suggest ed that students put more effort into their initial posts to reach higher level s of knowledge construction than when trying to co c onstruct knowledge with others. Such

PAGE 116

116 elaborated instruction put s less emphasis on collaboration, as observed in the forum used in this study, which may have triggered this leap -to -conclusion incident Implications R elated to O utcomes of IAM Past studies that utilized and reported results from IAM showed that the model is able to demonstrate level s of knowledge in the discussion ; that is, the results of the IAM emerged as holistic view s of discussion flow and knowledge construc tion that are easily understood (Ma rra et al., 2004) Specifically, two unique patter n s of IAM coding results emerged from literature : (a) high occurrence of interaction within the three lower phases and low or non-existent interaction in phase s IV and V, and ( b ) observed interaction in th ree lower phases and in the final phase but low or non -existent interaction in phase IV Despite the inconsistent findings and suggestions in the literature, common themes came to light when they were organized into two component s of teaching presence in t he community of inquiry framework, instructional design and organization, and facilitating discourse (Garrison & Arbaugh, 2007) T hus some suggestion for informed decisions related to online teaching and learning can be given. Implications R elated to I nstructional Design and Organization Instructional design and organization usually involve setting curriculum and choosing methods of content delivery (Garrison & Arbaugh, 2007) To set up an online discussion, the instructo r and instructional designer may choose activities such as debate format, case studies, project based discussion, or reflective discussion. W hile the debate format might hinder participants to reach higher le vels of knowledge construction (Gunawardena et al., 1997) research suggests this may not be the case with other forms of discussion (Clark & Mayer, 2008; Sing & Khine, 2006; Yang et al., 2005)

PAGE 117

117 F orm s of discussion : Clark and Mayer (2008) suggest that instructor s /facilitator s may resort to proven technique from collaborative face to -face learning such as structured controversy. This particular activity adds a twist to a traditional debate format by assigning students in to small team s of four, with each pair either taking the pro or con position. Each pair presents their argument, while another pair re states the argument. The pairs then reverse roles. Later the group reconvenes and synthesizes to develop a grou p report from both perspectives. Unlike traditional debates, structured controversy allows students to move into synthesis phase, which is level IV on the level of knowledge construction in IAM. This activity is appropriate for asynchronous discussion as r elevant resources can be provided: students can research, develop their case, and synthesize their perspectives in the discussion forums Many researchers (Clark & Mayer, 2008; Sing & Khine, 2006; Yang et al., 2005) also suggest that a sking students to write r e flective discu ssion s at least as part of the activity would allow students to r each higher levels of knowledge construction In this study, t he instructor prepared elaborated instructions and adopted the reflective approach. T he forum was the culminating discussion th at required students to take on the broad issues related to technology, education, and economy. E laborated instructions requiring students to carefully craft their responses may, however, have an adverse effect on knowledge co-construction. In this particu lar study, research demonstrated that although detailed instruction s which w ere rated high in the rubric modified from Jonas s en et al. s model (1995) helped students to reach higher phases in knowledge construction : it forced students to respond with length y, essay like postings which are not desirable in discussion whether in face to -face or online settings (Bender, 2003) Bender (2003) explains that ideal postings should be succinct and informal, similar to a face to -face discussion

PAGE 118

118 in which interruptions or re buttals are common, stimulating, and expected. On the other hand, excessive postings are not likely to be critiqued and are difficult to respond to From this perspective, a plausible explanation of the results of this study, where the initial posts were r ated higher in levels of knowledge construction compared to the replies, is that students gain knowledge by accumulation instead of by argumentation (Bender, 2003, p. 70) Use of rubric: The next suggestion is to provide self asse ssment for students. Many scholars contend that a rubric could promote better understanding of the task and help assess performance, especially for meaningful and authentic assessment (Jonassen Howland, Moore, & Marra, 2003; Palloff & Pratt, 2005) According to Jonassen et al. (2003) a rubric is a code, or a set of codes, designed to govern action by identifying important aspects of the performance. Rubric s help reduce chances for students to ask how they should interact with peers and course material as it provide[s] students with a concrete way of evaluating their own performance as well as the performan ce of the members of their team (Palloff & Pratt, 2005, p.44) Palloff and Pratt also suggest that a rubric should connect to the course expectations so that the students end the course with a clear picture about their performan ce. Providing such rubric would help guide stu dents to think about their work, allow them to regulate their behaviors and encourage them to develop their work with higher level s of knowledge construction. In this study, the forum instruction only focused on direction for individual students with l ittle emphasis on collaboration. I t stated P lease be sure to reply to the posting of another participant. A lthough the discussion activity rubric was given in the syllabus detailed in Appendix C it only stat ed that students will be judged according to the quality of their comments and level of par ticipation to the responses of others and required students to respon d to other comments in a constructive fashion. To promote collaborative activity, a rubric shoul d

PAGE 119

119 include detailed performance measures for the group. Palloff and Pratt (2005) give examples of exemplary rubric s for individual performance on a team with a scale of four of five objectives including general attitude, working with others, collaboration, preparedness, and focus on task and time management Palloff and Pratts rubric for individual performance on a team can be found in Table 5 2. Group d iscussion and role assignment : The next aspect of instructional design and organization involves creating a desirable mix of individual and group activities (Garrison & Arbaugh, 2007) Several recipe s for group activities were i ntroduced in the past studies almost all of which also incorporate role assignment. In such settings s tudents also seem to benefit greatly from the reflective function Schellens et al. (2007) found that when the assignment of roles to group members (e.g., moderator, summarizer or theoretician), also known as scripting, were introduced, students who assumed a role of summarizer reached a significantly higher mean level of knowledge construction Howe ver, there are some caveats in roles assignment as Schellens et al. reported that a significantly negative effect was found when students worked as a source searcher, a group member who look s for additional information to stimulate others to go beyond the reading material. McLoughlin and Luca (1999) speculated that students may cease to engage in conversation once they complete their roles. For example, the questioner may not follow along with the discussion once his/ he r role is fulfilled. Duration of discussion : The final aspect of asynchronous online discussion is time span of the activity. Of the seven studies that utilized IAM, one study set up an eight -week project based discussion with three phases where each phase address ed different stages of the pro ject including sharing theoretical issues, planning and implementing a lesson, and reflection once the project was complete (Sing & Khine, 2006) One study set up a two -week case study discussion

PAGE 120

120 (Yang et al., 2005) and two studies from Schellens and Valcke (2005) and Schellens et al. (2007) organized a three -week discus sion. The o ther three studies (Gunawardena et al., 1997; Marra et al., 2004; McLoughlin & Luca, 1999) used a one -week discussion format. With the e xcept ion of the Sing and Khine study (2006) which limited purpose s of discussion s to certain levels of IAM ( e.g., sharing information or reflection of learning) r esults from th ese em pirical studies albeit in small number, suggest the length of discussion activities should be between one to three weeks. The results also indicated mixed outcomes of levels of knowledge construction. While levels of knowledge construction during a three -week discussion were largely in the first three lower phases, about ten percent of a two -week discussion was in phase V. Results of o ne -week discussion activities including those for this study, were also mixed, suggesting that time span may not be a crucial factor, in comparison with disc ussion format and instructions. Implications R elated to Facilitating Discourse T he second component of teaching presence, facilitating discourse, is associated with sharing meaning, identifying areas of agreement an d disagreement, and seeking to reach consensus. Instructors / facilitator s should engage in reviewing and commenting on student responses, raising questions and making observations to efficiently move discussions in an appropriate direction (Garrison & Arbaugh, 2007) From this point of view, choosing a strategy for facilitating discourse is similar to cho osing a p edagogical approach of the activity. Researchers argue that social constructivism is a pedagogical approach that is well -suited for asynchronous discussion (Buraphadeja & Dawson, 2008; Schellens et al., 2007) Many researchers also employed a student led, minimal intervention approach in discussion (Marra et al., 2004; Schellens et al., 2007; Schellens & Valcke, 2005) but only a small proportion of high levels of knowledge construction were

PAGE 121

121 observed. Other studies happened in online or blended courses in which the instructors/facilitators took a hands -on approach. I n the studies reviewed by Sing and Khine (2006) and Yang et al. (2005) the instructor assume d an intermediary role and was extensively involved in the discus sion Yet the results of these two studies were conflicting While Yang et al. reported approximately 11 13% of phase V in all four treatments Sing and Khine observed only 2.65% Essentially simply declaring which pedagogical approach is used in a class or merely choosing whether to intervene may not be sufficient. The results suggested that methods of intervention play an important role in enhancing learning Lowes et al. (2007) state d that the content of intervention largely influence s types of interaction among students; that is, participants would be more likely to offer new information if the instructor questions or challenges them inst ead of giving them information. One technique tha t stand s out is the Socratic method (Bender, 2003; Yang et al., 2005) ; i nstead of asking whether and how much to intervene, the i nstructor should focus not on giving t he students information, but on how to get involved in the conversation and on challenging students with thought provoking questions Findings of SNA related to the Literature SNA is a versatile method that can be applied to many contexts and is suggested by many scholars as a useful tool that could prove benefici al to the online learning community. SNA could provide a snapshot of the structure of the learning community and help instructor s keep track of communication (Reffay & Chanier, 2002; Shen et al., 2008) ; a specific technique such as s ociograms could help instructor s better monitor and detect communication problem s in the community as well as reveal structure s of the community with and without the in structor s input (Nurmela et al., 1999; Shen et al., 2008) It was also reported that group activity discussion s yielded more interaction (degree c entrality ) when compare d to individual or peer review

PAGE 122

122 activities (Shen et al., 2008) Further, w hen low degree centralities, both out -degree and in degree centralities, were observed in a group discussion, this implied internal group c ommunication problem s (Harrer et al., 2006) SNA measures (density and network centralization) were also found to be highly correlated with one another and with students sa tisfaction ratings fr om survey (Lowes et al., 2007) Although SNA could p otentially be a starting point for extensive analysis of knowledge construction and acquisition processes (Nurmela et al., 1999) most of the advocates of using SNA in online learning environments focus on understanding social aspects of the community (e.g., degree of interaction, students satisfaction, or communication problem s ) (Harrer et al., 2006; Reffay & Chanier, 2002; Shen et al., 2008) Research applying SNA to understand the cognitive aspect of the community is sparse This study attempt ed to discern the nature of the relationship between measures in SNA and one of the more grounded techniques to understand cognitive aspects of the learning community, content analysis using IAM. Specifically it aim ed to examine the relationship of these two techniques had in measuring critical thinking skills in online learning environments using discussion forums. Based on the material selection s, detailed in chapter 3, the data of the forum that exemplifies a constructivist learning environment was retrieved and used to create a sociomatric which was later used to calculate centrality measures and generate sociograms. Normalized in and out -degree s revealed that a few students were acti vely engaged in the discussion. While only one student emerged as an influential actor (high out degrees), there were three students who were considered prominent (high in degrees) The normalized betweenness centrality revealed that four students acted as communication broker s in the network.

PAGE 123

123 Although previous studies reported that instruc tors sometimes play a crucial role in the discussion (Nurmela et al., 1999; Shen et al., 2008) the sociograms of this study (see Figures 4 1 to 4 3) showed that the instructor was on the outer edge of the network and all of the centrality measures were about average (see Table 4 3). Thus the instructor did not influence the flow of the discussion Implications related to outcomes of SNA : Results of this study support the benefit of using SNA to study aspects of the discussion More specifically, o utcomes of the SNA both network measures and sociograms help instructors/facilitators better understand the social structure of a certain activity in the classroom Actor -level analyses identify prominent actors who may hold struc tural advantages in the network. Network measures at this level would benefit instructors/facilitators in the early stages of the discussion activity As notable actors emerge f rom the early activities, instructor s can quickly identify actors who are actively participating in the activity (high out degree centrality), those who are promin ent (high in degree centrality) and others who may hold brokering advantages (high betweenness centrality). By identifying prominent actors, instructors/facilitators could restructure the subsequent acti vities as needed. For example, mixing students who received different scores in network measures in a group discussion is likely t o foster more dyna mic communication in the subsequent activities. Other techniques in SNA which were not utilized in this study may also uncover more subtle structural issues in online activity. For example, clique analysis, an SNA technique that is used to identify cohes ive subgroups within a network (Knoke & Yang, 2008) may help instructors/facilitators better understand students who may be disposed toward homogeneity of t hought and behavior.

PAGE 124

124 Network measure at the macro level known as centralizatio n measures help instructors/facilitators quickly recognize the amount of concentration in the whole network, and how diverse the structural power that actors in the network have in terms of degre e and betweenness centralities. These centralization measures are concrete indicators for instructors/facilitators to decide further interventions. For example, high centralization in degree but low centralization in betweenness implies tight relations among actors (high degree) who can reach other actors without or with little aid of an intermediary (low betweenness). These indicators are preferable for discussion activities where instructors/facilitators expect students to collaborate and co -co nstruct knowledge with peers. Not only is SNA beneficial to understanding social aspects of the community at the individual and whole -network levels it also has the potential to reveal cogn itive aspects of the community. In other words how knowledge is constructed and whether the prominent actors in the network play crucial role s in the process of knowledge construction are important To understand the cognitive aspect s of the community, the centrality measures were correlated with the resulting mean levels of knowledge construction f rom content analysis using IAM. The next s ection discusses findings and implications of the research question of this study: Do levels of knowledge construction in online learning environments relate to the centrality measures of social networks? Findings R elated to the Relationship between IAM an d SNA This study asserted that centrality measures in SNA and mean level of knowledge construction are related based on the notion that individuals construct knowledge by interacting with others and thus more interactions would result in higher level s of k nowledge construction (Gunawardena et al., 1997; Schellens & Valcke, 2005) Spearman s correlations were utilized to analyze results from the two methods. N o evidence of relationship between centrality measures

PAGE 125

125 and mean level of knowledge construction was found One of the plausible explanations for this phenomenon was the fact that the o verall interaction in the forum was low which may have been caused by the focus of the forum instruction. As discussed in the Materials Selection section in Chapter 3, the forum instruction only focused on individuals performance with little expectation for collaboration. Nonetheless, the correlations suggested systema t ic tendency (Huck, 2004) Specifically, the correlations represented the inverse relationship that low scores on the mean levels of knowledge construction tend to be paired with low sco res on the centrality meas ures. In other words, actors that possess more structural power tend to have low scores in mean le vels of knowledge construction. Despite the fact that no significant relationship was found between the results from IAM and centrality measures, results from both techniques shed light on how knowledge is constructed and provide an understanding of the depth of structure in a learning activity Essentially, results of IAM showed how students responded to certain types of instructions and how knowledge construc tion wa s shaped by design of activities and types of intervention, as in the teaching presence in the Community of Inquiry (Garrison & Arbaugh, 2007) As discussed earlier in this chapter, the distinct mean levels of knowledge construction between initial posts and reply messages implied that students put more effort into c omposing their initial response and less in interact ing with others in the class. Although the elaborated instructions foster students to reach higher levels of knowledge construction, evidence of interaction and knowledg e co -construction were lacking. The SNA measures provided an explanation for multiple layers of the network. A t the actor levels it reveal ed that a few actor s were prominent to some extent Low variation of the centrality measures suggested that degrees were evenly dispersed. T he whole network level

PAGE 126

126 measures (network degree centr alization) were very low (6.25% for out -degree and 8.00% for indegree) indicating that there was no significant amount of communication (degrees). In other words, results of SNA measures were congruent with those of IAM, indicating that interaction s were remarka bly low throughout the network and knowledge was mostly individually constructed. Further, the fact that the instructor was not a prominent actor in any measure and no significant relationship between centrality measures and IAM was found after rem oving the instructor record from the data set suggested that the instructor was not in control of network communication and that students were given opportunities to construct knowledge without the instructors intervention. Results of both IAM and SNA, a lbeit absent any relationship, provide us implications in many aspects. Some have already been addressed separately in earlier section s in this chapter. The following sections discuss broad implications and directions for future research. Implications R ela ted to Online Teaching and Learning T he results of this study give us suggestions for designing online discussion-based activities founded on the notion of teaching presence in the Community of Inquiry (Garrison & Arbaugh, 2007) Implication 1: The following recommendations can be extrapolated from the research finding s in terms of instructional design and organi zation : D iscussion format s (e.g., debate, elaborated instructions ) can help or hinder students to reach higher le vels of knowledge construction. Discussion r ubric s should be given as a self assessment tool for students for both individual and group evalua tion If the instructor considers assigning roles to students, meaningful responsibility that reflects high levels of knowledge construction should be employed (e.g., questioner and summarizer). Small g roup discussion with diverse background is desirable

PAGE 127

127 Further, based on extensive reviews of research in computer -supported collaborative learning (CSCL), Clark and Mayer (2008) argue that several enablers may promote better individual and group learning outcomes f rom collaborative environments. Many of these enablers could be directly applied in an asynchronous discussion forum: Group process structures that foster the accountability and participation of each member of the team Focus on ou tcomes that benefit from reflection and independent research Clear guidance and objectives for team processes to avoid extraneous mental processing Ultimately, the study developed the potential path s to higher levels of knowledge construction forum as a f low chart ( Figure 5 1 ). T he flow chart indicates four important aspects of designing forum discussion: role assignment, concise and controversial discussion topic, rubric with collaborative components, and reflective components. Implication 2: To facilitat e discourse in online discussion, the emphasis should be placed on quality of intervention (Lowes et al., 2007) rather than quantity of intervention. SNA measures provide indicators to help instructors/facilitators monitor and make decisions on when to intervene if necessary. Focusing on comments that pu sh stu dents out of their comfort zone or constructively challenge them can foster higher levels of knowledge construction and exercise of critical thinking skills. Essentially, instructional design and facilitat ion discourse are intertwine d components in o nline discussion that can inform one another throughout the course lifespan ; instructors/facilitators can continuously adjust and reorganize discussion activities based on the previous round of discussion s w ith a snapshot of SNA measures and sociograms Wi th the large quantities of data available in the LMS, instant feedback would be especially effective in improving course design and facilitation of the discussion activities The next section discusses

PAGE 128

128 technical aspects of the research findings and how the y impact issues in online teaching and learning Recommendations for LMS Developers Due to the low number of participants (N=21), t his study developed a socio matri c manually that was double check ed by running PHP snippets. Such snippets, if fully developed, could benefit larger audiences s uch as open source LMS communities The snippets could be integrated as an additional report module in the Moodle LMS to allow users to quickly generate sociomatrices Integrating other web application s such as NetVis Modu le (Cummings, 2009) an open source web -based tool to visualize social network data, or Flare (flare.prefuse.org, 2008) an ActionScript library for creating visualizations on Adobe Flash Player would allow sociograms to be quickly rendered. Sociograms or the visualized version of the networks would give instructors/facilitators the ability to keep track of the class and quickly reorganize participants in order to promote dynamic discussio n For example, if the whole class discussion were followed by a small group discussion, instructor s could improve the group dynamic by placing students who were located in the middle of the whole network (i.e., those who tend to have higher degr ee and bet weenness centralities) into different groups. Such students with structural advantage s have a tendency to generate more traffic in the network. It should be noted that such visualization tools mainly help the facilitation of discourse in terms of the flow of communication regardless of the content exchanged. Ultimately, instructors/facilitators would still be required to guide the discussion, introduce new concepts, and steer the dialogue (Bender, 2003) Recommendations for Research This dissertation served to outline the initial steps in search ing for alternative methods o f assessing critical thinking in online discussion. Although no evidence was found to claim that

PAGE 129

129 SNA has a relationship with content analysis techniques that measure level s of knowledge co nstruction, the results of this study concurred with findings of c ontemporary SNA studies in online learning environment s reporting that such technique s provide broad and deep understanding s of the social a spects of a learning community. Future research should continue to utilize IAM in different educational settings to add more evidence to emerging patterns of knowledge construction as dis cussed earlier in this chapter. Future research should also extend the scope of this study to examine larger populati on s for example, all forums in a course that ranked high in constructivist learning environments could be examined. Materials selection is crucial for future research since the underlying assumptions of SNA are alig ned with that of social constructivism, suggesting that more interaction s would result in higher levels of knowledge construction. Although this study was grounded in this assumption the selected forum placed its emphasis on students initial response s rather than their interaction The materi als may also come from prospective data where r esearcher s and instructor s develop self assessment for group evaluation to ensure that the participants focus on socia l negotiation and thus yield higher interaction Using a prospective data approach also giv e s advantage to employ experimental design research that bring s a mixture of various conditions from the notion of teaching presence (Garrison & Arbaugh, 2007) Specifically, conditions in instructional design and organization forms of discussion, rubrics, role assignm ent, and duration of discussion are key variables of interest that can be implemented. Other variables are found in the f acilitating discourse frequency and approach to facilitation For instance, t he research design may investigate the differences in using short or long prompts, in a small or large group discussion. The design may also involve forums that last from one to three weeks with different rate and

PAGE 130

130 technique (e.g., Socratic method, or simply provide information) of scaffolding from an instructor By focusing on unobtrusive data collection techniques, an e xtensive version of this study could be replicated by incorporating other methods of readily available data in the LMS. Similar to the study of Black et al. (2008) that suggested simple LMS data logs could be a predict or of sense of community in distance learning environment s based on the Classroom Community Scale (CCS) (R ovai, 2002a) other instruments that measure cognitive aspects or critical thinking skills of learners can be used to discern the nature of the relationship s with the SNA measure. The Moodle LMS automatically maintains learners activity logs which can be sorted by types of actions In forum activities activity logs can be categorized into add ( discussion or post ), update, view and delete A s ociomatric of this study was based on adding discussions (21 entries) and posts (72 entries) while log entries related to update s (18 entries) and view s (858 entries ) were ignored These log entries may give us further insight into the social structure of the community and provide more evidence of interaction that may have implications in the process and levels of knowledge construction of the community. Conclusion This study aimed to discern the nature of the relationship between two different ways of measuring critical thinking skills in an online discussion forum. Social network analysis (SNA) is an a lternative method that was proposed for the use in measuring level s of knowledge construction The proposed indicators centrality measures in social network analysis were used and the resulting measures were correlated with a well developed content analytical m o d el, interaction analysis model (IAM) of Gunawardena et al (1997) A rubric based on Jonassen et al.s (1995) criteria for constructivist learning environments was developed to find the best online discussion candidate for th e study Although both methods shed light on many

PAGE 131

131 perspectives in the community participating in online discussion, no relationship was found between the two methods It is important to consider that the absence of a relationship was found under conditions where online disc ussion activity was designed for individual responses rather than interaction among participants. Such a relationship under different conditions is still unexplored. SNA provides an explanation of social structure in online courses that m ay help course instructors/facilitators better monitor their classroom s Furthermore issues related to finding s were discussed including implications for instructional designers, instructors, and LMS developers. The findings also point to IAM and how it could explain individua l or collective construction of knowledge More studies should expand on these findings to gain insight into the level of knowledge construction and how to foster critical thinking skills in online discussion. Alternative methods, es pecially those that rely on unob trusive data collection a nd require less time to analyze should be explored to support the demand for online learning that has burgeoned over the last decade.

PAGE 132

132 Table 5 1 Components in teaching presence and IAM coding r esults from previous studies and the study Instructional Design and Organization Facilitating Discourse Duration Direct Instruction Treatment / Group Phases (percentage) Pattern I II III IV V Total Gunawardena et al. (1997) Online debate in pro fessional setting N/A One week N/A 92.72 2.43 1.94 0.97 1.94 100.00 I McLoughlin & Luca (1999) Discussion; assign roles to students Student centered, minimal intervention One week 30% of the final grade Week4 67.02 21.28 6.38 5.32 0.00 100. 00 I Week5 64.95 22.68 9.28 3.09 0.00 100.00 I Week6 66.07 19.64 8.93 3.57 1.79 100.00 I Marra, R. M., Moore, J. L., & Klimczak, A. K. (2004) Case studies discussion Student led discussion; each student lead a case study worth 4 % One week 5% of the final grade 22.73 36.36 31.82 9.09 0.00 100.00 I Yang, Newby and Bill (2005) Debates and case studies Socratic questioning Two weeks N/A A1 1 st half of semester 75.44 10.09 2.63 0.00 11.84 100.00 II B1 1 st half of semester 63.76 17.11 5.37 0.00 13.76 100.00 II A2 2 nd half of semester 63.89 15.74 6.48 0.46 13.43 100.00 II B2 2 nd half of semester 68.29 14.15 4.39 0.00 13.17 100.00 II Sing & Khine (2006) Discussion Facilitator actively contr ibuted to the discussion (most posted and reached higher phase) Eight weeks (three phases) N/A: Advanced Diploma 59.73 20.35 12.83 4.42 2.65 100.00 I Schellens & Valcke (2005) Case studies discussion Weekly scaffolding feedback Three wee ks 25% of the final grade 51.70 13.70 33.10 1.20 0.40 100.10 I Schellens, Van Keer, De Wever & Valcke (2007) Case studies discussion; assign roles to students Weekly scaffolding feedback Three weeks 25% of the final grade Non script 51.70 13.70 33.10 1.20 0.40 100.10 I Script 52.90 6.10 29.80 2.80 6.50 98.10 I The study Integrated discussion related to real world issues Minimal intervention One week 10% of the final grade 12.90 34.07 37.36 1.10 15.38 100.00 II

PAGE 133

133 Table 5 2. Rubric for individual performance on a team from Palloff and Pratt (2005) Needs Improvement: 1 Developing: 2 Accomplished: 3 Exemplary: 4 General Attitude Often is publicly critical of the project or the work of other members of the group. Often has a negative attitude about the task(s). Occasionally is publicly critical of the project or the work of other members of the group but most of the time has a positive attitude about the task(s). Rarely is publicly critical of the project or the work of others. Often has a positive attitude about the task(s). Never is publicly critical of the project or the work of others. Always has a positive attitude about the task(s). Working with Others Rarely listen to, shares with, or supports the efforts of others. Often is not a good team player. Often listen to, shares with, and supports the efforts of others, but sometimes is not a good team member. Usually listens to, shares with, and supports the efforts of others. Does not cause waves in the group. Almost always l istens to, shares with, and supports the efforts of others. Tries to keep people working well together. Collaboration Rarely provides useful ideas when participating in the group and in classroom discussion. May refuse to participate. Sometimes pro vides useful ideas when participating in the group and in classroom discussion. Usually provides useful ideas when participating in the group and in classroom discussion. A strong group member who tires hard. Routinely provides useful ideas when participat ing in the group and in classroom discussion. A definite leader who contributes a lot of effort. Preparedness Often forgets needed materials or is rarely ready to get to work. Almost always brings needed materials but sometimes needs to settle down and get to work. Almost always brings needed materials to class and is ready to work. Brings needed materials to class and is always ready to work. Focus on task and time management Rarely focuses on the task and what needs to be done, and does not respect deadlines. Lets others do the work. Group has to adjust deadlines or work responsibilities because of this persons inadequate time management and lack of collaboration. Focuses on the task and what needs to be done some of the time. Other group members must sometimes nag, prod, and remind to keep this person on task. Tends to procrastinate, but finally always gets things done by the deadlines. Focuses on the task and what needs to be done most of the time and uses time well throughout the projec t. other group members can count on this person. However, may have procrastinated on one thing or another. Consistently stays focused on the task and what needs to be done. Very self directed. Uses time well throughout the project to ensure things get done on time. Does not procrastinate.

PAGE 134

134 Figure 5 1. Potential path s to higher levels of knowledge construction forum

PAGE 135

135 APPENDIX A S OCIAL NETWORK ANALYS IS Network study, according to Knoke and Yang (2008) emphasizes structural relations re gularities in the patterns of relations or ties among concrete entities that influence perceptions, beliefs, decisions, and actions. Network analysts believe that (a) to understand observed behaviors, social relations are often more important than entities attributes such as age, gender, and ideology, (b) structural mechanisms that are socially constructed by relations among entities affect perceptions, beliefs, and actions, and (c) such relations should be viewed as dynamic processes ( Knoke & Yang, 2008) T wo crucial elements in network study are actors and relations (Knoke & Yang, 2008) Actors can be individuals such as students on a football team, staff of a company, or collective actors such as firms in an indu stry, or political parties holding seats in a parliament. A particular member in a certain network is called an actor or ego and other members with which the ego has direct relations are referred to as others or alters A relation is defined as a specific kind of connection or tie between a pair of actors, or dyad. Such dyadic connection may be either directed where one actor initiates contact and another receives (e.g., tutoring or emailing), or non-directed where the connection is mutual (e.g., chattin g or marrying). Levels of measurement in network study may vary from binary values to ordinal measures (Hanneman & Riddle, 2005) Binary measures examine the presence or absence of the relation, indicated by binary values 0 and 1 respectively. Nonbinary measures include categorical nominal measures of relations where actors report types of relationship. For example, an ego may be asked to identify kinds of relationship for each of their alters as a friend, a lover, or a relative.

PAGE 136

136 Network measures may also use ordinal scale, such as asking an ego to report frequency of interaction. Research in network study also differs in term of level of analysis. According to Knoke and Yang (2008) there a re micro and macro levels of analysis in network study. An egocentric is a micro level of analysis focusing on an actor (ego) and others (alters) with which an ego has direct relations, also called the egos first zone An egocentric is usually employed if tracking down every contact of the ego on t he network is not possible (Hanneman & Riddle, 2005) A complete network, also known as sociocentric is a macro level of network analysis that use s informati on of all relations among all N actors in the population to represent and explain an entire networks structural relation (Knoke & Yang, 2008, p.14) Unlike in statistical analysis, social network study does not draw sample s from some larger population. Rather, social network analysts refer to population s as the population of interest; that is, a defined group of members (Hanneman & Riddle, 2005) Formal methods in reporting results of social network analysis include graphs and matrices (Knoke & Yang, 2008) A graph or a sociogram is a two -dimensional diagram visualizing actors (nodes or points) and their relations (lines). A non-directed rel ation is usually depicted as a no arrow headed line, whereas a directed relation is represented by an arrow -headed line. A singleheaded arrow line indicates a directed relation from its tail to its arrowhead (e.g., a sender to a recipient). A double -heade d arrow line indicates a relation that mutually occurs (e.g., a married couple). A matrix, also called a sociomatri c or adjacency matrix is an algebraic representation of network relations (Knoke & Yang, 2008) Similar to the general concept of a matr ix in mathematics, a sociomatric is an array of numerical elements arranged in rows and columns

PAGE 137

137 representing the sequence of actors with dimensions of N2 actors. Typically, this sequence is identical across the rows and the columns rangin g from 1 to N in referring to a particular row and -column location or a cell, denoted Xij. For example, element X23 refers to the value in the 2nd row and 3rd column. A sociomatri c may be either symmetric or asymmetric In binary measures, defining X23 = 0 in a symmetric matrix indicates that actor #2 does not send a relation to actor #3 and vice versa. Such matrices are therefore used to report non -directed relations. In a directed network, however, X23 = 0 does not equal to X32 = 0. Such matrices are used for directed relations. It is conventional to refer to a row number as a sender and a column number as recipient in such a relation. Social n etwork analysis also report s several indicators of the relationship, with density and centrality being the most common measures. A graph density in a binary network is the proportion of possible ties that ar e actually present in the graph (Wasserman & Faust, 1994) For a valued network, density is defined as a sum of the ties (tie strength ) divided by the number of possible ties (Hanneman & Riddle, 2005) For a graph with g nodes, there are 2(1)/2ggg possible unordered pairs of nodes. That is, there are (1)/2 gg possible non-directed ties that could be presented in the graph. A directed graph, on the other hand, may have the maximum number of lines up to (1) gg T he density of a non-directed graph ( denoted ) is the ratio of the number of reported ties (L) to the maximum possible : 2 (1)/2(1) LL gggg The d irected graph (digraph) is calculated as: (1) L gg

PAGE 138

138 This proportion of ties is a fraction that goes from a minimum of 0, if no ties are present, to a maximum of 1, if all ties are present To generalize the notion of density to a valued digraph, one can average the values attached to the ties across all ties (1)kv gg where kv denotes th e value of each dyad It should be noted that the density of a valued digraph measures average strength of the ties; that is, the ratio is not restricted to a fraction of 0 and 1. In addition to density measures, centrality measures are common relationship measures that seek to quantify the notion of an actors prominence within a complete network (Knoke & Yang, 2008) One of the most widely used centralit y measures is degree centrality. Degree centrality can be measured in both actor and network levels. In non -directed data, actors are different from one another in how many relations they have. This formula applies to a non -directed binary graph with g actors (Knoke & Yang, 2008) : 1()()g Di ij jCNxij wher e ()DiCN denotes the degree centrality of an actor i The formula simply counts the number of direct relations (nodal degree) that actor i has to the 1 g in oth er j nodes except i s relation to itself () ij (i.e., the diagonal values). However, a degree centrality score reflects network size g ; that is, the larger the size the higher the sco re. Normalized process is applied to eliminate this effect: () () 1Di DiCN CN g

PAGE 139

139 An actor i s degree centrality is divided by the maximum number of possible connections with 1 g other actors to produ ce a proportion of the network actors with direct relations to actor i In directed data, however, actors differ by how many out degree relations, influential actors, and in -degree relations, prominent actors or actors who have hi gh prestige they are associated with (Hanneman & Riddle, 2005) Since the centrality measure focuses on the choices made by actors, out -degree which is used for non -directed relations, is used for measu ring both (in d egree) centrality and prestige (Wasserman & Faust, 1994) To mea sure centrality and prestige in a valued graph, frequency or strength of relations will be summed up in the same manner via a non -directed graph. N ormalizing degree centrality requires the following formula: () () (*)(1)Di Di DCN CN Cng where (*)DCn denotes the maximum reported relation al value in the data. In addition, group degree centralization measures the extent to which th e actors in a network differ from one another in their individual degree centralities, which closely resemble s measures of dispersion in descriptive statistics (Knoke & Yang, 2008) In other words, group degree centralization expresses the degree of inequity or variance in the network as a percentage of that of the most centralized, or a perfect star, network of the same size (Hanneman & Riddle, 2005) The following formula is a measur e of group degree centralization proposed by Wasserman and Faust (Wasserman & Faust, 1994) : 1[(*)()] (1)(2)g DDi i DCNCN C gg

PAGE 140

140 where (*)DCN denotes the largest actor degree centrality reported and the ()DiCN are degree centralities of the other 1 g actors. The numerator sums the observed differences between the largest act or centrality and all the other s and the denominator is the theoretically maximum possible sum of those differences or the most centralized, perfect star network (Knoke & Yang, 2008) For a binary graph, this d enominator is equal to (*)(1)DCng For a valued graph, the denominator is (*)(1)(1)DCngg Cl oseness refers to how quickly an actor can interact with other actors in a network. A n actors closeness centrality is a function of its geodesic distance to all other actors (Knoke & Yang, 2008) The total distance of an actor i from other actors is 1(,)g jijdnn where the sum is taken over ji and (,)ijdnn is a distance function linking actors i and j (Wasserman & Faust, 1994) The index of closeness is an inverse of the sum of the geodesic distances between actor i and the 1 g other actors: 11 () () (,)Ci g ij jCN ij dnn

PAGE 141

141 APPENDIX B ENTITY RELATIONSHIP DIAGRAM Three tables in the MySQL database were crucial to the organization o f discussion forums: forum table: keeps forum activity information (e.g., associated course, title, introduction text, time limit, subscription policy, grade policy) forum_discussions table: keeps discussion topic information (e.g., title of the topic, firs t post ID, timestamp) forum_posts table: keeps individual posting transactions with organizational information (e.g., discussion topic ID, parent post ID) Figure B 1. Entity relationship diagram of forum discussion tables used in this study

PAGE 142

142 APPENDIX C INSTRUCTION TO THE EME 6458 DISCUSSION PARTICIPA TION Discussion participation Discussions will use both asynchronous discussion board tools (located in Moodle) and synchronous discussion tools (located in Moodle and online). Discussions will involve cla ss members and occasional outside guests and experts. Active participation is a critical component of building an effective online learning community. You are expected to be a regular and active participant in online discussions. This means you will post original material and thoughts as well as reply to posts submitted by others. Reading assignments will be made from a variety of online resources to prepare you to engage in the discussion. The quality of online asynchronous discussion will be driven by the extent of your preparation. The purpose of the discussions is to promote a learning community and encourage critical thinking skills in order to assimilate the information that is being provided. The discussion questions have been specifically designed t o encourage critical thinking and group discussion, so hopefully you will be encouraged to play an integral role in this process. Discussion topics will be open as assignments are posted. Start early to allow everyone to complete the assigned postings and responses before they are due. Each topic will be officially closed following the due date, so that we can move on. This process leaves a few days for everyone to finish discussing the topic and keeps us from continually having to revisit old discussions. We have a lot of material to cover in this course in a short amount of time, so it will be important that we move through the material at a certain pace. You will not be responsible for reading postings in old topics once the topic has closed, however, if the conversation is still of interest to you, it can be continued in the Open Forum in Moodle.

PAGE 143

143 Assignment rubric: (10 points for each discussion) Value Meets/exceeds all criteria: 5 Meets some criteria: 3 Meets few/no criteria: 0 Quality of comments Re sponses address the initial question; additional information to advance the discussion Superficial or inappropriate discussion Requirement absent Level of participation Responses address other comments in constructive ways Responses address only the initial question or do not contribute constructively Requirement absent

PAGE 144

144 LIST OF REFERENCES Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., et al. (2008). Instructional interventions affecting critical thinking skills and dispositions: a stage 1 meta analysis. Review of educational research, 78(4), 1102 1134. Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing Teacher Presence in a Computer Conferencing Context. Journal of Async hronous Learning Networks, 5(2), 1 17. Aviv, R., Erlich, Z., Ravid, G., & Geva, A. (2003). Network analysis of knowledge construction in asynchronous learning networks. Journal of Asynchronous Learning Networks, 7(3), 1 23. Bender, T. (2003). Discussion -based online teaching to enhance student learning: theory, practice, and assessment Sterling, VA: Stylus Publishing. Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., M.Tamim, R., Surkes, M. A., et al. (2009). A meta analysis of three types of interaction treatments in distance education. Review of Educational Research, 79(3), 1243 1289. Black, E. W., Dawson, K., & Priem, J. (2008). Data for free: Using LMS activity logs to measure community in online courses. The Internet and Higher Education, 11 (2), 65 70. Bloom, B. S. (1956). Taxonomy of educational objectives (Vol. 1: Cognitive Domain). New York: David McKay Company, Inc. Borgatti, S. P. (2002). NetDraw: Graph Visualization Software. Harvard, MA: Analytic Technologies. Borgatti, S. P., E verett, M. G., & Freeman, L. C. (2002). Ucinet for Windows: Software for Social Network Analysis. Harvard, MA: Analytic Technologies. Bratitsis, T., & Dimitracopoulou, A. (2008). Interpretation issues in monitoring and analyzing group interactions in asyn chronous discussions. International Journal of e -Collaboration, 4 (1), 20 40. Bullen, M. (1998). Participation and critical thinking in online university distance education. Journal of Distance Education, 13(2), 1 32. Buraphadeja, V., & Dawson, K. (2008). Content analysis in computer -mediated communication: analyzing models for assessing critical thinking through the lens of social constructivism. American Journal of Distance Education, 22(3), 130 145. Celani, M. A. A., & Collins, H. (2005). Critical thinking in reflective sessions and in online interactions. AILA Review, 18 (1), 41 57.

PAGE 145

145 Cho, H., Gay, G., Davidson, B., & Ingraffea, A. (2007). Social networks, communication styles, and learning performance in a CSCL community. Computers & Education, 49(2 ), 309329. Clark, R. C., & Mayer, R. E. (2008). e -Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (2nd ed.). San Francisco, CA: Pfeiffer. Cummings, J. N. (2009). NetVis module dynamic visua lization of social networks. Retrieved June 14, 2010, from http://www.netvis.org/index.php De Wever, B., Schellens, T., Valcke, M., & Van Keer, H. (2006). Content analysis schemes to analyze transcripts o f online asynchronous discussion groups: A review. Computers & Education, 16(1), 6 28. De Wever, B., Van Keer, H., Schellens, T., & Valcke, M. (2009). Structuring asynchronous discussion groups: the impact of role assignment and self assessment on student s' levels of knowledge construction through social negotiation. Journal of Computer Assisted Learning, 25(2), 177188. Dewey, J. (1910). How we think Boston: D.C. Heath & Co. Dringus, L. P., & Ellis, T. (2005). Using data mining as a strategy for assess ing asynchronous discussion forums. Computers & Education, 45, 141 160. Ennis, R. H. (1986). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron & R. J. Sternberg (Eds.), Teaching Thinking Skills: Theory and Practice New York: W. H. Freeman and Company. Ennis, R. H. (1993). Critical thinking assessment. Theory into Practice, 32(3), 179189. Entwistle, N., Tait, H., & McCune, V. (2000). Patterns of response to an approaches to studying inventory across contrasting groups and cont exts. European Journal of Psychology of Education, 15, 33 48. Fahy, P. J. (2002). Assessing critical thinking processes in a computer conference : Centre for Distance Education, Athabasca University. Fahy, P. J., Ally, M., Crawford, G., Cookson, P. S., Keller, V., & Prosser, F. (2000). The development and testing of a tool for analysis of computer mediated conferencing transcripts. Alberta Journal of Educational Research, 46(1), 85 88. Fahy, P. J., Crawford, G., & Ally, M. (2001). Patterns of interactio n in a computer conference transcript. International Review of Research in Open and Distance Learning, 2(1).

PAGE 146

146 flare.prefuse.org. (2008). Flare data visualization for the web. Retrieved June 14, 2010, from http:// flare.prefuse.org/ Fleiss, J. L. (1981). Statistical methods for rates and proportions (2 ed.). New York: John Wiley. Fowler, J. H., & Christakis, N. A. (2009). Dynamic spread of happiness in a large social network: longitudinal analysis over 20 years i n the Framingham Heart Study. British Medical Journal, 338, 23 37. Freeman, L. C. (2004). The development of social network analysis: a study in the sociology of science North Charleston, SC: BookSurge. Garrison, D. R. (1992). Critical thinking and sel fdirected learning in adult education: An analysis of responsibility and control issues. Adult Education Quarterly, 42(3), 136148. Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text -based environment: Computer conferencing in higher education. Internet and Higher Education, 2(2 3), 87105. Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. The American Journal of Distance Educational, 15(1), 7 23. Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry framework: Review, issues, and future directions. Internet and Higher Education, 10, 157 172. Gokhale, A. (1995). Collaborative learning enhances critical thinking. Journal of Technology Education, 7(1), 22 30. Golden, C., & Katz, L. F. (2008). The Race between Education and Technology Cambridge, Massachusetts: The Belknap Press of Harvard University Press. Gould, R. V. (1987). Measures of betweenness in non-symmet ric networks. Social Networks, 9 (3), 277282. Gunawardena, C. N., Lowe, C., & Anderson, T. (1997). Analysis of global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 395429. Gunawardena, C. N., & McIsaac, M. S. (2004). Distance Education. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 355 395). Mahwah, N.J.: Lawrence Erlbaum. Hanneman, R. A., & Riddle, M. (2005). Introduction to social network methods Retrieved January 12, 2010, from http://faculty.ucr.edu/~hanneman/nettext/

PAGE 147

147 Hara, N., Bonk, C., & Angeli, C (2000). Content analyses of on-line discussion in an applied educational psychology course. Instructional Science, 28(2), 115 152. Harrer, A., Zeini, S., & Pinkwart, N. (2006). Evaluation of communication in web-supported learning communities an analy sis with triangulation research design. Int. J. Web Based Communities, 2 (4), 428 446. Havard, B., Du, J., & Olinzock, A. (2005). Deep learning: the knowledge, methods, and cognition process in instructor -led online discussion. Quarterly Review of Distan ce Education, 6(2), 125135. Henri, F. (1991). Computer conferencing and content analysis. In A. Kaye (Ed.), Collaborative learning through computer conferencing: The Najaden papers (pp. 117 136). NY: Springer -Verlag. Hrastinski, S. (2008). Asynchronous and synchronous e -learning. EDUCAUSE Quarterly, 31 (4), 5155. Huang, C. J., & Chuang, Y. T. (2008). Supporting the development of collaborative problem based learning environments with an intelligent diagnosis tool. Expert Systems with Applications, 35, 6 22 631. Huck, S. W. (2004). Reading statistics and research (4th ed.). New York: Longman. : Pearson Education Inc. Jonassen, D., Davidson, M., Collins, M., Campbell, J., & Haag, B. B. (1995). Constructivism and computer -mediated communication in distan ce education. American Journal of Distance Education, 9(2), 7 26. Jonassen, D., Howland, J., Moore, J., & Marra, R. M. (2003). Learning to solve problems with technology : a constructivist perspective (2 ed.). Upper Saddle River, N.J: Merrill. Kim, B. (2001). Social constructivism. In M. Orey (Ed.), Emerging perspectives on learning, teaching, and technology : Association for Educational Communications and Technology (http://www.aect.or g/Intranet/Publications/index.asp) Knoke, D., & Yang, S. (2008). Social network analysis Los Angeles: Sage Publications. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press. Lew is, K., Kaufman, J., Gonzalez, M., Wimmer, A., & Christakis, N. (2008). Tastes, ties, and time: A new social network dataset using Facebook.com. Social Networks, 30, 330 342. Lipman, M. (2003). Thinking in education. New York: Cambridge University Press.

PAGE 148

148 Liu, B. (2007). Web data mining: Exploring hyperlinks, contents, and usage data. Berlin: Springer. Lowes, S., Lin, P., & Wang, Y. (2007). Studying the Effectiveness of the Discussion Forum in Online Professional Development Courses. Journal of Interact ive Online Learning, 6(3), 181 210. Marra, R. M., Moore, J. L., & Klimczak, A. K. (2004). Content analysis of online discussion forums: A comparative analysis of protocols. Educational Technology Research & Development, 52 (2), 23 40. Martinez, S. (2009). Student Support of Laptop Programs Olympia, WA: Generation YES. McLoughlin, C., & Luca, J. (1999, 58 December 1999). Lonely outpourings or reasoned dialogue? An analysis of text -based conferencing as a tool to support learning. Paper presented at th e 16th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE '99), Brisbane, Australia (pp.217228). Newman, D. R., Johnson, C., Webb, B., & Cochrane, C. (1997). Evaluating the quality of learning in compu ter supported co-operative learning. Journal of the American Society for Information Science, 48(6), 484495. Newman, D. R., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face to -face and computer supported group learning. Interpersonal Computing and Technology, 3(2), 56 77. Nurmela, K., Lehtinen, E., & Palonen, T. (1999). Evaluating CSCL log files by social network analysis. Paper presented at the The 1999 conference on Computer support for collaborative lea rning, Palo Alto, California. Ormrod, J. E. (2008). Human Learning (5th ed.). Upper Saddle River, New Jersy: Pearson Education. Palloff, R. M., & Pratt, K. (2005). Collaborating online: Learning together in community San Francisco, CA: Jossey Bass. Par sad, B., & Lewis, L. (2008). Distance Education at Degree -Granting Postsecondary Institutions: 2006 2007 (No. NCES 2009044). Washington, DC: National Center for Education Statistics. Pena Shaff, J., Martin, W., & Gay, G. (2001). An Epistemological Framew ork for Analyzing Student Interactions in Computer -Medicated Communication Environments.(Internet/Web/Online Service Information). Journal of Interactive Learning Research, 12(1), 4168.

PAGE 149

149 Pena Shaff, J., & Nicholls, C. (2004). Analyzing student interaction s and meaning construction in computer bulletin board discussions. Computers & Education, 42(3), 243 265. php.net. (2010). What is PHP? Retrieved January 30, 2010, from http://us2.php.net/man ual/en/intro -whatis.php Reffay, C., & Chanier, T. (2002). Social Network Analysis Used for Modeling Collaboration in Distance Learning Groups. Paper presented at the 6th International Conference on Intelligent Tutoring Systems (pp.31 40). Rourke, L., & Anderson, T. (2004). Validity in Quantitative Content Analysis. Educational Technology Research & Development, 52 (1), 5 18. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological Issues in the Content Analysis of Computer Confere nce Transcripts. International Journal of Artificial Intelligence in Education, 12, 8 22. Rovai, A. P. (2002a). Building sense of community at a distance [Electronic Version]. International review of research in open and distance learning, 3(1) from http://www.irrodl.org/index.php/irrodl/article/view/79/152 Rovai, A. P. (2002b). Development of an instrument to measure classroom community. The Internet and Higher Education, 5(3), 197211. Schellens, T., Keer, H. V., Wever, B. D., & Valcke, M. (2007). Scripting by assigning roles: Does it improve knowledge construction in asynchronous discussion groups? International Journal of Computer -Supported Collaborative Learning, 2(2 3), 225 246. Schellens, T., & Valcke, M. (2005). Collaborative learning in asynchronous discussion groups: what about the impact on cognitive processing? Computers in Human Behavior, 21, 957 975. Schellens, T., Van Keer, H., & Valcke, M. (2005). The Impact of Role Assignment on Knowledge Construction in Asynchronous Discussion Groups: A Multilevel Analysis. Small Group Research, 36, 704 745. Schwier, R. A., & Balbar, S. (2002). The interplay of content and community in synchronous and asynchronous communica tion: virtual communication in a graduate seminar. Canadian Journal of Learning and Technology, 28(2), 2130. Shen, D., Nuankhieo, P., Huang, X., Amelung, C., & Laffey, J. (2008). Using social network analysis to understand sense of community in an online learning environment. Journal of Educational Computing Research, 39(1), 17 36. Sing, C. C., & Khine, M. S. (2006). An Analysis of Interaction and Participation Patterns in Online Community. Educational Technology and Society, 9(1), 250 261.

PAGE 150

150 Smith, C P. (2000). Content analysis and narrative analysis. In H. T. Reis & C. M. Judd (Eds.), Handbook of Research Methods in Social and Personality Psychology (pp. 313335). Cambridge: Cambridge University Press. Stefanone, M. A., & Gay, G. (2008). Structural reproduction of social networks in computer mediated communication forums. Behaviour & Information Technology, 27(2), 97106. Veerman, A., & Veldhuis -Diermanse, E. (2001). Collaborative learning through computer mediated communication in academic educati on. Paper presented at the Euro CSCL 2001, Maastricht: McLuhan institute, University of Maastricht (pp.625632). Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes Massachusetts: Harvard University Press. Waits, T ., & Lewis, L. (2003). Distance Education at Degree -Granting Postsecondary Institutions: 2000 2001 (No. NCES 2003017). Washington, DC: National Center for Education Statistics. Wasserman, S., & Faust, K. (1994). Social Network Analysis: Methods and Appli cations New York: Cambridge University Press. Wickersham, L. E., & Dooley, K. E. (2006). A content analysis of critical thinking skills as an indicator of quality of online discussion in virtual learning communities. Quarterly Review of Distance Education, 7 (2), 185193. Yang, Y. C., Newby, T. J., & Bill, R. L. (2005). Using Socratic questioning to promote critical thinking skills through asynchronous discussion forums in distance learning environments. American Journ al of Distance Education, 19(3), 163181. Zhu, E. (2006). Interaction and cognitive engagement: An analysis of four asynchronous online discussions. Instructional Science, 34(6), 451 480.

PAGE 151

151 BIOGRAPHICAL SKETCH Vasa Buraphadeja holds a Ph.D. in curriculu m and instruction from the University of Florida. He current ly serves as a faculty member in the School of Management at Assumption University of Thailand. His research agenda focuses on assessing quality in asynchronous online education, computer game s in education, content analysis, data mining in learning management systems, and psychology. In regard to assessing quality in asynchronous online environments, he is interested in traditional web tools (e.g., discussion forums) and web 2.0 platforms (e.g., w ikis, or blogs) and how these tools evolve and influence teaching and learning.