UFDC Home  Search all Groups  UF Institutional Repository  UF Institutional Repository  UF Theses & Dissertations  Vendor Digitized Files   Help 
Material Information
Subjects
Notes
Record Information

Full Text 
COLLABORATIVE LEARNING AND TESTING IN INTRODUCTORY GENERAL CHEMISTRY By KATIE ELIZABETH AMARAL A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2004 ACKNOWLEDGEMENTS I would like to thank Dr. Martin Vala for his encouragement and guidance during these past few years. I would also like to thank Tim Young and Mark Sciallo in the Admissions Office for collecting students' grades for me. Thanks are also extended to Larry Margerum of the University of San Francisco for the idea to use collaborative testing in the CHM 1025 class. Thanks go also to Jason Morris and Keith Zientek for proofreading the document. TABLE OF CONTENTS page ACKNOWLEDGEMENTS............................................................................ LIST O F TA BLES......................................................................................................v LIST O F FIG U R E S......................................................................................................... vii A B STR A C T ................................................................................................................ix CHAPTER 1 INTRODUCTION.......................... ............ ......................... Collaborative Learning..................................................................................... 1 Chemistry Readiness Assessment Exam............................................................. 11 2 ANALYSIS METHODS AND PROCEDURES................................. ....... 14 A analysis of V ariance........................................................................................ 14 Collaborative Learning.......................... ......................... ............. .............18 F all, 2002........................................................................................................18 Spring, 2003............................................ ........................... ... .. 23 Summ er, 2003.......................................................... ... ....... 24 Fall, 2003................................................................................. ..............26 Spring, 2004...................................................................................................28 C alculations....................................................................................................29 Chemistry Readiness Assessment Exam............................................... .......... 30 3 STATISTICAL RESULTS................................................... ....... 32 Effectiveness of CHM 1025...................................... .... ..................................32 Collaborative Learning...............................................................37 A ttendance.................................................................................................... ....40 Chemistry Readiness Assessment Exam.........................................................42 4 COLLABORATIVE TESTING..................................... ................................48 5 PEER MENTORS........................................................................71 6 CONCLUSIONS........................................................................................... 83 Chem istry Readiness Assessment Exam .................................... .............................83 Effectiveness of CHM 1025................................................................................ 84 Collaborative Learning................................. ................. ..............................85 Attendance...........................................................................................................86 Collaborative Testing................................................... ............................... 87 Peer M entors................................................................................................. 88 APPENDIX A KEY TO INSTRUCTOR INITIALS ........................................ ..............................90 B COURSE OUTLINES .................................................................................. 91 CHM 1025..................................................................................................... 91 CHM 2045..................................................................................................... 92 REFERENCE LIST................................................................................................. 94 BIOGRAPHICAL SKETCH.................................................................................... 98 LIST OF TABLES Table page 21 Sample ANOVA Table............................................................................... 16 22 CH M 1025 in fall 2002.................................................. .................................... 19 23 CHM 1025 in spring 2003.................................................................................. 23 24 CHM 1025 in fall 2003................................................................................. 26 25 CHM 1025 in spring 2004.................................................................................. 28 31 GPA in CHM 1025 based on group type...................................................... 37 32 Number of students continuing to CHM 2045 based on group type......................................................................................................38 33 Number of students continuing to CHM 2046 based on group type......................................................................................................39 34 Scores in CHM 2045 and CHM 2046 based on CHM 1025 section........................................................................................................... 39 41 Performance on exam 1.................................................................................55 42 Performance on exam 2...............................................................................56 43 Performance on exam 3............................................................................... 57 44 Performance on exam 4...............................................................................59 45 Performance on the final exam............................... ........................................60 46 Percent of students taking each exam.......................................... ............ 62 47 Performance on each section of the exam......................................................62 48 Grades on the makeup final.........................................................................65 v Table page 51 Percent of students withdrawing from each section during the fall of 2003.................................................... ............................. 78 52 Withdrawal rates versus mentor utilization.....................................................79 53 Mentor grades in CHM 2045.............................................................................. 81 LIST OF FIGURES Figure page 21 F probability distribution.............................. ................. ..............................15 22 C LB C 130..................................................................................................... 20 23 HITT remote control.................................. .................. ..............................21 24 Chemistry Learning Center...................................................... ...................25 31 Percent student attendance by class meeting in CHM 1025............................... 40 32 Percent of students receiving a grade in CHM 1025 by number of classes attended.................................................................. 41 33 Percent of students receiving each grade in CHM 2045 by number of correct ChRA math questions..................................................44 34 Percent of students receiving each grade in CHM 2045 by number of correct ChRA chemistry questions........................... ............ 46 35 Percent of grades in CHM 2045 based on ChRA score................................. 47 41 Percent of students earning each grade on exam 1............................................55 42 Percent of students earning each grade on exam 2............................................56 43 Percent of students earning each grade on exam 3............................................58 44 Percent of students earning each grade on exam 4............................................59 45 Percent of students earning each grade on the final exam.............................. 61 46 Percent of students earning each grade on the makeup final.......................... 66 47 Student reported anxiety levels when studying and when taking the collaborative exam s.............................. ..........................................67 Figure page 51 Percent of students rating the helpfulness of peer mentors........................... 79 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy COLLABORATIVE LEARNING AND TESTING IN INTRODUCTORY GENERAL CHEMISTRY By Katie Elizabeth Amaral August 2004 Chair: Martin T. Vala Major Department: Chemistry Students taking General chemistry at the University of Florida are either well prepared or underprepared. To meet the needs of the underprepared students, an introductory course (CHM 1025) was developed. An accurate method of placement into CHM 1025 or the mainstream course (CHM 2045) was needed. The Chemistry Readiness Assessment Exam was written and tested and students are advised to take either course based upon their scores. The accuracy of the cutoff scores was examined, with the minimum passing chemistry score lowered to six correct out of 18, and the math score raised to six correct out of eight. Collaborative problemsolving sessions were held during every CHM 1025 class. These sessions were shown to increase student achievement in CHM 1025. Group placement was also shown to have an effect on student achievement in the course. Students placed randomly into collaborative groups had the highest average GPA, while students placed by achievement had the lowest average GPA. ix The efficacy of CHM 1025 was examined to determine if the students who required the course do as well in CHM 2045 as those students who did not need it. Students who had taken CHM 1025 had a higher GPA in CHM 2045 than the students who went directly into CHM 2045. Students in the spring semester of 2004 took collaborative exams. Achievement levels of students who had collaborative exams were compared to students who took traditional exams to determine if collaborative testing had an effect on student achievement and retention in CHM 1025. There was no significant difference in achievement although the collaborative exams were harder. Percentages of students taking each exam were also compared, with more students taking the collaborative exams. Finally, undergraduate students called peer mentors, who had taken CHM 1025, were recruited to assist with the course. Mentors helped CHM 1025 students with the collaborative problems. The mentors' presence helped lower students' withdrawal rates in the class. The mentors also benefited from the program, as evidenced by their higher GPA in CHM 2045. CHAPTER 1 INTRODUCTION Collaborative Learning There are three ways that the actions of an individual may influence actions of another individual. The action may help another individual succeed, may hinder the success of another individual, or may have no effect on the other individual.' The first of these outcomes is usually the result of a cooperative effort on the part of the individuals in question, the second of competitive efforts, and the last of individual efforts. Although individual efforts are usually stressed in the learning environment, individuals who work near each other on the same task generally begin to compete. In the university setting, high grades are often made artificially scarce, with only a certain percentage of students able to achieve this level of success. This often fosters a competitive attitude among students. When individuals compete against each other, the outcome of their actions is often negatively linked. The success of a small minority of individuals means the automatic failure of the rest of the competitors. For example, if the students' grades in a class are normalized to a bell curve, only a select few students will receive an A. All other students will automatically not get the high grade, and these students usually know during the course of the semester that they will not achieve this level of performance. This knowledge of failure, as well as the failure itself, can leave the losers feeling angry, jealous, or inadequate about their own performance Competition can also have an adverse effect on the interactions between the individuals involved. Anything that helps one competitor succeed directly hurts another competitor's chance of success. Individuals who compete may realize that anything to their immediate advantage is better than cooperating with others engaged in the same task. Often, when there is a high level of importance placed on success, competitors will cheat in order to enhance their chances. In a survey of central Florida college students, 9 out of 10 stated that they had cheated on an exam or a homework assignment.' Competition can also lead to high anxiety levels among students. High anxiety may inhibit an individual's performance on the task. Anxiety disrupts a student's ability to cognitively reason and promotes avoidance of the situation that the student fears. In the case of the college student, it means skipping class, taking long breaks, or avoiding challenging learning situations. As early as 1927 it was known that individuals who fail in the early stages of competition did not overcome their initial failure and scored poorly on all subsequent tests, even though the tests were matched to competitors' abilities.2 It has also been demonstrated that individuals who fail consistently in a competitive situation tend to withdraw from the competition.3 In a scholastic setting, this means that students who perform poorly on the first exam are likely to continue with this poor performance in spite of their ability to achieve higher scores. These students often give up, and stop trying to achieve a higher score. A study performed at the University of North Carolina at Charlotte has shown that 80% of the students who failed their first exam in the first semester general chemistry class failed the course.4 Competitive students either work hard to do better than their peers or do only the minimum amount of work, because they do not believe they have a chance to succeed.5 Competitive situations also tend to increase student anxiety. In a nonexam situation, students who are working on homework or merely reviewing lecture notes have no other source of information. If a student does not understand or have the knowledge to complete a problem, work on that problem ceases. Other students, in this case competitors, are not likely to assist the student with the task because it will directly inhibit their own chances of success. Also, students who work alone may have a difficult time remaining motivated to complete a task, especially when they feel the task is unimportant to them. Competitive learning situations tend to isolate students from each other and create negative relationships both between the instructor and the students, and between the students themselves.5 These situations also tend to discourage active learning and do not help develop talent among students. Reviews of the literature have shown that college students are much more likely to acquire critical thinking skills and to develop learning strategies in a collaborative setting than in a competitive setting.6'7 Bligh has shown that college students spend more time integrating concepts and synthesizing connections between concepts in a collaborative setting than those students who listen to a traditional lecture.8 In fact, groups have been shown to be better at promoting student problem solving skills than the traditional lecture.9 With these thoughts in mind, we decided to implement collaborative learning in the introductory general chemistry class at the University of Florida. Collaborative learning is learning that takes place in small groups of cultural, academic, and occasionally linguistically heterogeneous populations.'1 The participants in these groups work together to accomplish the assigned task, both giving and receiving pertinent information from group members. Whether collaborative learning is effective or not depends upon several variables. The teacher's knowledge and skills will determine how the groups are guided through their tasks and will also affect classroom management. Classroom configuration is also important to collaborative learning. Some classrooms, especially auditorium style classrooms or classrooms where the seats are stationary, are difficult to arrange in a pattern that promotes communication among group members. As discussed below, the type of group greatly affects the outcome of the collaborative situation. There are several different types of groups that appear in the collaborative learning setting. These include the pseudo learning group, the traditional classroom group, the formal and informal collaborative learning groups, and collaborative base groups." In the pseudo learning group, the participants are assigned to work together for a varying length of time, but have no reason to help each other because they are directly competing for the highest grade. These students are usually graded on a bell curve. In these learning groups, students may mislead and confuse each other as well as try to hide what they know from each other. The students in these groups perform worse on the assignment than if they had actually worked by themselves." In the traditional classroom learning group, students are assigned to work together and do so because it is required. The length of time the students work together varies from situation to situation. Students are evaluated on individual performance only, and therefore may have no vested interest in helping other students. While they desire the information that other students have, they may have no incentive to share their own 5 information. Personally they desire to do well on the assignment, but may not want their colleagues to perform equally as well. Only a minimum of collaborative learning occurs in these groups and this may occasionally help the lower achieving students. However, the higher achieving and more conscientious students would usually perform much better on an assignment if they had worked alone.' Formal collaborative learning occurs when students work together for a period of time, lasting from one to several class periods, in order to achieve specific tasks and assignments. In order for formal collaborative learning to be successful, the teacher must decide in advance the size of the group, the objective of the group, how students are assigned to a group, and the role each student will play in the group. The objective needs to be explained to the students in such a manner that they understand how they are to function as a group and how their grade depends both on individual performance as well as group performance. The manner in which the instructor approaches formal collaborative learning plays a key role in these types of groups. The instructor needs to monitor the students while they are in the group to provide assistance with the objective and to help with student social skills. Informal collaborative learning occurs when students form their own groups and work together for only a short period of time, from a few minutes to one class period. This type of group can be used to focus students' attention on the lesson at hand, to ensure that they understand and are capable of applying the information they are being taught, and to break up an otherwise monotonous lecture into something more interesting. There is neither a defined evaluation procedure nor are there defined student roles in this type of collaborative learning. The final type of collaborative learning group is the collaborative base group. These groups consist of students of heterogeneous ability and backgrounds that work together for several semesters or years. These groups work best in the K12 setting, and thus will not be discussed here." The success of formal collaborative learning hinges upon several factors.' First, students must realize that they are connected to each other and that they cannot individually succeed unless the group succeeds. Usually this involves connecting an individual student's grade with the group grade. In order for the group to perform well, students must share their work, ideas, skills and information with each other. Students must also be individually accountable for their own work. This ensures that one student does not have to perform all the work in a group and that all students are contributing to the objective. The students must be comfortable in the group setting in order to take risks regarding learning and to encourage and support their peers. Students must also learn the social skills necessary to work in a group. The students must learn how to solve problems, they must discuss the nature of the concepts and strategies being learned, they must teach the knowledge to classmates and explain connections between past and present learning.12 Finally, students must discuss where the group is in terms of the group goals and what must be done to achieve the goals. These factors are not easily achieved and must be constantly addressed as the group works together. The instructor plays an important role in whether or not collaborative learning is achieved in the classroom.12 The instructor needs to ensure that students are actively learning and contributing towards the common goal of the group through careful observation of student interactions. The instructor is no longer just the source of information, but is also a facilitator of learning. The instructor must put a lot of effort into preparing for a collaborative class. All contingencies must be prepared for, and all students must be doing what the instructor has planned. If collaborative learning is implemented properly, the classroom will turn into a stimulating, challenging, and educational environment. The collaborative process must be continually monitored as it occurs, and any changes that need to be made should be made immediately. Group activities need to be focused around a central idea in order to foster conceptual understanding of difficult material.' Students should encounter the central idea in many different settings so they will have many different opportunities to work on the problem. They will also need to have different tasks that can help promote comprehension of the material. The group tasks should inherently be openended in order to ensure that students work together to arrive at a solution to the problem. If there is more than one right answer to the problem, the students must discuss the problem, decide upon a direction for the group to take and then work on the problem, arriving at a solution that may be different from other groups in the class. Openended tasks increase group cohesiveness because they force students to rely upon each other's knowledge and skills. To ensure that varying levels of knowledge and skills are present within the group, the groups should be heterogeneous. This should help the lower achieving students perform better. The higher achieving students also benefit because their knowledge and understanding of the objective is tested as they assist the lower achieving students. Students' reliance upon each other fosters higherorder learning skills because they have to communicate their thoughts effectively to one another, examine issues from different perspectives, and justify their arguments. The small collaborative groups also provide an opportunity for students to ask questions about the material in a safe setting. Greater than 60% of college students stated that having a large number of peers listening prevents them from asking questions in a class.3 The smaller group provides a much less threatening situation where students can ask questions. Difficulties can arise in the collaborative setting when lower achieving students or socially distant students are excluded from the learning group.14 Critics of collaborative learning state that the exclusion of certain individuals from a collaborative group reinforces existing social and educational problems. Expectation States Theory (EST) outlines how the status of an individual group member determines the value the group places upon that individual's work.15 The status of an individual depends upon status characteristics, which are agreed upon social rankings where high status is better than low status. Status characteristics are based on several things and can change from situation to situation. They can be general, based on gender or ethnicity, or they can be specific, based on perceived ability related to the task at hand. They can also be temporary, based upon perceived academic status. Contrary to what might be expected, minority students and low socioeconomic status students are not necessarily low status group members. In situations where students of minority or low socioeconomic backgrounds are considered high status, their relative status would carry into the groups in which they participated. It has also been demonstrated that collaborative learning can help reduce stereotypes because the individuals in the groups are exposed to others who disprove the stereotyped role.16 It is also possible for a teacher to equalize students' status within the collaborative learning group.14 If a teacher assigns competence to a student or group of students, there is a specific and public recognition of the intellectual contribution a student can make to the group assignment. It often has a positive side effect on the low status student because a teacher's high expectations can often raise the expectations of the student in question and other students in the peer group. The praise must be public, specific, and relevant to the work of the group for it to be effective. Cohen and Lotan have summarized research about the effectiveness of collaborative learning.17 They report on two data sets of five middle schools in the San Francisco Bay Area between 1991 and 1993. Students who participated in formal collaborative learning were compared to students who had not. All students covered the same topics in the social studies classroom and were given the same tests. The students who engaged in collaborative learning scored higher on items that required higherorder thinking than those students who did not. There was no significant difference in students' scores for questions requiring only factual recall. They also determined that the way in which the teacher implemented the collaborative learning program affected the students' achievement. Increasing the number of activities and reports the students completed had a positive effect on test scores. They also discovered that in classes where the instructor had difficulties with classroom management, student achievement diminished. Johnson and Johnson performed a metastudy on the 550 experimental and 100 correlational studies performed on the effects of collaborative, competitive and individual efforts on instructional outcomes.' They noted that the type of interaction among students determined the instructional outcome. For instance, structuring assignments so that they must be collaborative in nature resulted in students working together to achieve success. On the other hand, students who worked competitively tried to hinder the success of others while those students who worked independently had no effect on the success of other students. Johnson and Johnson noted that 375 studies in the last 100 years demonstrated that students who worked together had higher achievement and greater productivity than those students who worked alone. These studies also demonstrated that collaborative learning resulted in more highlevel reasoning, greater transfer of knowledge from one situation to another, and more time spent ontask than individual or competitive settings. They also concluded that collaborative learning ensured that all students were actively and productively involved in learning. As a side effect to the textbased learning that goes on in collaborative groups, Johnson and Johnson determined that groups promoted caring and committed relationships among students. These relationships were not fostered in the competitive or individual learning settings. They also stated that working collaboratively with peers resulted in greater psychological health, higher selfesteem and greater social competence of students than in the competitive or individual environments. Collaborative learning has been demonstrated to be an effective teaching tool in general chemistry,'828 organic chemistry,2932 analytical chemistry,33"37 and physical chemistry.3840 It is essential to ensure that true collaborative learning is present in order for it to be effective. Collaborative learning teams have been shown to outperform individual and competitive systems in mastering content within a time constrained system based upon set performance criteria.4143 Students in collaborative learning groups tend to think more critically and to learn less material by rote memorization.44 Collaborative learning is traditionally implemented in small classrooms of approximately 3050 students. One of the aims of this project is to demonstrate that collaborative learning can be implemented in a large introductory general chemistry class of approximately 300 students. Hogarth and Hartley have examined the effect of the method of placing students into groups.45 In this study, students in a British Grammar School, the equivalent of high school here in the States, either chose their own pairs or were assigned to a pair by the instructor. The researcher found no significant difference in the performance of the two types of groups. He did note, however, that students in ownchoice pairs stated that they enjoyed chemistry more than the assigned pairs. Aside from this study, there does not appear to be other studies discussing the effect of placement on the effectiveness of collaborative learning. Chemistry Readiness Assessment Exam Prior to the fall semester of 2002, there were two general chemistry tracks being offered at the University of Florida. In the first track, which was two semesters long, students initially took CHM 2045 with its associated laboratory class, CHM 2045L. After attaining a grade of C or better, most students took CHM 2046 with its laboratory class, CHM 2046L. The laboratories must be taken simultaneously with the corresponding class. In the second track, there were three semesters of general chemistry. Students first took CHM 2040, which does not have a corresponding laboratory class. After attaining a grade of C or better, the students took CHM 2041 with its associated laboratory class, CHM 2045L. Again, after attaining a grade of C or better, most students took CHM 2046 with CHM 2046L. The longer track split CHM 2045 into two semesters, spending more time on the foundation material for those students without an appropriate math and/or chemistry background. All students, regardless of their initial track, took CHM 2046/2046L if they continued on in chemistry. The advantage to the second track included a slower pace for learning the foundation material that was expanded upon in later classes. The disadvantage to this track was that students spent a longer time in general chemistry, which placed them a semester out of sequence for the rest of their chemistry classes. Students were initially placed into either CHM 2040 or CHM 2045 based on their SAT II scores. Students who achieved a certain score or higher were placed into CHM 2045 while those below that score were placed into CHM 2040. The Chemistry Department at the University of Florida (UF) stopped using this method of evaluation after it was determined that students' SAT II scores did not directly correlate to their achievement in chemistry. After this method was abandoned, the students decided which track to take based upon the evaluations of their advisors as well as by word of mouth advice from other students. Some advisors continued to use the discarded SAT II score method for placement. A significant number of students began the twosemester sequence but then had to drop back to CHM 2040 after the first exam. There were also students who did extremely well in CHM 2040, receiving all or most of the points available to them. These students could easily have done well in CHM 2045. In the fall of 2002, the course offerings for general chemistry were revised. All students now take CHM 2045 with CHM 2045L, followed by CHM 2046 with CHM 2046L. For those students without the appropriate chemistry and/or math background, a twocredit introductory class called CHM 1025 replaced the CHM 2040/2041 sequence. After students complete CHM 1025 with a grade of S (satisfactory), they take CHM 2045 followed by CHM 2046, with the appropriate laboratories, to finish their chemistry requirement. In order for this new system to be effective, a practical method was needed to place students into the appropriate class. There are a large number of students entering UF each year who expect to take general chemistry. The method used to place these students into the appropriate class must be both easily administered and easily scored to be of any use. In light of this, a voluntary webbased exam was administered to all students who took CHM 2040 in the fall of 2001 and the spring of 2002. The exam was administered at the beginning of the semester. This exam was used to determine a correlation between the preliminary exam scores and students' performance in the CHM 2040 class. The exam was modified into the Chemistry Readiness Assessment Exam (ChRA). The ChRA is a webbased exam and should be taken before the students arrive at UF. The University requires that all incoming first year students have a computer, and offers assistance in the purchase of one to those students with demonstrated financial need. This requirement, together with the fact that there are several computer clusters on campus that are open to students, ensures that all students will have access to a computer in order to take the preliminary exam. The students who perform well on the preliminary exam are registered for CHM 2045, while those who do not perform as well are registered for the introductory CHM 1025 class. Another goal of this project is to further refine the cutoff scores used for placement into general chemistry. Student achievement in CHM 2045 has been compared to ChRA scores to determine a more accurate cutoff score. CHAPTER 2 ANALYSIS METHODS AND PROCEDURES This chapter is comprised of three sections. The first section describes the calculation method used for the statistical tests performed during the course of this study. The second section describes the collaborative and noncollaborative sections of CHM 1025 during the fall semester of 2002, the spring, summer and fall semesters of 2003, and the spring semester of 2004. The third section discusses the ChRA and the suggested minimum requirements for admittance into CHM 2045. Analysis of Variance Analysis of Variance (ANOVA) is a statistical test designed to judge whether or not differences in sample means are significant.46 This calculation is based on both the null hypothesis and the test hypothesis. The null hypothesis assumes that all the means being tested are equal. The test hypothesis assumes that at least one sample mean is different from the others. These hypotheses are the same for all ANOVA calculations. The ANOVA calculation is based upon several premises. First, all the populations being tested have a bellshaped distribution, also called a normal distribution. Second, the sample variations (the square of the standard deviations) are approximately equal. Finally, the samples being compared must be independent from each other. An ANOVA calculation determines the F probability distribution for the samples. The F probability distribution, also called the F value, is an asymmetrical distribution based upon the degrees of freedom in the sample (Figure 21). Only a small percentage Probabilityp Figure 21: F probability distribution of the samples should fall above a certain value on this curve. The level of significance of the test, called the pvalue, is based upon the F distribution. The pvalue is the probability of obtaining a value of F that is as likely or more likely to reject the null hypothesis as the actual observed value of the test. The pvalue is based upon the idea that the null hypothesis is correct. If the pvalue for an ANOVA calculation is less than 0.05 (5.0%) the null hypothesis is usually rejected as incorrect, and the test hypothesis is determined to be correct. If the pvalue is higher than 0.05, the null hypothesis is considered correct. Another way of stating the pvalue would be that we are 95% confident that the samples are significantly different. In order to determine the F value in an ANOVA calculation, the degrees of freedom, sum of squares and mean square must be calculated for the samples (Table 21). Degrees of freedom (DF) determine how many measurements can vary in a sample. There are two types of degrees of freedom calculated in an ANOVA calculation. The Table 21: Sample ANOVA table Source Degrees of Sum of Mean F Test Freedom Squares Square Between t1 SSB s Samples Within n t SSW s Samples Total n, 1 TSS first type of degree of freedom denotes the freedom between samples and is given by DF = t 1, where t is the number of samples. The second type of degree of freedom denotes the freedom within the samples, given by DF = n, t, where n, = total sample size. These two degrees of freedom are summed into a total DFt = n, 1. There are three different sums of square calculations in the ANOVA calculation. The first is the withinsample sum of squares (SSW). This determines the variance within each sample. SSW= (y,,y,) = (ni1)s + (n21)s. +...+(n,1)s,2 where y, = thejth sample observation selected from population i, y, = the average of the n, sample observations drawn from population i, and is given by y, = and s = n, standard deviation of population i. The second type of sum of squares is the sum of squares between samples (SSB). This determines the variance between each sample. SSB = n(y, y)2 where n, is the number of sample observations obtained from population i, and y is the average of all sample observations, given by y = The final type is the total n, sum of squares (TSS) which determines the variance for the overall mean. TSS = (y, y)2 = (n )s2 ,=1 /=1 where s = standard deviation of the nt measurements y,y There are two types of mean squares, the mean square between samples and the mean square within samples. The mean squares are averages of squared deviations, and are usually calculated by dividing the sum of squares by its degree of freedom. The betweensample mean square is denoted as sB. 2 SSB t The withinsample mean square is denoted sW . 2 SSW ssw n t The F value is determined by dividing these two mean squares by each other. 2 F = _ 2 If the means have been found to be significantly different, either a Fisher's Least Significant Difference (LSD) test or a Tukey's W Procedure is run to determine which mean or means are different from the rest. The least significant difference is the smallest difference that can be observed between two sample means where the means are still statistically different. The LSD test often has a high error rate between experiments, but can often be used to get a general idea of which means are different. LSD, = to sj, + Hi H if y, y, 1> LSD,, then Pi and Ri are declared different, where tx is the mean for the sample x. Tukey's W Procedure is a much more conservative test than Fisher's LSD test. Tukey's controls the experimentwise error rate, and thus fewer means are declared different than with the LSD procedure. This test lowers the probability of declaring one pair of means significantly different that are not different when running multiple comparisons. Of the two, Tukey's W Procedure has been used more often than the LSD test. W = q, (t,v) Fn if I y, yj 12 W then gli and pj are declared different, where t = degrees of freedom between samples, v = degrees of freedom within samples, and qa= uppertail critical value of the F distribution. All ANOVA calculations, LSD calculations, and Tukey's calculations were run using SAS software version 8 for Windows.47 Collaborative Learning Fall, 2002 During the fall semester of 2002, five sections of CHM 1025 were offered, taught by three different instructors. One instructor taught one section of CHM 1025 (section 0675), and the other two instructors each taught two sections (sections 0676, 5253 and sections 0677, 4284). Students in CHM 1025 were not given a letter grade, but were instead graded on a Satisfactory/Unsatisfactory (S/U) basis. Statistics for each section are presented in Table 22. Table 22: CHM 1025 in fall 2002 Section Instructor # students # students Final # students Pass Rate enrolled withdrew Enrollment receiving S 0675 MV 298 19 279 237 84.9% 0676 KA 304 20 284 237 83.5% 0677 RB 298 24 274 240 87.6% 4284 RB 298 11 287 241 84.0% 5253 KA 219 18 201 152 75.6% All five sections were taught in Chemistry Laboratory Building (CLB) room C130, a lecturestyle auditorium that seats 297 students (Figure 22). The chairs are stationary and face towards a demonstration desk. There are four moveable chalkboards in the front of the room, plus a projector connected to a computer located in the front of the room. Infrared sensors, which line the walls of the auditorium, are also connected to the computer. The class met twice a week for 50 minutes. The five sections used common exams, jointly written by the three instructors. Exams were multiplechoice, usually comprised of 15 tenpoint questions. They were administered at night in rooms scattered throughout UF's campus and proctored by the instructors and the teaching assistants. There were four exams offered during the term and a final exam. The final exam was worth 200 points. The lowest of the four semester exams was dropped. All five sections were graded on the same scale, with 610 points out of 1000 possible needed to pass the class. At the beginning of the term, students were told that the maximum number of points that they needed to pass the class was 650 points out of a possible 1000. Many students reached this mark after the fourth exam, at which point they stopped coming to class. Those students that stopped attending also did not take the cumulative final. Each instructor approached collaborative learning in a different manner during this semester. All instructors were assigned peer mentors. In section 0675, students were Lab Demonstration Table Figure 22: CLB C130 allowed to choose their own groups of six students at the beginning of the term. Students either chose to join a group that included their friends or chose a group based upon their location. There were not enough peer mentors for each group to have their own mentor, so the peer mentors roamed assigned sections of C 130 and helped the groups in that area. The instructor lectured for a short period of time, then asked his students a question based upon the material just covered. All peer mentors were supplied with answers to the problems prior to the class session. After the problem was solved, the cycle was repeated. These problems were placed on a side overhead projector, and had five answer choices. The students worked with their group to solve the problems and each student entered an answer using the HITT remote controls (Figure 23). The HITT remote controls send signals to the infrared sensors. The sensors record both the students' unique identification numbers and the answer chosen by the student, and transmit this information to the computer. If a group experienced difficulty solving a problem, either the instructor or a nearby peer mentor would assist them. Figure 23: HITT remote control48 In section 0676, students were randomly placed into groups of six by the instructor prior to the second class meeting. Students were then told to sit with their groups for the rest of the term. These groups were balanced by gender, with no group having more males than females, although several groups had more females than males. In this section, the instructor lectured for approximately 1525 minutes, depending upon the topic being covered. The remainder of the 50 minute class period was spent working collaboratively on worksheets containing approximately 10 problems each, based on the material just covered. As in section 0675, there were not enough peer mentors to staff each group, so peer mentors patrolled certain areas of the classroom during the collaborative problemsolving sessions to assist students. The students were also asked HITT quiz questions during the collaborative session. In section 5253, taught by the same instructor as 0676, students were placed into groups of six based on achievement as determined by the students' SAT grades. The SAT grades were obtained from the Admissions Office at UF after the first class meeting. Each group had highachieving, middleachieving, and lowachieving students. The groups were rearranged after three of the four semester exams using the previous exam score as the new criterion for group placement. This section also had a brief lecture period, similar to the one in section 0676, followed by a collaborative problemsolving session assisted by peer mentors patrolling certain areas of the classroom. These students also had HITT quiz questions during the collaborative part of the class. The third instructor taught sections 0677 and 4284. This instructor lectured for a certain period of time, then asked a HITT quiz question. The students were not allowed to work together to solve these questions, and were considered to be cheating if they attempted to collaborate. Peer mentors who attempted to assist students were told to desist and were publicly chastised for talking in class. The peer mentors were expected to act as police and ensure that no students were cheating during the HITT quizzes. Eventually most peer mentors stopped attending these sections. Spring, 2003 There were two sections of CHM 1025 taught in the spring semester of 2003, both taught by the same instructor. Information about the sections is given in Table 23. Table 23: CHM 1025 in spring 2003 Section Instructor # students # students Final # students Pass Rate enrolled withdrew Enrollment receiving S 5267 KA 47 10 37 20 54.0% 5269 KA 126 21 105 67 63.8% Both sections received the same examinations. The semester exams were essay type examinations worth 150 points. The final exam was worth 200 points. The lowest of the four semester exams was dropped. The exams were administered at night in rooms throughout UF's campus and proctored by the instructor, TA and peer mentors. There were four exams and a final exam. Both sections were graded on the same scale, with 610 points out of 1000 possible needed to pass the class. As in the fall of 2002, students were told that they needed 650 points out of a possible 1000 to pass the class. In order to maintain student attendance past the fourth exam, students were told that the cumulative final exam for the course was mandatory. Students had to receive a minimum of 50% on the final in order to pass the class regardless of the number of points they had. There was also a poor performance forgiveness policy. Students who received 75% or greater on the cumulative final exam passed the class even if they had not reached the 650 point cutoff. Only one student took advantage of this policy. Section 5267 was placed into groups of six by location during the second class meeting. No attempt was made to balance these groups by gender. These groups remained constant throughout the semester. This section originally met in CLB C130, but due to its small size and early meeting time it was soon moved to the Chemistry Learning Center (CLC), a room with moveable tables and chairs (Figure 24). Students in section 5269 were placed into groups of six randomly by the instructor and were informed of their new groups during the second class meeting. The groups remained constant throughout the term. Most of this section met in CLB C130 for the entire semester. Four of the groups volunteered to meet in the CLC with two experienced peer mentors, one of whom was a senior chemistry major. The peer mentors were responsible for this minisection, giving lectures to the students based on the instructor's notes. Both sections met twice a week for 50 minutes. Sections 5267 and 5269, as well as the minisection of 5269, were all taught in the same manner. A brief lecture was given at the beginning of the class period that lasted from 15 to 25 minutes. After the lecture, students worked in their groups to solve problems based upon the material just covered. Peer mentors were assigned individual groups and got to know their students well during the term. The students also worked together on HITT quiz questions during the collaborative session. Group types were compared using the ANOVA table to determine if group type affected achievement or attendance. Summer, 2003 There was one section of CHM 1025 taught during the summer semester of 2003 (taught by KA). Section 4216 had an initial enrollment of 170 students, of whom four withdrew. Of these, 143 students received an S (86.1% pass rate). This section met four days per week for an hour and fifteen minutes over a period of six weeks in CLB C130. A N White Board A Figure 24: Chemistry Learning Center Section 4216 was given three long answer exams during class and a cumulative final examination on the last day of class. These exams were worth 150 points each, with the final worth 200 points. As in the spring 2003 semester, students were told that the final was mandatory and that a minimum of 50% was required on the exam in order to pass the class. The poor performance forgiveness policy was also offered in the summer 2003 semester. Students needed a total of 625 points out of a possible 1000 in order to pass the class. Students were placed into groups of six randomly by the instructor and were informed of their groups during the second class meeting. The individual class meetings were run in one of two ways. The first type of session involved a 30minute collaborative problemsolving session sandwiched between two brief 20minute lectures. The problems solved during the collaborative session were based on the first lecture given during the class. The second type of class had a 20minute lecture sandwiched by two 30minute collaborative problemsolving sessions. The first collaborative session had problems based on the second lecture from the previous day. The second collaborative session had problems based on the lecture immediately preceding it. Students were given HITT quiz questions during all collaborative learning sessions. Fall, 2003 There were five sections of CHM 1025 during the fall semester of 2003, taught by four different instructors. The same instructor taught section 0675 and 0677. Information about the sections is given in Table 24. Table 24: CHM 1025 in fall 2003 Section Instructor # students # students Final # students Pass Rate enrolled withdrew Enrollment receiving S 0675 LL 257 27 230 196 85.2% 0676 KW 303 37 266 202 75.9% 0677 LL 303 23 280 245 87.5 % 4284 KZ 297 10 287 234 81.5% 5253 KA 300 10 290 237 81.7% All five sections were taught in CLB C130 and met twice a week for 50 minutes. All sections took the same examinations. There were four multiplechoice exams during the semester, usually with 15 questions worth 10 points apiece. The lowest semester exam was dropped. There was also a cumulative final given during exam week. As in past semesters, this final was mandatory, and students were expected to receive a minimum of 50% on the final in order to pass the class. All exams were written jointly by the four instructors. However, at the end of the term, the grading scheme for the sections diverged. Three of the sections, 0675, 0676, and 0677, were graded on the same scale. These sections required 600 points out of a possible 1000 to pass the class and all students were given 88 points to help them towards the goal of 600 points. The students were given the points because these sections had no HITT quizzes. Essentially, a student only needed 512 points out of a possible 1000, or 51.2% of the available points, to pass in these sections. The other two sections, 4284 and 5253, were graded on a different scale. A student in these sections needed 620 points out of a possible 1000 to pass the class, and these students received no free points, since HITT quizzes were given during the class sessions. Sections 0675, 0676, and 0677, although taught by two different instructors, were taught in a similar manner. These instructors lectured for the whole class period and held no collaborative problemsolving sessions. Peer mentors were present in these classrooms, and had specific areas of the classroom to patrol. However, the peer mentors did not have an opportunity to assist students with problem solving. Peer mentors were mainly required to pass out inclass quizzes given weekly and soon stopped attending. Sections 4284 and 5253 were taught in a similar manner, although these sections also had two different instructors. These instructors lectured for 15 to 25 minutes, depending upon the topic, and then held collaborative problemsolving sessions during the remainder of the class periods. Students were randomly assigned to groups of six or seven by the instructors and informed of their groups during the second class meeting. Peer mentors patrolled certain areas of the classroom and assisted groups during the problemsolving sessions. As in prior semesters, the peer mentors were given the answers to the problems prior to the class meeting. HITT quizzes were also given during these collaborative sessions. Spring, 2004 There were two sections of CHM 1025 taught during the spring semester of 2004. A different instructor taught each section. Both sections met in CLB C130 twice a week for 50 minutes. Information about the sections is given in Table 25. Peer mentors were assigned individual groups and got to know their students well over the course of the semester. Both sections were comprised of a brief 15 to 25 minute lecture followed by a collaborative problemsolving session. Students were given HITT quizzes during the collaborative sessions. Table 25: CHM 1025 in spring 2004 Section Instructor # students # students Final # students Pass rate enrolled withdrew enrollment receiving S 5267 KZ 98 11 87 44 50.6% 5269 KA 154 13 141 107 75.9% The main difference between spring 2004 and prior semesters lay in the examinations. There were four multiplechoice examinations given during the semester, worth 150 points apiece. The lowest semester exam grade was dropped. There was also a mandatory cumulative final worth 200 points. The exams were written jointly by both instructors. The exams were administered at night in rooms dispersed throughout UF's campus and proctored by the instructors and peer mentors. Each exam had two sections, an individual section and a group section. The individual section was worth 70% of the exam points, and the group section was worth 30% of the exam points. For the first hour of the two hour exam, students sat with their groups and worked collaboratively on the group portion of the exam. At the end of the hour, the scantrons were collected and the individual exams were passed out. The students then worked alone on this second portion of the exam. For the first exam of the four semester exams, the individual and group portions were switched, with the indivjdtfal section taken first. Half of the questions on the group portion of the exam were taken from the individual portion, so students had an opportunity to rework the problems with their groups. However, students did not like having to wait quietly for all students to finish the individual portion. They also did not redo the repeated problems but instead chose the answer that the majority of students in the group had gotten for these problems. Therefore, the order was switched, with the group portion coming first for the remainder of the exams. This allowed the students to talk with each other while waiting for all the groups to finish. Once the sections were switched, however, the same problems no longer appeared on both portions of the test. Calculations The grades of all students who took CHM 2045 and/or CHM 2046 during the 2002/2003 school year as well as the fall semester of 2003 were obtained from the UF Admissions Office. It was also noted whether or not the student had taken CHM 1025 and passed. The students who had taken CHM 2045 were split into two groups based upon whether or not they had taken CHM 1025. The achievement of these groups of students was compared using an ANOVA table. Also, whether or not having taken CHM 1025 affected student retention in CHM 2045 was determined. The students who had taken CHM 2046 in the fall semester of 2003 were split into two groups based upon whether or not they had taken CHM 1025. The achievement of these groups of students was also compared. In addition, whether or not having taken CHM 1025 affected student retention in CHM 2046 was compared. Attendance data was available only for certain sections of CHM 1025. The attendance in these sections was compared to achievement in CHM 1025 using an ANOVA table. The sections involved in this calculation were 4284 and 5253 from the fall semester of 2003 and 5267 and 5269 from the spring semester of 2004. Chemistry Readiness Assessment Exam Students were administered a voluntary, webbased preliminary exam at the beginning of the fall 2001 and spring 2002 semesters. The exam consisted of eight math questions and 16 chemistry questions, and covered material in the first eight chapters of the textbook.49 The preliminary exam results were compared to the final course grade for the fall of 2001. This preliminary exam was modified into the Chemistry Readiness Assessment Exam (ChRA). The ChRA was placed online for prospective freshman chemistry students to take before arriving at UF. Based upon the results of the exam, students were advised to take either CHM 1025 or CHM 2045 beginning in the fall term of 2002. Students who received a score of four or better on the math portion of the exam and six or better on the chemistry portion of the exam were advised to take CHM 2045. Those students receiving less than these scores were advised to take CHM 1025. These cutoff scores were not absolute, however, and many students took CHM 2045 that did not meet the minimum ChRA requirements. For those students who did not 31 take CHM 1025, the grade received in CHM 2045 was compared to the results of the ChRA to further refine the cutoff scores. CHAPTER 3 STATISTICAL RESULTS Effectiveness of CHM 1025 The curriculum in the Introduction to Chemistry course (CHM 1025) was designed for those students who are underprepared for the mainstream general chemistry course (CHM 2045). Therefore, it was important to determine whether there was a difference in student achievement in CHM 2045 between those students who took the introductory course and those students who did not. In the fall of 2003, two CHM 2045 classes (taught by KA and MV) were given a quiz in the first week of the class based upon the material that students were expected to know before taking the course. The last question on the quiz allowed students to self identify as having taken the introductory course. The average score on the quiz for those students who selfidentified as having taken CHM 1025 was 5.36 out of 10.00. This was significantly different than the lower 4.41 score obtained by those students who had not taken CHM 1025 (pvalue <0.0001). This enhanced achievement in CHM 2045 for those students who had taken CHM 1025 continued throughout the course. The grades of all students who had taken CHM 2045 in the spring of 2003, summer of 2003 and fall of 2003 were compared for level of achievement. The students were split into three groups: those who had not taken CHM 1025; those who had taken CHM 1025 and passed; and those who had taken CHM 1025 and failed. The grade point averages (GPA) of these students were compared to determine the effect CHM 1025 had on achievement. On a 4.00 GPA scale, an A corresponds to a 4.00, a B+ to a 3.50, a B to a 3.00, etc. The students who had not taken CHM 1025 had an average GPA of 2.12 in CHM 2045. This is significantly lower than the 2.74 GPA of those students who had passed CHM 1025 (pvalue <0.0001). Those students who either failed or withdrew from CHM 1025 had an extremely low 0.59 GPA in CHM 2045. This abysmal success rate for students who failed or withdrew from CHM 1025 demonstrates that these students must not be allowed to enroll in CHM 2045. Once CHM 1025 is attempted, it must be passed before advancing to CHM 2045. Another measure of how well CHM 1025 prepares students (that does not rely on student grades) is to determine how many of these students continued on to the second semester of general chemistry (CHM 2046). Of the 1609 students who took CHM 2045 during the spring and summer semesters of 2003, 953 (59.2%) students earned a grade of C or better and were eligible to continue on to CHM 2046. Only 579 (60.8%) of these students had taken CHM 2046 by the fall semester of 2003. Of the 579 students that continued on to CHM 2046, 394 (68.0%) had originally taken CHM 1025. Students who had taken CHM 1025 were much more likely than their counterparts to continue on to CHM 2046 if they were eligible to do so. In these two semesters, 656 students took CHM 2045 but were ineligible to continue to CHM 2046. These ineligible students were approximately evenly split between former CHM 1025 students and nonCHM 1025 students (48.6% to 51.4%). Although there is no data available for what other chemistry classes were taken by the students who passed CHM 2046, it is possible to determine who was able to continue on in chemistry. There were 579 students enrolled in CHM 2046 over the summer and fall semesters of 2003. Of these, 471 (81.3%) passed with a grade of C or better. Of these 471 passing students, 327 (69.4%) were originally enrolled in CHM 1025. Former CHM 1025 students were much more likely to pass CHM 2046 than students who did not take the course. The fall semester of the school year has by far the highest enrollment of CHM 1025 students of all the semesters. The majority of these students that are eligible will take CHM 2045 the following spring. As a result, the spring semester is especially indicative of the performance of former CHM 1025 students in CHM 2045. The spring semester of 2003 is particularly telling, as there were approximately equal numbers of nonCHM 1025 students and former CHM 1025 students enrolled in CHM 2045. Of the 1408 students initially enrolled in CHM 2045 that semester, 846 of them had taken CHM 1025 (60.0%). Of the 829 students who passed CHM 2045 in the spring of 2003, 557 students had taken CHM 1025 (67.2%). Of the 579 students who failed or withdrew from CHM 2045 that semester, 287 (49.5%) had taken CHM 1025. Although these statistics suggest that students who pass CHM 2045 are more likely to be former CHM 1025 students, the statistics on withdrawals and failures is disheartening. Former CHM 1025 students are failing CHM 2045 at the same rate as students who did not need the introductory course. One of the possible methods that can be used to remedy this failure rate is to slow down in CHM 1025 and spend more time on fundamentals such as nomenclature and stoichiometry. Following the enrollment trend, the semester with the largest enrollment of former CHM 1025 students enrolled in CHM 2046 was the fall semester of 2003. Again, this semester is looked at individually because the numbers of both types of students is approximately equal. Of the 724 students initially enrolled in CHM 2046 this term, 348 had taken CHM 1025 (48.1%). Of the 607 students that passed CHM 2046, 300 of them had taken CHM 1025 (49.4%). Of the 117 students who failed or withdrew from the course, only 48 had taken CHM 1025 (41.0%). The pass rate of both the nonCHM 1025 students and the former CHM 1025 students during this semester is approximately the same. The failure rate of former CHM 1025 students during this semester is slightly lower than those students who did not take CHM 1025. Students who required an introductory course in chemistry are able to succeed just as well in CHM 2046 as those students who did not require it. The GPA scores of students enrolled in CHM 2045 in the spring of 2003 were compared to determine if CHM 1025 had an effect on the graded outcome of the class. Students who had taken CHM 1025 had an average GPA of 2.18 in CHM 2045 during the spring semester of 2003. This was significantly different than the average 1.59 GPA of the students who did not have CHM 1025 (pvalue <0.0001). The 2.18 GPA equates to a letter grade of a C, while the 1.59 GPA equates to a letter grade of D+. Students must earn at least a C or better in order to continue on to CHM 2046. Former CHM 1025 students performed better during this term as noted by their passing average GPA. The average GPA scores noted above include all students who had enrolled in CHM 2045 that semester, including those that failed the course. When only passing students are considered in the calculation of average GPA, the scores increase. Although the former CHM 1025 students still have a higher average GPA (3.18 versus 3.09), the scores are no longer significantly different (pvalue 0.0959). This again suggests that former CHM 1025 students have higher grades overall in CHM 2045, even for those students who fail the course. The GPA scores for students enrolled in CHM 2046 during the fall semester of 2003 were also compared to determine if CHM 1025 affected the graded outcome of that course. The average GPA for former CHM 1025 students enrolled in CHM 2046 was 3.01. Although higher than the average 2.86 GPA obtained by the students who had not taken CHM 1025, the scores were not significantly different (pvalue 0.1235). When only those students who passed were considered, the GPA of the former CHM 1025 students was only slightly higher than the nonCHM 1025 students (3.42 versus 3.39), and these scores were also not significantly different (pvalue 0.5639). However, CHM 1025 students are succeeding in CHM 2046 even though their initial chemistry backgrounds were not as strong as the students who did not need CHM 1025. Each semester there are students who take either CHM 2045 or CHM 2046 a second time, due to either a failure or a withdrawal in the course. These students were studied to determine whether or not CHM 1025 affected the numbers of students who had to repeat either CHM 2045 or CHM 2046. Over the course of this study, 439 students took CHM 2045 twice. Of those students, only 124 (28.2%) had taken CHM 1025 prior to their initial enrollment in CHM 2045. Additionally, 256 students repeated CHM 2046, with only 19 (7.4%) having taken CHM 1025. The low number of former CHM 1025 students who had to repeat one of these two courses demonstrates that the CHM 1025 students are better prepared to succeed in later chemistry classes on the first attempt. This is a desirable outcome because students who need to repeat a course end up behind in their majors, which can cause difficulty later in their college careers. Collaborative Learning In order to determine if collaborative learning affected achievement in CHM 1025, students' grades in that course were compared for the fall 2002 semester. There were five sections of CHM 1025 that semester: section 0675 had studentchosen groups; section 0676 had randomlychosen groups; section 5253 had achievementassigned groups; and sections 0677 and 4284 did not use collaborative learning. This was the first semester that collaborative learning was attempted on such a large scale. CHM 1025 is an S/U class, with an S being equivalent to a 2.00 or above on the 4.00 GPA scale, and a U is less than a 2.00 on the GPA scale, but is considered a 0.00 in the calculations. There was no significant difference in the achievement level of students as denoted by final grades in either the studentchosen, random, or noncollaborative classes (Table 31). However, all three of these types of classes had a higher achievement level than the students who were placed into groups by achievement (pvalue 0.0089). Clearly the collaborative effort hinders students when they are placed into groups using either SAT achievement or testscores as a method of placement. It is also disappointing to note that collaborative learning had no effect on student achievement over the non collaborative classes during the fall semester of 2002. However, with experience, collaborative learning was more effective by the fall semester of 2003. Table 31: GPA in CHM 1025 based on group type Section Instructor Group Type CHM 1025 GPA 0675 MV Student chosen 1.59 0676 KA Random 1.56 0677, 4284 RB None 1.62 5253 KA Achievement 1.39 During the fall semester of 2003, students were placed randomly into groups in sections 4284 and 5253, while sections 0675, 0676, and 0677 did not use collaborative learning. Students in the collaborative sections had a higher achievement level denoted by final grade than students in the noncollaborative sections (pvalue 0.0296). Students in the collaborative sections had an average GPA of 1.59, while the noncollaborative sections had a GPA of 1.49. Collaborative learning helped the students in sections 4284 and 5253 enjoy greater success in CHM 1025. Unfortunately, it is impossible to determine if this greater success carried into CHM 2045 or CHM 2046 because these students had not completed these courses by the end of this study. The students who enrolled in CHM 1025 in the fall semester of 2002 were the only students who were able to complete the entire general chemistry sequence (CHM 1025, CHM 2045 and CHM 2046) by the time this study ended. The number of students from each type of group that continued on to CHM 2045 and CHM 2046 were compared to determine if collaborative learning had an effect on student willingness to continue in chemistry. A slightly higher percentage of students from the three collaborative sections that were eligible to continue on the CHM 2045 and CHM 2046 did so when compared to the noncollaborative sections (Tables 32 and 33). Although collaborative learning had no effect on achievement during this semester, the collaborative students were slightly more likely to continue with chemistry than their noncollaborative counterparts. Table 32: Number of students continuing to CHM 2045 based on group type Section Instructor Group Type Students # in CHM % in CHM passing 2045 2045 0675 MV student chosen 238 186 78.2% 0676 KA Random 237 194 81.9% 0677 RB None 240 185 77.0% 4284 RB None 241 176 73.0% 5253 KA Achievement 152 122 80.3% Table 33: Number of students continuing to CHM 2046 based on group type Section Instructor Group Type Students # in CHM % in CHM passing 2046 2046 0675 MV student chosen 238 88 37.0% 0676 KA Random 237 88 37.1% 0677 RB None 240 75 31.3% 4284 RB None 241 76 31.5% 5253 KA Achievement 152 54 35.5% Student achievement in CHM 2045 and CHM 2046 was compared to the students' original section in CHM 1025 to determine if collaborative learning had an effect on student success in later classes. Although there was no significant difference in GPA scores for the students from the five fall 2002 and the two spring 2003 sections in CHM 2045 (pvalue 0.2049), the average scores for the students from the collaborative sections were slightly higher (Table 34). There was also no significant difference in average GPA scores for the students from these seven sections in CHM 2046 (pvalue 0.9735). It is interesting to note that section 5267, the section that met in a room that was easily rearranged and thus better for collaborative learning groups, had the highest average GPA scores of all seven sections in both CHM 2045 and CHM 2046. Room configuration is known to have an effect on the outcome of collaborative learning, and this idea is further confirmed by these grades. Table 34: Scores in CHM 2045 and CHM 2046 based on CHM 1025 section Section Semester Instructor 2045 GPA 2046 GPA 0675 Fall 2002 MV 2.08 2.88 0676 Fall 2002 KA 2.28 3.00 0677 Fall 2002 RB 2.07 2.99 4284 Fall 2002 RB 2.17 2.93 5253 Fall 2002 KA 2.38 2.91 5267 Spring 2003 KA 2.61 3.38 5269 Spring 2003 KA 2.21 2.60 Attendance Colloquially, it has always been known that students must attend class in order to succeed in the course. Getting the students to believe this simple fact is another matter altogether. Sections 4284 and 5253 from the fall of 2003 and sections 5267 and 5269 from the spring of 2004 had over 20 class meetings during their respective semesters (see Figure 31). The large dips on this graph come the day after an exam. Students' achievement in these classes was compared to their attendance as recorded by their H ITT remote controls. The students were split into four groups by number of classes attended: 0 5 classes attended; 6 10 classes attended; 11 16 classes attended; and >17 classes attended. The first group corresponds to students who attended less than 25% of the class meetings, the second group attended less than 50% of the time, the third  4284 a5253 5267 w 5269 100.00 90.00 70.00 60.00 50.00 40.00 30.00 . 0 5 10 15 20 25 Figure 31: Percent student attendance by class meeting in CHM 1025 group attended less than 75% of the time, and the fourth group attended most of the classes. As is expected, there is a significant difference in scores earned by the students who fall into these groups (pvalue <0.0001). Students who attend less than 25% of the time receive an average of 300 of the 1000 available points, earning a very low E. Students who attend less than 50% of the time receive an average of 521 of the available points, earning a high E. Students who attend less than 75% of the time receive an average of 589 of the available points, earning a D. Students who attend greater than 75% of the classes receive an average 758 of the available points, earning a B. As Figure 32 shows, only those students who attend class at least threequarters of the time can be expected to pass the class. It is important to recognize that students know exactly how 005 D610 1116 1722 90 80 70 60 50 40 30 20 A B+ B C+ C D+ D E Figure 32: Percent of students receiving a grade in CHM 1025 by number of classes attended many points they must earn to pass this S/U class. Most students earn this bare minimum number of points, and do not work one bit harder than they absolutely have to. Any students who earn over 650 points are doing extremely well in the class, and doing the amount of work that will help them succeed in CHM 2045. Students' attendance was compared to the type of collaborative group to determine if a particular group type affected attendance. Attendance data was only available for those students who were placed into groups randomly or by achievement. Students who were placed randomly into groups attended class an average of 83.3% of the time. This is significantly different from the students who were placed into groups by achievement (pvalue <0.0001). These students attended class only 68.2% of the time. As shown above, students who attend less than 75% of the time are likely to receive a D. This further demonstrates that placing students into groups by achievement is not an effective method of increasing student success. Chemistry Readiness Assessment Exam Most of the students who enrolled in either CHM 1025 or CHM 2045 beginning in the fall of 2002 took the Chemistry Readiness Assessment Exam (ChRA) before enrollment. The students were merely required to take the exam while there were guidelines, there were no absolute scores that directed students into either course. However, most students followed the suggested guidelines. Those students who received a score of four or greater out of a possible eight on the math portion and a score of nine or greater out of a possible 18 on the chemistry portion were advised to take CHM 2045. Those students who received scores below this level were advised to take CHM 1025. These levels were not absolute, and many students enrolled in whichever course they thought best suited their needs. The ChRA scores for all students who had taken CHM 2045 in the fall of 2002, the spring of 2003, and the summer of 2003 and who had not taken CHM 1025 were obtained from the Office of Admissions at UF. Students' achievement in CHM 2045 was compared to their ChRA scores. Students who received a score of four on the math portion of the ChRA had an average GPA of 1.85. This average GPA is not significantly different than the average 1.65 GPA of students who correctly answered less than four questions on the ChRA (pvalue 0.1717). The average GPA of students who received a four on the math portion converts to a D+ on the letter scale. Because students must receive a C or better in CHM 2045 in order to proceed to CHM 2046, this cutoff is unacceptable for the math portion of the ChRA. Students who received a five on the math portion of the ChRA had an average GPA of 2.13. This average GPA is significantly different than the average 1.76 GPA of the students who received less than five (pvalue 0.0006). While students who passed the math section of the ChRA with a score of five have an average grade of a C in CHM 2045, this is an extremely low C. Those students who received a six on the math portion of the ChRA had an average GPA of 2.41 in CHM 2045. This is significantly different than the average 1.95 GPA of the students who answered less than six questions correctly (pvalue <0.0001). This C grade is a much stronger one than the one received by students who earn a five on the math section. Therefore, the recommended cutoff for the ChRA math section is a score of six out of eight possible questions (Figure 33). 04 M5 M6 5 A B+ B C+ C D+ D E W Figure 33: Percent of students receiving each grade in CHM 2045 by number of correct ChRA math questions The chemistry score of the ChRA was determined to be not as crucial an indicator of performance in CHM 2045 as the math score. The initial cutoff score for the chemistry portion was nine out of 18 questions. However, students who received this score had an average GPA of 2.32 in CHM 2045. This was not significantly different than the average 2.06 GPA of the students who received less than a nine on the chemistry portion (pvalue 0.0109). This GPA is equivalent to a C, so this cutoff score can be lowered. Students who received a score of eight on the chemistry portion of the ChRA had an average GPA of 2.28 in CHM 1025. This was significantly different than the average 1.98 GPA of students who received less than this score (pvalue 0.0083). However this is still a reasonable C grade, so the minimum cutoff score can be lowered even further. Students who received a score of seven on the chemistry portion of the ChRA had an average GPA of 2.10. This is not significantly different than the average 1.95 GPA of the students who received less than a score of seven (pvalue 0.4993). This is a passing GPA, but it is only barely a C. However, the students who received a score of six on the ChRA had an average GPA of 2.30, a higher score than those students who earned a seven on the ChRA. The average 2.30 GPA of students who received a six on the ChRA is significantly different than the average 1.76 GPA of the students who received less than six (pvalue 0.0003). The students who received a score of five on the ChRA had an average GPA of 2.13 in CHM 2045. This is significantly different than the average 1.56 GPA of the students who received less than that score (pvalue 0.0038). Although a passing GPA, 2.13 is not a strong C grade. The average GPA of students who received a four on the chemistry portion of the ChRA is below passing (1.62). Therefore, the recommended cutoff for the chemistry portion of the ChRA is six out of a possible 18 (Figure 34). Although the cutoff values are the same, students must receive a 75% on the math portion of the ChRA and a 33% on the chemistry portion in order to succeed in CHM 2045. These results demonstrate that student success in beginning chemistry relies more on prior math knowledge than on prior chemistry knowledge. Therefore, although many students' high school chemistry backgrounds are poor, those with a strong math background can still succeed in chemistry. 05 06 E7 E8 09 A B+ B C+ C D+ D E W Figure 34: Percent of students receiving each grade in CHM 2045 by number of correct ChRA chemistry questions As shown in Figure 35, students who receive a six on both the math and the chemistry portions of the ChRA are more likely to get a grade of A in CHM 2045 than their counterparts. Those students who fail both sections of the ChRA are much more likely to withdraw from the class than were their classmates. The students who pass only one section of the ChRA have mixed results, with those students who passed only the math section performing better in CHM 2045 than the students who passed only the chemistry portion of the exam. 0 fail both Opass chem pass math U pass both A B+ B C+ C D+ D E W Figure 35: Percent of grades in CHM 2045 based on ChRA score CHAPTER 4 COLLABORATIVE TESTING Testing usually occurs in a traditional format. Students use a pencil and paper to take the exam. They usually work individually on the exam, and are seated to reduce the possibility of cheating. Any students working together are considered to be cheating. The penalties for cheating are severe, ranging from failing the examination to failing the course, or even expulsion from the university. Students are usually very concerned about the exams, and are anxious to do well on them, often to the point of asking, "Is this going to be on the exam?" A high anxiety level has been shown to decrease student performance. Using a testing system that is collaborative in nature can help to decrease student anxiety and increase student understanding of the subject material. Only a handful of studies have looked at collaborative testing as an alternative to traditional testing methods.5056 Most of these studies paired students, rather than using a larger group of four to six students. Also, few of these studies actually set up the exam in such a fashion that it had to be worked on collaboratively. Although many of the studies state that they used collaborative testing, few researchers actually wrote an exam where at least a portion of it was intended to be taken collaboratively. Muir and Tracy used paired testing in a course for preservice teachers at Oakland University.50 The students worked in studentchosen pairs on essaytype examinations, worth 25% of the students' total grade. The students were allowed to choose their own pairs to decrease teacher responsibility if students blamed their partner for a low exam score. Muir and Tracy found that test scores increased by 7.0% on the midterm examination, but only by 1.0% on the final examination. They do not state whether these results were significant. Muir and Tracy noted that student anxiety level was decreased, and students spent more time preparing for the examinations. However, they do not state how they came to the conclusions of lower student anxiety and longer preparation time. From a survey of students in their classes, they determined that 97% of the partners worked equally on the exam, with only 3% of the partners stating that one member was not exerting as much effort as the other member. Two aspects of collaborative testing that were noted as important by these researchers were that the desks and chairs must move in order to promote the collaborative effort and noise level must be monitored so that it does not interfere with student concentration. Muir and Tracy did not state whether students turned in individual copies of the exam, or only turned in one set of answers per pair. Another study conducted with nurses at Lake Superior State University assigned students randomly to pairs on the day of the test.51 The students did not work together prior to the exam. The researchers believed that this placement method would encourage all students to study and prevent one member of a group from carrying the pair. Students worked individually for most of the exam, and were only allowed to collaborate for the final third of the exam time. The partners did not have to agree on an answer and each member of the pair handed in an answer set. The students who compared answers had an equivalent comprehension of material as students who did not have the option to do so. Student achievement only increased slightly, from an average GPA of 3.24 in the non collaborative setting to an average of 3.27 in the paired setting. Students noted a decrease in anxiety in the setting where students were allowed to compare answers. A study at Stanford University used a partner testing system in a large, 300 student introductory psychology class.52 Students in this class were allowed to choose whether or not to take the test with a selfchosen partner. Those students that chose to work with a partner submitted only one set of answers per pair. The students in this course had to take the first midterm exam alone, and then had the option of taking the second and third midterm exams with a partner. Although 62% of the students chose to take the second test with a partner, only 48% chose to take the third exam collaboratively. By the second year of this study, only 30% of the students were choosing to take the exam collaboratively. The authors did not discuss why the numbers of students choosing collaborative testing decreased. The Stanford University study noted an increase in achievement of approximately nine percentage points in the partner exam situation over the noncollaborative situation. They also noted that low scores were eliminated in the collaborative setting. Students involved in this study that chose to work in pairs noted a reduced anxiety level both during testing and when studying prior to the exam. Unfortunately, the students involved in this study thought that their classmates were more likely to expend minimal effort when tested collaboratively. In a study conducted among medical students at Wayne State University School of Medicine, the students were assigned to pairs on the day of a quiz.53 Again, these students did not work with their partners at any other time during the course. The students took a quiz individually, and then took the same quiz again with a group. The individual portion of the quiz counted for 80% of the students' grades. The group portion accounted for the remaining 20% of the quiz grade. The students had a 6% increase in overall quiz scores. The score on the group portion was higher than that of the individual portion. The students involved in this study stated that they liked the collaborative format better than the traditional quiz format, but no mention was made of their anxiety levels. The only study that made use of a group larger than pairs was in a nonmajors statistics class at Colorado State University.54 Students were placed into groups in a random fashion, with the caveat that each group of four students would contain at least two females. Students were assigned to their groups on the day of their exam, and did not work with their groups at any other time or for any other exam. Each group handed in only one set of answers. There was a large increase in student achievement on the exams from previous semesters (approximately 15 percentage points). However, these students had to take an individual final, and the scores dropped dramatically. Not only were the final scores almost 30% lower than the collaborative exams, they were approximately 7% lower than the students who had never taken a collaborative exam. The author contributed this decrease in score to the lack of conscientious studying habits in his students. However, this attitude would have been apparent in all the exam scores. The students involved in this statistics course were in favor of collaborative testing. No mention was made of student anxiety levels. At least two additional studies discussed what the authors termed collaborative testing but were not similar to the collaborative testing formats described above.55,56 In one study performed in chemistry at several universities, including Florida State University, students watched a demonstration in class and were given exam questions to take home that had to be turned in fortyeight hours later. Students had the ability to work together if they wished, but did not have to, nor was the collaborative process monitored in any way. The researchers stated that their version of collaborative testing was effective because students in a comparable section of chemistry that did not have the takehome exams based on demonstrations were unable to answer questions on the final exam related to demonstrations they had not seen in class. This outcome is to be expected students should not be expected to answer questions on something that they had not seen or been taught. In the second study, students in a composition course at Widener University were allowed to freeform talk with each other during an exam on grammar skills and works cited pages. The collaborative process was not monitored and students were allowed to work with whomever they wished. Students were not required to work together, but the students who worked individually were seated in the same noisy environment as their collaborative counterparts. There was a slight increase in student achievement in this course, but no mention was made of student anxiety level. The studies discussed above noted an increase in student achievement, although the increases can be considered modest. But most of the studies omitted aspects of collaborative testing that could be important, such as handing in individual or group answers and student anxiety levels. Therefore collaborative testing was implemented in CHM 1025 during the spring semester of 2004 to test some of these aspects. Prior to this semester, students had worked together in groups during the class period. Once at the exam, however, students were told that collaborating was cheating, and were strongly discouraged from doing so. In many respects, this was unfair to the students. If collaboration is expected in the class, it should be carried into the situation where the students are evaluated on their progress. All exams given in CHM 1025 during the spring 2004 semester, including the final exam, were partially collaborative in nature. There were two sections to each exam, an individual section and a group section. The group section was administered during the first hour of the twohour exam block and was worth 30% of the exam grade. The individual section was administered during the second hour of the exam and was worth 70% of the exam grade. Questions were different on each section of the exam. The group section of the exam had harder questions than were asked to the students individually. It was believed that students would be better equipped to answer more difficult questions if they were working with a group than if they worked alone. The students worked in the same collaborative groups that they worked with every class period. The rationale behind the consistent grouping was that students would be more comfortable working with people they already knew in an otherwise tense exam situation. The first exam offered during the course was arranged in the reverse order, with the individual section offered first, and the group section offered second. On this exam, half of the questions on the group section were the same as on the individual section. It was believed that students would rework these problems in their groups and perhaps arrive at different answers than on the individual section. However, students merely recalled what they had chosen on the individual section and answered with a "majority rules" approach. Students also did not like waiting for all students to finish the individual portion before beginning the group section. In a survey given at the end of the semester, nearly 70% of the students stated that they did not like having the individual section of the exam first. After this first exam, the individual and group sections were switched, and no questions were repeated. The students still had to wait before beginning the individual section of the exam, but they were now able to talk with their group members while waiting. Several groups used this time to discuss concepts that members were having difficulty with. The grades on the collaborative exams were compared to grades on non collaborative exams given in the spring semester of 2003 to determine how the type of exam affected student achievement. Spring of 2003 was chosen as the comparison semester because the class size was approximately the same and the classes were comprised of similar types of students. Students who enroll in CHM 1025 in the spring semester are usually a semester behind in their majors. A comparison was not made to the fall semester because the class sizes are extremely different (approximately 250 versus approximately 1500) and the fall students are usually ontrack with their majors. On the first of four semester exams, the students who took the exam collaboratively earned an average of 116 points out of 150. This was significantly higher than the average 96 points earned by the spring 2003 students (pvalue <0.0001). The range of results on the collaborative exam also narrowed, with no students scoring below 55 points (Table 41). These results were promising, especially since the instructors considered the collaborative portion of the exam more difficult than exams offered in previous semesters. Table 41: Performance on exam 1 Term Instructor(s) Exam Average Standard Range Deviation Spring 2003 KA Individual 96 26.3 25142 Spring 2004 KA & KZ Collaborative 116 18.6 55150 The percent of students in each section that earned each letter grade was also compared (Figure 41). Overall, students in the collaborative sections had a much higher percentage of A and B grades than in the noncollaborative sections. As indicated by the decreased standard deviation spread, the collaborative section also had a much lower percentage of E grades than the noncollaborative section. The spring 2004 section also had a skewed bell curve distribution of grades, centered around a B+. The non collaborative section had the usual asymmetrical distribution of grades. *S03 S04 40 35 30 25 20 15  10  5 0 A B C D E Figure 41: Percent of students earning each grade on exam 1 By exam two, however, these promising initial results were reversed. As shown in Table 42, there was no significant difference in the grades earned in the spring 2003 and spring 2004 sections (pvalue 0.9008). However, the range was again narrowed, with the lowest scores being eliminated from the spring 2004 section. The standard deviation of the average scores was also narrowed, most likely due to the lack of extremely low grades. Table 42: Performance on exam 2 Term Instructor(s) Exam Average Standard Range Deviation Spring 2003 KA Individual 104 34 0150 Spring 2004 KA & KZ Collaborative 105 20 30150 The percent of students in the collaborative and noncollaborative sections that earned each letter grade was again compared (Figure 42). The collaborative section had 35 S03 S04 30 25  20  15  10  5 0 A B C D E Figure 42: Percent of students earning each grade on exam 2 fewer students receiving A and B grades than did the noncollaborative section. The collaborative section earned a higher percentage of C and D grades than the non collaborative section, but fewer E grades. Because the lower grades were eliminated, the students that would normally be earning E grades instead earned D grades, bringing that percentage up in the collaborative section. However, the skewed bellshaped distribution is not present in the collaborative section. Both sections now have the usual asymmetrical distribution of grades. Exam three in the collaborative section was an unusually difficult exam. The instructors in the spring semester of 2004 wrote a more elaborate exam than in 2003, and multiple steps were needed to solve the problems. This high degree of difficulty was reflected in the grades for this exam in the collaborative section. However, even with this increased level of difficulty for the collaborative exam, the average scores were slightly higher than those of the spring 2003 semester (Table 43). However, the difference was not significant (pvalue 0.6564). The range does not appear to be narrowed in the collaborative sections as in the other sections, however the student that earned only five points was a repeat student from the fall of 2003. This student had already attempted and failed CHM 1025. The student receiving the next lowest grade in the collaborative section earned 25 points on the exam. The majority of the lower scores were again eliminated on exam three. Table 43: Performance on exam 3 Term Instructor(s) Exam Average Standard Range Deviation Spring 2003 KA Individual 80 35 3150 Spring 2004 KA & KZ Collaborative 82 24 5150 The percentages of students receiving each grade were compared for differences in achievement between the two sections (Figure 43). The high difficulty level of exam three was reflected in the numbers of students earning each grade. Very few students earned an A grade, and the majority of the students earned an E grade. However, excluding the percentage of students that earned an A, the distribution of grades is not significantly different from the spring of 2003, which did not have as difficult of an exam. B S03 N S04 0  B C Figure 43: Percent of students earning each grade on exam 3 By the fourth semester exam, although the collaborative exam was more difficult than the noncollaborative exams, it was not as difficult as exam three. The student averages reflected this, and the average 107 points out of 150 possible earned by the collaborative section was significantly higher than the average 88 points earned by the noncollaborative section (pvalue <0.0001). As shown in Table 44, the ranges and the standard deviations were again narrowed, due to the elimination of the lower grades. Table 44: Performance on exam 4 Term Instructor(s) Exam Average Standard Range Deviation Spring 2003 KA Individual 88 33 19150 Spring 2004 KA & KZ Collaborative 106 24 50150 Students in the collaborative section earned more A, B and C grades than in the noncollaborative sections (Figure 44). The collaborative students also earned fewer D and E grades than did the noncollaborative students. Although both curves are asymmetrical, the distribution of grades is what was expected when collaborative testing was implemented. OS03 IS04 45 40 35 30 25 20 15  10 0 A B C D E Figure 44: Percent of students earning each grade on exam 4 However, by the time of the mandatory cumulative final, there was a decrease in student performance on the exam. As shown in Table 45, the students in the non collaborative section had a significantly higher 115 points out of a possible 200 than the average 106 points earned in the collaborative section (pvalue 0.0280). The range of scores in both the collaborative and noncollaborative sections was also approximately equal. However, the standard deviation in the collaborative section was again narrowed. It is important to note that students are told that they must earn 100 points on the final in order to pass the class. Those students who have already reached or are near the minimum cutoff for passing usually aim for exactly 100 points. This minimum grade is an E on the exam, thus the high number of E grades in both sections. Table 45: Performance on the final exam Term Instructor(s) Exam Average Standard Range Deviation Spring 2003 KA Individual 115 35 30190 Spring 2004 KA & KZ Collaborative 106 30 20190 Students in the collaborative section earned fewer passing grades on the final exam than did students in the noncollaborative section (Figure 45). They also earned more failing D and E grades than did the noncollaborative students. Again, this performance on the final is disappointing. However, starting in the summer of 2004, CHM 1025 will no longer be graded on an S/U basis and will be given a letter grade. The final, as well as all the other exams, is expected to become much more important to students when they will be receiving a letter grade for the course. There was one distinct advantage to the collaborative final noted by the instructors. Each semester, there are a few students who do not earn the minimum grade on the final but who would otherwise pass the class. These students either come in to the *S03 S04 50 45 40 35 30 25 20 15 10 A B C D E Figure 45: Percent of students earning each grade on the final exam instructors' offices or email the instructors pleading for points. The number of students who requested extra points in order to pass the class decreased markedly from prior semesters. The collaborative exams possibly made students more comfortable with the grades they had earned. Another benefit to the collaborative exam format is that the percent of students taking the exams increased over prior semesters. Generally, the largest numbers of students take exam one, and then the numbers decrease, occasionally increasing again for the final. Although the numbers did still decrease, the percent of students taking each exam was higher in spring 2004 versus spring 2003 (Table 46). The percentages of each student are based on initial, and not final, enrollments. More students were kept in the class in the collaborative setting than in the noncollaborative setting. Table 46: Percent of students taking each exam Term Exam Exam 1 Exam 2 Exam 3 Exam 4 Final Spring 2003 Individual 83.2% 85.6% 73.4% 59.5% 63.0% Spring 2004 Collaborative 91.7% 90.1% 83.3% 76.6% 74.2% In order to determine if students were performing better on the individual or group portion of the collaborative exams, the grades on each portion for section 5269 in the spring of 2004 were normalized to 100%. These percentages were then compared to determine if a difference in achievement level existed between the two sections of the exam. As shown in Table 47, there was no difference between the collaborative and individual sections of the first exam (pvalue 0.7954). Students performed equally as well on both portions of the exam. This demonstrates that students can answer more challenging questions as a group with as much accuracy as they are able to answer easier individual questions. When the percentage scores on the individual portion of the exam were compared to the percentage scores on the exam given in the spring semester of 2003, students who took the exam collaboratively had a significantly higher 77.2% average than the average 64.2% earned last spring (pvalue <0.0001). Students were better prepared for the exam during the semester it was offered collaboratively than when it was offered individually. Table 47: Performance on each section of the exam Section Exam 1 Exam 2 Exam 3 Exam 4 Final Individual 77.2% 60.5% 51.7% 57.4% 49.3% Group 78.2% 93.2% 50.2% 100% 68.0% On the second of the four semester exams, there was a significant difference between the average 93.2% earned on the collaborative portion of the exam and the average 60.5% earned on the individual section of the exam (pvalue <0.0001). The students were able to answer the more difficult collaborative questions with greater accuracy than the easier individual questions. However, when the individual scores were compared with the test scores from the spring 2003 semester, the students were less prepared for the individual section of the exam. The students in the spring of 2003 earned an average 69.6% on the exam. This was significantly higher than the average 60.5% earned on the individual portion in the spring of 2004 (pvalue <0.0001). After experiencing the first collaborative exam, students did not prepare as well for the second exam. However, this lack of preparation was discussed in class, and student achievement improved on the next two exams. On the most difficult exam, exam three, the students performed equally as well on both sections of the exam, earning an average 51.7% on the individual section, and an average 50.2% on the collaborative section (pvalue 0.7158). Again, this demonstrates that students can answer more challenging questions as a group with as much accuracy as they are able to answer easier individual questions. When the scores on the individual section are compared to the scores on exam three during the spring of 2003, the students in both semesters performed equally as well (pvalue 0.3571). On exam three in the spring of 2003, students earned an average of 53.9%. During the spring of 2004, students earned an average 51.7% on the individual section of the exam. The students who took the exam collaboratively were as prepared as were the students who took an individual exam during the spring of 2003. For the fourth of the semester exams, students' scores were significantly different on the collaborative and individual sections (pvalue <0.0001). The students earned an average 57.4% on the individual portion, and over 100% on the collaborative portion, due to the extra credit question on that section. Students correctly answered virtually every difficult question on the collaborative section of this exam. The students were also as prepared for the exam as the spring 2003 students were, earning essentially the same percentage on the individual section of the exam (pvalue 0.7407). The students in the spring of 2003 earned an average 58.4%, while the students in the spring of 2004 earned an average 57.4%. For the final exam, the students earned a higher 68.0% average on the group section of the exam, versus an average 49.3% on the individual section of the exam (p value <0.0001). This final was cumulative, therefore students were expected to answer more difficult questions on the collaborative section, tying several concepts that they had learned over the course together into one question. The students did this admirably, earning almost 70% on this section. However, most students were aiming for the minimum 50% that was required of them on the final, and this was reflected in their individual scores. Most students felt confident after taking the group portion of the exam, and did not try as hard on the individual portion of the exam. This is further reflected when the individual grades are compared to the exam grades in the spring of 2003. Students in the spring of 2004 earned an average 49.3 % on the individual section of the exam. This was significantly lower than the average 57.4% earned during the spring of 2003 (pvalue 0.0006). Overall, students were able to answer the more difficult questions with more accuracy when working in a group. The students had more problems with the individual section, even though those questions were not as difficult as the questions they had answered beforehand. For at least three of the five exams offered during the course, the students were at least as prepared as their counterparts from a year earlier. Using collaborative exams, students are able to answer harder questions and perform as well as expected on the individual sections. During the spring semester of 2004, eight students required a makeup exam for varying reasons. These students were a randomsampling of students from the two sections being offered that semester. They were given the same final as the students in the spring of 2003, with the noted exception that the students in 2004 had the exam split into collaborative and individual sections. Although eight students is not a large number, the results on this exam were compared to the section 5267 in spring 2003. Only 29 students took the exam in that section, so the numbers are roughly comparable when compared to the 80 students that took the final in section 5269 in the spring of 2003. The makeup students in the spring of 2004 had a significantly higher average of 134 out of a possible 200 than the 104 average of the spring 2003 students (pvalue 0.0281). As shown in Figure 46, students in the spring of 2004 had a higher percentage of A and C grades, and had approximately equal percent of B grades. Concerning the failing grades, although the students in the spring of 2004 had a higher percentage of D grades, there were no E grades for these students. Those E grades were brought up to D grades. The range and standard deviation of the scores in the spring 2004 section were also narrowed due to the elimination of the lowest grades (Table 48). Table 48: Grades on the makeup final Term Instructor(s) Exam Average Standard Range Deviation Spring 2003 KA Individual 104 36 40170 Spring 2004 KA & KZ Collaborative 134 20 110170 SS03 O S04 60  50  40  30  20  10  A B C D E Figure 46: Percent of students earning each grade on the makeup final The percent scores on the individual section of the makeup final in the spring of 2004 were compared to the percent scores on the final given in spring 2003. Students taking the makeup earned an average of 49.1% on the individual section of that exam. This was not significantly different than the 51.8% earned by the students in the spring of 2003 (pvalue 0.6942). Students in the makeup section were as prepared as their counterparts of the semester before for the individual section of the exam, but were able to perform much better on the group section. The difficulty of the questions remained the same between the two exams. Therefore students who took a collaborative exam had a higher achievement level than the students who took a noncollaborative exam. A study performed at the University of North Carolina at Charlotte has shown that 80% of the students who failed their first exam in the first semester general chemistry class failed the course.4 With this in mind, the failure rates of the students that failed the first exam in the spring of 2003 and the spring of 2004 were compared to determine if collaborative learning had an effect in this regard. During the spring semester of 2003, 88% of the students who failed the first exam failed the course. During the spring of 2004, 70% of the students who failed the first exam failed the course. The percent of students who failed the course after failing the first exam is decreased in the collaborative section. Collaborative testing can help increase the achievement of students who would otherwise fail the course. Students were given an anonymous survey regarding collaborative testing on the last day of classes during the spring semester of 2004. The first two questions on this survey were about student anxiety levels. As shown in Figure 47, the majority of U Studying M Exam 80% 70% 60% 50% 40% 30% 20% 10% 0% Reduced No Effect Increased Figure 47: Student reported anxiety levels when studying and when taking the collaborative exams students that responded believed that collaborative testing reduced their anxiety both when studying and during the actual exam. As discussed earlier, high anxiety levels lead to decreased student performance on examinations. As the collaborative method of testing reduced studentreported anxiety, even though the questions were more difficult, the students are more likely to perform better on these exams than on individual exams. The students were also asked if they would have preferred to take the entire exam individually. The majority of the students (68%) stated that they preferred the collaborative exam to an individual exam. Only 11% of the students stated that they would have preferred individual exams. The remaining 21% of the students wanted the ability to choose to take an exam either collaboratively or individually. Realistically, this would be virtually impossible to achieve with the current test design of more difficult Questions on the collaborative section of the exam. The students who opted to take the exam individually would most likely be unable to answer these difficult questions, and their achievement levels would drop considerably. One concern when implementing collaborative testing is that students will study less when they know they have other students to rely on during the exam. This is one reason behind the percent values given to the two portions of the exam it was believed that students would continue to study if 70% of their grade depended upon their own work. This was borne out in the students' response to a question on the survey about studying. Approximately 80% of the students reported that they studied for the collaborative exam about the same as they would have studied for an individual exam. As an unexpected result, 11% of the students reported that they studied more for the exam because it was collaborative. Perhaps this is due to the peer pressure of performing in a group, and not wanting to let one's groupmates down. Only 9% of the students reported that they decreased their usual amount of studying because the exams were collaborative. It is promising that students are studying the same amount as they usually would for the collaborative exams. This reduces the fear that students will not prepare as well when they are able to collaborate during an exam. Another problem that arises when collaborative testing is discussed is social loafing. Many students and instructors believe that when students work together there will be one or a few students who do all the work, while the rest of the group does nothing. These students would get the same grade as those students who worked hard and studied for the exam. This was another reason behind the percentage values given to both exam sections. There was no way that a student could abstain from working on the exam and still pass. When asked, only 23% of the students stated that they would have removed someone from their group for social loafing. Another 18% were unsure if they would have removed someone for not participating. The majority of the students (60%) stated that social loafing was not a problem in their group. When asked how the groups solved problems during the exam, the majority (59%) of the students responded that the questions were split up among group members. An additional 38% stated that every student in the group did every question. When group members arrived at different answers to the questions, these differences were solved with a mixture of a majority rules approach and using the answer of the student who had the best logic. Only 3% of the class (four students) stated that only one person answered all the questions. These four students were most likely in the same group. These results demonstrate that students are not expecting one member to do all the work, nor are some students relying on the results of their groupmates. A promising result from this survey was that 70% of the students believed that collaborative testing increased their grade. Perhaps because the questions were more difficult, students believed that they performed better working with a group than they would have if they had worked individually. Overall, with these promising survey results, the fact that more students are taking the exams, and the elimination of the lowest grades, collaborative testing should be continued in later semesters of this course. Collaborative testing has proven to be an effective assessment method in CHM 1025, and is fairer to the students who are expected to work collaboratively during class meetings. CHAPTER 5 PEER MENTORS As student enrollment in higher education increases and funding decreases, class sizes have increased dramatically. A professor facing 300 students often feels there is no choice but to lecture. The only active role that a student can take in a lecture format class is to take notes. As discussed earlier, collaborative learning is one way to transition from the traditional lecture into a more studentactive learning environment. Another method to increase student involvement is the use of peer mentors. Peer mentors are undergraduates of similar age and educational background as the students enrolled in the course who are recruited to help teach the students.57 After 45 years of research on universities, Newcomb stated that professors have little effect on the study habits and curriculum choices of their students.58 In general, the student's peer group has the highest influence on the student. Newcomb notes that there is little similarity between the interests that peer groups generate and support and the academic objectives supported by a university. Using peer mentors in a classroom can generate peer support for academic objectives of the university. Peer mentors can stress that students should spend more time on studying and homework. Although this message is no different from the professor's, Newcomb has shown that it will have more effect coming from peers. Peer mentors can help students take a more active approach to their own learning. In a traditional lecture class in chemistry, students meet once a week in a small recitation section, headed by a teaching assistant (TA). The purpose of the smaller class is to give the students an opportunity to ask questions in a small group without the professor present. However, it has been shown that discussion sections function mainly as small lectures, and the students are no more active in these types of sessions than in the traditional lecture.59 A peer mentor, on the other hand, works closely with students, both asking questions and offering information on a more informal basis than either a lecture or a discussion section. This giveandtake with a peer mentor helps students take a more active role in the learning process. Robin and Heselton have discussed three aspects of a good peer mentor.60 A peer mentor engages students in casual conversation in greeting and in closing a session. Although this can be related to events outside the class such as a recent football game, it can include generalizations about the course or an upcoming test. Casual conversation is important since it can highlight differences between the peer mentor and the instructor for the course. The instructor is often viewed as more of an authority figure (i.e. someone who would never ask about the football game) than the peer mentor, who is often viewed as more of a friend. A good peer mentor offers feedback to the students. This feedback should be personal and related directly to the student's progress in the course. The mentor can offer feedback on graded items, such as quizzes or exams, but also on verbal explanations or defenses of answers given by the students. In the introductory chemistry class, for instance, peer mentors should offer feedback on dimensional analysis as well as logic used in problem solving. The third aspect of a good peer mentor, as described by Robin and Heselton, is praising the student's performance. Peer mentors should offer immediate and positive feedback to the students whenever possible. This helps maintain students' morale and interest in the course. This aspect of a good peer mentor is truly an extension of the feedback aspect described above. Although other researchers agree upon these three aspects of a good peer mentor, often two additional aspects are described. A good peer mentor should also prompt the students.61 These prompts should consist of verbal clues that help students answer their own questions. The peer mentors should not simply provide answers. Instead, they should help students arrive at their own answers. The fifth aspect of a good peer mentor is assisting the instructor with administrative tasks.62 These administrative tasks can include proctoring and grading exams, handing out and collecting material from students, and gathering feedback about the course from the students. An important aspect of a good peer mentor that does not appear to be described in the literature is that the peer mentor should have a thorough knowledge of course content and material. Although the mentors are not expected to have the same indepth knowledge of subject material as the instructor, they must demonstrate a proficiency in the covered material. It would be difficult for a mentor to assist students with material that the mentor does not understand. This does not mean confining the mentor pool to subject majors. Often students in other majors are competently able to assist students with course content. In the introductory general chemistry class, only a very few peer mentors (approximately 7%) are chemistry majors. Most mentors are preprofessional or engineering majors with demonstrated proficiency in the course, although we have several business and finance majors who are excellent mentors. Several studies have been performed on the effectiveness of peer mentors versus staff instruction in small groups.6367 These studies were set up where the staff instructor was expected to act in a similar manner as the peer mentors. Students were assigned problems in small groups that were headed either by an instructor or a peer mentor. In most of these studies, the group leader was the only source of information in the class. Therefore, students in groups headed by a peer mentor did not meet with an instructor. The results of these studies were mixed, with results ranging from an increase in student achievement in the mentorled groups, no difference in achievement between the two types of groups, and increased student achievement in the instructorled groups. This mixture of results makes it difficult to draw conclusions about the effectiveness of the peer mentor method of instruction. Most of the studies noted positive side effects beyond the mixed student achievement in the mentorled groups. These studies noted that peer mentors were able to understand the nature of student difficulties with subjectmaterial better than the instructor, having recently gone through the course themselves. Peer mentors were also more interested in the daily lives of students, their study experiences, and student personalities. Peer mentors also directed students' attention to upcoming exams more than did the instructors. Additionally, peer mentors displayed more supportive behaviors than did the staff instructors. Although the quantitative academic benefits of peer mentors were originally not clear, peer mentors were introduced in CHM 1025 at the inception of the course. However, students did not meet solely with mentors. In CHM 1025 students meet with both an instructor and peer mentors every class session. For the first 15 to 25 minutes of the 50 minute class, the instructor delivers a brief lecture on the topic being covered. After the lecture, the students break into their small collaborative groups to solve problems based on the topic just covered. Peer mentors assist students with problem solving during the collaborative sessions. In this manner, students get the benefits of having an instructor impart background information, and also benefit by working closely with mentors who are able to assist them with difficult concepts or problems. Peer mentors are actively recruited at the end of every semester. Students who have passed the course and evidenced good helping behavior with their groupmates are asked to come back the following term and assist as a mentor. Current peer mentors usually know the students in their groups and have good suggestions for future mentors. These students do not have to be academically the best in the class. Some of our best mentors are students who struggled in the class but who demonstrated helping behavior in their groups. Mentors assist in the classroom for as many semesters as their schedules permit several have mentored every semester up until graduation. There were only six peer mentors during the first semester that this program was implemented. Now, over 100 students have volunteered to be mentors. Initially, the mentors were paid a stipend for helping in the class. The stipend money ran out after the second semester of the program. Now peer mentors enroll in a onecredit 4000 level chemistry class called Supervised Teaching. As an added incentive, the peer mentors work closely with an instructor who is then able to write an excellent recommendation based on the mentor's responsibility, leadership skills, helping behavior and teamwork. Most mentors begin the program during the second semester of their freshman year. For the first few years the mentors are enrolled in the program, the CHM 1025 instructor is the only professor who knows the mentors' names due to the large number of students at UF. The racial makeup of the peer mentors is diverse, although heavily weighted toward minorities and students of color, as is the CHM 1025 class. Many of the mentors speak a language other than English as their first language. Several mentors have been overheard explaining a concept in their native language to other nonnative English speaking students in the class. Often this added explanation in the student's native language makes the concept understandable to the student. Peer mentors attend a onehour training session during the first week of each semester. All peer mentors attend, including those who have previously participated in the program. During this session, peer mentors are taught how to get a whole group involved in problem solving, in large part by facilitating groupmember interactions. Peer mentors are informed that they are not expected to just give an answer to students. They should first check the student's logic and dimensional analysis setup. It is also stressed during this meeting that the peer mentors are not expected to have all the answers to the students' questions. The mentors are informed that "I don't know" is a perfectly acceptable response, as long as that mentor then finds someone who can answer the question for the student. After this training meeting, staff meetings are held every other week. At these staff meetings, peer mentors are given the answer keys to the problems that will be solved during the upcoming two weeks of class. Any issues that have arisen over the previous two weeks are also discussed. Experienced peer mentors are comfortable bringing up discussion points as early as the first meeting. As the new peer mentors see how well these ideas and criticisms are received, they begin bringing up issues as well. By the third or fourth meeting there is usually a good dialogue between the instructor and the mentors. Upcoming events such as exams and changes in curriculum are also discussed in these meetings. Students are much more likely to offer complaints and suggestions to the mentors during class than they are to offer them to the instructor. Therefore, it is important to have a time set aside when mentors can bring up their and the students' concerns. It is also helpful to discuss the reasoning behind policy decisions in the class with the mentors because they are able to relay this to the students when an unpopular decision is made. When the mentors understand the logic behind a decision, such as the minimum point requirement on the final, students are often more accepting of the policy than when the decision is only announced by the instructor. The peer mentors are also an excellent resource for suggestions about how the class can be changed for the better. Not only have they recently participated in the course, the students in the class are constantly offering suggestions to them. For example, the idea to reverse the group and individual portions of the collaborative exam was first brought up by a peer mentor, and the majority of the other mentors strongly agreed. It was believed that having the mentors in the class would help personalize the class for the students. Normally, the chemistry classes have one instructor and 297 students. With the mentors in the class, this ratio was brought down to one peer mentor for every six to 18 students, depending upon the semester. These mentors often know their students individually, and question the students on their progress as well as on their absences. It was believed that this individual attention would keep the students coming to class and learning the material. In order to test this assumption, the withdrawal rates of sections that used peer mentors were compared to sections that did not utilize them. During the fall of 2003, five sections of CHM 1025 were offered. Three of these sections did not utilize the peer mentors, while the other two sections did so extensively. As shown in Table 51, the withdrawal rates in the sections that did not use peer mentors was much higher than in the sections that did use them (pvalue <0.0001). Although it cannot be determined conclusively that the peer mentors affected the withdrawal rate, it is likely that having a peer mentor checking on students' progress helped lower this rate. Table 51: Percent of students withdrawing from each section during the fall of 2003 Section Instructor Peer # Students # Students Percent Mentors Enrolled Withdrew 0675 LL No 257 27 10.5% 0676 KW No 303 37 12.2% 0677 LL No 303 23 7.6% 4284 KZ Yes 297 10 3.4% 5253 KA Yes 300 10 3.3% The sections that utilized peer mentors during the fall of 2003 were held in the afternoon, while those sections that did not use the peer mentors were held during the first three 50minute periods of the day, beginning at 7:25 a.m. In order to ensure that the time of day did not skew withdrawal rates, the rates for fall of 2003 were compared to the rates for fall of 2002, where the classes were scheduled at exactly the same times. As shown in Table 52, the percent of students withdrawing from sections that did not utilize peer mentors was higher, regardless of the time of day the sections were held (pvalue 0.0823). Table 52: Withdrawal rates versus mentor utilization Fall 2003 Fall 2002 Mentors No Mentors No Mentors No Mentors (afternoon) (morning) (afternoon) (morning) # Enrolled 597 863 517 900 # Dropped 20 77 29 63 % Dropped 3.35% 8.92% 5.61% 7.00% In a voluntary, anonymous survey, students were asked to rate the helpfulness of the peer mentors. The majority of the students (74%) believed that the mentors were very helpful to have in class (Figure 51). An additional 17% thought the peer mentors were somewhat helpful. Only 9% of the students thought that the mentors were never helpful. The majority of students taking CHM 1025 believe that the peer mentors help them better understand the material. This is a very positive result. 80.00 70.00 60.00 50.00 40.00 30.00 20.00 10.00 0.00 very somewhat not at all Figure 51: Percent of students rating the helpfulness of peer mentors Although it is easy to see how the students benefit from having the peer mentors in class, it is important to recognize that the peer mentors also benefit from the program. The mentors have an opportunity to review the basic chemistry that they are expected to know in CHM 2045 and CHM 2046. Mentors who have progressed beyond this phase of chemistry have stated that mentoring has helped them review for such exams as the MCAT, DAT, and PCAT. The mentors also have a comfortable relationship with the CHM 1025 instructors, and have no problem asking for help in other chemistry subjects, such as organic and analytical chemistry. It is important to recognize that the peer mentors have mostly taken CHM 1025. In general, the mentors that are assisting in the classroom were underprepared for college chemistry when entering UF. These mentors are now much more confident in chemistry, and several have chosen to major in chemistry. Also, several mentors who took CHM 1025 in the summer of 2003, and CHM 2045 in the fall of that year took CHM 2051, an honors version of CHM 2046. Students who were initially under prepared have progressed far enough to take an honors chemistry class. When the CHM 2045 grades for students who have participated in the peer mentor program are compared to those students who are not in the program during the fall semester of 2003, there is a significant difference between these two groups (pvalue <0.0001). As shown in Table 53, the students who participated in the program have a higher average CHM 2045 grade of B+ versus the average grade of C for the students who have not participated in the program. When the peer mentor grades are compared to those students who also required the introductory course, the peer mentor grades are again significantly higher (pvalue <0.0001). Students who participate in the peer mentor program are more successful in later chemistry courses than students who do not participate in the program. Table 53: Mentor grades in CHM 2045 Mentors NonMentors Required Ready for Required CHM1025 CHM 2045 CHM 1025 GPA 3.71 2.32 2.14 Letter Grade B+ C C The mentors who are utilized in the CHM 1025 course have an overall positive experience with the program. Mentors comments from some of the required endof semester papers have included: There is something very special about helping people ... Sitting down with them and helping them to understand something that they couldn't before is just incredible. I brushed up on some of my basic chemistry and was able to help some students along the way. I liked being able to help students and at the same time let them know that there are people like them who can do chemistry. The students in the class were great to work with. I liked going to students who were struggling with a problem and helping them think it through and seeing them actually learn as they were getting the answer. It is a good feeling helping people gain knowledge. The mentoring program is an effective way to personalize the class for the students. The mentors help make the students more comfortable in the class, but the instructor still has the authority necessary to control the class. The mentors are able to help the students immediately with any questions and problems they have. In a large class of 300 students, the instructor would never be able to answer all the students' questions individually. However, the mentors provide students with an opportunity to ask questions in small groups, ensuring that students do not leave the class with unanswered questions. Confidence in the peer mentors among departmental instructors has also grown as the program has progressed. Initially, when peer mentors helped proctor exams some instructors in the department were skeptical. They did not like the idea of second semester freshman proctoring exams for their peers. As the mentors have proven themselves reliable and trustworthy this skepticism was replaced with an interest in the program. Instructors in CHM 2045 and CHM 2046, who have not used the peer mentors in their classes, now occasionally use the mentors to proctor exams when there are not enough graduate students available. During the spring semester of 2004, the department was short one TA in CHM 2046. An experienced peer mentor was asked to TA for this course, the first time that a nonchemistry major or minor has been a TA for a chemistry class. The mentoring program helps the peer mentors review basic chemistry and work on their social skills in the groups. Students who participate in the peer mentoring program also have higher GPA scores in later chemistry courses than their counterparts. Most students enrolled in CHM 1025 believe that the mentors helped them understand chemistry. Withdrawal rates also are lowered in the classes where mentors are checking on student progress and attendance. The program has had positive results both on the mentors and on the students, and should be continued in future CHM 1025 courses. CHAPTER 6 CONCLUSIONS Chemistry Readiness Assessment Exam The ChRA is the exam taken by potential chemistry students before enrollment in either the introductory CHM 1025 course or the mainstream CHM 2045 course. The initial cutoffs for this exam were four or greater (out of eight questions) correct on the math section of this exam, and nine or greater (out of 18 questions) correct on the chemistry section. Students should have earned 50% on both sections in order to take CHM 2045. These scores were only guidelines, and students were able to register for whichever course they wished. Students that earned only a four out of eight on the math section of the ChRA and did not take CHM 1025 earned an average grade of D+ in CHM 2045. In order to pass into CHM 2046, students must earn a grade of C, so this math cutoff is unacceptable. Students that correctly answered five out of eight on the math section earned an average grade of C. This C grade was extremely low. Students who correctly answered six out of eight questions on the math section also earned a C in CHM 2045, but this C grade was much stronger. Therefore, the minimum cutoff for the math section of the ChRA should be six out of eight questions. Students that correctly answered nine questions out of 18 on the chemistry portion earned an average grade of C in CHM 2045. This was a strong C, so the minimum cutoff for chemistry can be lowered. Students who correctly answered eight, seven, or six questions on the chemistry section of the ChRA also earned a C grade in CHM 2045. Students who correctly answered five of 18 on the chemistry portion earned an extremely low C in CHM 2045. Therefore, the minimum cutoff for the chemistry section of the ChRA should be six out of 18 questions. Although these cutoff values are the same, students must correctly answer 75% of the math section while only answering 33% of the chemistry section in order to succeed in general chemistry. Student success in chemistry depends more on students' prior math knowledge than their prior chemistry knowledge. Students who received a score of six on the math section and six on the chemistry section of the ChRA were more likely to receive an A in CHM 2045 than the students who did not receive these scores. Students who received less than a six on both math and chemistry were more likely to withdraw from CHM 2045 than their counterparts. Students who passed either the math section or the chemistry section of the exam had mixed grades, but the students who passed the math section performed better than the students who passed the chemistry section. Effectiveness of CHM 1025 CHM 1025 was designed for those students who are underprepared for the mainstream CHM 2045 course. Student achievement in CHM 2045 was compared to determine if CHM 1025 prepared these students for the mainstream course. The students who had taken CHM 1025 had an average GPA score of 2.74 in CHM 2045, while those students who had not taken CHM 1025 had an average GPA score of 2.12. Former CHM 1025 students are better equipped to succeed in CHM 2045 than the students who did not need the course. Students who failed CHM 1025 but enrolled in CHM 2045 anyway had an average GPA score of 0.59. Therefore, once CHM 1025 is attempted, it must be passed before the student can advance to CHM 2045. The average GPA 'scores of students in CHM 2046 that had taken CHM 1025 were approximately the same as those students that did not take the introductory course, and were about a B. Although these scores are not significantly different, it demonstrates that initially underprepared students are succeeding in the course at the same rate as their betterprepared classmates. The number of students who continued into CHM 2046 was compared to determine if CHM 1025 had an effect on this rate. The majority of the students (68.0%) that continued on to CHM 2046 in the fall semester of 2003 were initially enrolled in CHM 1025. Of the students that passed CHM 2046 and were eligible to continue in chemistry, the majority (69.4%) was initially enrolled in CHM 1025. Therefore, former CHM 1025 students are more likely to continue to CHM 2046, and are more likely to be eligible to continue into Organic than their counterparts. The students who had to repeat either CHM 2045 or CHM 2046 were also studied to determine if CHM 1025 had an effect on this rate. Only 28.2% of the students who had to repeat CHM 2045 had taken CHM 1025, and only 7.4% of the students who had to repeat CHM 2046 had taken CHM 1025. Therefore, CHM 1025 students are better prepared to succeed in later chemistry classes on the first attempt. Collaborative Learning During the fall semester of 2002, students were placed into groups in CHM 1025 in a random fashion, by studentchoice, or by achievement. There were two sections that did not use collaborative learning in this term. There was no difference in average final scores in the random, studentchoice, or noncollaborative classes. All three of these sections had a higher final GPA than the section grouped by achievement. Therefore, using student achievement to place students into groups is not an effective method of placement. The fact that collaborative learning did not have an effect on final GPA was disappointing, but by the fall of 2003 this was fixed. During the fall semester of 2003, there were two collaborative sections and three noncollaborative sections of CHM 1025. The students in the collaborative sections had a higher GPA score than in the noncollaborative section. Collaborative learning helped the students succeed in CHM 1025. Attendance Attendance rates in CHM 1025 were compared to determine if attendance had an effect on student achievement. Students who attended less than 25% of the time earned an extremely low E score in CHM 1025. Those who attended between 25% and 50% of the time earned a higher E score in CHM 1025. Students who attended between 50% and 75% of the time earned an average grade of D in CHM 1025. Students who attended more than 75% of the class sessions earned a B in the class. Only those students who attend class at least 75% of the time can be expected to pass. Group placement methods were compared to determine if one particular group type affected student attendance. Students who were placed randomly into groups attended class approximately 83% of the time. Those who were placed into groups by achievement attended class approximately only 68% of the time. This further demonstrates that placing students into groups by achievement is not an effective method of increasing student success. Collaborative Testing Prior studies that had used collaborative testing as an assessment method showed a modest increase in student achievement. Collaborative testing was implemented in CHM 1025 during the spring semester of 2004. The exams consisted of two sections, an individual section worth 70% of the student's grade, and a group section worth 30% of the student's grade. Students worked in the same groups during the exams as they worked with in class. The effect of collaborative testing on student achievement was mixed. On two of the exams (exam one and exam four), students had higher achievement on the collaborative exams than the noncollaborative exams offered in spring of 2003. On exams two and three, there was no difference in student achievement versus the prior semester. On the final exam, students who took the exam collaboratively performed worse than their counterparts of the year before. Although overall achievement was not conclusively affected by collaborative testing, there were some positive results to the study. The collaborative exams were more difficult, with more complex questions, than the exams from the spring of 2003. Generally, students performed just as well on these harder exams as they had on the easier individual exams. Students are better equipped to answer more difficult questions when in a group than when working individually. Also, the lowest scores were eliminated in the collaborative sections, as noted by the decreased ranges and standard deviations on the exams. Overall student achievement increases when collaborative exams are used. Finally, a greater percentage of students continue to take the exam when it is collaborative than when it is an individual effort. Students who took the collaborative exams stated that their testing anxiety was reduced both when taking the exam and when studying for it. Reduced anxiety levels has been shown to increase student performance. Students also stated that they preferred collaborative testing to individual testing, and studied for the collaborative tests about the same as they would have for an individual exam. Approximately 75% of the students stated that social loafing was not a problem in their group, and that all group members contributed to the exam. Collaborative testing is an effective assessment method and should be continued in CHM 1025. Peer Mentors Having peer mentors in the classroom helps personalize the class for the students, cutting down the studentinstructor ratio from 300 to one to about 12 to one. Students are more likely to ask questions in small groups and thus are more likely to understand difficult concepts with added explanations. The students also offer complaints and suggestions to the mentors, who are then able to relay this to the instructor. Approximately 75% of the students enrolled in CHM 1025 stated that peer mentors helped them understand the concepts covered in the class. The withdrawal rates for the CHM 1025 sections that used peer mentors in the fall of 2003 were much lower than in the sections that did not use them (approximately 3% versus approximately 10%). Although the difference in withdrawal rates cannot conclusively be attributed to the presence of peer mentors, having a mentor check on a student's progress and attendance certainly helps lower the rate. The peer mentors also benefit from participating in the program. The mentors have an opportunity to review basics that are used in later chemistry courses and that are 89 included in exams such as the MCAT and DAT. Students who participate in the peer mentoring program have a higher GPA in CHM 2045 than those students who are not in the program (3.71 versus 2.32). Overall, mentors enjoy the experience, and many continue to mentor long after they have stopped taking chemistry classes. APPENDIX A KEY TO INSTRUCTOR INITIALS KA .......................................................................................................... Katie Am aral KW .........................................................................................................K athryn W illiam s KZ ................................................................................................................ K eith Zientek LL ...................................... .. ..... ........... .................... ........................... Larry Land M V ................................................................................................................. M martin Vala RB .............................................................................................................C. Russ Bowers 