COLLABORATIVE LEARNING AND TESTING IN
INTRODUCTORY GENERAL CHEMISTRY
KATIE ELIZABETH AMARAL
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
I would like to thank Dr. Martin Vala for his encouragement and guidance during
these past few years. I would also like to thank Tim Young and Mark Sciallo in the
Admissions Office for collecting students' grades for me. Thanks are also extended to
Larry Margerum of the University of San Francisco for the idea to use collaborative
testing in the CHM 1025 class. Thanks go also to Jason Morris and Keith Zientek for
proofreading the document.
TABLE OF CONTENTS
LIST O F TA BLES......................................................................................................v
LIST O F FIG U R E S......................................................................................................... vii
A B STR A C T ................................................................................................................ix
1 INTRODUCTION.......................... ............ .........................
Collaborative Learning..................................................................................... 1
Chemistry Readiness Assessment Exam............................................................. 11
2 ANALYSIS METHODS AND PROCEDURES................................. ....... 14
A analysis of V ariance........................................................................................ 14
Collaborative Learning.......................... ......................... ............. .............18
F all, 2002........................................................................................................18
Spring, 2003............................................ ........................... ... .. 23
Summ er, 2003.......................................................... ... ....... 24
Fall, 2003................................................................................. ..............26
Chemistry Readiness Assessment Exam............................................... .......... 30
3 STATISTICAL RESULTS................................................... ....... 32
Effectiveness of CHM 1025...................................... .... ..................................32
A ttendance.................................................................................................... ....40
Chemistry Readiness Assessment Exam.........................................................42
4 COLLABORATIVE TESTING..................................... ................................48
5 PEER MENTORS........................................................................71
6 CONCLUSIONS........................................................................................... 83
Chem istry Readiness Assessment Exam .................................... .............................83
Effectiveness of CHM 1025................................................................................ 84
Collaborative Learning................................. ................. ..............................85
Collaborative Testing................................................... ............................... 87
Peer M entors................................................................................................. 88
A KEY TO INSTRUCTOR INITIALS ........................................ ..............................90
B COURSE OUTLINES .................................................................................. 91
CHM 1025..................................................................................................... 91
CHM 2045..................................................................................................... 92
REFERENCE LIST................................................................................................. 94
BIOGRAPHICAL SKETCH.................................................................................... 98
LIST OF TABLES
2-1 Sample ANOVA Table............................................................................... 16
2-2 CH M 1025 in fall 2002.................................................. .................................... 19
2-3 CHM 1025 in spring 2003.................................................................................. 23
2-4 CHM 1025 in fall 2003................................................................................. 26
2-5 CHM 1025 in spring 2004.................................................................................. 28
3-1 GPA in CHM 1025 based on group type...................................................... 37
3-2 Number of students continuing to CHM 2045 based on
3-3 Number of students continuing to CHM 2046 based on
3-4 Scores in CHM 2045 and CHM 2046 based on CHM 1025
4-1 Performance on exam 1.................................................................................55
4-2 Performance on exam 2...............................................................................56
4-3 Performance on exam 3............................................................................... 57
4-4 Performance on exam 4...............................................................................59
4-5 Performance on the final exam............................... ........................................60
4-6 Percent of students taking each exam.......................................... ............ 62
4-7 Performance on each section of the exam......................................................62
4-8 Grades on the makeup final.........................................................................65
5-1 Percent of students withdrawing from each section
during the fall of 2003.................................................... ............................. 78
5-2 Withdrawal rates versus mentor utilization.....................................................79
5-3 Mentor grades in CHM 2045.............................................................................. 81
LIST OF FIGURES
2-1 F probability distribution.............................. ................. ..............................15
2-2 C LB C 130..................................................................................................... 20
2-3 H-ITT remote control.................................. .................. ..............................21
2-4 Chemistry Learning Center...................................................... ...................25
3-1 Percent student attendance by class meeting in CHM 1025............................... 40
3-2 Percent of students receiving a grade in CHM 1025
by number of classes attended.................................................................. 41
3-3 Percent of students receiving each grade in CHM 2045
by number of correct ChRA math questions..................................................44
3-4 Percent of students receiving each grade in CHM 2045
by number of correct ChRA chemistry questions........................... ............ 46
3-5 Percent of grades in CHM 2045 based on ChRA score................................. 47
4-1 Percent of students earning each grade on exam 1............................................55
4-2 Percent of students earning each grade on exam 2............................................56
4-3 Percent of students earning each grade on exam 3............................................58
4-4 Percent of students earning each grade on exam 4............................................59
4-5 Percent of students earning each grade on the final exam.............................. 61
4-6 Percent of students earning each grade on the makeup final.......................... 66
4-7 Student reported anxiety levels when studying and when
taking the collaborative exam s.............................. ..........................................67
5-1 Percent of students rating the helpfulness of peer mentors........................... 79
Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy
COLLABORATIVE LEARNING AND TESTING IN
INTRODUCTORY GENERAL CHEMISTRY
Katie Elizabeth Amaral
Chair: Martin T. Vala
Major Department: Chemistry
Students taking General chemistry at the University of Florida are either well-
prepared or under-prepared. To meet the needs of the under-prepared students, an
introductory course (CHM 1025) was developed. An accurate method of placement into
CHM 1025 or the mainstream course (CHM 2045) was needed. The Chemistry
Readiness Assessment Exam was written and tested and students are advised to take
either course based upon their scores. The accuracy of the cutoff scores was examined,
with the minimum passing chemistry score lowered to six correct out of 18, and the math
score raised to six correct out of eight.
Collaborative problem-solving sessions were held during every CHM 1025 class.
These sessions were shown to increase student achievement in CHM 1025. Group
placement was also shown to have an effect on student achievement in the course.
Students placed randomly into collaborative groups had the highest average GPA, while
students placed by achievement had the lowest average GPA.
The efficacy of CHM 1025 was examined to determine if the students who
required the course do as well in CHM 2045 as those students who did not need it.
Students who had taken CHM 1025 had a higher GPA in CHM 2045 than the students
who went directly into CHM 2045.
Students in the spring semester of 2004 took collaborative exams. Achievement
levels of students who had collaborative exams were compared to students who took
traditional exams to determine if collaborative testing had an effect on student
achievement and retention in CHM 1025. There was no significant difference in
achievement although the collaborative exams were harder. Percentages of students
taking each exam were also compared, with more students taking the collaborative
Finally, undergraduate students called peer mentors, who had taken CHM 1025,
were recruited to assist with the course. Mentors helped CHM 1025 students with the
collaborative problems. The mentors' presence helped lower students' withdrawal rates
in the class. The mentors also benefited from the program, as evidenced by their higher
GPA in CHM 2045.
There are three ways that the actions of an individual may influence actions of
another individual. The action may help another individual succeed, may hinder the
success of another individual, or may have no effect on the other individual.' The first of
these outcomes is usually the result of a cooperative effort on the part of the individuals
in question, the second of competitive efforts, and the last of individual efforts. Although
individual efforts are usually stressed in the learning environment, individuals who work
near each other on the same task generally begin to compete. In the university setting,
high grades are often made artificially scarce, with only a certain percentage of students
able to achieve this level of success. This often fosters a competitive attitude among
When individuals compete against each other, the outcome of their actions is
often negatively linked. The success of a small minority of individuals means the
automatic failure of the rest of the competitors. For example, if the students' grades in a
class are normalized to a bell curve, only a select few students will receive an A. All
other students will automatically not get the high grade, and these students usually know
during the course of the semester that they will not achieve this level of performance.
This knowledge of failure, as well as the failure itself, can leave the losers feeling angry,
jealous, or inadequate about their own performance
Competition can also have an adverse effect on the interactions between the
individuals involved. Anything that helps one competitor succeed directly hurts another
competitor's chance of success. Individuals who compete may realize that anything to
their immediate advantage is better than cooperating with others engaged in the same
task. Often, when there is a high level of importance placed on success, competitors will
cheat in order to enhance their chances. In a survey of central Florida college students,
9 out of 10 stated that they had cheated on an exam or a homework assignment.'
Competition can also lead to high anxiety levels among students. High anxiety
may inhibit an individual's performance on the task. Anxiety disrupts a student's ability
to cognitively reason and promotes avoidance of the situation that the student fears. In the
case of the college student, it means skipping class, taking long breaks, or avoiding
challenging learning situations. As early as 1927 it was known that individuals who fail
in the early stages of competition did not overcome their initial failure and scored poorly
on all subsequent tests, even though the tests were matched to competitors' abilities.2 It
has also been demonstrated that individuals who fail consistently in a competitive
situation tend to withdraw from the competition.3 In a scholastic setting, this means that
students who perform poorly on the first exam are likely to continue with this poor
performance in spite of their ability to achieve higher scores. These students often give
up, and stop trying to achieve a higher score. A study performed at the University of
North Carolina at Charlotte has shown that 80% of the students who failed their first
exam in the first semester general chemistry class failed the course.4 Competitive
students either work hard to do better than their peers or do only the minimum amount of
work, because they do not believe they have a chance to succeed.5 Competitive situations
also tend to increase student anxiety.
In a non-exam situation, students who are working on homework or merely
reviewing lecture notes have no other source of information. If a student does not
understand or have the knowledge to complete a problem, work on that problem ceases.
Other students, in this case competitors, are not likely to assist the student with the task
because it will directly inhibit their own chances of success. Also, students who work
alone may have a difficult time remaining motivated to complete a task, especially when
they feel the task is unimportant to them.
Competitive learning situations tend to isolate students from each other and create
negative relationships both between the instructor and the students, and between the
students themselves.5 These situations also tend to discourage active learning and do not
help develop talent among students. Reviews of the literature have shown that college
students are much more likely to acquire critical thinking skills and to develop learning-
strategies in a collaborative setting than in a competitive setting.6'7 Bligh has shown that
college students spend more time integrating concepts and synthesizing connections
between concepts in a collaborative setting than those students who listen to a traditional
lecture.8 In fact, groups have been shown to be better at promoting student problem-
solving skills than the traditional lecture.9 With these thoughts in mind, we decided to
implement collaborative learning in the introductory general chemistry class at the
University of Florida.
Collaborative learning is learning that takes place in small groups of cultural,
academic, and occasionally linguistically heterogeneous populations.'1 The participants
in these groups work together to accomplish the assigned task, both giving and receiving
pertinent information from group members. Whether collaborative learning is effective or
not depends upon several variables. The teacher's knowledge and skills will determine
how the groups are guided through their tasks and will also affect classroom
management. Classroom configuration is also important to collaborative learning. Some
classrooms, especially auditorium style classrooms or classrooms where the seats are
stationary, are difficult to arrange in a pattern that promotes communication among group
members. As discussed below, the type of group greatly affects the outcome of the
There are several different types of groups that appear in the collaborative
learning setting. These include the pseudo learning group, the traditional classroom
group, the formal and informal collaborative learning groups, and collaborative base
groups." In the pseudo learning group, the participants are assigned to work together for
a varying length of time, but have no reason to help each other because they are directly
competing for the highest grade. These students are usually graded on a bell curve. In
these learning groups, students may mislead and confuse each other as well as try to hide
what they know from each other. The students in these groups perform worse on the
assignment than if they had actually worked by themselves."
In the traditional classroom learning group, students are assigned to work together
and do so because it is required. The length of time the students work together varies
from situation to situation. Students are evaluated on individual performance only, and
therefore may have no vested interest in helping other students. While they desire the
information that other students have, they may have no incentive to share their own
information. Personally they desire to do well on the assignment, but may not want their
colleagues to perform equally as well. Only a minimum of collaborative learning occurs
in these groups and this may occasionally help the lower achieving students. However,
the higher achieving and more conscientious students would usually perform much better
on an assignment if they had worked alone.'
Formal collaborative learning occurs when students work together for a period of
time, lasting from one to several class periods, in order to achieve specific tasks and
assignments. In order for formal collaborative learning to be successful, the teacher must
decide in advance the size of the group, the objective of the group, how students are
assigned to a group, and the role each student will play in the group. The objective needs
to be explained to the students in such a manner that they understand how they are to
function as a group and how their grade depends both on individual performance as well
as group performance. The manner in which the instructor approaches formal
collaborative learning plays a key role in these types of groups. The instructor needs to
monitor the students while they are in the group to provide assistance with the objective
and to help with student social skills.
Informal collaborative learning occurs when students form their own groups and
work together for only a short period of time, from a few minutes to one class period.
This type of group can be used to focus students' attention on the lesson at hand, to
ensure that they understand and are capable of applying the information they are being
taught, and to break up an otherwise monotonous lecture into something more interesting.
There is neither a defined evaluation procedure nor are there defined student roles in this
type of collaborative learning. The final type of collaborative learning group is the
collaborative base group. These groups consist of students of heterogeneous ability and
backgrounds that work together for several semesters or years. These groups work best
in the K-12 setting, and thus will not be discussed here."
The success of formal collaborative learning hinges upon several factors.' First,
students must realize that they are connected to each other and that they cannot
individually succeed unless the group succeeds. Usually this involves connecting an
individual student's grade with the group grade. In order for the group to perform well,
students must share their work, ideas, skills and information with each other. Students
must also be individually accountable for their own work. This ensures that one student
does not have to perform all the work in a group and that all students are contributing to
the objective. The students must be comfortable in the group setting in order to take risks
regarding learning and to encourage and support their peers. Students must also learn the
social skills necessary to work in a group. The students must learn how to solve
problems, they must discuss the nature of the concepts and strategies being learned, they
must teach the knowledge to classmates and explain connections between past and
present learning.12 Finally, students must discuss where the group is in terms of the
group goals and what must be done to achieve the goals. These factors are not easily
achieved and must be constantly addressed as the group works together.
The instructor plays an important role in whether or not collaborative learning is
achieved in the classroom.12 The instructor needs to ensure that students are actively
learning and contributing towards the common goal of the group through careful
observation of student interactions. The instructor is no longer just the source of
information, but is also a facilitator of learning. The instructor must put a lot of effort
into preparing for a collaborative class. All contingencies must be prepared for, and all
students must be doing what the instructor has planned. If collaborative learning is
implemented properly, the classroom will turn into a stimulating, challenging, and
educational environment. The collaborative process must be continually monitored as it
occurs, and any changes that need to be made should be made immediately.
Group activities need to be focused around a central idea in order to foster
conceptual understanding of difficult material.' Students should encounter the central
idea in many different settings so they will have many different opportunities to work on
the problem. They will also need to have different tasks that can help promote
comprehension of the material. The group tasks should inherently be open-ended in
order to ensure that students work together to arrive at a solution to the problem. If there
is more than one right answer to the problem, the students must discuss the problem,
decide upon a direction for the group to take and then work on the problem, arriving at a
solution that may be different from other groups in the class. Open-ended tasks increase
group cohesiveness because they force students to rely upon each other's knowledge and
To ensure that varying levels of knowledge and skills are present within the
group, the groups should be heterogeneous. This should help the lower achieving
students perform better. The higher achieving students also benefit because their
knowledge and understanding of the objective is tested as they assist the lower achieving
students. Students' reliance upon each other fosters higher-order learning skills because
they have to communicate their thoughts effectively to one another, examine issues from
different perspectives, and justify their arguments. The small collaborative groups also
provide an opportunity for students to ask questions about the material in a safe setting.
Greater than 60% of college students stated that having a large number of peers listening
prevents them from asking questions in a class.3 The smaller group provides a much less
threatening situation where students can ask questions.
Difficulties can arise in the collaborative setting when lower achieving students or
socially distant students are excluded from the learning group.14 Critics of collaborative
learning state that the exclusion of certain individuals from a collaborative group
reinforces existing social and educational problems. Expectation States Theory (EST)
outlines how the status of an individual group member determines the value the group
places upon that individual's work.15 The status of an individual depends upon status
characteristics, which are agreed upon social rankings where high status is better than low
status. Status characteristics are based on several things and can change from situation to
situation. They can be general, based on gender or ethnicity, or they can be specific,
based on perceived ability related to the task at hand. They can also be temporary, based
upon perceived academic status. Contrary to what might be expected, minority students
and low socioeconomic status students are not necessarily low status group members. In
situations where students of minority or low socioeconomic backgrounds are considered
high status, their relative status would carry into the groups in which they participated. It
has also been demonstrated that collaborative learning can help reduce stereotypes
because the individuals in the groups are exposed to others who disprove the stereotyped
It is also possible for a teacher to equalize students' status within the collaborative
learning group.14 If a teacher assigns competence to a student or group of students, there
is a specific and public recognition of the intellectual contribution a student can make to
the group assignment. It often has a positive side effect on the low status student because
a teacher's high expectations can often raise the expectations of the student in question
and other students in the peer group. The praise must be public, specific, and relevant to
the work of the group for it to be effective.
Cohen and Lotan have summarized research about the effectiveness of
collaborative learning.17 They report on two data sets of five middle schools in the San
Francisco Bay Area between 1991 and 1993. Students who participated in formal
collaborative learning were compared to students who had not. All students covered the
same topics in the social studies classroom and were given the same tests. The students
who engaged in collaborative learning scored higher on items that required higher-order
thinking than those students who did not. There was no significant difference in students'
scores for questions requiring only factual recall. They also determined that the way in
which the teacher implemented the collaborative learning program affected the students'
achievement. Increasing the number of activities and reports the students completed had
a positive effect on test scores. They also discovered that in classes where the instructor
had difficulties with classroom management, student achievement diminished.
Johnson and Johnson performed a meta-study on the 550 experimental and 100
correlational studies performed on the effects of collaborative, competitive and individual
efforts on instructional outcomes.' They noted that the type of interaction among
students determined the instructional outcome. For instance, structuring assignments so
that they must be collaborative in nature resulted in students working together to achieve
success. On the other hand, students who worked competitively tried to hinder the
success of others while those students who worked independently had no effect on the
success of other students. Johnson and Johnson noted that 375 studies in the last 100
years demonstrated that students who worked together had higher achievement and
greater productivity than those students who worked alone. These studies also
demonstrated that collaborative learning resulted in more high-level reasoning, greater
transfer of knowledge from one situation to another, and more time spent on-task than
individual or competitive settings. They also concluded that collaborative learning
ensured that all students were actively and productively involved in learning.
As a side effect to the text-based learning that goes on in collaborative groups,
Johnson and Johnson determined that groups promoted caring and committed
relationships among students. These relationships were not fostered in the competitive or
individual learning settings. They also stated that working collaboratively with peers
resulted in greater psychological health, higher self-esteem and greater social competence
of students than in the competitive or individual environments.
Collaborative learning has been demonstrated to be an effective teaching tool in
general chemistry,'8-28 organic chemistry,29-32 analytical chemistry,33"37 and physical
chemistry.3840 It is essential to ensure that true collaborative learning is present in order
for it to be effective. Collaborative learning teams have been shown to outperform
individual and competitive systems in mastering content within a time constrained system
based upon set performance criteria.4143 Students in collaborative learning groups tend
to think more critically and to learn less material by rote memorization.44 Collaborative
learning is traditionally implemented in small classrooms of approximately 30-50
students. One of the aims of this project is to demonstrate that collaborative learning can
be implemented in a large introductory general chemistry class of approximately 300
Hogarth and Hartley have examined the effect of the method of placing students
into groups.45 In this study, students in a British Grammar School, the equivalent of high
school here in the States, either chose their own pairs or were assigned to a pair by the
instructor. The researcher found no significant difference in the performance of the two
types of groups. He did note, however, that students in own-choice pairs stated that they
enjoyed chemistry more than the assigned pairs. Aside from this study, there does not
appear to be other studies discussing the effect of placement on the effectiveness of
Chemistry Readiness Assessment Exam
Prior to the fall semester of 2002, there were two general chemistry tracks being
offered at the University of Florida. In the first track, which was two semesters long,
students initially took CHM 2045 with its associated laboratory class, CHM 2045L.
After attaining a grade of C or better, most students took CHM 2046 with its laboratory
class, CHM 2046L. The laboratories must be taken simultaneously with the
corresponding class. In the second track, there were three semesters of general
chemistry. Students first took CHM 2040, which does not have a corresponding
laboratory class. After attaining a grade of C or better, the students took CHM 2041 with
its associated laboratory class, CHM 2045L. Again, after attaining a grade of C or better,
most students took CHM 2046 with CHM 2046L. The longer track split CHM 2045 into
two semesters, spending more time on the foundation material for those students without
an appropriate math and/or chemistry background. All students, regardless of their initial
track, took CHM 2046/2046L if they continued on in chemistry. The advantage to the
second track included a slower pace for learning the foundation material that was
expanded upon in later classes. The disadvantage to this track was that students spent a
longer time in general chemistry, which placed them a semester out of sequence for the
rest of their chemistry classes.
Students were initially placed into either CHM 2040 or CHM 2045 based on their
SAT II scores. Students who achieved a certain score or higher were placed into CHM
2045 while those below that score were placed into CHM 2040. The Chemistry
Department at the University of Florida (UF) stopped using this method of evaluation
after it was determined that students' SAT II scores did not directly correlate to their
achievement in chemistry. After this method was abandoned, the students decided which
track to take based upon the evaluations of their advisors as well as by word of mouth
advice from other students. Some advisors continued to use the discarded SAT II score
method for placement. A significant number of students began the two-semester
sequence but then had to drop back to CHM 2040 after the first exam. There were also
students who did extremely well in CHM 2040, receiving all or most of the points
available to them. These students could easily have done well in CHM 2045.
In the fall of 2002, the course offerings for general chemistry were revised. All
students now take CHM 2045 with CHM 2045L, followed by CHM 2046 with CHM
2046L. For those students without the appropriate chemistry and/or math background, a
two-credit introductory class called CHM 1025 replaced the CHM 2040/2041 sequence.
After students complete CHM 1025 with a grade of S (satisfactory), they take CHM 2045
followed by CHM 2046, with the appropriate laboratories, to finish their chemistry
In order for this new system to be effective, a practical method was needed to
place students into the appropriate class. There are a large number of students entering
UF each year who expect to take general chemistry. The method used to place these
students into the appropriate class must be both easily administered and easily scored to
be of any use. In light of this, a voluntary web-based exam was administered to all
students who took CHM 2040 in the fall of 2001 and the spring of 2002. The exam was
administered at the beginning of the semester. This exam was used to determine a
correlation between the preliminary exam scores and students' performance in the CHM
2040 class. The exam was modified into the Chemistry Readiness Assessment Exam
The ChRA is a web-based exam and should be taken before the students arrive at
UF. The University requires that all in-coming first year students have a computer, and
offers assistance in the purchase of one to those students with demonstrated financial
need. This requirement, together with the fact that there are several computer clusters on
campus that are open to students, ensures that all students will have access to a computer
in order to take the preliminary exam. The students who perform well on the preliminary
exam are registered for CHM 2045, while those who do not perform as well are
registered for the introductory CHM 1025 class. Another goal of this project is to further
refine the cut-off scores used for placement into general chemistry. Student achievement
in CHM 2045 has been compared to ChRA scores to determine a more accurate cut-off
ANALYSIS METHODS AND PROCEDURES
This chapter is comprised of three sections. The first section describes the
calculation method used for the statistical tests performed during the course of this study.
The second section describes the collaborative and non-collaborative sections of CHM
1025 during the fall semester of 2002, the spring, summer and fall semesters of 2003, and
the spring semester of 2004. The third section discusses the ChRA and the suggested
minimum requirements for admittance into CHM 2045.
Analysis of Variance
Analysis of Variance (ANOVA) is a statistical test designed to judge whether or
not differences in sample means are significant.46 This calculation is based on both the
null hypothesis and the test hypothesis. The null hypothesis assumes that all the means
being tested are equal. The test hypothesis assumes that at least one sample mean is
different from the others. These hypotheses are the same for all ANOVA calculations.
The ANOVA calculation is based upon several premises. First, all the populations
being tested have a bell-shaped distribution, also called a normal distribution. Second,
the sample variations (the square of the standard deviations) are approximately equal.
Finally, the samples being compared must be independent from each other.
An ANOVA calculation determines the F probability distribution for the samples.
The F probability distribution, also called the F value, is an asymmetrical distribution
based upon the degrees of freedom in the sample (Figure 2-1). Only a small percentage
Figure 2-1: F probability distribution
of the samples should fall above a certain value on this curve. The level of significance
of the test, called the p-value, is based upon the F distribution. The p-value is the
probability of obtaining a value of F that is as likely or more likely to reject the null
hypothesis as the actual observed value of the test. The p-value is based upon the idea
that the null hypothesis is correct. If the p-value for an ANOVA calculation is less than
0.05 (5.0%) the null hypothesis is usually rejected as incorrect, and the test hypothesis is
determined to be correct. If the p-value is higher than 0.05, the null hypothesis is
considered correct. Another way of stating the p-value would be that we are 95%
confident that the samples are significantly different.
In order to determine the F value in an ANOVA calculation, the degrees of
freedom, sum of squares and mean square must be calculated for the samples (Table 2-1).
Degrees of freedom (DF) determine how many measurements can vary in a sample.
There are two types of degrees of freedom calculated in an ANOVA calculation. The
Table 2-1: Sample ANOVA table
Source Degrees of Sum of Mean F Test
Freedom Squares Square
Between t-1 SSB s
Within n t SSW s
Total n, -1 TSS
first type of degree of freedom denotes the freedom between samples and is given by DF
= t 1, where t is the number of samples. The second type of degree of freedom denotes
the freedom within the samples, given by DF = n, t, where n, = total sample size. These
two degrees of freedom are summed into a total DFt = n, 1.
There are three different sums of square calculations in the ANOVA calculation.
The first is the within-sample sum of squares (SSW). This determines the variance
within each sample.
SSW= (y,,-y,)- = (ni-1)s- + (n2-1)s. +...+(n,-1)s,2
where y, = thejth sample observation selected from population i, y, = the average of the
n, sample observations drawn from population i, and is given by y, = and s =
standard deviation of population i. The second type of sum of squares is the sum of
squares between samples (SSB). This determines the variance between each sample.
SSB = n(y, -y)2
where n, is the number of sample observations obtained from population i, and y is the
average of all sample observations, given by y = The final type is the total
sum of squares (TSS) which determines the variance for the overall mean.
TSS = (y, y)2 = (n -)s2
where s = standard deviation of the nt measurements y,y
There are two types of mean squares, the mean square between samples and the
mean square within samples. The mean squares are averages of squared deviations, and
are usually calculated by dividing the sum of squares by its degree of freedom. The
between-sample mean square is denoted as sB.
The within-sample mean square is denoted sW .
The F value is determined by dividing these two mean squares by each other.
F = _
If the means have been found to be significantly different, either a Fisher's Least
Significant Difference (LSD) test or a Tukey's W Procedure is run to determine which
mean or means are different from the rest. The least significant difference is the smallest
difference that can be observed between two sample means where the means are still
statistically different. The LSD test often has a high error rate between experiments, but
can often be used to get a general idea of which means are different.
LSD, = to sj, +-
if y, y, 1> LSD,, then Pi and Ri are declared different, where tx is the mean for the
Tukey's W Procedure is a much more conservative test than Fisher's LSD test.
Tukey's controls the experiment-wise error rate, and thus fewer means are declared
different than with the LSD procedure. This test lowers the probability of declaring one
pair of means significantly different that are not different when running multiple
comparisons. Of the two, Tukey's W Procedure has been used more often than the LSD
W = q, (t,v)
if I y, yj 12 W then gli and pj are declared different, where t = degrees of freedom
between samples, v = degrees of freedom within samples, and qa= upper-tail critical
value of the F distribution. All ANOVA calculations, LSD calculations, and Tukey's
calculations were run using SAS software version 8 for Windows.47
During the fall semester of 2002, five sections of CHM 1025 were offered, taught
by three different instructors. One instructor taught one section of CHM 1025 (section
0675), and the other two instructors each taught two sections (sections 0676, 5253 and
sections 0677, 4284). Students in CHM 1025 were not given a letter grade, but were
instead graded on a Satisfactory/Unsatisfactory (S/U) basis. Statistics for each section are
presented in Table 2-2.
Table 2-2: CHM 1025 in fall 2002
Section Instructor # students # students Final # students Pass Rate
enrolled withdrew Enrollment receiving S
0675 MV 298 19 279 237 84.9%
0676 KA 304 20 284 237 83.5%
0677 RB 298 24 274 240 87.6%
4284 RB 298 11 287 241 84.0%
5253 KA 219 18 201 152 75.6%
All five sections were taught in Chemistry Laboratory Building (CLB) room
C130, a lecture-style auditorium that seats 297 students (Figure 2-2). The chairs are
stationary and face towards a demonstration desk. There are four moveable chalkboards
in the front of the room, plus a projector connected to a computer located in the front of
the room. Infrared sensors, which line the walls of the auditorium, are also connected to
the computer. The class met twice a week for 50 minutes.
The five sections used common exams, jointly written by the three instructors.
Exams were multiple-choice, usually comprised of 15 ten-point questions. They were
administered at night in rooms scattered throughout UF's campus and proctored by the
instructors and the teaching assistants. There were four exams offered during the term
and a final exam. The final exam was worth 200 points. The lowest of the four semester
exams was dropped. All five sections were graded on the same scale, with 610 points out
of 1000 possible needed to pass the class.
At the beginning of the term, students were told that the maximum number of
points that they needed to pass the class was 650 points out of a possible 1000. Many
students reached this mark after the fourth exam, at which point they stopped coming to
class. Those students that stopped attending also did not take the cumulative final.
Each instructor approached collaborative learning in a different manner during
this semester. All instructors were assigned peer mentors. In section 0675, students were
Lab Demonstration Table
Figure 2-2: CLB C130
allowed to choose their own groups of six students at the beginning of the term. Students
either chose to join a group that included their friends or chose a group based upon their
location. There were not enough peer mentors for each group to have their own mentor,
so the peer mentors roamed assigned sections of C 130 and helped the groups in that area.
The instructor lectured for a short period of time, then asked his students a question based
upon the material just covered. All peer mentors were supplied with answers to the
problems prior to the class session. After the problem was solved, the cycle was
repeated. These problems were placed on a side overhead projector, and had five answer
choices. The students worked with their group to solve the problems and each student
entered an answer using the H-ITT remote controls (Figure 2-3). The H-ITT remote
controls send signals to the infrared sensors. The sensors record both the students' unique
identification numbers and the answer chosen by the student, and transmit this
information to the computer. If a group experienced difficulty solving a problem, either
the instructor or a nearby peer mentor would assist them.
Figure 2-3: H-ITT remote control48
In section 0676, students were randomly placed into groups of six by the
instructor prior to the second class meeting. Students were then told to sit with their
groups for the rest of the term. These groups were balanced by gender, with no group
having more males than females, although several groups had more females than males.
In this section, the instructor lectured for approximately 15-25 minutes, depending upon
the topic being covered. The remainder of the 50 minute class period was spent working
collaboratively on worksheets containing approximately 10 problems each, based on the
material just covered. As in section 0675, there were not enough peer mentors to staff
each group, so peer mentors patrolled certain areas of the classroom during the
collaborative problem-solving sessions to assist students. The students were also asked
H-ITT quiz questions during the collaborative session.
In section 5253, taught by the same instructor as 0676, students were placed into
groups of six based on achievement as determined by the students' SAT grades. The SAT
grades were obtained from the Admissions Office at UF after the first class meeting.
Each group had high-achieving, middle-achieving, and low-achieving students. The
groups were rearranged after three of the four semester exams using the previous exam
score as the new criterion for group placement. This section also had a brief lecture
period, similar to the one in section 0676, followed by a collaborative problem-solving
session assisted by peer mentors patrolling certain areas of the classroom. These students
also had H-ITT quiz questions during the collaborative part of the class.
The third instructor taught sections 0677 and 4284. This instructor lectured for a
certain period of time, then asked a H-ITT quiz question. The students were not allowed
to work together to solve these questions, and were considered to be cheating if they
attempted to collaborate. Peer mentors who attempted to assist students were told to
desist and were publicly chastised for talking in class. The peer mentors were expected
to act as police and ensure that no students were cheating during the H-ITT quizzes.
Eventually most peer mentors stopped attending these sections.
There were two sections of CHM 1025 taught in the spring semester of 2003, both
taught by the same instructor. Information about the sections is given in Table 2-3.
Table 2-3: CHM 1025 in spring 2003
Section Instructor # students # students Final # students Pass Rate
enrolled withdrew Enrollment receiving S
5267 KA 47 10 37 20 54.0%
5269 KA 126 21 105 67 63.8%
Both sections received the same examinations. The semester exams were essay-
type examinations worth 150 points. The final exam was worth 200 points. The lowest of
the four semester exams was dropped. The exams were administered at night in rooms
throughout UF's campus and proctored by the instructor, TA and peer mentors. There
were four exams and a final exam. Both sections were graded on the same scale, with
610 points out of 1000 possible needed to pass the class.
As in the fall of 2002, students were told that they needed 650 points out of a
possible 1000 to pass the class. In order to maintain student attendance past the fourth
exam, students were told that the cumulative final exam for the course was mandatory.
Students had to receive a minimum of 50% on the final in order to pass the class
regardless of the number of points they had. There was also a poor performance
forgiveness policy. Students who received 75% or greater on the cumulative final exam
passed the class even if they had not reached the 650 point cutoff. Only one student took
advantage of this policy.
Section 5267 was placed into groups of six by location during the second class
meeting. No attempt was made to balance these groups by gender. These groups
remained constant throughout the semester. This section originally met in CLB C130,
but due to its small size and early meeting time it was soon moved to the Chemistry
Learning Center (CLC), a room with moveable tables and chairs (Figure 2-4). Students
in section 5269 were placed into groups of six randomly by the instructor and were
informed of their new groups during the second class meeting. The groups remained
constant throughout the term. Most of this section met in CLB C130 for the entire
semester. Four of the groups volunteered to meet in the CLC with two experienced peer
mentors, one of whom was a senior chemistry major. The peer mentors were responsible
for this mini-section, giving lectures to the students based on the instructor's notes. Both
sections met twice a week for 50 minutes.
Sections 5267 and 5269, as well as the mini-section of 5269, were all taught in the
same manner. A brief lecture was given at the beginning of the class period that lasted
from 15 to 25 minutes. After the lecture, students worked in their groups to solve
problems based upon the material just covered. Peer mentors were assigned individual
groups and got to know their students well during the term. The students also worked
together on H-ITT quiz questions during the collaborative session. Group types were
compared using the ANOVA table to determine if group type affected achievement or
There was one section of CHM 1025 taught during the summer semester of 2003
(taught by KA). Section 4216 had an initial enrollment of 170 students, of whom four
withdrew. Of these, 143 students received an S (86.1% pass rate). This section met four
days per week for an hour and fifteen minutes over a period of six weeks in CLB C130.
Figure 2-4: Chemistry Learning Center
Section 4216 was given three long answer exams during class and a cumulative
final examination on the last day of class. These exams were worth 150 points each, with
the final worth 200 points. As in the spring 2003 semester, students were told that the
final was mandatory and that a minimum of 50% was required on the exam in order to
pass the class. The poor performance forgiveness policy was also offered in the summer
2003 semester. Students needed a total of 625 points out of a possible 1000 in order to
pass the class.
Students were placed into groups of six randomly by the instructor and were
informed of their groups during the second class meeting. The individual class meetings
were run in one of two ways. The first type of session involved a 30-minute
collaborative problem-solving session sandwiched between two brief 20-minute lectures.
The problems solved during the collaborative session were based on the first lecture
given during the class. The second type of class had a 20-minute lecture sandwiched by
two 30-minute collaborative problem-solving sessions. The first collaborative session
had problems based on the second lecture from the previous day. The second
collaborative session had problems based on the lecture immediately preceding it.
Students were given H-ITT quiz questions during all collaborative learning sessions.
There were five sections of CHM 1025 during the fall semester of 2003, taught by
four different instructors. The same instructor taught section 0675 and 0677. Information
about the sections is given in Table 2-4.
Table 2-4: CHM 1025 in fall 2003
Section Instructor # students # students Final # students Pass Rate
enrolled withdrew Enrollment receiving S
0675 LL 257 27 230 196 85.2%
0676 KW 303 37 266 202 75.9%
0677 LL 303 23 280 245 87.5 %
4284 KZ 297 10 287 234 81.5%
5253 KA 300 10 290 237 81.7%
All five sections were taught in CLB C130 and met twice a week for 50 minutes.
All sections took the same examinations. There were four multiple-choice exams during
the semester, usually with 15 questions worth 10 points apiece. The lowest semester
exam was dropped. There was also a cumulative final given during exam week. As in
past semesters, this final was mandatory, and students were expected to receive a
minimum of 50% on the final in order to pass the class. All exams were written jointly
by the four instructors. However, at the end of the term, the grading scheme for the
Three of the sections, 0675, 0676, and 0677, were graded on the same scale.
These sections required 600 points out of a possible 1000 to pass the class and all
students were given 88 points to help them towards the goal of 600 points. The students
were given the points because these sections had no H-ITT quizzes. Essentially, a student
only needed 512 points out of a possible 1000, or 51.2% of the available points, to pass in
these sections. The other two sections, 4284 and 5253, were graded on a different scale.
A student in these sections needed 620 points out of a possible 1000 to pass the class, and
these students received no free points, since H-ITT quizzes were given during the class
Sections 0675, 0676, and 0677, although taught by two different instructors, were
taught in a similar manner. These instructors lectured for the whole class period and held
no collaborative problem-solving sessions. Peer mentors were present in these
classrooms, and had specific areas of the classroom to patrol. However, the peer mentors
did not have an opportunity to assist students with problem solving. Peer mentors were
mainly required to pass out in-class quizzes given weekly and soon stopped attending.
Sections 4284 and 5253 were taught in a similar manner, although these sections
also had two different instructors. These instructors lectured for 15 to 25 minutes,
depending upon the topic, and then held collaborative problem-solving sessions during
the remainder of the class periods. Students were randomly assigned to groups of six or
seven by the instructors and informed of their groups during the second class meeting.
Peer mentors patrolled certain areas of the classroom and assisted groups during the
problem-solving sessions. As in prior semesters, the peer mentors were given the
answers to the problems prior to the class meeting. H-ITT quizzes were also given
during these collaborative sessions.
There were two sections of CHM 1025 taught during the spring semester of 2004.
A different instructor taught each section. Both sections met in CLB C130 twice a week
for 50 minutes. Information about the sections is given in Table 2-5. Peer mentors were
assigned individual groups and got to know their students well over the course of the
semester. Both sections were comprised of a brief 15 to 25 minute lecture followed by a
collaborative problem-solving session. Students were given H-ITT quizzes during the
Table 2-5: CHM 1025 in spring 2004
Section Instructor # students # students Final # students Pass rate
enrolled withdrew enrollment receiving S
5267 KZ 98 11 87 44 50.6%
5269 KA 154 13 141 107 75.9%
The main difference between spring 2004 and prior semesters lay in the
examinations. There were four multiple-choice examinations given during the semester,
worth 150 points apiece. The lowest semester exam grade was dropped. There was also
a mandatory cumulative final worth 200 points. The exams were written jointly by both
instructors. The exams were administered at night in rooms dispersed throughout UF's
campus and proctored by the instructors and peer mentors. Each exam had two sections,
an individual section and a group section. The individual section was worth 70% of the
exam points, and the group section was worth 30% of the exam points. For the first hour
of the two hour exam, students sat with their groups and worked collaboratively on the
group portion of the exam. At the end of the hour, the scantrons were collected and the
individual exams were passed out. The students then worked alone on this second
portion of the exam.
For the first exam of the four semester exams, the individual and group portions
were switched, with the indivjdtfal section taken first. Half of the questions on the group
portion of the exam were taken from the individual portion, so students had an
opportunity to rework the problems with their groups. However, students did not like
having to wait quietly for all students to finish the individual portion. They also did not
redo the repeated problems but instead chose the answer that the majority of students in
the group had gotten for these problems. Therefore, the order was switched, with the
group portion coming first for the remainder of the exams. This allowed the students to
talk with each other while waiting for all the groups to finish. Once the sections were
switched, however, the same problems no longer appeared on both portions of the test.
The grades of all students who took CHM 2045 and/or CHM 2046 during the
2002/2003 school year as well as the fall semester of 2003 were obtained from the UF
Admissions Office. It was also noted whether or not the student had taken CHM 1025
and passed. The students who had taken CHM 2045 were split into two groups based
upon whether or not they had taken CHM 1025. The achievement of these groups of
students was compared using an ANOVA table. Also, whether or not having taken CHM
1025 affected student retention in CHM 2045 was determined. The students who had
taken CHM 2046 in the fall semester of 2003 were split into two groups based upon
whether or not they had taken CHM 1025. The achievement of these groups of students
was also compared. In addition, whether or not having taken CHM 1025 affected student
retention in CHM 2046 was compared.
Attendance data was available only for certain sections of CHM 1025. The
attendance in these sections was compared to achievement in CHM 1025 using an
ANOVA table. The sections involved in this calculation were 4284 and 5253 from the
fall semester of 2003 and 5267 and 5269 from the spring semester of 2004.
Chemistry Readiness Assessment Exam
Students were administered a voluntary, web-based preliminary exam at the
beginning of the fall 2001 and spring 2002 semesters. The exam consisted of eight math
questions and 16 chemistry questions, and covered material in the first eight chapters of
the textbook.49 The preliminary exam results were compared to the final course grade for
the fall of 2001. This preliminary exam was modified into the Chemistry Readiness
Assessment Exam (ChRA).
The ChRA was placed online for prospective freshman chemistry students to take
before arriving at UF. Based upon the results of the exam, students were advised to take
either CHM 1025 or CHM 2045 beginning in the fall term of 2002. Students who
received a score of four or better on the math portion of the exam and six or better on the
chemistry portion of the exam were advised to take CHM 2045. Those students receiving
less than these scores were advised to take CHM 1025.
These cutoff scores were not absolute, however, and many students took CHM
2045 that did not meet the minimum ChRA requirements. For those students who did not
take CHM 1025, the grade received in CHM 2045 was compared to the results of the
ChRA to further refine the cutoff scores.
Effectiveness of CHM 1025
The curriculum in the Introduction to Chemistry course (CHM 1025) was
designed for those students who are under-prepared for the mainstream general chemistry
course (CHM 2045). Therefore, it was important to determine whether there was a
difference in student achievement in CHM 2045 between those students who took the
introductory course and those students who did not.
In the fall of 2003, two CHM 2045 classes (taught by KA and MV) were given a
quiz in the first week of the class based upon the material that students were expected to
know before taking the course. The last question on the quiz allowed students to self-
identify as having taken the introductory course. The average score on the quiz for those
students who self-identified as having taken CHM 1025 was 5.36 out of 10.00. This was
significantly different than the lower 4.41 score obtained by those students who had not
taken CHM 1025 (p-value <0.0001). This enhanced achievement in CHM 2045 for those
students who had taken CHM 1025 continued throughout the course.
The grades of all students who had taken CHM 2045 in the spring of 2003,
summer of 2003 and fall of 2003 were compared for level of achievement. The students
were split into three groups: those who had not taken CHM 1025; those who had taken
CHM 1025 and passed; and those who had taken CHM 1025 and failed. The grade point
averages (GPA) of these students were compared to determine the effect CHM 1025 had
on achievement. On a 4.00 GPA scale, an A corresponds to a 4.00, a B+ to a 3.50, a B to
a 3.00, etc. The students who had not taken CHM 1025 had an average GPA of 2.12 in
CHM 2045. This is significantly lower than the 2.74 GPA of those students who had
passed CHM 1025 (p-value <0.0001). Those students who either failed or withdrew from
CHM 1025 had an extremely low 0.59 GPA in CHM 2045. This abysmal success rate for
students who failed or withdrew from CHM 1025 demonstrates that these students must
not be allowed to enroll in CHM 2045. Once CHM 1025 is attempted, it must be passed
before advancing to CHM 2045.
Another measure of how well CHM 1025 prepares students (that does not rely on
student grades) is to determine how many of these students continued on to the second
semester of general chemistry (CHM 2046). Of the 1609 students who took CHM 2045
during the spring and summer semesters of 2003, 953 (59.2%) students earned a grade of
C or better and were eligible to continue on to CHM 2046. Only 579 (60.8%) of these
students had taken CHM 2046 by the fall semester of 2003. Of the 579 students that
continued on to CHM 2046, 394 (68.0%) had originally taken CHM 1025. Students who
had taken CHM 1025 were much more likely than their counterparts to continue on to
CHM 2046 if they were eligible to do so. In these two semesters, 656 students took
CHM 2045 but were ineligible to continue to CHM 2046. These ineligible students were
approximately evenly split between former CHM 1025 students and non-CHM 1025
students (48.6% to 51.4%).
Although there is no data available for what other chemistry classes were taken by
the students who passed CHM 2046, it is possible to determine who was able to continue
on in chemistry. There were 579 students enrolled in CHM 2046 over the summer and
fall semesters of 2003. Of these, 471 (81.3%) passed with a grade of C or better. Of these
471 passing students, 327 (69.4%) were originally enrolled in CHM 1025. Former CHM
1025 students were much more likely to pass CHM 2046 than students who did not take
The fall semester of the school year has by far the highest enrollment of CHM
1025 students of all the semesters. The majority of these students that are eligible will
take CHM 2045 the following spring. As a result, the spring semester is especially
indicative of the performance of former CHM 1025 students in CHM 2045. The spring
semester of 2003 is particularly telling, as there were approximately equal numbers of
non-CHM 1025 students and former CHM 1025 students enrolled in CHM 2045. Of the
1408 students initially enrolled in CHM 2045 that semester, 846 of them had taken CHM
1025 (60.0%). Of the 829 students who passed CHM 2045 in the spring of 2003, 557
students had taken CHM 1025 (67.2%). Of the 579 students who failed or withdrew
from CHM 2045 that semester, 287 (49.5%) had taken CHM 1025. Although these
statistics suggest that students who pass CHM 2045 are more likely to be former CHM
1025 students, the statistics on withdrawals and failures is disheartening. Former CHM
1025 students are failing CHM 2045 at the same rate as students who did not need the
introductory course. One of the possible methods that can be used to remedy this failure
rate is to slow down in CHM 1025 and spend more time on fundamentals such as
nomenclature and stoichiometry.
Following the enrollment trend, the semester with the largest enrollment of former
CHM 1025 students enrolled in CHM 2046 was the fall semester of 2003. Again, this
semester is looked at individually because the numbers of both types of students is
approximately equal. Of the 724 students initially enrolled in CHM 2046 this term, 348
had taken CHM 1025 (48.1%). Of the 607 students that passed CHM 2046, 300 of them
had taken CHM 1025 (49.4%). Of the 117 students who failed or withdrew from the
course, only 48 had taken CHM 1025 (41.0%). The pass rate of both the non-CHM 1025
students and the former CHM 1025 students during this semester is approximately the
same. The failure rate of former CHM 1025 students during this semester is slightly
lower than those students who did not take CHM 1025. Students who required an
introductory course in chemistry are able to succeed just as well in CHM 2046 as those
students who did not require it.
The GPA scores of students enrolled in CHM 2045 in the spring of 2003 were
compared to determine if CHM 1025 had an effect on the graded outcome of the class.
Students who had taken CHM 1025 had an average GPA of 2.18 in CHM 2045 during
the spring semester of 2003. This was significantly different than the average 1.59 GPA
of the students who did not have CHM 1025 (p-value <0.0001). The 2.18 GPA equates
to a letter grade of a C, while the 1.59 GPA equates to a letter grade of D+. Students
must earn at least a C or better in order to continue on to CHM 2046. Former CHM 1025
students performed better during this term as noted by their passing average GPA. The
average GPA scores noted above include all students who had enrolled in CHM 2045 that
semester, including those that failed the course. When only passing students are
considered in the calculation of average GPA, the scores increase. Although the former
CHM 1025 students still have a higher average GPA (3.18 versus 3.09), the scores are no
longer significantly different (p-value 0.0959). This again suggests that former CHM
1025 students have higher grades overall in CHM 2045, even for those students who fail
The GPA scores for students enrolled in CHM 2046 during the fall semester of
2003 were also compared to determine if CHM 1025 affected the graded outcome of that
course. The average GPA for former CHM 1025 students enrolled in CHM 2046 was
3.01. Although higher than the average 2.86 GPA obtained by the students who had not
taken CHM 1025, the scores were not significantly different (p-value 0.1235). When
only those students who passed were considered, the GPA of the former CHM 1025
students was only slightly higher than the non-CHM 1025 students (3.42 versus 3.39),
and these scores were also not significantly different (p-value 0.5639). However, CHM
1025 students are succeeding in CHM 2046 even though their initial chemistry
backgrounds were not as strong as the students who did not need CHM 1025.
Each semester there are students who take either CHM 2045 or CHM 2046 a
second time, due to either a failure or a withdrawal in the course. These students were
studied to determine whether or not CHM 1025 affected the numbers of students who had
to repeat either CHM 2045 or CHM 2046. Over the course of this study, 439 students
took CHM 2045 twice. Of those students, only 124 (28.2%) had taken CHM 1025 prior
to their initial enrollment in CHM 2045. Additionally, 256 students repeated CHM 2046,
with only 19 (7.4%) having taken CHM 1025. The low number of former CHM 1025
students who had to repeat one of these two courses demonstrates that the CHM 1025
students are better prepared to succeed in later chemistry classes on the first attempt.
This is a desirable outcome because students who need to repeat a course end up behind
in their majors, which can cause difficulty later in their college careers.
In order to determine if collaborative learning affected achievement in CHM
1025, students' grades in that course were compared for the fall 2002 semester. There
were five sections of CHM 1025 that semester: section 0675 had student-chosen groups;
section 0676 had randomly-chosen groups; section 5253 had achievement-assigned
groups; and sections 0677 and 4284 did not use collaborative learning. This was the first
semester that collaborative learning was attempted on such a large scale. CHM 1025 is an
S/U class, with an S being equivalent to a 2.00 or above on the 4.00 GPA scale, and a U
is less than a 2.00 on the GPA scale, but is considered a 0.00 in the calculations.
There was no significant difference in the achievement level of students as
denoted by final grades in either the student-chosen, random, or non-collaborative classes
(Table 3-1). However, all three of these types of classes had a higher achievement level
than the students who were placed into groups by achievement (p-value 0.0089). Clearly
the collaborative effort hinders students when they are placed into groups using either
SAT achievement or test-scores as a method of placement. It is also disappointing to
note that collaborative learning had no effect on student achievement over the non-
collaborative classes during the fall semester of 2002. However, with experience,
collaborative learning was more effective by the fall semester of 2003.
Table 3-1: GPA in CHM 1025 based on group type
Section Instructor Group Type CHM 1025 GPA
0675 MV Student chosen 1.59
0676 KA Random 1.56
0677, 4284 RB None 1.62
5253 KA Achievement 1.39
During the fall semester of 2003, students were placed randomly into groups in
sections 4284 and 5253, while sections 0675, 0676, and 0677 did not use collaborative
learning. Students in the collaborative sections had a higher achievement level denoted
by final grade than students in the non-collaborative sections (p-value 0.0296). Students
in the collaborative sections had an average GPA of 1.59, while the non-collaborative
sections had a GPA of 1.49. Collaborative learning helped the students in sections 4284
and 5253 enjoy greater success in CHM 1025. Unfortunately, it is impossible to
determine if this greater success carried into CHM 2045 or CHM 2046 because these
students had not completed these courses by the end of this study.
The students who enrolled in CHM 1025 in the fall semester of 2002 were the
only students who were able to complete the entire general chemistry sequence (CHM
1025, CHM 2045 and CHM 2046) by the time this study ended. The number of students
from each type of group that continued on to CHM 2045 and CHM 2046 were compared
to determine if collaborative learning had an effect on student willingness to continue in
chemistry. A slightly higher percentage of students from the three collaborative sections
that were eligible to continue on the CHM 2045 and CHM 2046 did so when compared to
the non-collaborative sections (Tables 3-2 and 3-3). Although collaborative learning had
no effect on achievement during this semester, the collaborative students were slightly
more likely to continue with chemistry than their non-collaborative counterparts.
Table 3-2: Number of students continuing to CHM 2045 based on group type
Section Instructor Group Type Students # in CHM % in CHM
passing 2045 2045
0675 MV student chosen 238 186 78.2%
0676 KA Random 237 194 81.9%
0677 RB None 240 185 77.0%
4284 RB None 241 176 73.0%
5253 KA Achievement 152 122 80.3%
Table 3-3: Number of students continuing to CHM 2046 based on group type
Section Instructor Group Type Students # in CHM % in CHM
passing 2046 2046
0675 MV student chosen 238 88 37.0%
0676 KA Random 237 88 37.1%
0677 RB None 240 75 31.3%
4284 RB None 241 76 31.5%
5253 KA Achievement 152 54 35.5%
Student achievement in CHM 2045 and CHM 2046 was compared to the students'
original section in CHM 1025 to determine if collaborative learning had an effect on
student success in later classes. Although there was no significant difference in GPA
scores for the students from the five fall 2002 and the two spring 2003 sections in CHM
2045 (p-value 0.2049), the average scores for the students from the collaborative sections
were slightly higher (Table 3-4). There was also no significant difference in average
GPA scores for the students from these seven sections in CHM 2046 (p-value 0.9735). It
is interesting to note that section 5267, the section that met in a room that was easily
rearranged and thus better for collaborative learning groups, had the highest average GPA
scores of all seven sections in both CHM 2045 and CHM 2046. Room configuration is
known to have an effect on the outcome of collaborative learning, and this idea is further
confirmed by these grades.
Table 3-4: Scores in CHM 2045 and CHM 2046 based on CHM 1025 section
Section Semester Instructor 2045 GPA 2046 GPA
0675 Fall 2002 MV 2.08 2.88
0676 Fall 2002 KA 2.28 3.00
0677 Fall 2002 RB 2.07 2.99
4284 Fall 2002 RB 2.17 2.93
5253 Fall 2002 KA 2.38 2.91
5267 Spring 2003 KA 2.61 3.38
5269 Spring 2003 KA 2.21 2.60
Colloquially, it has always been known that students must attend class in order to
succeed in the course. Getting the students to believe this simple fact is another matter
altogether. Sections 4284 and 5253 from the fall of 2003 and sections 5267 and 5269
from the spring of 2004 had over 20 class meetings during their respective semesters (see
Figure 3-1). The large dips on this graph come the day after an exam. Students'
achievement in these classes was compared to their attendance as recorded by their H-
ITT remote controls. The students were split into four groups by number of classes
attended: 0 5 classes attended; 6 10 classes attended; 11 16 classes attended; and
>17 classes attended. The first group corresponds to students who attended less than
25% of the class meetings, the second group attended less than 50% of the time, the third
-- 4284 -a-5253 5267 -w-- 5269
0 5 10 15 20 25
Figure 3-1: Percent student attendance by class meeting in CHM 1025
group attended less than 75% of the time, and the fourth group attended most of the
As is expected, there is a significant difference in scores earned by the students
who fall into these groups (p-value <0.0001). Students who attend less than 25% of the
time receive an average of 300 of the 1000 available points, earning a very low E.
Students who attend less than 50% of the time receive an average of 521 of the available
points, earning a high E. Students who attend less than 75% of the time receive an
average of 589 of the available points, earning a D. Students who attend greater than
75% of the classes receive an average 758 of the available points, earning a B. As Figure
3-2 shows, only those students who attend class at least three-quarters of the time can be
expected to pass the class. It is important to recognize that students know exactly how
00-5 D6-10 11-16 17-22
A B+ B C+ C D+ D E
Figure 3-2: Percent of students receiving a grade in CHM 1025 by number of
many points they must earn to pass this S/U class. Most students earn this bare minimum
number of points, and do not work one bit harder than they absolutely have to. Any
students who earn over 650 points are doing extremely well in the class, and doing the
amount of work that will help them succeed in CHM 2045.
Students' attendance was compared to the type of collaborative group to
determine if a particular group type affected attendance. Attendance data was only
available for those students who were placed into groups randomly or by achievement.
Students who were placed randomly into groups attended class an average of 83.3% of
the time. This is significantly different from the students who were placed into groups by
achievement (p-value <0.0001). These students attended class only 68.2% of the time.
As shown above, students who attend less than 75% of the time are likely to receive a D.
This further demonstrates that placing students into groups by achievement is not an
effective method of increasing student success.
Chemistry Readiness Assessment Exam
Most of the students who enrolled in either CHM 1025 or CHM 2045 beginning
in the fall of 2002 took the Chemistry Readiness Assessment Exam (ChRA) before
enrollment. The students were merely required to take the exam while there were
guidelines, there were no absolute scores that directed students into either course.
However, most students followed the suggested guidelines. Those students who received
a score of four or greater out of a possible eight on the math portion and a score of nine or
greater out of a possible 18 on the chemistry portion were advised to take CHM 2045.
Those students who received scores below this level were advised to take CHM 1025.
These levels were not absolute, and many students enrolled in whichever course they
thought best suited their needs.
The ChRA scores for all students who had taken CHM 2045 in the fall of 2002,
the spring of 2003, and the summer of 2003 and who had not taken CHM 1025 were
obtained from the Office of Admissions at UF. Students' achievement in CHM 2045 was
compared to their ChRA scores. Students who received a score of four on the math
portion of the ChRA had an average GPA of 1.85. This average GPA is not significantly
different than the average 1.65 GPA of students who correctly answered less than four
questions on the ChRA (p-value 0.1717). The average GPA of students who received a
four on the math portion converts to a D+ on the letter scale. Because students must
receive a C or better in CHM 2045 in order to proceed to CHM 2046, this cutoff is
unacceptable for the math portion of the ChRA.
Students who received a five on the math portion of the ChRA had an average
GPA of 2.13. This average GPA is significantly different than the average 1.76 GPA of
the students who received less than five (p-value 0.0006). While students who passed the
math section of the ChRA with a score of five have an average grade of a C in CHM
2045, this is an extremely low C.
Those students who received a six on the math portion of the ChRA had an
average GPA of 2.41 in CHM 2045. This is significantly different than the average 1.95
GPA of the students who answered less than six questions correctly (p-value <0.0001).
This C grade is a much stronger one than the one received by students who earn a five on
the math section. Therefore, the recommended cutoff for the ChRA math section is a
score of six out of eight possible questions (Figure 3-3).
04 M5 M6
A B+ B C+ C D+ D E W
Figure 3-3: Percent of students receiving each grade in CHM 2045 by number of
correct ChRA math questions
The chemistry score of the ChRA was determined to be not as crucial an indicator
of performance in CHM 2045 as the math score. The initial cutoff score for the
chemistry portion was nine out of 18 questions. However, students who received this
score had an average GPA of 2.32 in CHM 2045. This was not significantly different
than the average 2.06 GPA of the students who received less than a nine on the chemistry
portion (p-value 0.0109). This GPA is equivalent to a C, so this cutoff score can be
Students who received a score of eight on the chemistry portion of the ChRA had
an average GPA of 2.28 in CHM 1025. This was significantly different than the average
1.98 GPA of students who received less than this score (p-value 0.0083). However this is
still a reasonable C grade, so the minimum cutoff score can be lowered even further.
Students who received a score of seven on the chemistry portion of the ChRA had
an average GPA of 2.10. This is not significantly different than the average 1.95 GPA of
the students who received less than a score of seven (p-value 0.4993). This is a passing
GPA, but it is only barely a C. However, the students who received a score of six on the
ChRA had an average GPA of 2.30, a higher score than those students who earned a
seven on the ChRA. The average 2.30 GPA of students who received a six on the ChRA
is significantly different than the average 1.76 GPA of the students who received less
than six (p-value 0.0003).
The students who received a score of five on the ChRA had an average GPA of
2.13 in CHM 2045. This is significantly different than the average 1.56 GPA of the
students who received less than that score (p-value 0.0038). Although a passing GPA,
2.13 is not a strong C grade. The average GPA of students who received a four on the
chemistry portion of the ChRA is below passing (1.62). Therefore, the recommended
cutoff for the chemistry portion of the ChRA is six out of a possible 18 (Figure 3-4).
Although the cutoff values are the same, students must receive a 75% on the math
portion of the ChRA and a 33% on the chemistry portion in order to succeed in CHM
2045. These results demonstrate that student success in beginning chemistry relies more
on prior math knowledge than on prior chemistry knowledge. Therefore, although many
students' high school chemistry backgrounds are poor, those with a strong math
background can still succeed in chemistry.
05 06 E7 E8 09
A B+ B C+ C D+ D E W
Figure 3-4: Percent of students receiving each grade in CHM 2045 by number of
correct ChRA chemistry questions
As shown in Figure 3-5, students who receive a six on both the math and the
chemistry portions of the ChRA are more likely to get a grade of A in CHM 2045 than
their counterparts. Those students who fail both sections of the ChRA are much more
likely to withdraw from the class than were their classmates. The students who pass only
one section of the ChRA have mixed results, with those students who passed only the
math section performing better in CHM 2045 than the students who passed only the
chemistry portion of the exam.
0 fail both Opass chem pass math U pass both
A B+ B C+ C D+ D E W
Figure 3-5: Percent of grades in CHM 2045 based on ChRA score
Testing usually occurs in a traditional format. Students use a pencil and paper to
take the exam. They usually work individually on the exam, and are seated to reduce the
possibility of cheating. Any students working together are considered to be cheating.
The penalties for cheating are severe, ranging from failing the examination to failing the
course, or even expulsion from the university. Students are usually very concerned about
the exams, and are anxious to do well on them, often to the point of asking, "Is this going
to be on the exam?" A high anxiety level has been shown to decrease student
performance. Using a testing system that is collaborative in nature can help to decrease
student anxiety and increase student understanding of the subject material.
Only a handful of studies have looked at collaborative testing as an alternative to
traditional testing methods.50-56 Most of these studies paired students, rather than using a
larger group of four to six students. Also, few of these studies actually set up the exam in
such a fashion that it had to be worked on collaboratively. Although many of the studies
state that they used collaborative testing, few researchers actually wrote an exam where at
least a portion of it was intended to be taken collaboratively.
Muir and Tracy used paired testing in a course for preservice teachers at Oakland
University.50 The students worked in student-chosen pairs on essay-type examinations,
worth 25% of the students' total grade. The students were allowed to choose their own
pairs to decrease teacher responsibility if students blamed their partner for a low exam
score. Muir and Tracy found that test scores increased by 7.0% on the midterm
examination, but only by 1.0% on the final examination. They do not state whether these
results were significant. Muir and Tracy noted that student anxiety level was decreased,
and students spent more time preparing for the examinations. However, they do not state
how they came to the conclusions of lower student anxiety and longer preparation time.
From a survey of students in their classes, they determined that 97% of the partners
worked equally on the exam, with only 3% of the partners stating that one member was
not exerting as much effort as the other member. Two aspects of collaborative testing
that were noted as important by these researchers were that the desks and chairs must
move in order to promote the collaborative effort and noise level must be monitored so
that it does not interfere with student concentration. Muir and Tracy did not state
whether students turned in individual copies of the exam, or only turned in one set of
answers per pair.
Another study conducted with nurses at Lake Superior State University assigned
students randomly to pairs on the day of the test.51 The students did not work together
prior to the exam. The researchers believed that this placement method would encourage
all students to study and prevent one member of a group from carrying the pair. Students
worked individually for most of the exam, and were only allowed to collaborate for the
final third of the exam time. The partners did not have to agree on an answer and each
member of the pair handed in an answer set. The students who compared answers had an
equivalent comprehension of material as students who did not have the option to do so.
Student achievement only increased slightly, from an average GPA of 3.24 in the non-
collaborative setting to an average of 3.27 in the paired setting. Students noted a
decrease in anxiety in the setting where students were allowed to compare answers.
A study at Stanford University used a partner testing system in a large, 300-
student introductory psychology class.52 Students in this class were allowed to choose
whether or not to take the test with a self-chosen partner. Those students that chose to
work with a partner submitted only one set of answers per pair. The students in this
course had to take the first midterm exam alone, and then had the option of taking the
second and third midterm exams with a partner. Although 62% of the students chose to
take the second test with a partner, only 48% chose to take the third exam collaboratively.
By the second year of this study, only 30% of the students were choosing to take the
exam collaboratively. The authors did not discuss why the numbers of students choosing
collaborative testing decreased.
The Stanford University study noted an increase in achievement of approximately
nine percentage points in the partner exam situation over the non-collaborative situation.
They also noted that low scores were eliminated in the collaborative setting. Students
involved in this study that chose to work in pairs noted a reduced anxiety level both
during testing and when studying prior to the exam. Unfortunately, the students involved
in this study thought that their classmates were more likely to expend minimal effort
when tested collaboratively.
In a study conducted among medical students at Wayne State University School
of Medicine, the students were assigned to pairs on the day of a quiz.53 Again, these
students did not work with their partners at any other time during the course. The
students took a quiz individually, and then took the same quiz again with a group. The
individual portion of the quiz counted for 80% of the students' grades. The group portion
accounted for the remaining 20% of the quiz grade. The students had a 6% increase in
overall quiz scores. The score on the group portion was higher than that of the individual
portion. The students involved in this study stated that they liked the collaborative
format better than the traditional quiz format, but no mention was made of their anxiety
The only study that made use of a group larger than pairs was in a non-majors
statistics class at Colorado State University.54 Students were placed into groups in a
random fashion, with the caveat that each group of four students would contain at least
two females. Students were assigned to their groups on the day of their exam, and did
not work with their groups at any other time or for any other exam. Each group handed
in only one set of answers. There was a large increase in student achievement on the
exams from previous semesters (approximately 15 percentage points). However, these
students had to take an individual final, and the scores dropped dramatically. Not only
were the final scores almost 30% lower than the collaborative exams, they were
approximately 7% lower than the students who had never taken a collaborative exam.
The author contributed this decrease in score to the lack of conscientious studying habits
in his students. However, this attitude would have been apparent in all the exam scores.
The students involved in this statistics course were in favor of collaborative testing. No
mention was made of student anxiety levels.
At least two additional studies discussed what the authors termed collaborative
testing but were not similar to the collaborative testing formats described above.55,56 In
one study performed in chemistry at several universities, including Florida State
University, students watched a demonstration in class and were given exam questions to
take home that had to be turned in forty-eight hours later. Students had the ability to
work together if they wished, but did not have to, nor was the collaborative process
monitored in any way. The researchers stated that their version of collaborative testing
was effective because students in a comparable section of chemistry that did not have the
take-home exams based on demonstrations were unable to answer questions on the final
exam related to demonstrations they had not seen in class. This outcome is to be
expected students should not be expected to answer questions on something that they
had not seen or been taught.
In the second study, students in a composition course at Widener University were
allowed to free-form talk with each other during an exam on grammar skills and works
cited pages. The collaborative process was not monitored and students were allowed to
work with whomever they wished. Students were not required to work together, but the
students who worked individually were seated in the same noisy environment as their
collaborative counterparts. There was a slight increase in student achievement in this
course, but no mention was made of student anxiety level.
The studies discussed above noted an increase in student achievement, although
the increases can be considered modest. But most of the studies omitted aspects of
collaborative testing that could be important, such as handing in individual or group
answers and student anxiety levels. Therefore collaborative testing was implemented in
CHM 1025 during the spring semester of 2004 to test some of these aspects. Prior to this
semester, students had worked together in groups during the class period. Once at the
exam, however, students were told that collaborating was cheating, and were strongly
discouraged from doing so. In many respects, this was unfair to the students. If
collaboration is expected in the class, it should be carried into the situation where the
students are evaluated on their progress. All exams given in CHM 1025 during the
spring 2004 semester, including the final exam, were partially collaborative in nature.
There were two sections to each exam, an individual section and a group section.
The group section was administered during the first hour of the two-hour exam block and
was worth 30% of the exam grade. The individual section was administered during the
second hour of the exam and was worth 70% of the exam grade. Questions were
different on each section of the exam. The group section of the exam had harder
questions than were asked to the students individually. It was believed that students
would be better equipped to answer more difficult questions if they were working with a
group than if they worked alone. The students worked in the same collaborative groups
that they worked with every class period. The rationale behind the consistent grouping
was that students would be more comfortable working with people they already knew in
an otherwise tense exam situation.
The first exam offered during the course was arranged in the reverse order, with
the individual section offered first, and the group section offered second. On this exam,
half of the questions on the group section were the same as on the individual section. It
was believed that students would re-work these problems in their groups and perhaps
arrive at different answers than on the individual section. However, students merely
recalled what they had chosen on the individual section and answered with a "majority
rules" approach. Students also did not like waiting for all students to finish the individual
portion before beginning the group section. In a survey given at the end of the semester,
nearly 70% of the students stated that they did not like having the individual section of
the exam first. After this first exam, the individual and group sections were switched,
and no questions were repeated. The students still had to wait before beginning the
individual section of the exam, but they were now able to talk with their group members
while waiting. Several groups used this time to discuss concepts that members were
having difficulty with.
The grades on the collaborative exams were compared to grades on non-
collaborative exams given in the spring semester of 2003 to determine how the type of
exam affected student achievement. Spring of 2003 was chosen as the comparison
semester because the class size was approximately the same and the classes were
comprised of similar types of students. Students who enroll in CHM 1025 in the spring
semester are usually a semester behind in their majors. A comparison was not made to the
fall semester because the class sizes are extremely different (approximately 250 versus
approximately 1500) and the fall students are usually on-track with their majors.
On the first of four semester exams, the students who took the exam
collaboratively earned an average of 116 points out of 150. This was significantly higher
than the average 96 points earned by the spring 2003 students (p-value <0.0001). The
range of results on the collaborative exam also narrowed, with no students scoring below
55 points (Table 4-1). These results were promising, especially since the instructors
considered the collaborative portion of the exam more difficult than exams offered in
Table 4-1: Performance on exam 1
Term Instructor(s) Exam Average Standard Range
Spring 2003 KA Individual 96 26.3 25-142
Spring 2004 KA & KZ Collaborative 116 18.6 55-150
The percent of students in each section that earned each letter grade was also
compared (Figure 4-1). Overall, students in the collaborative sections had a much higher
percentage of A and B grades than in the non-collaborative sections. As indicated by the
decreased standard deviation spread, the collaborative section also had a much lower
percentage of E grades than the non-collaborative section. The spring 2004 section also
had a skewed bell curve distribution of grades, centered around a B+. The non-
collaborative section had the usual asymmetrical distribution of grades.
A B C D E
Figure 4-1: Percent of students earning each grade on exam 1
By exam two, however, these promising initial results were reversed. As shown
in Table 4-2, there was no significant difference in the grades earned in the spring 2003
and spring 2004 sections (p-value 0.9008). However, the range was again narrowed, with
the lowest scores being eliminated from the spring 2004 section. The standard deviation
of the average scores was also narrowed, most likely due to the lack of extremely low
Table 4-2: Performance on exam 2
Term Instructor(s) Exam Average Standard Range
Spring 2003 KA Individual 104 34 0-150
Spring 2004 KA & KZ Collaborative 105 20 30-150
The percent of students in the collaborative and non-collaborative sections that
earned each letter grade was again compared (Figure 4-2). The collaborative section had
35 S03 S04
A B C D E
Figure 4-2: Percent of students earning each grade on exam 2
fewer students receiving A and B grades than did the non-collaborative section. The
collaborative section earned a higher percentage of C and D grades than the non-
collaborative section, but fewer E grades. Because the lower grades were eliminated, the
students that would normally be earning E grades instead earned D grades, bringing that
percentage up in the collaborative section. However, the skewed bell-shaped distribution
is not present in the collaborative section. Both sections now have the usual
asymmetrical distribution of grades.
Exam three in the collaborative section was an unusually difficult exam. The
instructors in the spring semester of 2004 wrote a more elaborate exam than in 2003, and
multiple steps were needed to solve the problems. This high degree of difficulty was
reflected in the grades for this exam in the collaborative section. However, even with this
increased level of difficulty for the collaborative exam, the average scores were slightly
higher than those of the spring 2003 semester (Table 4-3). However, the difference was
not significant (p-value 0.6564). The range does not appear to be narrowed in the
collaborative sections as in the other sections, however the student that earned only five
points was a repeat student from the fall of 2003. This student had already attempted and
failed CHM 1025. The student receiving the next lowest grade in the collaborative
section earned 25 points on the exam. The majority of the lower scores were again
eliminated on exam three.
Table 4-3: Performance on exam 3
Term Instructor(s) Exam Average Standard Range
Spring 2003 KA Individual 80 35 3-150
Spring 2004 KA & KZ Collaborative 82 24 5-150
The percentages of students receiving each grade were compared for differences
in achievement between the two sections (Figure 4-3). The high difficulty level of exam
three was reflected in the numbers of students earning each grade. Very few students
earned an A grade, and the majority of the students earned an E grade. However,
excluding the percentage of students that earned an A, the distribution of grades is not
significantly different from the spring of 2003, which did not have as difficult of an
B S03 N S04
Figure 4-3: Percent of students earning each grade on exam 3
By the fourth semester exam, although the collaborative exam was more difficult
than the non-collaborative exams, it was not as difficult as exam three. The student
averages reflected this, and the average 107 points out of 150 possible earned by the
collaborative section was significantly higher than the average 88 points earned by the
non-collaborative section (p-value <0.0001). As shown in Table 4-4, the ranges and the
standard deviations were again narrowed, due to the elimination of the lower grades.
Table 4-4: Performance on exam 4
Term Instructor(s) Exam Average Standard Range
Spring 2003 KA Individual 88 33 19-150
Spring 2004 KA & KZ Collaborative 106 24 50-150
Students in the collaborative section earned more A, B and C grades than in the
non-collaborative sections (Figure 4-4). The collaborative students also earned fewer D
and E grades than did the non-collaborative students. Although both curves are
asymmetrical, the distribution of grades is what was expected when collaborative testing
A B C D E
Figure 4-4: Percent of students earning each grade on exam 4
However, by the time of the mandatory cumulative final, there was a decrease in
student performance on the exam. As shown in Table 4-5, the students in the non-
collaborative section had a significantly higher 115 points out of a possible 200 than the
average 106 points earned in the collaborative section (p-value 0.0280). The range of
scores in both the collaborative and non-collaborative sections was also approximately
equal. However, the standard deviation in the collaborative section was again narrowed.
It is important to note that students are told that they must earn 100 points on the final in
order to pass the class. Those students who have already reached or are near the
minimum cutoff for passing usually aim for exactly 100 points. This minimum grade is
an E on the exam, thus the high number of E grades in both sections.
Table 4-5: Performance on the final exam
Term Instructor(s) Exam Average Standard Range
Spring 2003 KA Individual 115 35 30-190
Spring 2004 KA & KZ Collaborative 106 30 20-190
Students in the collaborative section earned fewer passing grades on the final
exam than did students in the non-collaborative section (Figure 4-5). They also earned
more failing D and E grades than did the non-collaborative students. Again, this
performance on the final is disappointing. However, starting in the summer of 2004,
CHM 1025 will no longer be graded on an S/U basis and will be given a letter grade. The
final, as well as all the other exams, is expected to become much more important to
students when they will be receiving a letter grade for the course.
There was one distinct advantage to the collaborative final noted by the
instructors. Each semester, there are a few students who do not earn the minimum grade
on the final but who would otherwise pass the class. These students either come in to the
A B C D E
Figure 4-5: Percent of students earning each grade on the final exam
instructors' offices or email the instructors pleading for points. The number of students
who requested extra points in order to pass the class decreased markedly from prior
semesters. The collaborative exams possibly made students more comfortable with the
grades they had earned.
Another benefit to the collaborative exam format is that the percent of students
taking the exams increased over prior semesters. Generally, the largest numbers of
students take exam one, and then the numbers decrease, occasionally increasing again for
the final. Although the numbers did still decrease, the percent of students taking each
exam was higher in spring 2004 versus spring 2003 (Table 4-6). The percentages of each
student are based on initial, and not final, enrollments. More students were kept in the
class in the collaborative setting than in the non-collaborative setting.
Table 4-6: Percent of students taking each exam
Term Exam Exam 1 Exam 2 Exam 3 Exam 4 Final
Spring 2003 Individual 83.2% 85.6% 73.4% 59.5% 63.0%
Spring 2004 Collaborative 91.7% 90.1% 83.3% 76.6% 74.2%
In order to determine if students were performing better on the individual or
group portion of the collaborative exams, the grades on each portion for section 5269 in
the spring of 2004 were normalized to 100%. These percentages were then compared to
determine if a difference in achievement level existed between the two sections of the
exam. As shown in Table 4-7, there was no difference between the collaborative and
individual sections of the first exam (p-value 0.7954). Students performed equally as
well on both portions of the exam. This demonstrates that students can answer more
challenging questions as a group with as much accuracy as they are able to answer easier
individual questions. When the percentage scores on the individual portion of the exam
were compared to the percentage scores on the exam given in the spring semester of
2003, students who took the exam collaboratively had a significantly higher 77.2%
average than the average 64.2% earned last spring (p-value <0.0001). Students were
better prepared for the exam during the semester it was offered collaboratively than when
it was offered individually.
Table 4-7: Performance on each section of the exam
Section Exam 1 Exam 2 Exam 3 Exam 4 Final
Individual 77.2% 60.5% 51.7% 57.4% 49.3%
Group 78.2% 93.2% 50.2% 100% 68.0%
On the second of the four semester exams, there was a significant difference
between the average 93.2% earned on the collaborative portion of the exam and the
average 60.5% earned on the individual section of the exam (p-value <0.0001). The
students were able to answer the more difficult collaborative questions with greater
accuracy than the easier individual questions. However, when the individual scores were
compared with the test scores from the spring 2003 semester, the students were less
prepared for the individual section of the exam. The students in the spring of 2003
earned an average 69.6% on the exam. This was significantly higher than the average
60.5% earned on the individual portion in the spring of 2004 (p-value <0.0001). After
experiencing the first collaborative exam, students did not prepare as well for the second
exam. However, this lack of preparation was discussed in class, and student achievement
improved on the next two exams.
On the most difficult exam, exam three, the students performed equally as well on
both sections of the exam, earning an average 51.7% on the individual section, and an
average 50.2% on the collaborative section (p-value 0.7158). Again, this demonstrates
that students can answer more challenging questions as a group with as much accuracy as
they are able to answer easier individual questions. When the scores on the individual
section are compared to the scores on exam three during the spring of 2003, the students
in both semesters performed equally as well (p-value 0.3571). On exam three in the
spring of 2003, students earned an average of 53.9%. During the spring of 2004, students
earned an average 51.7% on the individual section of the exam. The students who took
the exam collaboratively were as prepared as were the students who took an individual
exam during the spring of 2003.
For the fourth of the semester exams, students' scores were significantly different
on the collaborative and individual sections (p-value <0.0001). The students earned an
average 57.4% on the individual portion, and over 100% on the collaborative portion, due
to the extra credit question on that section. Students correctly answered virtually every
difficult question on the collaborative section of this exam. The students were also as
prepared for the exam as the spring 2003 students were, earning essentially the same
percentage on the individual section of the exam (p-value 0.7407). The students in the
spring of 2003 earned an average 58.4%, while the students in the spring of 2004 earned
an average 57.4%.
For the final exam, the students earned a higher 68.0% average on the group
section of the exam, versus an average 49.3% on the individual section of the exam (p-
value <0.0001). This final was cumulative, therefore students were expected to answer
more difficult questions on the collaborative section, tying several concepts that they had
learned over the course together into one question. The students did this admirably,
earning almost 70% on this section. However, most students were aiming for the
minimum 50% that was required of them on the final, and this was reflected in their
individual scores. Most students felt confident after taking the group portion of the exam,
and did not try as hard on the individual portion of the exam. This is further reflected
when the individual grades are compared to the exam grades in the spring of 2003.
Students in the spring of 2004 earned an average 49.3 % on the individual section of the
exam. This was significantly lower than the average 57.4% earned during the spring of
2003 (p-value 0.0006).
Overall, students were able to answer the more difficult questions with more
accuracy when working in a group. The students had more problems with the individual
section, even though those questions were not as difficult as the questions they had
answered beforehand. For at least three of the five exams offered during the course, the
students were at least as prepared as their counterparts from a year earlier. Using
collaborative exams, students are able to answer harder questions and perform as well as
expected on the individual sections.
During the spring semester of 2004, eight students required a makeup exam for
varying reasons. These students were a random-sampling of students from the two
sections being offered that semester. They were given the same final as the students in
the spring of 2003, with the noted exception that the students in 2004 had the exam split
into collaborative and individual sections. Although eight students is not a large number,
the results on this exam were compared to the section 5267 in spring 2003. Only 29
students took the exam in that section, so the numbers are roughly comparable when
compared to the 80 students that took the final in section 5269 in the spring of 2003.
The makeup students in the spring of 2004 had a significantly higher average of 134 out
of a possible 200 than the 104 average of the spring 2003 students (p-value 0.0281). As
shown in Figure 4-6, students in the spring of 2004 had a higher percentage of A and C
grades, and had approximately equal percent of B grades. Concerning the failing grades,
although the students in the spring of 2004 had a higher percentage of D grades, there
were no E grades for these students. Those E grades were brought up to D grades. The
range and standard deviation of the scores in the spring 2004 section were also narrowed
due to the elimination of the lowest grades (Table 4-8).
Table 4-8: Grades on the makeup final
Term Instructor(s) Exam Average Standard Range
Spring 2003 KA Individual 104 36 40-170
Spring 2004 KA & KZ Collaborative 134 20 110-170
SS03 O S04
A B C D E
Figure 4-6: Percent of students earning each grade on the makeup final
The percent scores on the individual section of the makeup final in the spring of
2004 were compared to the percent scores on the final given in spring 2003. Students
taking the makeup earned an average of 49.1% on the individual section of that exam.
This was not significantly different than the 51.8% earned by the students in the spring of
2003 (p-value 0.6942). Students in the makeup section were as prepared as their
counterparts of the semester before for the individual section of the exam, but were able
to perform much better on the group section. The difficulty of the questions remained the
same between the two exams. Therefore students who took a collaborative exam had a
higher achievement level than the students who took a non-collaborative exam.
A study performed at the University of North Carolina at Charlotte has shown that
80% of the students who failed their first exam in the first semester general chemistry
class failed the course.4 With this in mind, the failure rates of the students that failed the
first exam in the spring of 2003 and the spring of 2004 were compared to determine if
collaborative learning had an effect in this regard. During the spring semester of 2003,
88% of the students who failed the first exam failed the course. During the spring of
2004, 70% of the students who failed the first exam failed the course. The percent of
students who failed the course after failing the first exam is decreased in the collaborative
section. Collaborative testing can help increase the achievement of students who would
otherwise fail the course.
Students were given an anonymous survey regarding collaborative testing on the
last day of classes during the spring semester of 2004. The first two questions on this
survey were about student anxiety levels. As shown in Figure 4-7, the majority of
U Studying M Exam
Reduced No Effect Increased
Figure 4-7: Student reported anxiety levels when studying and when taking the
students that responded believed that collaborative testing reduced their anxiety both
when studying and during the actual exam. As discussed earlier, high anxiety levels lead
to decreased student performance on examinations. As the collaborative method of
testing reduced student-reported anxiety, even though the questions were more difficult,
the students are more likely to perform better on these exams than on individual exams.
The students were also asked if they would have preferred to take the entire exam
individually. The majority of the students (68%) stated that they preferred the
collaborative exam to an individual exam. Only 11% of the students stated that they
would have preferred individual exams. The remaining 21% of the students wanted the
ability to choose to take an exam either collaboratively or individually. Realistically, this
would be virtually impossible to achieve with the current test design of more difficult
Questions on the collaborative section of the exam. The students who opted to take the
exam individually would most likely be unable to answer these difficult questions, and
their achievement levels would drop considerably.
One concern when implementing collaborative testing is that students will study
less when they know they have other students to rely on during the exam. This is one
reason behind the percent values given to the two portions of the exam it was believed
that students would continue to study if 70% of their grade depended upon their own
work. This was borne out in the students' response to a question on the survey about
studying. Approximately 80% of the students reported that they studied for the
collaborative exam about the same as they would have studied for an individual exam.
As an unexpected result, 11% of the students reported that they studied more for the
exam because it was collaborative. Perhaps this is due to the peer pressure of performing
in a group, and not wanting to let one's group-mates down. Only 9% of the students
reported that they decreased their usual amount of studying because the exams were
collaborative. It is promising that students are studying the same amount as they usually
would for the collaborative exams. This reduces the fear that students will not prepare as
well when they are able to collaborate during an exam.
Another problem that arises when collaborative testing is discussed is social
loafing. Many students and instructors believe that when students work together there
will be one or a few students who do all the work, while the rest of the group does
nothing. These students would get the same grade as those students who worked hard
and studied for the exam. This was another reason behind the percentage values given to
both exam sections. There was no way that a student could abstain from working on the
exam and still pass. When asked, only 23% of the students stated that they would have
removed someone from their group for social loafing. Another 18% were unsure if they
would have removed someone for not participating. The majority of the students (60%)
stated that social loafing was not a problem in their group.
When asked how the groups solved problems during the exam, the majority
(59%) of the students responded that the questions were split up among group members.
An additional 38% stated that every student in the group did every question. When group
members arrived at different answers to the questions, these differences were solved with
a mixture of a majority rules approach and using the answer of the student who had the
best logic. Only 3% of the class (four students) stated that only one person answered all
the questions. These four students were most likely in the same group. These results
demonstrate that students are not expecting one member to do all the work, nor are some
students relying on the results of their group-mates.
A promising result from this survey was that 70% of the students believed that
collaborative testing increased their grade. Perhaps because the questions were more
difficult, students believed that they performed better working with a group than they
would have if they had worked individually. Overall, with these promising survey
results, the fact that more students are taking the exams, and the elimination of the lowest
grades, collaborative testing should be continued in later semesters of this course.
Collaborative testing has proven to be an effective assessment method in CHM 1025, and
is fairer to the students who are expected to work collaboratively during class meetings.
As student enrollment in higher education increases and funding decreases, class
sizes have increased dramatically. A professor facing 300 students often feels there is no
choice but to lecture. The only active role that a student can take in a lecture format class
is to take notes. As discussed earlier, collaborative learning is one way to transition from
the traditional lecture into a more student-active learning environment. Another method
to increase student involvement is the use of peer mentors. Peer mentors are
undergraduates of similar age and educational background as the students enrolled in the
course who are recruited to help teach the students.57
After 45 years of research on universities, Newcomb stated that professors have
little effect on the study habits and curriculum choices of their students.58 In general, the
student's peer group has the highest influence on the student. Newcomb notes that there
is little similarity between the interests that peer groups generate and support and the
academic objectives supported by a university. Using peer mentors in a classroom can
generate peer support for academic objectives of the university. Peer mentors can stress
that students should spend more time on studying and homework. Although this message
is no different from the professor's, Newcomb has shown that it will have more effect
coming from peers. Peer mentors can help students take a more active approach to their
In a traditional lecture class in chemistry, students meet once a week in a small
recitation section, headed by a teaching assistant (TA). The purpose of the smaller class
is to give the students an opportunity to ask questions in a small group without the
professor present. However, it has been shown that discussion sections function mainly
as small lectures, and the students are no more active in these types of sessions than in the
traditional lecture.59 A peer mentor, on the other hand, works closely with students, both
asking questions and offering information on a more informal basis than either a lecture
or a discussion section. This give-and-take with a peer mentor helps students take a more
active role in the learning process.
Robin and Heselton have discussed three aspects of a good peer mentor.60 A peer
mentor engages students in casual conversation in greeting and in closing a session.
Although this can be related to events outside the class such as a recent football game, it
can include generalizations about the course or an upcoming test. Casual conversation is
important since it can highlight differences between the peer mentor and the instructor for
the course. The instructor is often viewed as more of an authority figure (i.e. someone
who would never ask about the football game) than the peer mentor, who is often viewed
as more of a friend.
A good peer mentor offers feedback to the students. This feedback should be
personal and related directly to the student's progress in the course. The mentor can offer
feedback on graded items, such as quizzes or exams, but also on verbal explanations or
defenses of answers given by the students. In the introductory chemistry class, for
instance, peer mentors should offer feedback on dimensional analysis as well as logic
used in problem solving.
The third aspect of a good peer mentor, as described by Robin and Heselton, is
praising the student's performance. Peer mentors should offer immediate and positive
feedback to the students whenever possible. This helps maintain students' morale and
interest in the course. This aspect of a good peer mentor is truly an extension of the
feedback aspect described above.
Although other researchers agree upon these three aspects of a good peer mentor,
often two additional aspects are described. A good peer mentor should also prompt the
students.61 These prompts should consist of verbal clues that help students answer their
own questions. The peer mentors should not simply provide answers. Instead, they
should help students arrive at their own answers.
The fifth aspect of a good peer mentor is assisting the instructor with
administrative tasks.62 These administrative tasks can include proctoring and grading
exams, handing out and collecting material from students, and gathering feedback about
the course from the students.
An important aspect of a good peer mentor that does not appear to be described in
the literature is that the peer mentor should have a thorough knowledge of course content
and material. Although the mentors are not expected to have the same in-depth
knowledge of subject material as the instructor, they must demonstrate a proficiency in
the covered material. It would be difficult for a mentor to assist students with material
that the mentor does not understand. This does not mean confining the mentor pool to
subject majors. Often students in other majors are competently able to assist students
with course content. In the introductory general chemistry class, only a very few peer
mentors (approximately 7%) are chemistry majors. Most mentors are pre-professional or
engineering majors with demonstrated proficiency in the course, although we have
several business and finance majors who are excellent mentors.
Several studies have been performed on the effectiveness of peer mentors versus
staff instruction in small groups.63-67 These studies were set up where the staff instructor
was expected to act in a similar manner as the peer mentors. Students were assigned
problems in small groups that were headed either by an instructor or a peer mentor. In
most of these studies, the group leader was the only source of information in the class.
Therefore, students in groups headed by a peer mentor did not meet with an instructor.
The results of these studies were mixed, with results ranging from an increase in student
achievement in the mentor-led groups, no difference in achievement between the two
types of groups, and increased student achievement in the instructor-led groups. This
mixture of results makes it difficult to draw conclusions about the effectiveness of the
peer mentor method of instruction.
Most of the studies noted positive side effects beyond the mixed student
achievement in the mentor-led groups. These studies noted that peer mentors were able
to understand the nature of student difficulties with subject-material better than the
instructor, having recently gone through the course themselves. Peer mentors were also
more interested in the daily lives of students, their study experiences, and student
personalities. Peer mentors also directed students' attention to upcoming exams more
than did the instructors. Additionally, peer mentors displayed more supportive behaviors
than did the staff instructors. Although the quantitative academic benefits of peer
mentors were originally not clear, peer mentors were introduced in CHM 1025 at the
inception of the course. However, students did not meet solely with mentors.
In CHM 1025 students meet with both an instructor and peer mentors every class
session. For the first 15 to 25 minutes of the 50 minute class, the instructor delivers a
brief lecture on the topic being covered. After the lecture, the students break into their
small collaborative groups to solve problems based on the topic just covered. Peer
mentors assist students with problem solving during the collaborative sessions. In this
manner, students get the benefits of having an instructor impart background information,
and also benefit by working closely with mentors who are able to assist them with
difficult concepts or problems.
Peer mentors are actively recruited at the end of every semester. Students who
have passed the course and evidenced good helping behavior with their group-mates are
asked to come back the following term and assist as a mentor. Current peer mentors
usually know the students in their groups and have good suggestions for future mentors.
These students do not have to be academically the best in the class. Some of our best
mentors are students who struggled in the class but who demonstrated helping behavior
in their groups. Mentors assist in the classroom for as many semesters as their schedules
permit several have mentored every semester up until graduation. There were only six
peer mentors during the first semester that this program was implemented. Now, over
100 students have volunteered to be mentors.
Initially, the mentors were paid a stipend for helping in the class. The stipend
money ran out after the second semester of the program. Now peer mentors enroll in a
one-credit 4000 level chemistry class called Supervised Teaching. As an added
incentive, the peer mentors work closely with an instructor who is then able to write an
excellent recommendation based on the mentor's responsibility, leadership skills, helping
behavior and teamwork. Most mentors begin the program during the second semester of
their freshman year. For the first few years the mentors are enrolled in the program, the
CHM 1025 instructor is the only professor who knows the mentors' names due to the
large number of students at UF.
The racial makeup of the peer mentors is diverse, although heavily weighted
toward minorities and students of color, as is the CHM 1025 class. Many of the mentors
speak a language other than English as their first language. Several mentors have been
overheard explaining a concept in their native language to other non-native English
speaking students in the class. Often this added explanation in the student's native
language makes the concept understandable to the student.
Peer mentors attend a one-hour training session during the first week of each
semester. All peer mentors attend, including those who have previously participated in
the program. During this session, peer mentors are taught how to get a whole group
involved in problem solving, in large part by facilitating group-member interactions. Peer
mentors are informed that they are not expected to just give an answer to students. They
should first check the student's logic and dimensional analysis setup. It is also stressed
during this meeting that the peer mentors are not expected to have all the answers to the
students' questions. The mentors are informed that "I don't know" is a perfectly
acceptable response, as long as that mentor then finds someone who can answer the
question for the student.
After this training meeting, staff meetings are held every other week. At these
staff meetings, peer mentors are given the answer keys to the problems that will be solved
during the upcoming two weeks of class. Any issues that have arisen over the previous
two weeks are also discussed. Experienced peer mentors are comfortable bringing up
discussion points as early as the first meeting. As the new peer mentors see how well
these ideas and criticisms are received, they begin bringing up issues as well. By the
third or fourth meeting there is usually a good dialogue between the instructor and the
mentors. Upcoming events such as exams and changes in curriculum are also discussed
in these meetings.
Students are much more likely to offer complaints and suggestions to the mentors
during class than they are to offer them to the instructor. Therefore, it is important to
have a time set aside when mentors can bring up their and the students' concerns. It is
also helpful to discuss the reasoning behind policy decisions in the class with the mentors
because they are able to relay this to the students when an unpopular decision is made.
When the mentors understand the logic behind a decision, such as the minimum point
requirement on the final, students are often more accepting of the policy than when the
decision is only announced by the instructor.
The peer mentors are also an excellent resource for suggestions about how the
class can be changed for the better. Not only have they recently participated in the
course, the students in the class are constantly offering suggestions to them. For
example, the idea to reverse the group and individual portions of the collaborative exam
was first brought up by a peer mentor, and the majority of the other mentors strongly
It was believed that having the mentors in the class would help personalize the
class for the students. Normally, the chemistry classes have one instructor and 297
students. With the mentors in the class, this ratio was brought down to one peer mentor
for every six to 18 students, depending upon the semester. These mentors often know
their students individually, and question the students on their progress as well as on their
absences. It was believed that this individual attention would keep the students coming
to class and learning the material. In order to test this assumption, the withdrawal rates of
sections that used peer mentors were compared to sections that did not utilize them.
During the fall of 2003, five sections of CHM 1025 were offered. Three of these
sections did not utilize the peer mentors, while the other two sections did so extensively.
As shown in Table 5-1, the withdrawal rates in the sections that did not use peer mentors
was much higher than in the sections that did use them (p-value <0.0001). Although it
cannot be determined conclusively that the peer mentors affected the withdrawal rate, it is
likely that having a peer mentor checking on students' progress helped lower this rate.
Table 5-1: Percent of students withdrawing from each section during the fall of
Section Instructor Peer # Students # Students Percent
Mentors Enrolled Withdrew
0675 LL No 257 27 10.5%
0676 KW No 303 37 12.2%
0677 LL No 303 23 7.6%
4284 KZ Yes 297 10 3.4%
5253 KA Yes 300 10 3.3%
The sections that utilized peer mentors during the fall of 2003 were held in the
afternoon, while those sections that did not use the peer mentors were held during the
first three 50-minute periods of the day, beginning at 7:25 a.m. In order to ensure that the
time of day did not skew withdrawal rates, the rates for fall of 2003 were compared to the
rates for fall of 2002, where the classes were scheduled at exactly the same times. As
shown in Table 5-2, the percent of students withdrawing from sections that did not utilize
peer mentors was higher, regardless of the time of day the sections were held (p-value
Table 5-2: Withdrawal rates versus mentor utilization
Fall 2003 Fall 2002
Mentors No Mentors No Mentors No Mentors
(afternoon) (morning) (afternoon) (morning)
# Enrolled 597 863 517 900
# Dropped 20 77 29 63
% Dropped 3.35% 8.92% 5.61% 7.00%
In a voluntary, anonymous survey, students were asked to rate the helpfulness of
the peer mentors. The majority of the students (74%) believed that the mentors were very
helpful to have in class (Figure 5-1). An additional 17% thought the peer mentors were
somewhat helpful. Only 9% of the students thought that the mentors were never helpful.
The majority of students taking CHM 1025 believe that the peer mentors help them better
understand the material. This is a very positive result.
very somewhat not at all
Figure 5-1: Percent of students rating the helpfulness of peer mentors
Although it is easy to see how the students benefit from having the peer mentors
in class, it is important to recognize that the peer mentors also benefit from the program.
The mentors have an opportunity to review the basic chemistry that they are expected to
know in CHM 2045 and CHM 2046. Mentors who have progressed beyond this phase of
chemistry have stated that mentoring has helped them review for such exams as the
MCAT, DAT, and PCAT. The mentors also have a comfortable relationship with the
CHM 1025 instructors, and have no problem asking for help in other chemistry subjects,
such as organic and analytical chemistry.
It is important to recognize that the peer mentors have mostly taken CHM 1025.
In general, the mentors that are assisting in the classroom were under-prepared for
college chemistry when entering UF. These mentors are now much more confident in
chemistry, and several have chosen to major in chemistry. Also, several mentors who
took CHM 1025 in the summer of 2003, and CHM 2045 in the fall of that year took
CHM 2051, an honors version of CHM 2046. Students who were initially under-
prepared have progressed far enough to take an honors chemistry class.
When the CHM 2045 grades for students who have participated in the peer
mentor program are compared to those students who are not in the program during the
fall semester of 2003, there is a significant difference between these two groups (p-value
<0.0001). As shown in Table 5-3, the students who participated in the program have a
higher average CHM 2045 grade of B+ versus the average grade of C for the students
who have not participated in the program. When the peer mentor grades are compared to
those students who also required the introductory course, the peer mentor grades are
again significantly higher (p-value <0.0001). Students who participate in the peer mentor
program are more successful in later chemistry courses than students who do not
participate in the program.
Table 5-3: Mentor grades in CHM 2045
Required Ready for Required
CHM1025 CHM 2045 CHM 1025
GPA 3.71 2.32 2.14
Letter Grade B+ C C
The mentors who are utilized in the CHM 1025 course have an overall positive
experience with the program. Mentors comments from some of the required end-of-
semester papers have included:
There is something very special about helping people ...
Sitting down with them and helping them to understand
something that they couldn't before is just incredible.
I brushed up on some of my basic chemistry and was able
to help some students along the way.
I liked being able to help students and at the same time let
them know that there are people like them who can do
The students in the class were great to work with. I liked
going to students who were struggling with a problem and
helping them think it through and seeing them actually
learn as they were getting the answer. It is a good feeling
helping people gain knowledge.
The mentoring program is an effective way to personalize the class for the
students. The mentors help make the students more comfortable in the class, but the
instructor still has the authority necessary to control the class. The mentors are able to
help the students immediately with any questions and problems they have. In a large
class of 300 students, the instructor would never be able to answer all the students'
questions individually. However, the mentors provide students with an opportunity to
ask questions in small groups, ensuring that students do not leave the class with
Confidence in the peer mentors among departmental instructors has also grown as
the program has progressed. Initially, when peer mentors helped proctor exams some
instructors in the department were skeptical. They did not like the idea of second
semester freshman proctoring exams for their peers. As the mentors have proven
themselves reliable and trustworthy this skepticism was replaced with an interest in the
program. Instructors in CHM 2045 and CHM 2046, who have not used the peer mentors
in their classes, now occasionally use the mentors to proctor exams when there are not
enough graduate students available. During the spring semester of 2004, the department
was short one TA in CHM 2046. An experienced peer mentor was asked to TA for this
course, the first time that a non-chemistry major or minor has been a TA for a chemistry
The mentoring program helps the peer mentors review basic chemistry and work
on their social skills in the groups. Students who participate in the peer mentoring
program also have higher GPA scores in later chemistry courses than their counterparts.
Most students enrolled in CHM 1025 believe that the mentors helped them understand
chemistry. Withdrawal rates also are lowered in the classes where mentors are checking
on student progress and attendance. The program has had positive results both on the
mentors and on the students, and should be continued in future CHM 1025 courses.
Chemistry Readiness Assessment Exam
The ChRA is the exam taken by potential chemistry students before enrollment in
either the introductory CHM 1025 course or the mainstream CHM 2045 course. The
initial cutoffs for this exam were four or greater (out of eight questions) correct on the
math section of this exam, and nine or greater (out of 18 questions) correct on the
chemistry section. Students should have earned 50% on both sections in order to take
CHM 2045. These scores were only guidelines, and students were able to register for
whichever course they wished.
Students that earned only a four out of eight on the math section of the ChRA and
did not take CHM 1025 earned an average grade of D+ in CHM 2045. In order to pass
into CHM 2046, students must earn a grade of C, so this math cutoff is unacceptable.
Students that correctly answered five out of eight on the math section earned an average
grade of C. This C grade was extremely low. Students who correctly answered six out of
eight questions on the math section also earned a C in CHM 2045, but this C grade was
much stronger. Therefore, the minimum cutoff for the math section of the ChRA should
be six out of eight questions.
Students that correctly answered nine questions out of 18 on the chemistry portion
earned an average grade of C in CHM 2045. This was a strong C, so the minimum cutoff
for chemistry can be lowered. Students who correctly answered eight, seven, or six
questions on the chemistry section of the ChRA also earned a C grade in CHM 2045.
Students who correctly answered five of 18 on the chemistry portion earned an extremely
low C in CHM 2045. Therefore, the minimum cutoff for the chemistry section of the
ChRA should be six out of 18 questions.
Although these cutoff values are the same, students must correctly answer 75% of
the math section while only answering 33% of the chemistry section in order to succeed
in general chemistry. Student success in chemistry depends more on students' prior math
knowledge than their prior chemistry knowledge. Students who received a score of six
on the math section and six on the chemistry section of the ChRA were more likely to
receive an A in CHM 2045 than the students who did not receive these scores. Students
who received less than a six on both math and chemistry were more likely to withdraw
from CHM 2045 than their counterparts. Students who passed either the math section or
the chemistry section of the exam had mixed grades, but the students who passed the
math section performed better than the students who passed the chemistry section.
Effectiveness of CHM 1025
CHM 1025 was designed for those students who are under-prepared for the
mainstream CHM 2045 course. Student achievement in CHM 2045 was compared to
determine if CHM 1025 prepared these students for the mainstream course. The students
who had taken CHM 1025 had an average GPA score of 2.74 in CHM 2045, while those
students who had not taken CHM 1025 had an average GPA score of 2.12. Former CHM
1025 students are better equipped to succeed in CHM 2045 than the students who did not
need the course. Students who failed CHM 1025 but enrolled in CHM 2045 anyway had
an average GPA score of 0.59. Therefore, once CHM 1025 is attempted, it must be
passed before the student can advance to CHM 2045.
The average GPA 'scores of students in CHM 2046 that had taken CHM 1025
were approximately the same as those students that did not take the introductory course,
and were about a B. Although these scores are not significantly different, it demonstrates
that initially under-prepared students are succeeding in the course at the same rate as their
The number of students who continued into CHM 2046 was compared to
determine if CHM 1025 had an effect on this rate. The majority of the students (68.0%)
that continued on to CHM 2046 in the fall semester of 2003 were initially enrolled in
CHM 1025. Of the students that passed CHM 2046 and were eligible to continue in
chemistry, the majority (69.4%) was initially enrolled in CHM 1025. Therefore, former
CHM 1025 students are more likely to continue to CHM 2046, and are more likely to be
eligible to continue into Organic than their counterparts.
The students who had to repeat either CHM 2045 or CHM 2046 were also studied
to determine if CHM 1025 had an effect on this rate. Only 28.2% of the students who
had to repeat CHM 2045 had taken CHM 1025, and only 7.4% of the students who had to
repeat CHM 2046 had taken CHM 1025. Therefore, CHM 1025 students are better
prepared to succeed in later chemistry classes on the first attempt.
During the fall semester of 2002, students were placed into groups in CHM 1025 in a
random fashion, by student-choice, or by achievement. There were two sections that did
not use collaborative learning in this term. There was no difference in average final
scores in the random, student-choice, or non-collaborative classes. All three of these
sections had a higher final GPA than the section grouped by achievement. Therefore,
using student achievement to place students into groups is not an effective method of
placement. The fact that collaborative learning did not have an effect on final GPA was
disappointing, but by the fall of 2003 this was fixed.
During the fall semester of 2003, there were two collaborative sections and three
non-collaborative sections of CHM 1025. The students in the collaborative sections had
a higher GPA score than in the non-collaborative section. Collaborative learning helped
the students succeed in CHM 1025.
Attendance rates in CHM 1025 were compared to determine if attendance had an
effect on student achievement. Students who attended less than 25% of the time earned
an extremely low E score in CHM 1025. Those who attended between 25% and 50% of
the time earned a higher E score in CHM 1025. Students who attended between 50% and
75% of the time earned an average grade of D in CHM 1025. Students who attended
more than 75% of the class sessions earned a B in the class. Only those students who
attend class at least 75% of the time can be expected to pass.
Group placement methods were compared to determine if one particular group
type affected student attendance. Students who were placed randomly into groups
attended class approximately 83% of the time. Those who were placed into groups by
achievement attended class approximately only 68% of the time. This further
demonstrates that placing students into groups by achievement is not an effective method
of increasing student success.
Prior studies that had used collaborative testing as an assessment method showed
a modest increase in student achievement. Collaborative testing was implemented in
CHM 1025 during the spring semester of 2004. The exams consisted of two sections, an
individual section worth 70% of the student's grade, and a group section worth 30% of
the student's grade. Students worked in the same groups during the exams as they
worked with in class.
The effect of collaborative testing on student achievement was mixed. On two of
the exams (exam one and exam four), students had higher achievement on the
collaborative exams than the non-collaborative exams offered in spring of 2003. On
exams two and three, there was no difference in student achievement versus the prior
semester. On the final exam, students who took the exam collaboratively performed
worse than their counterparts of the year before.
Although overall achievement was not conclusively affected by collaborative
testing, there were some positive results to the study. The collaborative exams were more
difficult, with more complex questions, than the exams from the spring of 2003.
Generally, students performed just as well on these harder exams as they had on the
easier individual exams. Students are better equipped to answer more difficult questions
when in a group than when working individually. Also, the lowest scores were
eliminated in the collaborative sections, as noted by the decreased ranges and standard
deviations on the exams. Overall student achievement increases when collaborative
exams are used. Finally, a greater percentage of students continue to take the exam when
it is collaborative than when it is an individual effort.
Students who took the collaborative exams stated that their testing anxiety was
reduced both when taking the exam and when studying for it. Reduced anxiety levels has
been shown to increase student performance. Students also stated that they preferred
collaborative testing to individual testing, and studied for the collaborative tests about the
same as they would have for an individual exam. Approximately 75% of the students
stated that social loafing was not a problem in their group, and that all group members
contributed to the exam. Collaborative testing is an effective assessment method and
should be continued in CHM 1025.
Having peer mentors in the classroom helps personalize the class for the students,
cutting down the student-instructor ratio from 300 to one to about 12 to one. Students are
more likely to ask questions in small groups and thus are more likely to understand
difficult concepts with added explanations. The students also offer complaints and
suggestions to the mentors, who are then able to relay this to the instructor.
Approximately 75% of the students enrolled in CHM 1025 stated that peer mentors
helped them understand the concepts covered in the class.
The withdrawal rates for the CHM 1025 sections that used peer mentors in the fall
of 2003 were much lower than in the sections that did not use them (approximately 3%
versus approximately 10%). Although the difference in withdrawal rates cannot
conclusively be attributed to the presence of peer mentors, having a mentor check on a
student's progress and attendance certainly helps lower the rate.
The peer mentors also benefit from participating in the program. The mentors
have an opportunity to review basics that are used in later chemistry courses and that are
included in exams such as the MCAT and DAT. Students who participate in the peer
mentoring program have a higher GPA in CHM 2045 than those students who are not in
the program (3.71 versus 2.32). Overall, mentors enjoy the experience, and many
continue to mentor long after they have stopped taking chemistry classes.
KEY TO INSTRUCTOR INITIALS
KA .......................................................................................................... Katie Am aral
KW .........................................................................................................K athryn W illiam s
KZ ................................................................................................................ K eith Zientek
LL ...................................... .. ..... ........... .................... ........................... Larry Land
M V ................................................................................................................. M martin Vala
RB .............................................................................................................C. Russ Bowers