Collaborative learning and testing in introductory general chemistry:

MISSING IMAGE

Material Information

Title:
Collaborative learning and testing in introductory general chemistry:
Physical Description:
x, 98 leaves : ill. ; 29 cm.
Language:
English
Creator:
Amaral, Katie Elizabeth
Publication Date:

Subjects

Subjects / Keywords:
Chemistry thesis, Ph. D   ( lcsh )
Dissertations, Academic -- Chemistry -- UF   ( lcsh )
Genre:
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Thesis:
Thesis (Ph. D.)--University of Florida, 2004.
Bibliography:
Includes bibliographical references.
Statement of Responsibility:
by Katie Elizabeth Amaral.
General Note:
Printout.
General Note:
Vita.

Record Information

Source Institution:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 003100318
sobekcm - AA00004686_00001
System ID:
AA00004686:00001

Full Text










COLLABORATIVE LEARNING AND TESTING IN
INTRODUCTORY GENERAL CHEMISTRY














By

KATIE ELIZABETH AMARAL


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2004















ACKNOWLEDGEMENTS

I would like to thank Dr. Martin Vala for his encouragement and guidance during

these past few years. I would also like to thank Tim Young and Mark Sciallo in the

Admissions Office for collecting students' grades for me. Thanks are also extended to

Larry Margerum of the University of San Francisco for the idea to use collaborative

testing in the CHM 1025 class. Thanks go also to Jason Morris and Keith Zientek for

proofreading the document.
















TABLE OF CONTENTS
page

ACKNOWLEDGEMENTS............................................................................

LIST O F TA BLES......................................................................................................v

LIST O F FIG U R E S......................................................................................................... vii

A B STR A C T ................................................................................................................ix

CHAPTER

1 INTRODUCTION.......................... ............ .........................

Collaborative Learning..................................................................................... 1
Chemistry Readiness Assessment Exam............................................................. 11

2 ANALYSIS METHODS AND PROCEDURES................................. ....... 14

A analysis of V ariance........................................................................................ 14
Collaborative Learning.......................... ......................... ............. .............18
F all, 2002........................................................................................................18
Spring, 2003............................................ ........................... ... .. 23
Summ er, 2003.......................................................... ... ....... 24
Fall, 2003................................................................................. ..............26
Spring, 2004...................................................................................................28
C alculations....................................................................................................29
Chemistry Readiness Assessment Exam............................................... .......... 30

3 STATISTICAL RESULTS................................................... ....... 32

Effectiveness of CHM 1025...................................... .... ..................................32
Collaborative Learning...............................................................37
A ttendance.................................................................................................... ....40
Chemistry Readiness Assessment Exam.........................................................42

4 COLLABORATIVE TESTING..................................... ................................48

5 PEER MENTORS........................................................................71









6 CONCLUSIONS........................................................................................... 83

Chem istry Readiness Assessment Exam .................................... .............................83
Effectiveness of CHM 1025................................................................................ 84
Collaborative Learning................................. ................. ..............................85
Attendance...........................................................................................................86
Collaborative Testing................................................... ............................... 87
Peer M entors................................................................................................. 88

APPENDIX

A KEY TO INSTRUCTOR INITIALS ........................................ ..............................90

B COURSE OUTLINES .................................................................................. 91

CHM 1025..................................................................................................... 91
CHM 2045..................................................................................................... 92

REFERENCE LIST................................................................................................. 94

BIOGRAPHICAL SKETCH.................................................................................... 98















LIST OF TABLES


Table page

2-1 Sample ANOVA Table............................................................................... 16

2-2 CH M 1025 in fall 2002.................................................. .................................... 19

2-3 CHM 1025 in spring 2003.................................................................................. 23

2-4 CHM 1025 in fall 2003................................................................................. 26

2-5 CHM 1025 in spring 2004.................................................................................. 28

3-1 GPA in CHM 1025 based on group type...................................................... 37

3-2 Number of students continuing to CHM 2045 based on
group type......................................................................................................38

3-3 Number of students continuing to CHM 2046 based on
group type......................................................................................................39

3-4 Scores in CHM 2045 and CHM 2046 based on CHM 1025
section........................................................................................................... 39

4-1 Performance on exam 1.................................................................................55

4-2 Performance on exam 2...............................................................................56

4-3 Performance on exam 3............................................................................... 57

4-4 Performance on exam 4...............................................................................59

4-5 Performance on the final exam............................... ........................................60

4-6 Percent of students taking each exam.......................................... ............ 62

4-7 Performance on each section of the exam......................................................62

4-8 Grades on the makeup final.........................................................................65

v










Table page

5-1 Percent of students withdrawing from each section
during the fall of 2003.................................................... ............................. 78

5-2 Withdrawal rates versus mentor utilization.....................................................79

5-3 Mentor grades in CHM 2045.............................................................................. 81















LIST OF FIGURES


Figure page

2-1 F probability distribution.............................. ................. ..............................15

2-2 C LB C 130..................................................................................................... 20

2-3 H-ITT remote control.................................. .................. ..............................21

2-4 Chemistry Learning Center...................................................... ...................25

3-1 Percent student attendance by class meeting in CHM 1025............................... 40

3-2 Percent of students receiving a grade in CHM 1025
by number of classes attended.................................................................. 41

3-3 Percent of students receiving each grade in CHM 2045
by number of correct ChRA math questions..................................................44

3-4 Percent of students receiving each grade in CHM 2045
by number of correct ChRA chemistry questions........................... ............ 46

3-5 Percent of grades in CHM 2045 based on ChRA score................................. 47

4-1 Percent of students earning each grade on exam 1............................................55

4-2 Percent of students earning each grade on exam 2............................................56

4-3 Percent of students earning each grade on exam 3............................................58

4-4 Percent of students earning each grade on exam 4............................................59

4-5 Percent of students earning each grade on the final exam.............................. 61

4-6 Percent of students earning each grade on the makeup final.......................... 66

4-7 Student reported anxiety levels when studying and when
taking the collaborative exam s.............................. ..........................................67









Figure page

5-1 Percent of students rating the helpfulness of peer mentors........................... 79














Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

COLLABORATIVE LEARNING AND TESTING IN
INTRODUCTORY GENERAL CHEMISTRY

By

Katie Elizabeth Amaral

August 2004

Chair: Martin T. Vala
Major Department: Chemistry

Students taking General chemistry at the University of Florida are either well-

prepared or under-prepared. To meet the needs of the under-prepared students, an

introductory course (CHM 1025) was developed. An accurate method of placement into

CHM 1025 or the mainstream course (CHM 2045) was needed. The Chemistry

Readiness Assessment Exam was written and tested and students are advised to take

either course based upon their scores. The accuracy of the cutoff scores was examined,

with the minimum passing chemistry score lowered to six correct out of 18, and the math

score raised to six correct out of eight.

Collaborative problem-solving sessions were held during every CHM 1025 class.

These sessions were shown to increase student achievement in CHM 1025. Group

placement was also shown to have an effect on student achievement in the course.

Students placed randomly into collaborative groups had the highest average GPA, while

students placed by achievement had the lowest average GPA.

ix









The efficacy of CHM 1025 was examined to determine if the students who

required the course do as well in CHM 2045 as those students who did not need it.

Students who had taken CHM 1025 had a higher GPA in CHM 2045 than the students

who went directly into CHM 2045.

Students in the spring semester of 2004 took collaborative exams. Achievement

levels of students who had collaborative exams were compared to students who took

traditional exams to determine if collaborative testing had an effect on student

achievement and retention in CHM 1025. There was no significant difference in

achievement although the collaborative exams were harder. Percentages of students

taking each exam were also compared, with more students taking the collaborative

exams.

Finally, undergraduate students called peer mentors, who had taken CHM 1025,

were recruited to assist with the course. Mentors helped CHM 1025 students with the

collaborative problems. The mentors' presence helped lower students' withdrawal rates

in the class. The mentors also benefited from the program, as evidenced by their higher

GPA in CHM 2045.














CHAPTER 1
INTRODUCTION

Collaborative Learning

There are three ways that the actions of an individual may influence actions of

another individual. The action may help another individual succeed, may hinder the

success of another individual, or may have no effect on the other individual.' The first of

these outcomes is usually the result of a cooperative effort on the part of the individuals

in question, the second of competitive efforts, and the last of individual efforts. Although

individual efforts are usually stressed in the learning environment, individuals who work

near each other on the same task generally begin to compete. In the university setting,

high grades are often made artificially scarce, with only a certain percentage of students

able to achieve this level of success. This often fosters a competitive attitude among

students.

When individuals compete against each other, the outcome of their actions is

often negatively linked. The success of a small minority of individuals means the

automatic failure of the rest of the competitors. For example, if the students' grades in a

class are normalized to a bell curve, only a select few students will receive an A. All

other students will automatically not get the high grade, and these students usually know

during the course of the semester that they will not achieve this level of performance.

This knowledge of failure, as well as the failure itself, can leave the losers feeling angry,

jealous, or inadequate about their own performance








Competition can also have an adverse effect on the interactions between the

individuals involved. Anything that helps one competitor succeed directly hurts another

competitor's chance of success. Individuals who compete may realize that anything to

their immediate advantage is better than cooperating with others engaged in the same

task. Often, when there is a high level of importance placed on success, competitors will

cheat in order to enhance their chances. In a survey of central Florida college students,

9 out of 10 stated that they had cheated on an exam or a homework assignment.'

Competition can also lead to high anxiety levels among students. High anxiety

may inhibit an individual's performance on the task. Anxiety disrupts a student's ability

to cognitively reason and promotes avoidance of the situation that the student fears. In the

case of the college student, it means skipping class, taking long breaks, or avoiding

challenging learning situations. As early as 1927 it was known that individuals who fail

in the early stages of competition did not overcome their initial failure and scored poorly

on all subsequent tests, even though the tests were matched to competitors' abilities.2 It

has also been demonstrated that individuals who fail consistently in a competitive

situation tend to withdraw from the competition.3 In a scholastic setting, this means that

students who perform poorly on the first exam are likely to continue with this poor

performance in spite of their ability to achieve higher scores. These students often give

up, and stop trying to achieve a higher score. A study performed at the University of

North Carolina at Charlotte has shown that 80% of the students who failed their first

exam in the first semester general chemistry class failed the course.4 Competitive

students either work hard to do better than their peers or do only the minimum amount of








work, because they do not believe they have a chance to succeed.5 Competitive situations

also tend to increase student anxiety.

In a non-exam situation, students who are working on homework or merely

reviewing lecture notes have no other source of information. If a student does not

understand or have the knowledge to complete a problem, work on that problem ceases.

Other students, in this case competitors, are not likely to assist the student with the task

because it will directly inhibit their own chances of success. Also, students who work

alone may have a difficult time remaining motivated to complete a task, especially when

they feel the task is unimportant to them.

Competitive learning situations tend to isolate students from each other and create

negative relationships both between the instructor and the students, and between the

students themselves.5 These situations also tend to discourage active learning and do not

help develop talent among students. Reviews of the literature have shown that college

students are much more likely to acquire critical thinking skills and to develop learning-

strategies in a collaborative setting than in a competitive setting.6'7 Bligh has shown that

college students spend more time integrating concepts and synthesizing connections

between concepts in a collaborative setting than those students who listen to a traditional

lecture.8 In fact, groups have been shown to be better at promoting student problem-

solving skills than the traditional lecture.9 With these thoughts in mind, we decided to

implement collaborative learning in the introductory general chemistry class at the

University of Florida.

Collaborative learning is learning that takes place in small groups of cultural,

academic, and occasionally linguistically heterogeneous populations.'1 The participants








in these groups work together to accomplish the assigned task, both giving and receiving

pertinent information from group members. Whether collaborative learning is effective or

not depends upon several variables. The teacher's knowledge and skills will determine

how the groups are guided through their tasks and will also affect classroom

management. Classroom configuration is also important to collaborative learning. Some

classrooms, especially auditorium style classrooms or classrooms where the seats are

stationary, are difficult to arrange in a pattern that promotes communication among group

members. As discussed below, the type of group greatly affects the outcome of the

collaborative situation.

There are several different types of groups that appear in the collaborative

learning setting. These include the pseudo learning group, the traditional classroom

group, the formal and informal collaborative learning groups, and collaborative base

groups." In the pseudo learning group, the participants are assigned to work together for

a varying length of time, but have no reason to help each other because they are directly

competing for the highest grade. These students are usually graded on a bell curve. In

these learning groups, students may mislead and confuse each other as well as try to hide

what they know from each other. The students in these groups perform worse on the

assignment than if they had actually worked by themselves."

In the traditional classroom learning group, students are assigned to work together

and do so because it is required. The length of time the students work together varies

from situation to situation. Students are evaluated on individual performance only, and

therefore may have no vested interest in helping other students. While they desire the

information that other students have, they may have no incentive to share their own






5


information. Personally they desire to do well on the assignment, but may not want their

colleagues to perform equally as well. Only a minimum of collaborative learning occurs

in these groups and this may occasionally help the lower achieving students. However,

the higher achieving and more conscientious students would usually perform much better

on an assignment if they had worked alone.'

Formal collaborative learning occurs when students work together for a period of

time, lasting from one to several class periods, in order to achieve specific tasks and

assignments. In order for formal collaborative learning to be successful, the teacher must

decide in advance the size of the group, the objective of the group, how students are

assigned to a group, and the role each student will play in the group. The objective needs

to be explained to the students in such a manner that they understand how they are to

function as a group and how their grade depends both on individual performance as well

as group performance. The manner in which the instructor approaches formal

collaborative learning plays a key role in these types of groups. The instructor needs to

monitor the students while they are in the group to provide assistance with the objective

and to help with student social skills.

Informal collaborative learning occurs when students form their own groups and

work together for only a short period of time, from a few minutes to one class period.

This type of group can be used to focus students' attention on the lesson at hand, to

ensure that they understand and are capable of applying the information they are being

taught, and to break up an otherwise monotonous lecture into something more interesting.

There is neither a defined evaluation procedure nor are there defined student roles in this

type of collaborative learning. The final type of collaborative learning group is the









collaborative base group. These groups consist of students of heterogeneous ability and

backgrounds that work together for several semesters or years. These groups work best

in the K-12 setting, and thus will not be discussed here."

The success of formal collaborative learning hinges upon several factors.' First,

students must realize that they are connected to each other and that they cannot

individually succeed unless the group succeeds. Usually this involves connecting an

individual student's grade with the group grade. In order for the group to perform well,

students must share their work, ideas, skills and information with each other. Students

must also be individually accountable for their own work. This ensures that one student

does not have to perform all the work in a group and that all students are contributing to

the objective. The students must be comfortable in the group setting in order to take risks

regarding learning and to encourage and support their peers. Students must also learn the

social skills necessary to work in a group. The students must learn how to solve

problems, they must discuss the nature of the concepts and strategies being learned, they

must teach the knowledge to classmates and explain connections between past and

present learning.12 Finally, students must discuss where the group is in terms of the

group goals and what must be done to achieve the goals. These factors are not easily

achieved and must be constantly addressed as the group works together.

The instructor plays an important role in whether or not collaborative learning is

achieved in the classroom.12 The instructor needs to ensure that students are actively

learning and contributing towards the common goal of the group through careful

observation of student interactions. The instructor is no longer just the source of

information, but is also a facilitator of learning. The instructor must put a lot of effort








into preparing for a collaborative class. All contingencies must be prepared for, and all

students must be doing what the instructor has planned. If collaborative learning is

implemented properly, the classroom will turn into a stimulating, challenging, and

educational environment. The collaborative process must be continually monitored as it

occurs, and any changes that need to be made should be made immediately.

Group activities need to be focused around a central idea in order to foster

conceptual understanding of difficult material.' Students should encounter the central

idea in many different settings so they will have many different opportunities to work on

the problem. They will also need to have different tasks that can help promote

comprehension of the material. The group tasks should inherently be open-ended in

order to ensure that students work together to arrive at a solution to the problem. If there

is more than one right answer to the problem, the students must discuss the problem,

decide upon a direction for the group to take and then work on the problem, arriving at a

solution that may be different from other groups in the class. Open-ended tasks increase

group cohesiveness because they force students to rely upon each other's knowledge and

skills.

To ensure that varying levels of knowledge and skills are present within the

group, the groups should be heterogeneous. This should help the lower achieving

students perform better. The higher achieving students also benefit because their

knowledge and understanding of the objective is tested as they assist the lower achieving

students. Students' reliance upon each other fosters higher-order learning skills because

they have to communicate their thoughts effectively to one another, examine issues from

different perspectives, and justify their arguments. The small collaborative groups also









provide an opportunity for students to ask questions about the material in a safe setting.

Greater than 60% of college students stated that having a large number of peers listening

prevents them from asking questions in a class.3 The smaller group provides a much less

threatening situation where students can ask questions.

Difficulties can arise in the collaborative setting when lower achieving students or

socially distant students are excluded from the learning group.14 Critics of collaborative

learning state that the exclusion of certain individuals from a collaborative group

reinforces existing social and educational problems. Expectation States Theory (EST)

outlines how the status of an individual group member determines the value the group

places upon that individual's work.15 The status of an individual depends upon status

characteristics, which are agreed upon social rankings where high status is better than low

status. Status characteristics are based on several things and can change from situation to

situation. They can be general, based on gender or ethnicity, or they can be specific,

based on perceived ability related to the task at hand. They can also be temporary, based

upon perceived academic status. Contrary to what might be expected, minority students

and low socioeconomic status students are not necessarily low status group members. In

situations where students of minority or low socioeconomic backgrounds are considered

high status, their relative status would carry into the groups in which they participated. It

has also been demonstrated that collaborative learning can help reduce stereotypes

because the individuals in the groups are exposed to others who disprove the stereotyped

role.16

It is also possible for a teacher to equalize students' status within the collaborative

learning group.14 If a teacher assigns competence to a student or group of students, there








is a specific and public recognition of the intellectual contribution a student can make to

the group assignment. It often has a positive side effect on the low status student because

a teacher's high expectations can often raise the expectations of the student in question

and other students in the peer group. The praise must be public, specific, and relevant to

the work of the group for it to be effective.

Cohen and Lotan have summarized research about the effectiveness of

collaborative learning.17 They report on two data sets of five middle schools in the San

Francisco Bay Area between 1991 and 1993. Students who participated in formal

collaborative learning were compared to students who had not. All students covered the

same topics in the social studies classroom and were given the same tests. The students

who engaged in collaborative learning scored higher on items that required higher-order

thinking than those students who did not. There was no significant difference in students'

scores for questions requiring only factual recall. They also determined that the way in

which the teacher implemented the collaborative learning program affected the students'

achievement. Increasing the number of activities and reports the students completed had

a positive effect on test scores. They also discovered that in classes where the instructor

had difficulties with classroom management, student achievement diminished.

Johnson and Johnson performed a meta-study on the 550 experimental and 100

correlational studies performed on the effects of collaborative, competitive and individual

efforts on instructional outcomes.' They noted that the type of interaction among

students determined the instructional outcome. For instance, structuring assignments so

that they must be collaborative in nature resulted in students working together to achieve

success. On the other hand, students who worked competitively tried to hinder the









success of others while those students who worked independently had no effect on the

success of other students. Johnson and Johnson noted that 375 studies in the last 100

years demonstrated that students who worked together had higher achievement and

greater productivity than those students who worked alone. These studies also

demonstrated that collaborative learning resulted in more high-level reasoning, greater

transfer of knowledge from one situation to another, and more time spent on-task than

individual or competitive settings. They also concluded that collaborative learning

ensured that all students were actively and productively involved in learning.

As a side effect to the text-based learning that goes on in collaborative groups,

Johnson and Johnson determined that groups promoted caring and committed

relationships among students. These relationships were not fostered in the competitive or

individual learning settings. They also stated that working collaboratively with peers

resulted in greater psychological health, higher self-esteem and greater social competence

of students than in the competitive or individual environments.

Collaborative learning has been demonstrated to be an effective teaching tool in

general chemistry,'8-28 organic chemistry,29-32 analytical chemistry,33"37 and physical

chemistry.3840 It is essential to ensure that true collaborative learning is present in order

for it to be effective. Collaborative learning teams have been shown to outperform

individual and competitive systems in mastering content within a time constrained system

based upon set performance criteria.4143 Students in collaborative learning groups tend

to think more critically and to learn less material by rote memorization.44 Collaborative

learning is traditionally implemented in small classrooms of approximately 30-50

students. One of the aims of this project is to demonstrate that collaborative learning can









be implemented in a large introductory general chemistry class of approximately 300

students.

Hogarth and Hartley have examined the effect of the method of placing students

into groups.45 In this study, students in a British Grammar School, the equivalent of high

school here in the States, either chose their own pairs or were assigned to a pair by the

instructor. The researcher found no significant difference in the performance of the two

types of groups. He did note, however, that students in own-choice pairs stated that they

enjoyed chemistry more than the assigned pairs. Aside from this study, there does not

appear to be other studies discussing the effect of placement on the effectiveness of

collaborative learning.

Chemistry Readiness Assessment Exam

Prior to the fall semester of 2002, there were two general chemistry tracks being

offered at the University of Florida. In the first track, which was two semesters long,

students initially took CHM 2045 with its associated laboratory class, CHM 2045L.

After attaining a grade of C or better, most students took CHM 2046 with its laboratory

class, CHM 2046L. The laboratories must be taken simultaneously with the

corresponding class. In the second track, there were three semesters of general

chemistry. Students first took CHM 2040, which does not have a corresponding

laboratory class. After attaining a grade of C or better, the students took CHM 2041 with

its associated laboratory class, CHM 2045L. Again, after attaining a grade of C or better,

most students took CHM 2046 with CHM 2046L. The longer track split CHM 2045 into

two semesters, spending more time on the foundation material for those students without

an appropriate math and/or chemistry background. All students, regardless of their initial









track, took CHM 2046/2046L if they continued on in chemistry. The advantage to the

second track included a slower pace for learning the foundation material that was

expanded upon in later classes. The disadvantage to this track was that students spent a

longer time in general chemistry, which placed them a semester out of sequence for the

rest of their chemistry classes.

Students were initially placed into either CHM 2040 or CHM 2045 based on their

SAT II scores. Students who achieved a certain score or higher were placed into CHM

2045 while those below that score were placed into CHM 2040. The Chemistry

Department at the University of Florida (UF) stopped using this method of evaluation

after it was determined that students' SAT II scores did not directly correlate to their

achievement in chemistry. After this method was abandoned, the students decided which

track to take based upon the evaluations of their advisors as well as by word of mouth

advice from other students. Some advisors continued to use the discarded SAT II score

method for placement. A significant number of students began the two-semester

sequence but then had to drop back to CHM 2040 after the first exam. There were also

students who did extremely well in CHM 2040, receiving all or most of the points

available to them. These students could easily have done well in CHM 2045.

In the fall of 2002, the course offerings for general chemistry were revised. All

students now take CHM 2045 with CHM 2045L, followed by CHM 2046 with CHM

2046L. For those students without the appropriate chemistry and/or math background, a

two-credit introductory class called CHM 1025 replaced the CHM 2040/2041 sequence.

After students complete CHM 1025 with a grade of S (satisfactory), they take CHM 2045









followed by CHM 2046, with the appropriate laboratories, to finish their chemistry

requirement.

In order for this new system to be effective, a practical method was needed to

place students into the appropriate class. There are a large number of students entering

UF each year who expect to take general chemistry. The method used to place these

students into the appropriate class must be both easily administered and easily scored to

be of any use. In light of this, a voluntary web-based exam was administered to all

students who took CHM 2040 in the fall of 2001 and the spring of 2002. The exam was

administered at the beginning of the semester. This exam was used to determine a

correlation between the preliminary exam scores and students' performance in the CHM

2040 class. The exam was modified into the Chemistry Readiness Assessment Exam

(ChRA).

The ChRA is a web-based exam and should be taken before the students arrive at

UF. The University requires that all in-coming first year students have a computer, and

offers assistance in the purchase of one to those students with demonstrated financial

need. This requirement, together with the fact that there are several computer clusters on

campus that are open to students, ensures that all students will have access to a computer

in order to take the preliminary exam. The students who perform well on the preliminary

exam are registered for CHM 2045, while those who do not perform as well are

registered for the introductory CHM 1025 class. Another goal of this project is to further

refine the cut-off scores used for placement into general chemistry. Student achievement

in CHM 2045 has been compared to ChRA scores to determine a more accurate cut-off

score.














CHAPTER 2
ANALYSIS METHODS AND PROCEDURES

This chapter is comprised of three sections. The first section describes the

calculation method used for the statistical tests performed during the course of this study.

The second section describes the collaborative and non-collaborative sections of CHM

1025 during the fall semester of 2002, the spring, summer and fall semesters of 2003, and

the spring semester of 2004. The third section discusses the ChRA and the suggested

minimum requirements for admittance into CHM 2045.

Analysis of Variance

Analysis of Variance (ANOVA) is a statistical test designed to judge whether or

not differences in sample means are significant.46 This calculation is based on both the

null hypothesis and the test hypothesis. The null hypothesis assumes that all the means

being tested are equal. The test hypothesis assumes that at least one sample mean is

different from the others. These hypotheses are the same for all ANOVA calculations.

The ANOVA calculation is based upon several premises. First, all the populations

being tested have a bell-shaped distribution, also called a normal distribution. Second,

the sample variations (the square of the standard deviations) are approximately equal.

Finally, the samples being compared must be independent from each other.

An ANOVA calculation determines the F probability distribution for the samples.

The F probability distribution, also called the F value, is an asymmetrical distribution

based upon the degrees of freedom in the sample (Figure 2-1). Only a small percentage



















Probabilityp


Figure 2-1: F probability distribution

of the samples should fall above a certain value on this curve. The level of significance

of the test, called the p-value, is based upon the F distribution. The p-value is the

probability of obtaining a value of F that is as likely or more likely to reject the null

hypothesis as the actual observed value of the test. The p-value is based upon the idea

that the null hypothesis is correct. If the p-value for an ANOVA calculation is less than

0.05 (5.0%) the null hypothesis is usually rejected as incorrect, and the test hypothesis is

determined to be correct. If the p-value is higher than 0.05, the null hypothesis is

considered correct. Another way of stating the p-value would be that we are 95%

confident that the samples are significantly different.

In order to determine the F value in an ANOVA calculation, the degrees of

freedom, sum of squares and mean square must be calculated for the samples (Table 2-1).

Degrees of freedom (DF) determine how many measurements can vary in a sample.

There are two types of degrees of freedom calculated in an ANOVA calculation. The









Table 2-1: Sample ANOVA table
Source Degrees of Sum of Mean F Test
Freedom Squares Square
Between t-1 SSB s
Samples

Within n t SSW s
Samples
Total n, -1 TSS

first type of degree of freedom denotes the freedom between samples and is given by DF

= t 1, where t is the number of samples. The second type of degree of freedom denotes

the freedom within the samples, given by DF = n, t, where n, = total sample size. These

two degrees of freedom are summed into a total DFt = n, 1.

There are three different sums of square calculations in the ANOVA calculation.

The first is the within-sample sum of squares (SSW). This determines the variance

within each sample.


SSW= (y,,-y,)- = (ni-1)s- + (n2-1)s. +...+(n,-1)s,2


where y, = thejth sample observation selected from population i, y, = the average of the


n, sample observations drawn from population i, and is given by y, = and s =
n,

standard deviation of population i. The second type of sum of squares is the sum of

squares between samples (SSB). This determines the variance between each sample.

SSB = n(y, -y)2


where n, is the number of sample observations obtained from population i, and y is the


average of all sample observations, given by y = The final type is the total
n,









sum of squares (TSS) which determines the variance for the overall mean.


TSS = (y, y)2 = (n -)s2
,=1 /=1

where s = standard deviation of the nt measurements y,y

There are two types of mean squares, the mean square between samples and the

mean square within samples. The mean squares are averages of squared deviations, and

are usually calculated by dividing the sum of squares by its degree of freedom. The

between-sample mean square is denoted as sB.

2 SSB
t--


The within-sample mean square is denoted sW .

2 SSW
ssw
n t

The F value is determined by dividing these two mean squares by each other.

2
F = _
2


If the means have been found to be significantly different, either a Fisher's Least

Significant Difference (LSD) test or a Tukey's W Procedure is run to determine which

mean or means are different from the rest. The least significant difference is the smallest

difference that can be observed between two sample means where the means are still

statistically different. The LSD test often has a high error rate between experiments, but

can often be used to get a general idea of which means are different.


LSD, = to sj, +-
Hi H









if y, y, 1> LSD,, then Pi and Ri are declared different, where tx is the mean for the

sample x.

Tukey's W Procedure is a much more conservative test than Fisher's LSD test.

Tukey's controls the experiment-wise error rate, and thus fewer means are declared

different than with the LSD procedure. This test lowers the probability of declaring one

pair of means significantly different that are not different when running multiple

comparisons. Of the two, Tukey's W Procedure has been used more often than the LSD

test.


W = q, (t,v)
Fn

if I y, yj 12 W then gli and pj are declared different, where t = degrees of freedom

between samples, v = degrees of freedom within samples, and qa= upper-tail critical

value of the F distribution. All ANOVA calculations, LSD calculations, and Tukey's

calculations were run using SAS software version 8 for Windows.47

Collaborative Learning

Fall, 2002

During the fall semester of 2002, five sections of CHM 1025 were offered, taught

by three different instructors. One instructor taught one section of CHM 1025 (section

0675), and the other two instructors each taught two sections (sections 0676, 5253 and

sections 0677, 4284). Students in CHM 1025 were not given a letter grade, but were

instead graded on a Satisfactory/Unsatisfactory (S/U) basis. Statistics for each section are

presented in Table 2-2.









Table 2-2: CHM 1025 in fall 2002
Section Instructor # students # students Final # students Pass Rate
enrolled withdrew Enrollment receiving S
0675 MV 298 19 279 237 84.9%
0676 KA 304 20 284 237 83.5%
0677 RB 298 24 274 240 87.6%
4284 RB 298 11 287 241 84.0%
5253 KA 219 18 201 152 75.6%

All five sections were taught in Chemistry Laboratory Building (CLB) room

C130, a lecture-style auditorium that seats 297 students (Figure 2-2). The chairs are

stationary and face towards a demonstration desk. There are four moveable chalkboards

in the front of the room, plus a projector connected to a computer located in the front of

the room. Infrared sensors, which line the walls of the auditorium, are also connected to

the computer. The class met twice a week for 50 minutes.

The five sections used common exams, jointly written by the three instructors.

Exams were multiple-choice, usually comprised of 15 ten-point questions. They were

administered at night in rooms scattered throughout UF's campus and proctored by the

instructors and the teaching assistants. There were four exams offered during the term

and a final exam. The final exam was worth 200 points. The lowest of the four semester

exams was dropped. All five sections were graded on the same scale, with 610 points out

of 1000 possible needed to pass the class.

At the beginning of the term, students were told that the maximum number of

points that they needed to pass the class was 650 points out of a possible 1000. Many

students reached this mark after the fourth exam, at which point they stopped coming to

class. Those students that stopped attending also did not take the cumulative final.

Each instructor approached collaborative learning in a different manner during

this semester. All instructors were assigned peer mentors. In section 0675, students were

















Lab Demonstration Table


Figure 2-2: CLB C130








allowed to choose their own groups of six students at the beginning of the term. Students

either chose to join a group that included their friends or chose a group based upon their

location. There were not enough peer mentors for each group to have their own mentor,

so the peer mentors roamed assigned sections of C 130 and helped the groups in that area.

The instructor lectured for a short period of time, then asked his students a question based

upon the material just covered. All peer mentors were supplied with answers to the

problems prior to the class session. After the problem was solved, the cycle was

repeated. These problems were placed on a side overhead projector, and had five answer

choices. The students worked with their group to solve the problems and each student

entered an answer using the H-ITT remote controls (Figure 2-3). The H-ITT remote

controls send signals to the infrared sensors. The sensors record both the students' unique

identification numbers and the answer chosen by the student, and transmit this

information to the computer. If a group experienced difficulty solving a problem, either

the instructor or a nearby peer mentor would assist them.







Figure 2-3: H-ITT remote control48

In section 0676, students were randomly placed into groups of six by the

instructor prior to the second class meeting. Students were then told to sit with their

groups for the rest of the term. These groups were balanced by gender, with no group

having more males than females, although several groups had more females than males.

In this section, the instructor lectured for approximately 15-25 minutes, depending upon

the topic being covered. The remainder of the 50 minute class period was spent working









collaboratively on worksheets containing approximately 10 problems each, based on the

material just covered. As in section 0675, there were not enough peer mentors to staff

each group, so peer mentors patrolled certain areas of the classroom during the

collaborative problem-solving sessions to assist students. The students were also asked

H-ITT quiz questions during the collaborative session.

In section 5253, taught by the same instructor as 0676, students were placed into

groups of six based on achievement as determined by the students' SAT grades. The SAT

grades were obtained from the Admissions Office at UF after the first class meeting.

Each group had high-achieving, middle-achieving, and low-achieving students. The

groups were rearranged after three of the four semester exams using the previous exam

score as the new criterion for group placement. This section also had a brief lecture

period, similar to the one in section 0676, followed by a collaborative problem-solving

session assisted by peer mentors patrolling certain areas of the classroom. These students

also had H-ITT quiz questions during the collaborative part of the class.

The third instructor taught sections 0677 and 4284. This instructor lectured for a

certain period of time, then asked a H-ITT quiz question. The students were not allowed

to work together to solve these questions, and were considered to be cheating if they

attempted to collaborate. Peer mentors who attempted to assist students were told to

desist and were publicly chastised for talking in class. The peer mentors were expected

to act as police and ensure that no students were cheating during the H-ITT quizzes.

Eventually most peer mentors stopped attending these sections.









Spring, 2003

There were two sections of CHM 1025 taught in the spring semester of 2003, both

taught by the same instructor. Information about the sections is given in Table 2-3.

Table 2-3: CHM 1025 in spring 2003
Section Instructor # students # students Final # students Pass Rate
enrolled withdrew Enrollment receiving S
5267 KA 47 10 37 20 54.0%
5269 KA 126 21 105 67 63.8%

Both sections received the same examinations. The semester exams were essay-

type examinations worth 150 points. The final exam was worth 200 points. The lowest of

the four semester exams was dropped. The exams were administered at night in rooms

throughout UF's campus and proctored by the instructor, TA and peer mentors. There

were four exams and a final exam. Both sections were graded on the same scale, with

610 points out of 1000 possible needed to pass the class.

As in the fall of 2002, students were told that they needed 650 points out of a

possible 1000 to pass the class. In order to maintain student attendance past the fourth

exam, students were told that the cumulative final exam for the course was mandatory.

Students had to receive a minimum of 50% on the final in order to pass the class

regardless of the number of points they had. There was also a poor performance

forgiveness policy. Students who received 75% or greater on the cumulative final exam

passed the class even if they had not reached the 650 point cutoff. Only one student took

advantage of this policy.

Section 5267 was placed into groups of six by location during the second class

meeting. No attempt was made to balance these groups by gender. These groups

remained constant throughout the semester. This section originally met in CLB C130,








but due to its small size and early meeting time it was soon moved to the Chemistry

Learning Center (CLC), a room with moveable tables and chairs (Figure 2-4). Students

in section 5269 were placed into groups of six randomly by the instructor and were

informed of their new groups during the second class meeting. The groups remained

constant throughout the term. Most of this section met in CLB C130 for the entire

semester. Four of the groups volunteered to meet in the CLC with two experienced peer

mentors, one of whom was a senior chemistry major. The peer mentors were responsible

for this mini-section, giving lectures to the students based on the instructor's notes. Both

sections met twice a week for 50 minutes.

Sections 5267 and 5269, as well as the mini-section of 5269, were all taught in the

same manner. A brief lecture was given at the beginning of the class period that lasted

from 15 to 25 minutes. After the lecture, students worked in their groups to solve

problems based upon the material just covered. Peer mentors were assigned individual

groups and got to know their students well during the term. The students also worked

together on H-ITT quiz questions during the collaborative session. Group types were

compared using the ANOVA table to determine if group type affected achievement or

attendance.

Summer, 2003

There was one section of CHM 1025 taught during the summer semester of 2003

(taught by KA). Section 4216 had an initial enrollment of 170 students, of whom four

withdrew. Of these, 143 students received an S (86.1% pass rate). This section met four

days per week for an hour and fifteen minutes over a period of six weeks in CLB C130.










A
N














White Board




A
Figure 2-4: Chemistry Learning Center
Section 4216 was given three long answer exams during class and a cumulative
final examination on the last day of class. These exams were worth 150 points each, with
the final worth 200 points. As in the spring 2003 semester, students were told that the
final was mandatory and that a minimum of 50% was required on the exam in order to
pass the class. The poor performance forgiveness policy was also offered in the summer
2003 semester. Students needed a total of 625 points out of a possible 1000 in order to
pass the class.









Students were placed into groups of six randomly by the instructor and were

informed of their groups during the second class meeting. The individual class meetings

were run in one of two ways. The first type of session involved a 30-minute

collaborative problem-solving session sandwiched between two brief 20-minute lectures.

The problems solved during the collaborative session were based on the first lecture

given during the class. The second type of class had a 20-minute lecture sandwiched by

two 30-minute collaborative problem-solving sessions. The first collaborative session

had problems based on the second lecture from the previous day. The second

collaborative session had problems based on the lecture immediately preceding it.

Students were given H-ITT quiz questions during all collaborative learning sessions.

Fall, 2003

There were five sections of CHM 1025 during the fall semester of 2003, taught by

four different instructors. The same instructor taught section 0675 and 0677. Information

about the sections is given in Table 2-4.

Table 2-4: CHM 1025 in fall 2003
Section Instructor # students # students Final # students Pass Rate
enrolled withdrew Enrollment receiving S
0675 LL 257 27 230 196 85.2%
0676 KW 303 37 266 202 75.9%
0677 LL 303 23 280 245 87.5 %
4284 KZ 297 10 287 234 81.5%
5253 KA 300 10 290 237 81.7%

All five sections were taught in CLB C130 and met twice a week for 50 minutes.

All sections took the same examinations. There were four multiple-choice exams during

the semester, usually with 15 questions worth 10 points apiece. The lowest semester

exam was dropped. There was also a cumulative final given during exam week. As in

past semesters, this final was mandatory, and students were expected to receive a









minimum of 50% on the final in order to pass the class. All exams were written jointly

by the four instructors. However, at the end of the term, the grading scheme for the

sections diverged.

Three of the sections, 0675, 0676, and 0677, were graded on the same scale.

These sections required 600 points out of a possible 1000 to pass the class and all

students were given 88 points to help them towards the goal of 600 points. The students

were given the points because these sections had no H-ITT quizzes. Essentially, a student

only needed 512 points out of a possible 1000, or 51.2% of the available points, to pass in

these sections. The other two sections, 4284 and 5253, were graded on a different scale.

A student in these sections needed 620 points out of a possible 1000 to pass the class, and

these students received no free points, since H-ITT quizzes were given during the class

sessions.

Sections 0675, 0676, and 0677, although taught by two different instructors, were

taught in a similar manner. These instructors lectured for the whole class period and held

no collaborative problem-solving sessions. Peer mentors were present in these

classrooms, and had specific areas of the classroom to patrol. However, the peer mentors

did not have an opportunity to assist students with problem solving. Peer mentors were

mainly required to pass out in-class quizzes given weekly and soon stopped attending.

Sections 4284 and 5253 were taught in a similar manner, although these sections

also had two different instructors. These instructors lectured for 15 to 25 minutes,

depending upon the topic, and then held collaborative problem-solving sessions during

the remainder of the class periods. Students were randomly assigned to groups of six or

seven by the instructors and informed of their groups during the second class meeting.









Peer mentors patrolled certain areas of the classroom and assisted groups during the

problem-solving sessions. As in prior semesters, the peer mentors were given the

answers to the problems prior to the class meeting. H-ITT quizzes were also given

during these collaborative sessions.

Spring, 2004

There were two sections of CHM 1025 taught during the spring semester of 2004.

A different instructor taught each section. Both sections met in CLB C130 twice a week

for 50 minutes. Information about the sections is given in Table 2-5. Peer mentors were

assigned individual groups and got to know their students well over the course of the

semester. Both sections were comprised of a brief 15 to 25 minute lecture followed by a

collaborative problem-solving session. Students were given H-ITT quizzes during the

collaborative sessions.

Table 2-5: CHM 1025 in spring 2004
Section Instructor # students # students Final # students Pass rate
enrolled withdrew enrollment receiving S
5267 KZ 98 11 87 44 50.6%
5269 KA 154 13 141 107 75.9%

The main difference between spring 2004 and prior semesters lay in the

examinations. There were four multiple-choice examinations given during the semester,

worth 150 points apiece. The lowest semester exam grade was dropped. There was also

a mandatory cumulative final worth 200 points. The exams were written jointly by both

instructors. The exams were administered at night in rooms dispersed throughout UF's

campus and proctored by the instructors and peer mentors. Each exam had two sections,

an individual section and a group section. The individual section was worth 70% of the

exam points, and the group section was worth 30% of the exam points. For the first hour









of the two hour exam, students sat with their groups and worked collaboratively on the

group portion of the exam. At the end of the hour, the scantrons were collected and the

individual exams were passed out. The students then worked alone on this second

portion of the exam.

For the first exam of the four semester exams, the individual and group portions

were switched, with the indivjdtfal section taken first. Half of the questions on the group

portion of the exam were taken from the individual portion, so students had an

opportunity to rework the problems with their groups. However, students did not like

having to wait quietly for all students to finish the individual portion. They also did not

redo the repeated problems but instead chose the answer that the majority of students in

the group had gotten for these problems. Therefore, the order was switched, with the

group portion coming first for the remainder of the exams. This allowed the students to

talk with each other while waiting for all the groups to finish. Once the sections were

switched, however, the same problems no longer appeared on both portions of the test.

Calculations

The grades of all students who took CHM 2045 and/or CHM 2046 during the

2002/2003 school year as well as the fall semester of 2003 were obtained from the UF

Admissions Office. It was also noted whether or not the student had taken CHM 1025

and passed. The students who had taken CHM 2045 were split into two groups based

upon whether or not they had taken CHM 1025. The achievement of these groups of

students was compared using an ANOVA table. Also, whether or not having taken CHM

1025 affected student retention in CHM 2045 was determined. The students who had

taken CHM 2046 in the fall semester of 2003 were split into two groups based upon









whether or not they had taken CHM 1025. The achievement of these groups of students

was also compared. In addition, whether or not having taken CHM 1025 affected student

retention in CHM 2046 was compared.

Attendance data was available only for certain sections of CHM 1025. The

attendance in these sections was compared to achievement in CHM 1025 using an

ANOVA table. The sections involved in this calculation were 4284 and 5253 from the

fall semester of 2003 and 5267 and 5269 from the spring semester of 2004.

Chemistry Readiness Assessment Exam

Students were administered a voluntary, web-based preliminary exam at the

beginning of the fall 2001 and spring 2002 semesters. The exam consisted of eight math

questions and 16 chemistry questions, and covered material in the first eight chapters of

the textbook.49 The preliminary exam results were compared to the final course grade for

the fall of 2001. This preliminary exam was modified into the Chemistry Readiness

Assessment Exam (ChRA).

The ChRA was placed online for prospective freshman chemistry students to take

before arriving at UF. Based upon the results of the exam, students were advised to take

either CHM 1025 or CHM 2045 beginning in the fall term of 2002. Students who

received a score of four or better on the math portion of the exam and six or better on the

chemistry portion of the exam were advised to take CHM 2045. Those students receiving

less than these scores were advised to take CHM 1025.

These cutoff scores were not absolute, however, and many students took CHM

2045 that did not meet the minimum ChRA requirements. For those students who did not






31


take CHM 1025, the grade received in CHM 2045 was compared to the results of the

ChRA to further refine the cutoff scores.














CHAPTER 3
STATISTICAL RESULTS

Effectiveness of CHM 1025

The curriculum in the Introduction to Chemistry course (CHM 1025) was

designed for those students who are under-prepared for the mainstream general chemistry

course (CHM 2045). Therefore, it was important to determine whether there was a

difference in student achievement in CHM 2045 between those students who took the

introductory course and those students who did not.

In the fall of 2003, two CHM 2045 classes (taught by KA and MV) were given a

quiz in the first week of the class based upon the material that students were expected to

know before taking the course. The last question on the quiz allowed students to self-

identify as having taken the introductory course. The average score on the quiz for those

students who self-identified as having taken CHM 1025 was 5.36 out of 10.00. This was

significantly different than the lower 4.41 score obtained by those students who had not

taken CHM 1025 (p-value <0.0001). This enhanced achievement in CHM 2045 for those

students who had taken CHM 1025 continued throughout the course.

The grades of all students who had taken CHM 2045 in the spring of 2003,

summer of 2003 and fall of 2003 were compared for level of achievement. The students

were split into three groups: those who had not taken CHM 1025; those who had taken

CHM 1025 and passed; and those who had taken CHM 1025 and failed. The grade point

averages (GPA) of these students were compared to determine the effect CHM 1025 had








on achievement. On a 4.00 GPA scale, an A corresponds to a 4.00, a B+ to a 3.50, a B to

a 3.00, etc. The students who had not taken CHM 1025 had an average GPA of 2.12 in

CHM 2045. This is significantly lower than the 2.74 GPA of those students who had

passed CHM 1025 (p-value <0.0001). Those students who either failed or withdrew from

CHM 1025 had an extremely low 0.59 GPA in CHM 2045. This abysmal success rate for

students who failed or withdrew from CHM 1025 demonstrates that these students must

not be allowed to enroll in CHM 2045. Once CHM 1025 is attempted, it must be passed

before advancing to CHM 2045.

Another measure of how well CHM 1025 prepares students (that does not rely on

student grades) is to determine how many of these students continued on to the second

semester of general chemistry (CHM 2046). Of the 1609 students who took CHM 2045

during the spring and summer semesters of 2003, 953 (59.2%) students earned a grade of

C or better and were eligible to continue on to CHM 2046. Only 579 (60.8%) of these

students had taken CHM 2046 by the fall semester of 2003. Of the 579 students that

continued on to CHM 2046, 394 (68.0%) had originally taken CHM 1025. Students who

had taken CHM 1025 were much more likely than their counterparts to continue on to

CHM 2046 if they were eligible to do so. In these two semesters, 656 students took

CHM 2045 but were ineligible to continue to CHM 2046. These ineligible students were

approximately evenly split between former CHM 1025 students and non-CHM 1025

students (48.6% to 51.4%).

Although there is no data available for what other chemistry classes were taken by

the students who passed CHM 2046, it is possible to determine who was able to continue

on in chemistry. There were 579 students enrolled in CHM 2046 over the summer and









fall semesters of 2003. Of these, 471 (81.3%) passed with a grade of C or better. Of these

471 passing students, 327 (69.4%) were originally enrolled in CHM 1025. Former CHM

1025 students were much more likely to pass CHM 2046 than students who did not take

the course.

The fall semester of the school year has by far the highest enrollment of CHM

1025 students of all the semesters. The majority of these students that are eligible will

take CHM 2045 the following spring. As a result, the spring semester is especially

indicative of the performance of former CHM 1025 students in CHM 2045. The spring

semester of 2003 is particularly telling, as there were approximately equal numbers of

non-CHM 1025 students and former CHM 1025 students enrolled in CHM 2045. Of the

1408 students initially enrolled in CHM 2045 that semester, 846 of them had taken CHM

1025 (60.0%). Of the 829 students who passed CHM 2045 in the spring of 2003, 557

students had taken CHM 1025 (67.2%). Of the 579 students who failed or withdrew

from CHM 2045 that semester, 287 (49.5%) had taken CHM 1025. Although these

statistics suggest that students who pass CHM 2045 are more likely to be former CHM

1025 students, the statistics on withdrawals and failures is disheartening. Former CHM

1025 students are failing CHM 2045 at the same rate as students who did not need the

introductory course. One of the possible methods that can be used to remedy this failure

rate is to slow down in CHM 1025 and spend more time on fundamentals such as

nomenclature and stoichiometry.

Following the enrollment trend, the semester with the largest enrollment of former

CHM 1025 students enrolled in CHM 2046 was the fall semester of 2003. Again, this

semester is looked at individually because the numbers of both types of students is









approximately equal. Of the 724 students initially enrolled in CHM 2046 this term, 348

had taken CHM 1025 (48.1%). Of the 607 students that passed CHM 2046, 300 of them

had taken CHM 1025 (49.4%). Of the 117 students who failed or withdrew from the

course, only 48 had taken CHM 1025 (41.0%). The pass rate of both the non-CHM 1025

students and the former CHM 1025 students during this semester is approximately the

same. The failure rate of former CHM 1025 students during this semester is slightly

lower than those students who did not take CHM 1025. Students who required an

introductory course in chemistry are able to succeed just as well in CHM 2046 as those

students who did not require it.

The GPA scores of students enrolled in CHM 2045 in the spring of 2003 were

compared to determine if CHM 1025 had an effect on the graded outcome of the class.

Students who had taken CHM 1025 had an average GPA of 2.18 in CHM 2045 during

the spring semester of 2003. This was significantly different than the average 1.59 GPA

of the students who did not have CHM 1025 (p-value <0.0001). The 2.18 GPA equates

to a letter grade of a C, while the 1.59 GPA equates to a letter grade of D+. Students

must earn at least a C or better in order to continue on to CHM 2046. Former CHM 1025

students performed better during this term as noted by their passing average GPA. The

average GPA scores noted above include all students who had enrolled in CHM 2045 that

semester, including those that failed the course. When only passing students are

considered in the calculation of average GPA, the scores increase. Although the former

CHM 1025 students still have a higher average GPA (3.18 versus 3.09), the scores are no

longer significantly different (p-value 0.0959). This again suggests that former CHM








1025 students have higher grades overall in CHM 2045, even for those students who fail

the course.

The GPA scores for students enrolled in CHM 2046 during the fall semester of

2003 were also compared to determine if CHM 1025 affected the graded outcome of that

course. The average GPA for former CHM 1025 students enrolled in CHM 2046 was

3.01. Although higher than the average 2.86 GPA obtained by the students who had not

taken CHM 1025, the scores were not significantly different (p-value 0.1235). When

only those students who passed were considered, the GPA of the former CHM 1025

students was only slightly higher than the non-CHM 1025 students (3.42 versus 3.39),

and these scores were also not significantly different (p-value 0.5639). However, CHM

1025 students are succeeding in CHM 2046 even though their initial chemistry

backgrounds were not as strong as the students who did not need CHM 1025.

Each semester there are students who take either CHM 2045 or CHM 2046 a

second time, due to either a failure or a withdrawal in the course. These students were

studied to determine whether or not CHM 1025 affected the numbers of students who had

to repeat either CHM 2045 or CHM 2046. Over the course of this study, 439 students

took CHM 2045 twice. Of those students, only 124 (28.2%) had taken CHM 1025 prior

to their initial enrollment in CHM 2045. Additionally, 256 students repeated CHM 2046,

with only 19 (7.4%) having taken CHM 1025. The low number of former CHM 1025

students who had to repeat one of these two courses demonstrates that the CHM 1025

students are better prepared to succeed in later chemistry classes on the first attempt.

This is a desirable outcome because students who need to repeat a course end up behind

in their majors, which can cause difficulty later in their college careers.








Collaborative Learning

In order to determine if collaborative learning affected achievement in CHM

1025, students' grades in that course were compared for the fall 2002 semester. There

were five sections of CHM 1025 that semester: section 0675 had student-chosen groups;

section 0676 had randomly-chosen groups; section 5253 had achievement-assigned

groups; and sections 0677 and 4284 did not use collaborative learning. This was the first

semester that collaborative learning was attempted on such a large scale. CHM 1025 is an

S/U class, with an S being equivalent to a 2.00 or above on the 4.00 GPA scale, and a U

is less than a 2.00 on the GPA scale, but is considered a 0.00 in the calculations.

There was no significant difference in the achievement level of students as

denoted by final grades in either the student-chosen, random, or non-collaborative classes

(Table 3-1). However, all three of these types of classes had a higher achievement level

than the students who were placed into groups by achievement (p-value 0.0089). Clearly

the collaborative effort hinders students when they are placed into groups using either

SAT achievement or test-scores as a method of placement. It is also disappointing to

note that collaborative learning had no effect on student achievement over the non-

collaborative classes during the fall semester of 2002. However, with experience,

collaborative learning was more effective by the fall semester of 2003.

Table 3-1: GPA in CHM 1025 based on group type
Section Instructor Group Type CHM 1025 GPA
0675 MV Student chosen 1.59
0676 KA Random 1.56
0677, 4284 RB None 1.62
5253 KA Achievement 1.39

During the fall semester of 2003, students were placed randomly into groups in

sections 4284 and 5253, while sections 0675, 0676, and 0677 did not use collaborative








learning. Students in the collaborative sections had a higher achievement level denoted

by final grade than students in the non-collaborative sections (p-value 0.0296). Students

in the collaborative sections had an average GPA of 1.59, while the non-collaborative

sections had a GPA of 1.49. Collaborative learning helped the students in sections 4284

and 5253 enjoy greater success in CHM 1025. Unfortunately, it is impossible to

determine if this greater success carried into CHM 2045 or CHM 2046 because these

students had not completed these courses by the end of this study.

The students who enrolled in CHM 1025 in the fall semester of 2002 were the

only students who were able to complete the entire general chemistry sequence (CHM

1025, CHM 2045 and CHM 2046) by the time this study ended. The number of students

from each type of group that continued on to CHM 2045 and CHM 2046 were compared

to determine if collaborative learning had an effect on student willingness to continue in

chemistry. A slightly higher percentage of students from the three collaborative sections

that were eligible to continue on the CHM 2045 and CHM 2046 did so when compared to

the non-collaborative sections (Tables 3-2 and 3-3). Although collaborative learning had

no effect on achievement during this semester, the collaborative students were slightly

more likely to continue with chemistry than their non-collaborative counterparts.

Table 3-2: Number of students continuing to CHM 2045 based on group type
Section Instructor Group Type Students # in CHM % in CHM
passing 2045 2045
0675 MV student chosen 238 186 78.2%
0676 KA Random 237 194 81.9%
0677 RB None 240 185 77.0%
4284 RB None 241 176 73.0%
5253 KA Achievement 152 122 80.3%








Table 3-3: Number of students continuing to CHM 2046 based on group type
Section Instructor Group Type Students # in CHM % in CHM
passing 2046 2046
0675 MV student chosen 238 88 37.0%
0676 KA Random 237 88 37.1%
0677 RB None 240 75 31.3%
4284 RB None 241 76 31.5%
5253 KA Achievement 152 54 35.5%

Student achievement in CHM 2045 and CHM 2046 was compared to the students'

original section in CHM 1025 to determine if collaborative learning had an effect on

student success in later classes. Although there was no significant difference in GPA

scores for the students from the five fall 2002 and the two spring 2003 sections in CHM

2045 (p-value 0.2049), the average scores for the students from the collaborative sections

were slightly higher (Table 3-4). There was also no significant difference in average

GPA scores for the students from these seven sections in CHM 2046 (p-value 0.9735). It

is interesting to note that section 5267, the section that met in a room that was easily

rearranged and thus better for collaborative learning groups, had the highest average GPA

scores of all seven sections in both CHM 2045 and CHM 2046. Room configuration is

known to have an effect on the outcome of collaborative learning, and this idea is further

confirmed by these grades.

Table 3-4: Scores in CHM 2045 and CHM 2046 based on CHM 1025 section
Section Semester Instructor 2045 GPA 2046 GPA
0675 Fall 2002 MV 2.08 2.88
0676 Fall 2002 KA 2.28 3.00
0677 Fall 2002 RB 2.07 2.99
4284 Fall 2002 RB 2.17 2.93
5253 Fall 2002 KA 2.38 2.91
5267 Spring 2003 KA 2.61 3.38
5269 Spring 2003 KA 2.21 2.60









Attendance

Colloquially, it has always been known that students must attend class in order to

succeed in the course. Getting the students to believe this simple fact is another matter

altogether. Sections 4284 and 5253 from the fall of 2003 and sections 5267 and 5269

from the spring of 2004 had over 20 class meetings during their respective semesters (see

Figure 3-1). The large dips on this graph come the day after an exam. Students'

achievement in these classes was compared to their attendance as recorded by their H-

ITT remote controls. The students were split into four groups by number of classes

attended: 0 5 classes attended; 6 10 classes attended; 11 16 classes attended; and

>17 classes attended. The first group corresponds to students who attended less than

25% of the class meetings, the second group attended less than 50% of the time, the third




-- 4284 -a-5253 5267 -w-- 5269
100.00


90.00




70.00


60.00


50.00


40.00


30.00 .
0 5 10 15 20 25

Figure 3-1: Percent student attendance by class meeting in CHM 1025










group attended less than 75% of the time, and the fourth group attended most of the

classes.

As is expected, there is a significant difference in scores earned by the students

who fall into these groups (p-value <0.0001). Students who attend less than 25% of the

time receive an average of 300 of the 1000 available points, earning a very low E.

Students who attend less than 50% of the time receive an average of 521 of the available

points, earning a high E. Students who attend less than 75% of the time receive an

average of 589 of the available points, earning a D. Students who attend greater than

75% of the classes receive an average 758 of the available points, earning a B. As Figure

3-2 shows, only those students who attend class at least three-quarters of the time can be

expected to pass the class. It is important to recognize that students know exactly how


00-5 D6-10 11-16 17-22

90

80

70

60

50

40

30

20




A B+ B C+ C D+ D E
Figure 3-2: Percent of students receiving a grade in CHM 1025 by number of
classes attended










many points they must earn to pass this S/U class. Most students earn this bare minimum

number of points, and do not work one bit harder than they absolutely have to. Any

students who earn over 650 points are doing extremely well in the class, and doing the

amount of work that will help them succeed in CHM 2045.

Students' attendance was compared to the type of collaborative group to

determine if a particular group type affected attendance. Attendance data was only

available for those students who were placed into groups randomly or by achievement.

Students who were placed randomly into groups attended class an average of 83.3% of

the time. This is significantly different from the students who were placed into groups by

achievement (p-value <0.0001). These students attended class only 68.2% of the time.

As shown above, students who attend less than 75% of the time are likely to receive a D.

This further demonstrates that placing students into groups by achievement is not an

effective method of increasing student success.

Chemistry Readiness Assessment Exam

Most of the students who enrolled in either CHM 1025 or CHM 2045 beginning

in the fall of 2002 took the Chemistry Readiness Assessment Exam (ChRA) before

enrollment. The students were merely required to take the exam while there were

guidelines, there were no absolute scores that directed students into either course.

However, most students followed the suggested guidelines. Those students who received

a score of four or greater out of a possible eight on the math portion and a score of nine or

greater out of a possible 18 on the chemistry portion were advised to take CHM 2045.

Those students who received scores below this level were advised to take CHM 1025.








These levels were not absolute, and many students enrolled in whichever course they

thought best suited their needs.

The ChRA scores for all students who had taken CHM 2045 in the fall of 2002,

the spring of 2003, and the summer of 2003 and who had not taken CHM 1025 were

obtained from the Office of Admissions at UF. Students' achievement in CHM 2045 was

compared to their ChRA scores. Students who received a score of four on the math

portion of the ChRA had an average GPA of 1.85. This average GPA is not significantly

different than the average 1.65 GPA of students who correctly answered less than four

questions on the ChRA (p-value 0.1717). The average GPA of students who received a

four on the math portion converts to a D+ on the letter scale. Because students must

receive a C or better in CHM 2045 in order to proceed to CHM 2046, this cutoff is

unacceptable for the math portion of the ChRA.

Students who received a five on the math portion of the ChRA had an average

GPA of 2.13. This average GPA is significantly different than the average 1.76 GPA of

the students who received less than five (p-value 0.0006). While students who passed the

math section of the ChRA with a score of five have an average grade of a C in CHM

2045, this is an extremely low C.

Those students who received a six on the math portion of the ChRA had an

average GPA of 2.41 in CHM 2045. This is significantly different than the average 1.95

GPA of the students who answered less than six questions correctly (p-value <0.0001).

This C grade is a much stronger one than the one received by students who earn a five on

the math section. Therefore, the recommended cutoff for the ChRA math section is a

score of six out of eight possible questions (Figure 3-3).











04 M5 M6


5




A B+ B C+ C D+ D E W



Figure 3-3: Percent of students receiving each grade in CHM 2045 by number of
correct ChRA math questions

The chemistry score of the ChRA was determined to be not as crucial an indicator

of performance in CHM 2045 as the math score. The initial cutoff score for the

chemistry portion was nine out of 18 questions. However, students who received this

score had an average GPA of 2.32 in CHM 2045. This was not significantly different

than the average 2.06 GPA of the students who received less than a nine on the chemistry

portion (p-value 0.0109). This GPA is equivalent to a C, so this cutoff score can be

lowered.

Students who received a score of eight on the chemistry portion of the ChRA had

an average GPA of 2.28 in CHM 1025. This was significantly different than the average








1.98 GPA of students who received less than this score (p-value 0.0083). However this is

still a reasonable C grade, so the minimum cutoff score can be lowered even further.

Students who received a score of seven on the chemistry portion of the ChRA had

an average GPA of 2.10. This is not significantly different than the average 1.95 GPA of

the students who received less than a score of seven (p-value 0.4993). This is a passing

GPA, but it is only barely a C. However, the students who received a score of six on the

ChRA had an average GPA of 2.30, a higher score than those students who earned a

seven on the ChRA. The average 2.30 GPA of students who received a six on the ChRA

is significantly different than the average 1.76 GPA of the students who received less

than six (p-value 0.0003).

The students who received a score of five on the ChRA had an average GPA of

2.13 in CHM 2045. This is significantly different than the average 1.56 GPA of the

students who received less than that score (p-value 0.0038). Although a passing GPA,

2.13 is not a strong C grade. The average GPA of students who received a four on the

chemistry portion of the ChRA is below passing (1.62). Therefore, the recommended

cutoff for the chemistry portion of the ChRA is six out of a possible 18 (Figure 3-4).

Although the cutoff values are the same, students must receive a 75% on the math

portion of the ChRA and a 33% on the chemistry portion in order to succeed in CHM

2045. These results demonstrate that student success in beginning chemistry relies more

on prior math knowledge than on prior chemistry knowledge. Therefore, although many

students' high school chemistry backgrounds are poor, those with a strong math

background can still succeed in chemistry.










05 06 E7 E8 09


A B+ B C+ C D+ D E W
Figure 3-4: Percent of students receiving each grade in CHM 2045 by number of
correct ChRA chemistry questions

As shown in Figure 3-5, students who receive a six on both the math and the

chemistry portions of the ChRA are more likely to get a grade of A in CHM 2045 than

their counterparts. Those students who fail both sections of the ChRA are much more

likely to withdraw from the class than were their classmates. The students who pass only

one section of the ChRA have mixed results, with those students who passed only the

math section performing better in CHM 2045 than the students who passed only the

chemistry portion of the exam.











0 fail both Opass chem pass math U pass both


A B+ B C+ C D+ D E W


Figure 3-5: Percent of grades in CHM 2045 based on ChRA score














CHAPTER 4
COLLABORATIVE TESTING

Testing usually occurs in a traditional format. Students use a pencil and paper to

take the exam. They usually work individually on the exam, and are seated to reduce the

possibility of cheating. Any students working together are considered to be cheating.

The penalties for cheating are severe, ranging from failing the examination to failing the

course, or even expulsion from the university. Students are usually very concerned about

the exams, and are anxious to do well on them, often to the point of asking, "Is this going

to be on the exam?" A high anxiety level has been shown to decrease student

performance. Using a testing system that is collaborative in nature can help to decrease

student anxiety and increase student understanding of the subject material.

Only a handful of studies have looked at collaborative testing as an alternative to

traditional testing methods.50-56 Most of these studies paired students, rather than using a

larger group of four to six students. Also, few of these studies actually set up the exam in

such a fashion that it had to be worked on collaboratively. Although many of the studies

state that they used collaborative testing, few researchers actually wrote an exam where at

least a portion of it was intended to be taken collaboratively.

Muir and Tracy used paired testing in a course for preservice teachers at Oakland

University.50 The students worked in student-chosen pairs on essay-type examinations,

worth 25% of the students' total grade. The students were allowed to choose their own

pairs to decrease teacher responsibility if students blamed their partner for a low exam








score. Muir and Tracy found that test scores increased by 7.0% on the midterm

examination, but only by 1.0% on the final examination. They do not state whether these

results were significant. Muir and Tracy noted that student anxiety level was decreased,

and students spent more time preparing for the examinations. However, they do not state

how they came to the conclusions of lower student anxiety and longer preparation time.

From a survey of students in their classes, they determined that 97% of the partners

worked equally on the exam, with only 3% of the partners stating that one member was

not exerting as much effort as the other member. Two aspects of collaborative testing

that were noted as important by these researchers were that the desks and chairs must

move in order to promote the collaborative effort and noise level must be monitored so

that it does not interfere with student concentration. Muir and Tracy did not state

whether students turned in individual copies of the exam, or only turned in one set of

answers per pair.

Another study conducted with nurses at Lake Superior State University assigned

students randomly to pairs on the day of the test.51 The students did not work together

prior to the exam. The researchers believed that this placement method would encourage

all students to study and prevent one member of a group from carrying the pair. Students

worked individually for most of the exam, and were only allowed to collaborate for the

final third of the exam time. The partners did not have to agree on an answer and each

member of the pair handed in an answer set. The students who compared answers had an

equivalent comprehension of material as students who did not have the option to do so.

Student achievement only increased slightly, from an average GPA of 3.24 in the non-








collaborative setting to an average of 3.27 in the paired setting. Students noted a

decrease in anxiety in the setting where students were allowed to compare answers.

A study at Stanford University used a partner testing system in a large, 300-

student introductory psychology class.52 Students in this class were allowed to choose

whether or not to take the test with a self-chosen partner. Those students that chose to

work with a partner submitted only one set of answers per pair. The students in this

course had to take the first midterm exam alone, and then had the option of taking the

second and third midterm exams with a partner. Although 62% of the students chose to

take the second test with a partner, only 48% chose to take the third exam collaboratively.

By the second year of this study, only 30% of the students were choosing to take the

exam collaboratively. The authors did not discuss why the numbers of students choosing

collaborative testing decreased.

The Stanford University study noted an increase in achievement of approximately

nine percentage points in the partner exam situation over the non-collaborative situation.

They also noted that low scores were eliminated in the collaborative setting. Students

involved in this study that chose to work in pairs noted a reduced anxiety level both

during testing and when studying prior to the exam. Unfortunately, the students involved

in this study thought that their classmates were more likely to expend minimal effort

when tested collaboratively.

In a study conducted among medical students at Wayne State University School

of Medicine, the students were assigned to pairs on the day of a quiz.53 Again, these

students did not work with their partners at any other time during the course. The

students took a quiz individually, and then took the same quiz again with a group. The








individual portion of the quiz counted for 80% of the students' grades. The group portion

accounted for the remaining 20% of the quiz grade. The students had a 6% increase in

overall quiz scores. The score on the group portion was higher than that of the individual

portion. The students involved in this study stated that they liked the collaborative

format better than the traditional quiz format, but no mention was made of their anxiety

levels.

The only study that made use of a group larger than pairs was in a non-majors

statistics class at Colorado State University.54 Students were placed into groups in a

random fashion, with the caveat that each group of four students would contain at least

two females. Students were assigned to their groups on the day of their exam, and did

not work with their groups at any other time or for any other exam. Each group handed

in only one set of answers. There was a large increase in student achievement on the

exams from previous semesters (approximately 15 percentage points). However, these

students had to take an individual final, and the scores dropped dramatically. Not only

were the final scores almost 30% lower than the collaborative exams, they were

approximately 7% lower than the students who had never taken a collaborative exam.

The author contributed this decrease in score to the lack of conscientious studying habits

in his students. However, this attitude would have been apparent in all the exam scores.

The students involved in this statistics course were in favor of collaborative testing. No

mention was made of student anxiety levels.

At least two additional studies discussed what the authors termed collaborative

testing but were not similar to the collaborative testing formats described above.55,56 In

one study performed in chemistry at several universities, including Florida State








University, students watched a demonstration in class and were given exam questions to

take home that had to be turned in forty-eight hours later. Students had the ability to

work together if they wished, but did not have to, nor was the collaborative process

monitored in any way. The researchers stated that their version of collaborative testing

was effective because students in a comparable section of chemistry that did not have the

take-home exams based on demonstrations were unable to answer questions on the final

exam related to demonstrations they had not seen in class. This outcome is to be

expected students should not be expected to answer questions on something that they

had not seen or been taught.

In the second study, students in a composition course at Widener University were

allowed to free-form talk with each other during an exam on grammar skills and works

cited pages. The collaborative process was not monitored and students were allowed to

work with whomever they wished. Students were not required to work together, but the

students who worked individually were seated in the same noisy environment as their

collaborative counterparts. There was a slight increase in student achievement in this

course, but no mention was made of student anxiety level.

The studies discussed above noted an increase in student achievement, although

the increases can be considered modest. But most of the studies omitted aspects of

collaborative testing that could be important, such as handing in individual or group

answers and student anxiety levels. Therefore collaborative testing was implemented in

CHM 1025 during the spring semester of 2004 to test some of these aspects. Prior to this

semester, students had worked together in groups during the class period. Once at the

exam, however, students were told that collaborating was cheating, and were strongly








discouraged from doing so. In many respects, this was unfair to the students. If

collaboration is expected in the class, it should be carried into the situation where the

students are evaluated on their progress. All exams given in CHM 1025 during the

spring 2004 semester, including the final exam, were partially collaborative in nature.

There were two sections to each exam, an individual section and a group section.

The group section was administered during the first hour of the two-hour exam block and

was worth 30% of the exam grade. The individual section was administered during the

second hour of the exam and was worth 70% of the exam grade. Questions were

different on each section of the exam. The group section of the exam had harder

questions than were asked to the students individually. It was believed that students

would be better equipped to answer more difficult questions if they were working with a

group than if they worked alone. The students worked in the same collaborative groups

that they worked with every class period. The rationale behind the consistent grouping

was that students would be more comfortable working with people they already knew in

an otherwise tense exam situation.

The first exam offered during the course was arranged in the reverse order, with

the individual section offered first, and the group section offered second. On this exam,

half of the questions on the group section were the same as on the individual section. It

was believed that students would re-work these problems in their groups and perhaps

arrive at different answers than on the individual section. However, students merely

recalled what they had chosen on the individual section and answered with a "majority

rules" approach. Students also did not like waiting for all students to finish the individual

portion before beginning the group section. In a survey given at the end of the semester,








nearly 70% of the students stated that they did not like having the individual section of

the exam first. After this first exam, the individual and group sections were switched,

and no questions were repeated. The students still had to wait before beginning the

individual section of the exam, but they were now able to talk with their group members

while waiting. Several groups used this time to discuss concepts that members were

having difficulty with.

The grades on the collaborative exams were compared to grades on non-

collaborative exams given in the spring semester of 2003 to determine how the type of

exam affected student achievement. Spring of 2003 was chosen as the comparison

semester because the class size was approximately the same and the classes were

comprised of similar types of students. Students who enroll in CHM 1025 in the spring

semester are usually a semester behind in their majors. A comparison was not made to the

fall semester because the class sizes are extremely different (approximately 250 versus

approximately 1500) and the fall students are usually on-track with their majors.

On the first of four semester exams, the students who took the exam

collaboratively earned an average of 116 points out of 150. This was significantly higher

than the average 96 points earned by the spring 2003 students (p-value <0.0001). The

range of results on the collaborative exam also narrowed, with no students scoring below

55 points (Table 4-1). These results were promising, especially since the instructors

considered the collaborative portion of the exam more difficult than exams offered in

previous semesters.









Table 4-1: Performance on exam 1
Term Instructor(s) Exam Average Standard Range
Deviation
Spring 2003 KA Individual 96 26.3 25-142
Spring 2004 KA & KZ Collaborative 116 18.6 55-150

The percent of students in each section that earned each letter grade was also

compared (Figure 4-1). Overall, students in the collaborative sections had a much higher

percentage of A and B grades than in the non-collaborative sections. As indicated by the

decreased standard deviation spread, the collaborative section also had a much lower

percentage of E grades than the non-collaborative section. The spring 2004 section also

had a skewed bell curve distribution of grades, centered around a B+. The non-

collaborative section had the usual asymmetrical distribution of grades.

*S03 S04
40

35

30

25

20

15 -


10 -


5-

0
A B C D E

Figure 4-1: Percent of students earning each grade on exam 1









By exam two, however, these promising initial results were reversed. As shown

in Table 4-2, there was no significant difference in the grades earned in the spring 2003

and spring 2004 sections (p-value 0.9008). However, the range was again narrowed, with

the lowest scores being eliminated from the spring 2004 section. The standard deviation

of the average scores was also narrowed, most likely due to the lack of extremely low

grades.

Table 4-2: Performance on exam 2
Term Instructor(s) Exam Average Standard Range
Deviation
Spring 2003 KA Individual 104 34 0-150
Spring 2004 KA & KZ Collaborative 105 20 30-150

The percent of students in the collaborative and non-collaborative sections that

earned each letter grade was again compared (Figure 4-2). The collaborative section had


35 S03 S04


30


25 -


20 -


15 -


10 -


5


0
A B C D E

Figure 4-2: Percent of students earning each grade on exam 2








fewer students receiving A and B grades than did the non-collaborative section. The

collaborative section earned a higher percentage of C and D grades than the non-

collaborative section, but fewer E grades. Because the lower grades were eliminated, the

students that would normally be earning E grades instead earned D grades, bringing that

percentage up in the collaborative section. However, the skewed bell-shaped distribution

is not present in the collaborative section. Both sections now have the usual

asymmetrical distribution of grades.

Exam three in the collaborative section was an unusually difficult exam. The

instructors in the spring semester of 2004 wrote a more elaborate exam than in 2003, and

multiple steps were needed to solve the problems. This high degree of difficulty was

reflected in the grades for this exam in the collaborative section. However, even with this

increased level of difficulty for the collaborative exam, the average scores were slightly

higher than those of the spring 2003 semester (Table 4-3). However, the difference was

not significant (p-value 0.6564). The range does not appear to be narrowed in the

collaborative sections as in the other sections, however the student that earned only five

points was a repeat student from the fall of 2003. This student had already attempted and

failed CHM 1025. The student receiving the next lowest grade in the collaborative

section earned 25 points on the exam. The majority of the lower scores were again

eliminated on exam three.

Table 4-3: Performance on exam 3
Term Instructor(s) Exam Average Standard Range
Deviation
Spring 2003 KA Individual 80 35 3-150
Spring 2004 KA & KZ Collaborative 82 24 5-150









The percentages of students receiving each grade were compared for differences

in achievement between the two sections (Figure 4-3). The high difficulty level of exam

three was reflected in the numbers of students earning each grade. Very few students

earned an A grade, and the majority of the students earned an E grade. However,

excluding the percentage of students that earned an A, the distribution of grades is not

significantly different from the spring of 2003, which did not have as difficult of an

exam.


B S03 N S04


0 -


B C


Figure 4-3: Percent of students earning each grade on exam 3

By the fourth semester exam, although the collaborative exam was more difficult

than the non-collaborative exams, it was not as difficult as exam three. The student

averages reflected this, and the average 107 points out of 150 possible earned by the

collaborative section was significantly higher than the average 88 points earned by the









non-collaborative section (p-value <0.0001). As shown in Table 4-4, the ranges and the

standard deviations were again narrowed, due to the elimination of the lower grades.

Table 4-4: Performance on exam 4
Term Instructor(s) Exam Average Standard Range
Deviation
Spring 2003 KA Individual 88 33 19-150
Spring 2004 KA & KZ Collaborative 106 24 50-150

Students in the collaborative section earned more A, B and C grades than in the

non-collaborative sections (Figure 4-4). The collaborative students also earned fewer D

and E grades than did the non-collaborative students. Although both curves are

asymmetrical, the distribution of grades is what was expected when collaborative testing

was implemented.


OS03 IS04
45

40

35

30

25

20

15 -

10



0-
A B C D E

Figure 4-4: Percent of students earning each grade on exam 4








However, by the time of the mandatory cumulative final, there was a decrease in

student performance on the exam. As shown in Table 4-5, the students in the non-

collaborative section had a significantly higher 115 points out of a possible 200 than the

average 106 points earned in the collaborative section (p-value 0.0280). The range of

scores in both the collaborative and non-collaborative sections was also approximately

equal. However, the standard deviation in the collaborative section was again narrowed.

It is important to note that students are told that they must earn 100 points on the final in

order to pass the class. Those students who have already reached or are near the

minimum cutoff for passing usually aim for exactly 100 points. This minimum grade is

an E on the exam, thus the high number of E grades in both sections.

Table 4-5: Performance on the final exam
Term Instructor(s) Exam Average Standard Range
Deviation
Spring 2003 KA Individual 115 35 30-190
Spring 2004 KA & KZ Collaborative 106 30 20-190

Students in the collaborative section earned fewer passing grades on the final

exam than did students in the non-collaborative section (Figure 4-5). They also earned

more failing D and E grades than did the non-collaborative students. Again, this

performance on the final is disappointing. However, starting in the summer of 2004,

CHM 1025 will no longer be graded on an S/U basis and will be given a letter grade. The

final, as well as all the other exams, is expected to become much more important to

students when they will be receiving a letter grade for the course.

There was one distinct advantage to the collaborative final noted by the

instructors. Each semester, there are a few students who do not earn the minimum grade

on the final but who would otherwise pass the class. These students either come in to the









*S03 S04
50

45

40

35

30

25

20

15

10




A B C D E

Figure 4-5: Percent of students earning each grade on the final exam

instructors' offices or email the instructors pleading for points. The number of students

who requested extra points in order to pass the class decreased markedly from prior

semesters. The collaborative exams possibly made students more comfortable with the

grades they had earned.

Another benefit to the collaborative exam format is that the percent of students

taking the exams increased over prior semesters. Generally, the largest numbers of

students take exam one, and then the numbers decrease, occasionally increasing again for

the final. Although the numbers did still decrease, the percent of students taking each

exam was higher in spring 2004 versus spring 2003 (Table 4-6). The percentages of each

student are based on initial, and not final, enrollments. More students were kept in the

class in the collaborative setting than in the non-collaborative setting.








Table 4-6: Percent of students taking each exam
Term Exam Exam 1 Exam 2 Exam 3 Exam 4 Final
Spring 2003 Individual 83.2% 85.6% 73.4% 59.5% 63.0%
Spring 2004 Collaborative 91.7% 90.1% 83.3% 76.6% 74.2%

In order to determine if students were performing better on the individual or

group portion of the collaborative exams, the grades on each portion for section 5269 in

the spring of 2004 were normalized to 100%. These percentages were then compared to

determine if a difference in achievement level existed between the two sections of the

exam. As shown in Table 4-7, there was no difference between the collaborative and

individual sections of the first exam (p-value 0.7954). Students performed equally as

well on both portions of the exam. This demonstrates that students can answer more

challenging questions as a group with as much accuracy as they are able to answer easier

individual questions. When the percentage scores on the individual portion of the exam

were compared to the percentage scores on the exam given in the spring semester of

2003, students who took the exam collaboratively had a significantly higher 77.2%

average than the average 64.2% earned last spring (p-value <0.0001). Students were

better prepared for the exam during the semester it was offered collaboratively than when

it was offered individually.

Table 4-7: Performance on each section of the exam
Section Exam 1 Exam 2 Exam 3 Exam 4 Final
Individual 77.2% 60.5% 51.7% 57.4% 49.3%
Group 78.2% 93.2% 50.2% 100% 68.0%

On the second of the four semester exams, there was a significant difference

between the average 93.2% earned on the collaborative portion of the exam and the

average 60.5% earned on the individual section of the exam (p-value <0.0001). The

students were able to answer the more difficult collaborative questions with greater








accuracy than the easier individual questions. However, when the individual scores were

compared with the test scores from the spring 2003 semester, the students were less

prepared for the individual section of the exam. The students in the spring of 2003

earned an average 69.6% on the exam. This was significantly higher than the average

60.5% earned on the individual portion in the spring of 2004 (p-value <0.0001). After

experiencing the first collaborative exam, students did not prepare as well for the second

exam. However, this lack of preparation was discussed in class, and student achievement

improved on the next two exams.

On the most difficult exam, exam three, the students performed equally as well on

both sections of the exam, earning an average 51.7% on the individual section, and an

average 50.2% on the collaborative section (p-value 0.7158). Again, this demonstrates

that students can answer more challenging questions as a group with as much accuracy as

they are able to answer easier individual questions. When the scores on the individual

section are compared to the scores on exam three during the spring of 2003, the students

in both semesters performed equally as well (p-value 0.3571). On exam three in the

spring of 2003, students earned an average of 53.9%. During the spring of 2004, students

earned an average 51.7% on the individual section of the exam. The students who took

the exam collaboratively were as prepared as were the students who took an individual

exam during the spring of 2003.

For the fourth of the semester exams, students' scores were significantly different

on the collaborative and individual sections (p-value <0.0001). The students earned an

average 57.4% on the individual portion, and over 100% on the collaborative portion, due

to the extra credit question on that section. Students correctly answered virtually every








difficult question on the collaborative section of this exam. The students were also as

prepared for the exam as the spring 2003 students were, earning essentially the same

percentage on the individual section of the exam (p-value 0.7407). The students in the

spring of 2003 earned an average 58.4%, while the students in the spring of 2004 earned

an average 57.4%.

For the final exam, the students earned a higher 68.0% average on the group

section of the exam, versus an average 49.3% on the individual section of the exam (p-

value <0.0001). This final was cumulative, therefore students were expected to answer

more difficult questions on the collaborative section, tying several concepts that they had

learned over the course together into one question. The students did this admirably,

earning almost 70% on this section. However, most students were aiming for the

minimum 50% that was required of them on the final, and this was reflected in their

individual scores. Most students felt confident after taking the group portion of the exam,

and did not try as hard on the individual portion of the exam. This is further reflected

when the individual grades are compared to the exam grades in the spring of 2003.

Students in the spring of 2004 earned an average 49.3 % on the individual section of the

exam. This was significantly lower than the average 57.4% earned during the spring of

2003 (p-value 0.0006).

Overall, students were able to answer the more difficult questions with more

accuracy when working in a group. The students had more problems with the individual

section, even though those questions were not as difficult as the questions they had

answered beforehand. For at least three of the five exams offered during the course, the

students were at least as prepared as their counterparts from a year earlier. Using









collaborative exams, students are able to answer harder questions and perform as well as

expected on the individual sections.

During the spring semester of 2004, eight students required a makeup exam for

varying reasons. These students were a random-sampling of students from the two

sections being offered that semester. They were given the same final as the students in

the spring of 2003, with the noted exception that the students in 2004 had the exam split

into collaborative and individual sections. Although eight students is not a large number,

the results on this exam were compared to the section 5267 in spring 2003. Only 29

students took the exam in that section, so the numbers are roughly comparable when

compared to the 80 students that took the final in section 5269 in the spring of 2003.

The makeup students in the spring of 2004 had a significantly higher average of 134 out

of a possible 200 than the 104 average of the spring 2003 students (p-value 0.0281). As

shown in Figure 4-6, students in the spring of 2004 had a higher percentage of A and C

grades, and had approximately equal percent of B grades. Concerning the failing grades,

although the students in the spring of 2004 had a higher percentage of D grades, there

were no E grades for these students. Those E grades were brought up to D grades. The

range and standard deviation of the scores in the spring 2004 section were also narrowed

due to the elimination of the lowest grades (Table 4-8).

Table 4-8: Grades on the makeup final
Term Instructor(s) Exam Average Standard Range
Deviation
Spring 2003 KA Individual 104 36 40-170
Spring 2004 KA & KZ Collaborative 134 20 110-170











SS03 O S04
60 -


50 -


40 -


30 -


20 -


10 -



A B C D E
Figure 4-6: Percent of students earning each grade on the makeup final

The percent scores on the individual section of the makeup final in the spring of

2004 were compared to the percent scores on the final given in spring 2003. Students

taking the makeup earned an average of 49.1% on the individual section of that exam.

This was not significantly different than the 51.8% earned by the students in the spring of

2003 (p-value 0.6942). Students in the makeup section were as prepared as their

counterparts of the semester before for the individual section of the exam, but were able

to perform much better on the group section. The difficulty of the questions remained the

same between the two exams. Therefore students who took a collaborative exam had a

higher achievement level than the students who took a non-collaborative exam.

A study performed at the University of North Carolina at Charlotte has shown that

80% of the students who failed their first exam in the first semester general chemistry









class failed the course.4 With this in mind, the failure rates of the students that failed the

first exam in the spring of 2003 and the spring of 2004 were compared to determine if

collaborative learning had an effect in this regard. During the spring semester of 2003,

88% of the students who failed the first exam failed the course. During the spring of

2004, 70% of the students who failed the first exam failed the course. The percent of

students who failed the course after failing the first exam is decreased in the collaborative

section. Collaborative testing can help increase the achievement of students who would

otherwise fail the course.

Students were given an anonymous survey regarding collaborative testing on the

last day of classes during the spring semester of 2004. The first two questions on this

survey were about student anxiety levels. As shown in Figure 4-7, the majority of



U Studying M Exam
80%

70%

60%

50%

40%

30%

20%

10%

0%
Reduced No Effect Increased

Figure 4-7: Student reported anxiety levels when studying and when taking the
collaborative exams









students that responded believed that collaborative testing reduced their anxiety both

when studying and during the actual exam. As discussed earlier, high anxiety levels lead

to decreased student performance on examinations. As the collaborative method of

testing reduced student-reported anxiety, even though the questions were more difficult,

the students are more likely to perform better on these exams than on individual exams.

The students were also asked if they would have preferred to take the entire exam

individually. The majority of the students (68%) stated that they preferred the

collaborative exam to an individual exam. Only 11% of the students stated that they

would have preferred individual exams. The remaining 21% of the students wanted the

ability to choose to take an exam either collaboratively or individually. Realistically, this

would be virtually impossible to achieve with the current test design of more difficult

Questions on the collaborative section of the exam. The students who opted to take the

exam individually would most likely be unable to answer these difficult questions, and

their achievement levels would drop considerably.

One concern when implementing collaborative testing is that students will study

less when they know they have other students to rely on during the exam. This is one

reason behind the percent values given to the two portions of the exam it was believed

that students would continue to study if 70% of their grade depended upon their own

work. This was borne out in the students' response to a question on the survey about

studying. Approximately 80% of the students reported that they studied for the

collaborative exam about the same as they would have studied for an individual exam.

As an unexpected result, 11% of the students reported that they studied more for the

exam because it was collaborative. Perhaps this is due to the peer pressure of performing








in a group, and not wanting to let one's group-mates down. Only 9% of the students

reported that they decreased their usual amount of studying because the exams were

collaborative. It is promising that students are studying the same amount as they usually

would for the collaborative exams. This reduces the fear that students will not prepare as

well when they are able to collaborate during an exam.

Another problem that arises when collaborative testing is discussed is social

loafing. Many students and instructors believe that when students work together there

will be one or a few students who do all the work, while the rest of the group does

nothing. These students would get the same grade as those students who worked hard

and studied for the exam. This was another reason behind the percentage values given to

both exam sections. There was no way that a student could abstain from working on the

exam and still pass. When asked, only 23% of the students stated that they would have

removed someone from their group for social loafing. Another 18% were unsure if they

would have removed someone for not participating. The majority of the students (60%)

stated that social loafing was not a problem in their group.

When asked how the groups solved problems during the exam, the majority

(59%) of the students responded that the questions were split up among group members.

An additional 38% stated that every student in the group did every question. When group

members arrived at different answers to the questions, these differences were solved with

a mixture of a majority rules approach and using the answer of the student who had the

best logic. Only 3% of the class (four students) stated that only one person answered all

the questions. These four students were most likely in the same group. These results









demonstrate that students are not expecting one member to do all the work, nor are some

students relying on the results of their group-mates.

A promising result from this survey was that 70% of the students believed that

collaborative testing increased their grade. Perhaps because the questions were more

difficult, students believed that they performed better working with a group than they

would have if they had worked individually. Overall, with these promising survey

results, the fact that more students are taking the exams, and the elimination of the lowest

grades, collaborative testing should be continued in later semesters of this course.

Collaborative testing has proven to be an effective assessment method in CHM 1025, and

is fairer to the students who are expected to work collaboratively during class meetings.














CHAPTER 5
PEER MENTORS

As student enrollment in higher education increases and funding decreases, class

sizes have increased dramatically. A professor facing 300 students often feels there is no

choice but to lecture. The only active role that a student can take in a lecture format class

is to take notes. As discussed earlier, collaborative learning is one way to transition from

the traditional lecture into a more student-active learning environment. Another method

to increase student involvement is the use of peer mentors. Peer mentors are

undergraduates of similar age and educational background as the students enrolled in the

course who are recruited to help teach the students.57

After 45 years of research on universities, Newcomb stated that professors have

little effect on the study habits and curriculum choices of their students.58 In general, the

student's peer group has the highest influence on the student. Newcomb notes that there

is little similarity between the interests that peer groups generate and support and the

academic objectives supported by a university. Using peer mentors in a classroom can

generate peer support for academic objectives of the university. Peer mentors can stress

that students should spend more time on studying and homework. Although this message

is no different from the professor's, Newcomb has shown that it will have more effect

coming from peers. Peer mentors can help students take a more active approach to their

own learning.








In a traditional lecture class in chemistry, students meet once a week in a small

recitation section, headed by a teaching assistant (TA). The purpose of the smaller class

is to give the students an opportunity to ask questions in a small group without the

professor present. However, it has been shown that discussion sections function mainly

as small lectures, and the students are no more active in these types of sessions than in the

traditional lecture.59 A peer mentor, on the other hand, works closely with students, both

asking questions and offering information on a more informal basis than either a lecture

or a discussion section. This give-and-take with a peer mentor helps students take a more

active role in the learning process.

Robin and Heselton have discussed three aspects of a good peer mentor.60 A peer

mentor engages students in casual conversation in greeting and in closing a session.

Although this can be related to events outside the class such as a recent football game, it

can include generalizations about the course or an upcoming test. Casual conversation is

important since it can highlight differences between the peer mentor and the instructor for

the course. The instructor is often viewed as more of an authority figure (i.e. someone

who would never ask about the football game) than the peer mentor, who is often viewed

as more of a friend.

A good peer mentor offers feedback to the students. This feedback should be

personal and related directly to the student's progress in the course. The mentor can offer

feedback on graded items, such as quizzes or exams, but also on verbal explanations or

defenses of answers given by the students. In the introductory chemistry class, for

instance, peer mentors should offer feedback on dimensional analysis as well as logic

used in problem solving.








The third aspect of a good peer mentor, as described by Robin and Heselton, is

praising the student's performance. Peer mentors should offer immediate and positive

feedback to the students whenever possible. This helps maintain students' morale and

interest in the course. This aspect of a good peer mentor is truly an extension of the

feedback aspect described above.

Although other researchers agree upon these three aspects of a good peer mentor,

often two additional aspects are described. A good peer mentor should also prompt the

students.61 These prompts should consist of verbal clues that help students answer their

own questions. The peer mentors should not simply provide answers. Instead, they

should help students arrive at their own answers.

The fifth aspect of a good peer mentor is assisting the instructor with

administrative tasks.62 These administrative tasks can include proctoring and grading

exams, handing out and collecting material from students, and gathering feedback about

the course from the students.

An important aspect of a good peer mentor that does not appear to be described in

the literature is that the peer mentor should have a thorough knowledge of course content

and material. Although the mentors are not expected to have the same in-depth

knowledge of subject material as the instructor, they must demonstrate a proficiency in

the covered material. It would be difficult for a mentor to assist students with material

that the mentor does not understand. This does not mean confining the mentor pool to

subject majors. Often students in other majors are competently able to assist students

with course content. In the introductory general chemistry class, only a very few peer

mentors (approximately 7%) are chemistry majors. Most mentors are pre-professional or









engineering majors with demonstrated proficiency in the course, although we have

several business and finance majors who are excellent mentors.

Several studies have been performed on the effectiveness of peer mentors versus

staff instruction in small groups.63-67 These studies were set up where the staff instructor

was expected to act in a similar manner as the peer mentors. Students were assigned

problems in small groups that were headed either by an instructor or a peer mentor. In

most of these studies, the group leader was the only source of information in the class.

Therefore, students in groups headed by a peer mentor did not meet with an instructor.

The results of these studies were mixed, with results ranging from an increase in student

achievement in the mentor-led groups, no difference in achievement between the two

types of groups, and increased student achievement in the instructor-led groups. This

mixture of results makes it difficult to draw conclusions about the effectiveness of the

peer mentor method of instruction.

Most of the studies noted positive side effects beyond the mixed student

achievement in the mentor-led groups. These studies noted that peer mentors were able

to understand the nature of student difficulties with subject-material better than the

instructor, having recently gone through the course themselves. Peer mentors were also

more interested in the daily lives of students, their study experiences, and student

personalities. Peer mentors also directed students' attention to upcoming exams more

than did the instructors. Additionally, peer mentors displayed more supportive behaviors

than did the staff instructors. Although the quantitative academic benefits of peer

mentors were originally not clear, peer mentors were introduced in CHM 1025 at the

inception of the course. However, students did not meet solely with mentors.








In CHM 1025 students meet with both an instructor and peer mentors every class

session. For the first 15 to 25 minutes of the 50 minute class, the instructor delivers a

brief lecture on the topic being covered. After the lecture, the students break into their

small collaborative groups to solve problems based on the topic just covered. Peer

mentors assist students with problem solving during the collaborative sessions. In this

manner, students get the benefits of having an instructor impart background information,

and also benefit by working closely with mentors who are able to assist them with

difficult concepts or problems.

Peer mentors are actively recruited at the end of every semester. Students who

have passed the course and evidenced good helping behavior with their group-mates are

asked to come back the following term and assist as a mentor. Current peer mentors

usually know the students in their groups and have good suggestions for future mentors.

These students do not have to be academically the best in the class. Some of our best

mentors are students who struggled in the class but who demonstrated helping behavior

in their groups. Mentors assist in the classroom for as many semesters as their schedules

permit several have mentored every semester up until graduation. There were only six

peer mentors during the first semester that this program was implemented. Now, over

100 students have volunteered to be mentors.

Initially, the mentors were paid a stipend for helping in the class. The stipend

money ran out after the second semester of the program. Now peer mentors enroll in a

one-credit 4000 level chemistry class called Supervised Teaching. As an added

incentive, the peer mentors work closely with an instructor who is then able to write an

excellent recommendation based on the mentor's responsibility, leadership skills, helping









behavior and teamwork. Most mentors begin the program during the second semester of

their freshman year. For the first few years the mentors are enrolled in the program, the

CHM 1025 instructor is the only professor who knows the mentors' names due to the

large number of students at UF.

The racial makeup of the peer mentors is diverse, although heavily weighted

toward minorities and students of color, as is the CHM 1025 class. Many of the mentors

speak a language other than English as their first language. Several mentors have been

overheard explaining a concept in their native language to other non-native English

speaking students in the class. Often this added explanation in the student's native

language makes the concept understandable to the student.

Peer mentors attend a one-hour training session during the first week of each

semester. All peer mentors attend, including those who have previously participated in

the program. During this session, peer mentors are taught how to get a whole group

involved in problem solving, in large part by facilitating group-member interactions. Peer

mentors are informed that they are not expected to just give an answer to students. They

should first check the student's logic and dimensional analysis setup. It is also stressed

during this meeting that the peer mentors are not expected to have all the answers to the

students' questions. The mentors are informed that "I don't know" is a perfectly

acceptable response, as long as that mentor then finds someone who can answer the

question for the student.

After this training meeting, staff meetings are held every other week. At these

staff meetings, peer mentors are given the answer keys to the problems that will be solved

during the upcoming two weeks of class. Any issues that have arisen over the previous









two weeks are also discussed. Experienced peer mentors are comfortable bringing up

discussion points as early as the first meeting. As the new peer mentors see how well

these ideas and criticisms are received, they begin bringing up issues as well. By the

third or fourth meeting there is usually a good dialogue between the instructor and the

mentors. Upcoming events such as exams and changes in curriculum are also discussed

in these meetings.

Students are much more likely to offer complaints and suggestions to the mentors

during class than they are to offer them to the instructor. Therefore, it is important to

have a time set aside when mentors can bring up their and the students' concerns. It is

also helpful to discuss the reasoning behind policy decisions in the class with the mentors

because they are able to relay this to the students when an unpopular decision is made.

When the mentors understand the logic behind a decision, such as the minimum point

requirement on the final, students are often more accepting of the policy than when the

decision is only announced by the instructor.

The peer mentors are also an excellent resource for suggestions about how the

class can be changed for the better. Not only have they recently participated in the

course, the students in the class are constantly offering suggestions to them. For

example, the idea to reverse the group and individual portions of the collaborative exam

was first brought up by a peer mentor, and the majority of the other mentors strongly

agreed.

It was believed that having the mentors in the class would help personalize the

class for the students. Normally, the chemistry classes have one instructor and 297

students. With the mentors in the class, this ratio was brought down to one peer mentor








for every six to 18 students, depending upon the semester. These mentors often know

their students individually, and question the students on their progress as well as on their

absences. It was believed that this individual attention would keep the students coming

to class and learning the material. In order to test this assumption, the withdrawal rates of

sections that used peer mentors were compared to sections that did not utilize them.

During the fall of 2003, five sections of CHM 1025 were offered. Three of these

sections did not utilize the peer mentors, while the other two sections did so extensively.

As shown in Table 5-1, the withdrawal rates in the sections that did not use peer mentors

was much higher than in the sections that did use them (p-value <0.0001). Although it

cannot be determined conclusively that the peer mentors affected the withdrawal rate, it is

likely that having a peer mentor checking on students' progress helped lower this rate.

Table 5-1: Percent of students withdrawing from each section during the fall of
2003
Section Instructor Peer # Students # Students Percent
Mentors Enrolled Withdrew
0675 LL No 257 27 10.5%
0676 KW No 303 37 12.2%
0677 LL No 303 23 7.6%
4284 KZ Yes 297 10 3.4%
5253 KA Yes 300 10 3.3%

The sections that utilized peer mentors during the fall of 2003 were held in the

afternoon, while those sections that did not use the peer mentors were held during the

first three 50-minute periods of the day, beginning at 7:25 a.m. In order to ensure that the

time of day did not skew withdrawal rates, the rates for fall of 2003 were compared to the

rates for fall of 2002, where the classes were scheduled at exactly the same times. As

shown in Table 5-2, the percent of students withdrawing from sections that did not utilize









peer mentors was higher, regardless of the time of day the sections were held (p-value

0.0823).


Table 5-2: Withdrawal rates versus mentor utilization


Fall 2003 Fall 2002
Mentors No Mentors No Mentors No Mentors
(afternoon) (morning) (afternoon) (morning)
# Enrolled 597 863 517 900
# Dropped 20 77 29 63
% Dropped 3.35% 8.92% 5.61% 7.00%


In a voluntary, anonymous survey, students were asked to rate the helpfulness of

the peer mentors. The majority of the students (74%) believed that the mentors were very

helpful to have in class (Figure 5-1). An additional 17% thought the peer mentors were

somewhat helpful. Only 9% of the students thought that the mentors were never helpful.

The majority of students taking CHM 1025 believe that the peer mentors help them better

understand the material. This is a very positive result.


80.00

70.00

60.00

50.00

40.00

30.00

20.00

10.00

0.00
very somewhat not at all

Figure 5-1: Percent of students rating the helpfulness of peer mentors








Although it is easy to see how the students benefit from having the peer mentors

in class, it is important to recognize that the peer mentors also benefit from the program.

The mentors have an opportunity to review the basic chemistry that they are expected to

know in CHM 2045 and CHM 2046. Mentors who have progressed beyond this phase of

chemistry have stated that mentoring has helped them review for such exams as the

MCAT, DAT, and PCAT. The mentors also have a comfortable relationship with the

CHM 1025 instructors, and have no problem asking for help in other chemistry subjects,

such as organic and analytical chemistry.

It is important to recognize that the peer mentors have mostly taken CHM 1025.

In general, the mentors that are assisting in the classroom were under-prepared for

college chemistry when entering UF. These mentors are now much more confident in

chemistry, and several have chosen to major in chemistry. Also, several mentors who

took CHM 1025 in the summer of 2003, and CHM 2045 in the fall of that year took

CHM 2051, an honors version of CHM 2046. Students who were initially under-

prepared have progressed far enough to take an honors chemistry class.

When the CHM 2045 grades for students who have participated in the peer

mentor program are compared to those students who are not in the program during the

fall semester of 2003, there is a significant difference between these two groups (p-value

<0.0001). As shown in Table 5-3, the students who participated in the program have a

higher average CHM 2045 grade of B+ versus the average grade of C for the students

who have not participated in the program. When the peer mentor grades are compared to

those students who also required the introductory course, the peer mentor grades are

again significantly higher (p-value <0.0001). Students who participate in the peer mentor









program are more successful in later chemistry courses than students who do not

participate in the program.

Table 5-3: Mentor grades in CHM 2045
Mentors Non-Mentors
Required Ready for Required
CHM1025 CHM 2045 CHM 1025
GPA 3.71 2.32 2.14
Letter Grade B+ C C

The mentors who are utilized in the CHM 1025 course have an overall positive

experience with the program. Mentors comments from some of the required end-of-

semester papers have included:

There is something very special about helping people ...
Sitting down with them and helping them to understand
something that they couldn't before is just incredible.

I brushed up on some of my basic chemistry and was able
to help some students along the way.

I liked being able to help students and at the same time let
them know that there are people like them who can do
chemistry.

The students in the class were great to work with. I liked
going to students who were struggling with a problem and
helping them think it through and seeing them actually
learn as they were getting the answer. It is a good feeling
helping people gain knowledge.

The mentoring program is an effective way to personalize the class for the

students. The mentors help make the students more comfortable in the class, but the

instructor still has the authority necessary to control the class. The mentors are able to

help the students immediately with any questions and problems they have. In a large

class of 300 students, the instructor would never be able to answer all the students'

questions individually. However, the mentors provide students with an opportunity to








ask questions in small groups, ensuring that students do not leave the class with

unanswered questions.

Confidence in the peer mentors among departmental instructors has also grown as

the program has progressed. Initially, when peer mentors helped proctor exams some

instructors in the department were skeptical. They did not like the idea of second

semester freshman proctoring exams for their peers. As the mentors have proven

themselves reliable and trustworthy this skepticism was replaced with an interest in the

program. Instructors in CHM 2045 and CHM 2046, who have not used the peer mentors

in their classes, now occasionally use the mentors to proctor exams when there are not

enough graduate students available. During the spring semester of 2004, the department

was short one TA in CHM 2046. An experienced peer mentor was asked to TA for this

course, the first time that a non-chemistry major or minor has been a TA for a chemistry

class.

The mentoring program helps the peer mentors review basic chemistry and work

on their social skills in the groups. Students who participate in the peer mentoring

program also have higher GPA scores in later chemistry courses than their counterparts.

Most students enrolled in CHM 1025 believe that the mentors helped them understand

chemistry. Withdrawal rates also are lowered in the classes where mentors are checking

on student progress and attendance. The program has had positive results both on the

mentors and on the students, and should be continued in future CHM 1025 courses.














CHAPTER 6
CONCLUSIONS

Chemistry Readiness Assessment Exam

The ChRA is the exam taken by potential chemistry students before enrollment in

either the introductory CHM 1025 course or the mainstream CHM 2045 course. The

initial cutoffs for this exam were four or greater (out of eight questions) correct on the

math section of this exam, and nine or greater (out of 18 questions) correct on the

chemistry section. Students should have earned 50% on both sections in order to take

CHM 2045. These scores were only guidelines, and students were able to register for

whichever course they wished.

Students that earned only a four out of eight on the math section of the ChRA and

did not take CHM 1025 earned an average grade of D+ in CHM 2045. In order to pass

into CHM 2046, students must earn a grade of C, so this math cutoff is unacceptable.

Students that correctly answered five out of eight on the math section earned an average

grade of C. This C grade was extremely low. Students who correctly answered six out of

eight questions on the math section also earned a C in CHM 2045, but this C grade was

much stronger. Therefore, the minimum cutoff for the math section of the ChRA should

be six out of eight questions.

Students that correctly answered nine questions out of 18 on the chemistry portion

earned an average grade of C in CHM 2045. This was a strong C, so the minimum cutoff

for chemistry can be lowered. Students who correctly answered eight, seven, or six








questions on the chemistry section of the ChRA also earned a C grade in CHM 2045.

Students who correctly answered five of 18 on the chemistry portion earned an extremely

low C in CHM 2045. Therefore, the minimum cutoff for the chemistry section of the

ChRA should be six out of 18 questions.

Although these cutoff values are the same, students must correctly answer 75% of

the math section while only answering 33% of the chemistry section in order to succeed

in general chemistry. Student success in chemistry depends more on students' prior math

knowledge than their prior chemistry knowledge. Students who received a score of six

on the math section and six on the chemistry section of the ChRA were more likely to

receive an A in CHM 2045 than the students who did not receive these scores. Students

who received less than a six on both math and chemistry were more likely to withdraw

from CHM 2045 than their counterparts. Students who passed either the math section or

the chemistry section of the exam had mixed grades, but the students who passed the

math section performed better than the students who passed the chemistry section.

Effectiveness of CHM 1025

CHM 1025 was designed for those students who are under-prepared for the

mainstream CHM 2045 course. Student achievement in CHM 2045 was compared to

determine if CHM 1025 prepared these students for the mainstream course. The students

who had taken CHM 1025 had an average GPA score of 2.74 in CHM 2045, while those

students who had not taken CHM 1025 had an average GPA score of 2.12. Former CHM

1025 students are better equipped to succeed in CHM 2045 than the students who did not

need the course. Students who failed CHM 1025 but enrolled in CHM 2045 anyway had








an average GPA score of 0.59. Therefore, once CHM 1025 is attempted, it must be

passed before the student can advance to CHM 2045.

The average GPA 'scores of students in CHM 2046 that had taken CHM 1025

were approximately the same as those students that did not take the introductory course,

and were about a B. Although these scores are not significantly different, it demonstrates

that initially under-prepared students are succeeding in the course at the same rate as their

better-prepared classmates.

The number of students who continued into CHM 2046 was compared to

determine if CHM 1025 had an effect on this rate. The majority of the students (68.0%)

that continued on to CHM 2046 in the fall semester of 2003 were initially enrolled in

CHM 1025. Of the students that passed CHM 2046 and were eligible to continue in

chemistry, the majority (69.4%) was initially enrolled in CHM 1025. Therefore, former

CHM 1025 students are more likely to continue to CHM 2046, and are more likely to be

eligible to continue into Organic than their counterparts.

The students who had to repeat either CHM 2045 or CHM 2046 were also studied

to determine if CHM 1025 had an effect on this rate. Only 28.2% of the students who

had to repeat CHM 2045 had taken CHM 1025, and only 7.4% of the students who had to

repeat CHM 2046 had taken CHM 1025. Therefore, CHM 1025 students are better

prepared to succeed in later chemistry classes on the first attempt.

Collaborative Learning

During the fall semester of 2002, students were placed into groups in CHM 1025 in a

random fashion, by student-choice, or by achievement. There were two sections that did

not use collaborative learning in this term. There was no difference in average final








scores in the random, student-choice, or non-collaborative classes. All three of these

sections had a higher final GPA than the section grouped by achievement. Therefore,

using student achievement to place students into groups is not an effective method of

placement. The fact that collaborative learning did not have an effect on final GPA was

disappointing, but by the fall of 2003 this was fixed.

During the fall semester of 2003, there were two collaborative sections and three

non-collaborative sections of CHM 1025. The students in the collaborative sections had

a higher GPA score than in the non-collaborative section. Collaborative learning helped

the students succeed in CHM 1025.

Attendance

Attendance rates in CHM 1025 were compared to determine if attendance had an

effect on student achievement. Students who attended less than 25% of the time earned

an extremely low E score in CHM 1025. Those who attended between 25% and 50% of

the time earned a higher E score in CHM 1025. Students who attended between 50% and

75% of the time earned an average grade of D in CHM 1025. Students who attended

more than 75% of the class sessions earned a B in the class. Only those students who

attend class at least 75% of the time can be expected to pass.

Group placement methods were compared to determine if one particular group

type affected student attendance. Students who were placed randomly into groups

attended class approximately 83% of the time. Those who were placed into groups by

achievement attended class approximately only 68% of the time. This further

demonstrates that placing students into groups by achievement is not an effective method

of increasing student success.








Collaborative Testing

Prior studies that had used collaborative testing as an assessment method showed

a modest increase in student achievement. Collaborative testing was implemented in

CHM 1025 during the spring semester of 2004. The exams consisted of two sections, an

individual section worth 70% of the student's grade, and a group section worth 30% of

the student's grade. Students worked in the same groups during the exams as they

worked with in class.

The effect of collaborative testing on student achievement was mixed. On two of

the exams (exam one and exam four), students had higher achievement on the

collaborative exams than the non-collaborative exams offered in spring of 2003. On

exams two and three, there was no difference in student achievement versus the prior

semester. On the final exam, students who took the exam collaboratively performed

worse than their counterparts of the year before.

Although overall achievement was not conclusively affected by collaborative

testing, there were some positive results to the study. The collaborative exams were more

difficult, with more complex questions, than the exams from the spring of 2003.

Generally, students performed just as well on these harder exams as they had on the

easier individual exams. Students are better equipped to answer more difficult questions

when in a group than when working individually. Also, the lowest scores were

eliminated in the collaborative sections, as noted by the decreased ranges and standard

deviations on the exams. Overall student achievement increases when collaborative

exams are used. Finally, a greater percentage of students continue to take the exam when

it is collaborative than when it is an individual effort.








Students who took the collaborative exams stated that their testing anxiety was

reduced both when taking the exam and when studying for it. Reduced anxiety levels has

been shown to increase student performance. Students also stated that they preferred

collaborative testing to individual testing, and studied for the collaborative tests about the

same as they would have for an individual exam. Approximately 75% of the students

stated that social loafing was not a problem in their group, and that all group members

contributed to the exam. Collaborative testing is an effective assessment method and

should be continued in CHM 1025.

Peer Mentors

Having peer mentors in the classroom helps personalize the class for the students,

cutting down the student-instructor ratio from 300 to one to about 12 to one. Students are

more likely to ask questions in small groups and thus are more likely to understand

difficult concepts with added explanations. The students also offer complaints and

suggestions to the mentors, who are then able to relay this to the instructor.

Approximately 75% of the students enrolled in CHM 1025 stated that peer mentors

helped them understand the concepts covered in the class.

The withdrawal rates for the CHM 1025 sections that used peer mentors in the fall

of 2003 were much lower than in the sections that did not use them (approximately 3%

versus approximately 10%). Although the difference in withdrawal rates cannot

conclusively be attributed to the presence of peer mentors, having a mentor check on a

student's progress and attendance certainly helps lower the rate.

The peer mentors also benefit from participating in the program. The mentors

have an opportunity to review basics that are used in later chemistry courses and that are






89


included in exams such as the MCAT and DAT. Students who participate in the peer

mentoring program have a higher GPA in CHM 2045 than those students who are not in

the program (3.71 versus 2.32). Overall, mentors enjoy the experience, and many

continue to mentor long after they have stopped taking chemistry classes.














APPENDIX A
KEY TO INSTRUCTOR INITIALS

KA .......................................................................................................... Katie Am aral

KW .........................................................................................................K athryn W illiam s

KZ ................................................................................................................ K eith Zientek

LL ...................................... .. ..... ........... .................... ........................... Larry Land

M V ................................................................................................................. M martin Vala

RB .............................................................................................................C. Russ Bowers