ASSOCIATION BETWEEN STUDENT RELIANCE UPON NOPENALTY RETAKING
OF MODULE TESTS AND FINAL EXAMINATION SCORES IN
FLEXIBLY PACED ENGINEERING MECHANICS COURSES
by
WILLIAM CLIFFORD DOBY
A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1976
DEDICATION
To Marcia and Ann ..
ACKNOWLEDGEMENTS
The writer wishes to express his gratitude to Dr. James Hensel
for his help and direction, to Dr. Martin Eisenberg for his assistance
and sharing of records, and to Dr. Robert Ramey for his guidance and
positive attitude.
Appreciation is expressed to Professor Walter Bond of the
University of North Florida for his assistance in statistical analysis.
To all the above the writer gives thanks for the generous amount
of freely given time.
TABLE OF CONTENTS
ACKNOWLEDGEMENTS . . . . . . . . .
LIST OF TABLES. . . . . . . . . .
ABSTRACT . . . . . . . . . .. ..
CHAPTER I INTRODUCTION . . . . . . . .
Background . . . . . . . . . .
Statement of the Problem . . . . . ..
Need for the Study . . . . . . .
Assumptions. . . . . . . . . .
Definition of Terms . . . . . .
Boundaries of this Investigation . .. ....
Method . . . . . . . . . . .
Summary . . . . . . . .. . .
CHAPTER II REVIEW OF RESEARCH AND RELATED LITERATURE
Introduction . . . . . . . . .
Predictors of Academic Success . . . . .
Comparisons of Various Teaching Methods . .
Additional Topics . . .. .......
Summary . . . . . . . . . ..
Page
iii
viii
ix
1
1
Page
CHAPTER III THE MODULARIZED INSTRUCTIONAL SYSTEM AND
THE TEST POPULATION. . . . . . .... ... . 22
Introduction . . . . . . . . . . . 22
The Modular Instruction System . . . . .. ... 22
The Test Population. . . ... ... . . 28
Summary. . . . . . . . . . . . .. 30
CHAPTER IV METHODOLOGY, ANALYSIS OF DATA, AND FINDINGS . 31
Introduction . . .... .... ....... .. 31
Nature of Study. . . . . ... .. .. .. .. 31
List of Hypotheses . . . . . . . . ... 32
Rationale for Using "Modified Junior Year GPA" . . 33
Analysis of the Uniformity of the Modules and Units. .. 35
Data Collection and Selection. . . . . . ... .36
Characteristics of the Data. . . . . . . ... 37
Graphical Presentation of Selected Data. . . . ... 37
Summary of an Initial Chi Square Analysis. . . .. 38
Summary of a Revised Chi Square Analysis . . ... .39
Multiple Linear Regression Analysis for All Ten Sections 40
Effect of Date of Completion of the Modular Path . .. 52
Student Attitude . . . . . . . .... . 52
Summary . .... . . . .... 54
CHAPTER V SUMMARY. . . . . . . . . ... . 56
Introduction . . . . . . . . .. . . 56
The Instructional System . . . . . . . .. 56
Need for the Study . . . . . . .. . . 57
Page
Nature of the Study. . . . . ... .. . . . 58
Review of the Literature . . . . . . . .. 59
Uniformity of Difficulty of the Module Tests . . .. 60
Data Collection and Selection. . . . . . . .. 60
Results of Chi Square Analysis . . . . .... .. . 61
Results of Linear Regression Analysis. . . . . ... 62
CHAPTER VI CONCLUSIONS AND RECOMMENDATIONS . . . ... .66
Introduction . . . . . . . . . . . 66
Conclusions. . . . . . . . . . . . 66
Recommendations. . . . .. . . . . . 68
APPENDIX A
EXCERPTS FROM "A MODULAR INSTRUCTIONAL SYSTEM FOR
INTRODUCTORY COURSES IN ENGINEERING MECHANICS" ...... 71
APPENDIX B
STUDENT RECORD SHEET. . . . . . . . . . 88
APPENDIX C
SAMPLE MODULE PROFICIENCY TEST . . . . . ... 90
APPENDIX D
TABULATION OF OVERALL MODULE DIFFICULTY. . . . ... .92
APPENDIX E
TABULATION OF ENDOFMODULE PROFICIENCY TEST DIFFICULTY
WITHIN EACH MODULE . . . . . . . .... . 94
APPENDIX F
SELECTION OF STUDENT RECORDS FOR THE DATA BASE . . .. 98
APPENDIX G
TWO DIMENSIONAL PLOTS OF THE DATA FROM SELECTED SECTIONS 102
Page
APPENDIX H
Student Attitude Questionnaire . . . . . ... 115
REFERENCES. ............... ... . . 116
BIOGRAPHICAL SKETCH . . . . . . . . ... . 121
LIST OF TABLES
Page
TABLE 1 MODULES OF THE VERSIONS FOR VARIOUS DEGREE
PROGRAMS. . . . . . . . . .. . . 24
TABLE 2 NUMBER OF MODULES REQUIRED FOR INDIVIDUALIZATION
BY SUBDISCIPLINE . . . . . . . . .. 26
TABLE 3 ESM SECTIONS USABLE FOR ANALYSIS. . . . . . 37
TABLE 4 ANALYSIS OF VARIANCE SUMMARY. . . . . . . 43
TABLE 5 CONCOMITANT DATA FOR THE ANALYSIS OF VARIANCE
SUMMARY . . . . . . . . . . . 45
TABLE 6 MULTIPLE REGRESSION ANALYSIS RESULTS FOR SECTIONS
SHOWING OVERALL STATISTICAL SIGNIFICANCE. . . ... 47
TABLE 7 ANALYSIS OF VARIANCE RESULTS FOR REGRESSION
ANALYSIS PERFORMED FOR GPA ALONE AND RETAKES ALONE. 49
TABLE 8 MULTIPLE LINEAR REGRESSION ANALYSIS WITH DATE OF
COMPLETION OF MODULES INCLUDED. . . . . ... 52
TABLE 9 MULTIPLE LINEAR REGRESSION ANALYSIS WITH ATTITUDE
INCLUDED. . . . . . . . . . . . 54
TABLE 10 TABULATION OF OVERALL MODULE DIFFICULTY . . .. 92
TABLE 11 TABULATION OF ENDOFMODULE PROFICIENCY TEST DIFFICULTY
WITHIN EACH MODULE. . . . . . . . .. 94
viii
Abstract of Dissertation Presented to the Graduate Council
of the University of Florida in Partial Fulfillment of the Requirements
for the Degree of Doctor of Philosophy
ASSOCIATION BETWEEN STUDENT RELIANCE UPON NOPENALTY RETAKING
OF MODULE TESTS AND FINAL EXAMINATION SCORES IN
FLEXIBLY PACED ENGINEERING MECHANICS COURSES
by
William Clifford Doby
August, 1976
Chairman: James W. Hensel
Major Department: Curriculum and Instruction
This research investigated a modularized instructional system to
determine if there is an association between the extent of retaking of
module tests and the score on the final examination of a course. The
instructional system consisted of three junior level engineering mechan
ics courses in statics, dynamics, and strength of materials. The
courses were taught at the University of Florida in a flexible modu
larized form that offered several student options and individualization
by student engineering specialty. Several endofmodule proficiency
tests were provided for each module. One of these tests was selected
at random when requested by a student. A student failing a module test
could without penalty retake other tests until mastery of the module
was demonstrated. Only the passing grade was recorded. A different
option involving intracourse examinations could be followed by students
not selecting or continuing the modular proficiency test option. All
students took the same endofcourse examination. No sections of tra
ditionally taught students were available as a control group.
The number of retakes referred to how many times a student attempted
to pass a module proficiency test after failing an initial attempt. The
number of retakes was used as an independent variable for multiple
regression analysis.
Score on the final examination was the dependent variable. Grade
point average and number of retakes were used to predict the observed
final examination score. GPA was assumed to be the major determinant.
With its effects statistically controlled,the effect of the number of
retakes was sought as a secondorder effect.
Ten sections of the modularized courses yielded usable data. GPA
was a statistically significant predictor of final examination score
in six sections. In three of these six sections, the number of retakes
was also statistically significant at the 95% level of confidence.
In two of these three cases, an increasing number of retakes was
associated with a decreasing score on the final examination. In the
third case, the opposite was unexpectedly found to be true. Students
who seemingly had difficulty mastering the modules did unexpectedly
well on the final examination
Investigation of circumstances relating to the three sections
yielded the finding that students in the first two sections frequently
took two, three, or in some cases four versions of a module completion
test on the same day. Their demonstrated mastery of the subject matter
could reasonably be attributed to happenstance rather than to restudying
and increased learning. Moreover, the subject matter was statics, which
was the easiest of the three courses. The students were beginning engi
neering students and many of them were recent transfers from the junior
colleges. Circumstances were different for the third section, the one
in which greater retaking of module tests was associated with increased
achievement. Changes in administrative procedure hindered immediate
retaking. Students almost never took two tests of a module on the same
day. Moreover, the subject matter was mechanics of materials, which
was the most difficult of the three courses. Mastery upon initial
studying might well not be expected. The students were more mature.
Although this section was small, it yielded a highly statistically
significant result that could not be disregarded.
The researcher concluded that there is a small but discernible
association between the extent of module test retaking and the score on
the final examination of a course. The manner of retaking rather than
the amount of retaking appeared to be the condition that influenced
endofcourse achievement.
CHAPTER I
INTRODUCTION
Background
Academic course offerings at all levels have tended to change
from rigid to flexible formats in recent years. One of the innov
ations has been the use of modularized packaging of course content.
This innovation has provided opportunities for individualization,
flexible pacing, and mastery learning. Use of these techniques at
the college level has been less extensive than at lower levels.
Three introductory engineering courses at the University of
Florida were restructured and offered in a modularized format that
permitted selfpacing and a degree of individualization. For a period
of approximately two years, an unlimited retaking of randomly selected
module tests was permitted, thus allowing students to maintain an
average of A or B up to the point of taking the final examination.
Statement of the Problem
The College of Engineering of the University of Florida in 1973
offered three basic undergraduate engineering courses in a modularized
form. The three courses were engineering mechanics courses in statics,
dynamics, and mechanics of materials and are described in Chapter III.
Minor variations in the selection of modules assigned to students pro
vided a type of individualization of course content. These variations
provided a specialization according to a student's major within the
several departments of the College of Engineering. Student selfpacing
and unlimited retaking of module tests were the main student options
provided. This dissertation was undertaken to provide an indepth
study of the nopenalty retaking of module tests. This dissertation
primarily seeks to answer the question, Is student use of the unlimited
nopenalty retaking of module tests associated with student achievement
an the final examination when the effects of other reference or causal
variables are statistically controlled? No causal relationship between
achievement and extent of module test retaking was hypothesized.
Need for the Study
The modularized sequence of engineering mechanics courses at the
University of Florida was implemented after considerable curriculum
development effort. The unlimited retaking of module tests without
penalty was a feature that had been included because of the belief
that it was a desirable liberalization from the previous instructional
method. In 1975, this feature was discontinued because reduced funding
curtailed personnel services necessary to maintain the testing system.
The question remained open as to the relationship between student
achievement and the nopenalty retaking of module tests. This dis
sertation provided an indepth investigation of results associated
with retaking of module tests. The associations that were found permit
prediction of achievement based upon known patterns. Counseling and
early remedial guidance can he facilitated.
Assumptions
The assumption of this investigation was that a student's academic
achievement is the best predictor of that student's expected academic
achievement in other courses under identical circumstances. Achievement
in standard courses was used to calculate the expected achievement in
other courses taken concurrently. The effects of ability as well as
environment were thus assumedly controlled by using known representative
achievement during a given time period to estimate an achievement that
would be expected in separate courses taken during the same time period.
Selections from the literature concerning this assumption are presented
in Chapter II.
Definition of Terms
The expression "modified junior year GPA" refers to a specially
calculated gradepoint average earned in traditionally taught courses
taken during the same time period as were the modularized engineering
mechanics courses. The normal time for taking these engineering
mechanics courses was the junior year. Some students began the course
sequence early, during their sophomore year. Other students finished
the sequence late, during their senior year. Some students began
early and finished late. The many variations required the researcher
to select three representative quarters for each student. These
quarters were those in which the student took the three engineering
mechanics courses or a portion of them. The selected three quarters
approximate the junior year time period and are so labeled.
The term "nopenalty" in the expression "nopenalty retaking of
module tests" refers to the unlimited retaking of randomly selected
endofmodule tests until a score of 80% or better was obtained. Only
this last score was recorded for the module concerned. The scores of
previous attempts had no weight in determining the letter grade for
the course.
A retaking of an endofmodule completion test occurred after a
student failed in his first attempt to pass one of the randomly
selected tests. Each subsequent attempt to pass another of the tests
for that module was a retake. In an eight module version of a course,
a student requiring a total of 10 attempts had two retakes. When
used as an independent variable in regression equations, the number
of retakes is represented by the capitalized word "Retakes."
This investigation was an associational study. It was not an
experiment based on random assignment. A cause and effect relation
ship was not being sought. Somewhat more general words like "result"
and "outcome" were usually used herein in lieu of "effect."
The term "independent variable" referred to the number of module
test retakes during a course. The term "dependent variable" referred
to a student's score on the final examination of the course. The
latter definition was augmented when chi square analysis was used. In
that situation, a student's final examination score as compared to his
expected score was used as the dependent variable. The standard math
ematical terms "independent variable" and "dependent variable" were
used in this associational study although some researchers prefer to
use these terms only for experiments seeking cause and effect relation
ships.
Boundaries of this Investigation
A multitude of factors influence or are associated with student
achievement. This investigation examined one of these, the number of
times individual students relied upon the nopenalty retaking of mod
ule tests. This study was limited to those students who took one or
more of the three modularized engineering mechanics courses at the
University of Florida during the period from September 1973 to March
1975. This study considered only those students who pursued the mod
ular path to completion and did not investigate dropouts or students
who chose to follow the alternate path involving scheduled intracourse
examinations.
Method
Student achievement is multidetermined. This investigation des
ignated student achievement in the modularized instructional system
as the dependent variable. The extent of retaking of nopenalty mod
ule tests was one independent variable. The many other influences
were grouped into a single variable termed modified junior year grade
point average. Statistical analyses were performed to see if indi
vidual student reliance upon the nopenalty retaking of module tests
was significantly associated with student achievement when the other
influences were controlled. The design and methodology are described
in Chapter IV.
Summary
A modularized instructional system was developed and implemented
at the University of Florida for three junior level engineering
mechanics courses. The effectiveness of the innovations is a relevant
6
question. An analysis was made of the way that academic achievement of
students was associated with an instructional feature allowing unlimited
nopenalty retaking of module tests.
CHAPTER II
REVIEW OF RESEARCH AND RELATED LITERATURE
Introduction
An abundance of literature directly relates to predictors of
academic success. Because this dissertation compared observed student
performance with a calculated expected (predicted) performance, the
subject of methods of prediction of academic success is relevant.
Selections from the literature are reported herein.
A meager amount of literature relates to flexibly paced modu
larized courses at the college level. The effects of unlimited no
penalty retaking of module tests in college level courses is an
aspect that has seldom been reported.
A computer search of The Educational Resources Information Center
(ERIC) showed a listing of 31,238 articles on colleges, universities
and higher education of which 1180 related to testing and related
index terms such as testing methods. When the index term "modular"
or related words were added to those involving "college level" and
"testing" for an inclusive search specification, the number of articles
decreased to three. The index word "retake" and its variation when
used alone produced 10 references of which only two related to college
level testing. A character by character stringing was used to produce
the composite index term "retake module tests." No articles were
found for this term. It is noted that the Thesaurus of ERIC
8
Descriptors does not list the key words "modular" or "modularized" in
uses related to course organization or the key words "retake" or
"repeatable" in uses related to testing.
A computer search of dissertation abstracts showed 194 entries
having the key word "modular" or "modularized" and 4701 having the
key word "test," "testing" or variations. When variations of "mod
ular" and "test" were used together, the number of entries decreased
to three, and the subject matter of one of these pertained to the
testing of electronic modules.
A manual search of dissertation abstracts and the ERIC system
was therefore the primary means of reviewing the literature.
Predictors of Academic Success
Prediction of academic success is one of the most extensively
explored areas of educational research. A large number of variables
have been investigated for their predictive efficacy. Sophisticated
statistical methods with use of computer processing of extensive data
have been used.
Of particular interest to this dissertation is its use of a
modified junior year gradepoint average to estimate expected
achievement. Mann (21) found that the best single variable for use
in determining admission to a professional engineering program was
sophomore gradepoint average. His study had examined 26 predictors.
Chapman (8) in a study of engineering students found that combi
nations of several psychometric predictors offered no improvement
over prediction by any of his single predictors alone. Of interest
to this dissertation is that Chapman's predictors could establish
excellent group distinctions even though individual student predict
ability was small.
For community college graduates transferring to the University
of Florida, Sitzman (31) found that the most powerful predictor of
success was the gradepoint average earned prior to transfer. Post
transfer variables such as age, marital status, and local residence
were included among the predictors tested.
In observing the validity of the 1966 precollege testing program
for students who entered Walla Walla College, Wagner (39) found that
the test score was a valid predictor of the allcollege GPA, espe
cially for female students.. The cumulative GPA at the end of the
third quarter was better predicted than the final GPA.
Schroeder and Sledge (30) found that intellective variables were
better predictors of college achievement than nonintellective vari
ables. Ronald G. Taylor (35) supported the conclusions of many when
he found that ability factors were the best determinants of student
success in collegiate programs, especially so for academically
oriented curricula rather than for vocationally oriented programs.
Stone (33) reported that the first semester gradepoint average
had a statistically significant relationship with continuance in col
lege. Fairchild (13) found that the total gradepoint average was the
better predictor of academic performance as compared to gradepoint
average in the student's major.
For students within the community colleges in the state of Washington,
VanDruff (37) found that high school GPA and initial GPA at the com
munity college were the best predictors of success in calculus. Only
a few predictor variables were studied. The multiple regression
equations derived were better predictors of the "A" grade or of the
nocredit grade than of the intermediate grades.
Another study to determine predictors of success in calculus was
conducted by Sommers (32) at Hope College. He related several pre
course factors to the score obtained on the final examination in the
calculus course. The best precourse factor found was high school
GPA. A locally prepared test as well as the verbal and mathematics
scores from the SAT were also found to be valid predictors of success
in calculus.
For students transferring as juniors to the College of Engi
neering at Oklahoma State University, Mouser (23) compared previous
academic aptitude variables and previous academic achievement vari
ables to see which better predicted success in undergraduate engi
neering courses. The previous aptitude variables were scores from
the ACT. The previous achievement variables were overall GPA and
subject GPA. The findings suggested that previous academic achieve
ment is more closely related to subsequent GPA than aptitude as mea
sured by the ACT. Use of both in multiple regression equations
produced the best predictive capacity.
Other studies have reached rather different conclusions about
efficacy of predictors of academic success. Elkins (12) showed that
the mathematics portion but not the verbal portion of the SAT could
discriminate between persisters and dropouts among freshman engi
neering students at the University of Maryland. In studying achieve
ment in the general educational requirements of lower division college
students, Cloninger (9) found that noncognitive factors play an
important role in predicting academic success. These factors
accounted for most of the variance between students with respect to
the regression equations predicting their achievement.
Comparisons of Various Teaching Methods
Another much researched area is the comparison of two teaching
methods. Attempts to reveal significant differences by statistical
analysis of observed data often do not indicate a difference of
learning effectiveness attributable to teaching method. Some
researchers finding this result conclude that student ability rather
than teaching method is what matters.
An individually paced instructional system in an engineering
college was compared by Venable (38) with group paced classes pro
ceeding in a more traditional manner. The classes involved sophomore
engineering courses of statics and dynamics. Twentyseven programmed
instructional units were prepared for each course. The classes
taught in the more traditional manner proceeded on a published
schedule. Students in the selfpaced classes took quizzes available
40 hours per week. Examinations were given in sequence when an
individual student was ready. This research found no difference
between the instructional methods as indicated by the examination
scores. It also showed no clear relationship between the number of
attempts which a student in the selfpaced classes made on unit
quizzes and his examination performance.
Eide (11) studied the effect of two different methods of
teaching engineering graphics. His experimental study compared the
learningachievement of freshman students in a conventional "lecture
and problem" class with achievement of similar students using a
series of 25 learning packages with audiovisual tapes. Students in
the latter group could control the amount of time for individual study
used to complete a unit. No significant difference in the amount of
learning was found between the two instructional methods. High school
scholastic achievement was a good indicator of achievement for students
using either method.
In another study comparing two methods of teaching engineering
graphics, Walker (40) compared the effectiveness of an ideacommuni
cation method with that of the traditional method. No significant
differences in any of three dependent variables (results) were found.
Walker concluded only that students with higher levels of critical
thinking ability attained a greater degree of general drafting knowl
edge than did students with lower levels of critical thinking ability.
Harris (17) compared student performance in a college engineering
science course in which students were randomly assigned to classes
using one of two different teaching methods. One method used an
audiotutorial approach. The other used a printed transcript of the
same material. There was no significant difference between the
learning achievement of the two types of classes.
A similar finding was reported by Otten (26) who compared three
instructional models for teaching a sophomore electrical engineering
AC circuit course. One method used the traditional lecture approach
without written objectives or use of computer. A second method utilized
measurable behavioral objectives. A third method used the computer as
a computational tool to illustrate the material and motivate the stu
dents. All three methods were found to be effective. No statistically
significant differences in achievement appeared among the groups
instructed by the three different strategies.
A different conclusion was reached by Aird (1) in comparing tradi
tionally taught engineering students with students using selfstudy
involving computer based instruction. Mechanics of solids was the
engineering courses taught by the two methods. The findings seemed to
justify the conclusion that the computer based instructional mode pro
duced students who performed better than traditionally taught students.
Related to these comparisons of teaching methods is the finding by
Tovey (36) that postadmission factors have little influence on perfor
mance of incoming high school graduates and that the best predictor of
college success is rank in high school rather than postadmission events.
Simarily, a study of the use of repeatable testing for college
chemistry students by Donovan (10) revealed no difference in final
achievement as compared with the control group students who took non
repeatable tests.
Compulsory attendance in an audiotutorial college biology course
was compared by Nord (25) with noncompulsory attendance. He found no
significant difference in achievement between students participating
in the two methods.
Compulsory homework assignments for college mathematics students
was compared to no homework assignments in a study by Hasen (18). He
found no significant difference between group achievement means.
Comparisons of teaching methods involving selfpacing and mastery
learning are included in the following section.
Additional Topics
Selfpacing as well as repeatable testing is a part of the modu
larized course offerings studied in this dissertation. Bass (4) in
his study of engineering graphics taught with selfpacing as compared
to the traditional method could reject no null hypotheses and concluded
that the new method was similar in effectiveness to the traditional
method.
Gallegos (14) investigated pacing and found that forced pacing at
a rate greater than students would choose for themselves was less effec
tive than selfpacing or slow prescribed pacing. He found that self
pacing was particularly beneficial for high ability students. On the
other hand, slow but prescribed pacing was better for low ability stu
dents than selfpacing alone.
Lasco (20) reported that external pacing (selfpacing) caused an
increase of student time devoted to study in a learning laboratory but
did not affect endofcourse group achievement. His experiment used
an individualized instructional system consisting of nine units in
audiotutorial format for the teaching of college level geology. As a
student's characteristic work rate became slower, total unit achieve
ment tended to increase but endofcourse achievement did not. As a
student's characteristic work rate became slower and in addition his
general ability increased, total unit achievement as well as endof
course achievement tended to increase. The above findings suggest
that those engineering students of this dissertation who finished the
modules early and hence seemingly without difficulty and those stu
dents who progressed mainly by persistence will not necessarily be
high achievers on the final examination.
Selfpacing has the inherent disadvantage of permitting procrasti
nation. Some personality types would be expected to be more prone to
15
difficulties caused by procrastination than other students. Gehlausen
(15) analyzed the personality types of beginning engineering students
at TriState College. Despite many similarities between the successful
students and unsuccessful students regarding their backgrounds and
interests, the former as compared to the latter were found to have a
history of academic success, higher aptitudes, higher expectancies,
and better study habits. The high achievers in engineering seem to do
less procrastinating, waste less time, have fewer distractions, and in
general have better study orientation.
A selfpaced instructional system also using an open learning
laboratory and repeatable mastery examinations like the instructional
system of this dissertation was investigated by Naegele (24). The
subject matter was introductory college physics. His findings support
Ausbelian learning theory that the most important factor influencing
learning is the learner's possession of those concepts and skills
which have a clear and direct relationship to the subject matter under
consideration.
Mastery learning is important in the instructional system inves
tigated by this dissertation. It is a strategy that permits variations
of the kind, quality, and duration of instruction so as to fit an indi
vidual's need as measured by frequent evaluations with immediate feed
back.
Mastery learning was compared with the traditional method of
teaching freshman mathematics in a study by Price (28). He found no
significant difference in his experimental study. Student profile did
not matter either. It appeared, however, that as students had time to
adjust to the mastery learning procedure, their performance on achieve
ment tests improved. Price felt that the middle ability student in
particular reflected this trend.
Caponigri (7) reached a different conclusion in an investigation
of mastery learning methods for teaching college statistics. He com
pared two methods of mastery learning with the traditional lecture and
demonstration method. Both methods of mastery learning showed a sig
nificant improvement over the traditional method. Nonetheless, even
though mastery learning had been obtained, the endofcourse exami
nation scores correlated directly with a precourse mathematics aptitude
test. This finding conflicts with the hypothesis that in a mastery
learning system the relationship between aptitude and achievement
should approach zero.
Rowberry (29) studied an adjunctive autoinstruction method for
dental students and found that every student failing a topic of his
course but willing to spend sufficient time could achieve mastery by
means of the adjunct method. Although failers became achievers, some
of the initial achievers failed to retain mastery to the time of a
review test. Apparently some of the initial achievers did not review
because pressure no longer existed. This indicates that retention is
a problem. Of interest to this dissertation is the question suggested
by the above that some seemingly best achieving engineering students
who finish the modules early with few retakes of module tests may not
necessarily perform best on the course finalexamination.
This dissertation combined junior college transfer students with
native university students in the same test population. Wermers (41)
compared the achievement of both types within the upper division stu
dents at the UrbanaChampaign campus of the University of Illinois.
17
He found no difference in general between these two types of students.
Similarly, Philip W. Taylor (34) concluded that transfer students
at East Carolina University experienced relatively the same diffi
culties during their junior and senior years as native students.
The student population used for this dissertation was composed
almost exclusively of engineering students. These students controlled
their own efforts and pace. Achievement was a personal matter and
would obviously be influenced by the personality profile of each indi
vidual. Brown (6) has studied the personality characteristics of engi
neering students who succeed. He reports that the successful engi
neering student sets high goals for himself and is motivated to attain
them. He tends to be orderly and selfsufficient. He relies on per
sonal resources rather than looking to others. The engineering stu
dent is aggressive and satisfies this drive through personal exploit
rather than by engaging in activities which involve group social or
political action.
Personality factors were considered by Kirkpatrick (19) for begin
ning students of electrical and electronics technology students. He
investigated how personality traits and personality types (introversion
versus extroversion) are related to academic achievement in two dif
ferent methods of instruction. One method was an individually paced
type; the other involved lectures, discussion, and demonstrations for
groups of students proceeding at a group pace. A general finding not
statistically significant was that the individually paced method was
best suited for students with introversion tendencies. Students with
extroversion tendencies generally achieved better in the group method.
When personality traits such as Order, Abasement, Change and Endurance
were included with personality types, certain grouping were found to
be statistically significant. This research like others shows that
individual student achievement is multidetermined.
Braun (5) in his study of engineering and engineering technology
students supported the contention that the selfconcept of individuals
is related to their personal behavior and that measurement of this
selfconcept should be useful as an aid to curriculum choice.
Peterson (27) considered the hypothesis that factors other than
those of an intellectual nature contribute to persistence in an under
graduate engineering program. His study concluded that nonpersisters
in engineering tend toward greater independence and nonconformity than
do persisters. This finding suggests that the selfpacing feature of
the instructional system herein would decrease attrition.
A related finding by Augustine (3) was that both persisters and
nonpersisters among academically proficient engineering students are
frequently dissatisfied with highly structured inflexible engineering
curricula.
Student attitude is a matter of interest in applying the findings
of this dissertation. A few additional comments about attitude follow.
A secondary finding of the previously reported research by Eide
(11) was that freshman engineering students preferred the modularized
version of an engineering graphics course and that those students
experiencing it had a lower attrition that did similar students taking
the traditionally taught version.
A contrary finding appeared in the previously reported research
by Venable (38). In his study of certain engineering students, fewer
students in a selfpaced instructional system completed the courses
19
successfully as compared with students in regularly scheduled progression
through the identical subject matter.
The Arizona State Department of Education (2) sponsored research
that reported results of using modularized versions of five vocationally
oriented courses at Eastern Arizona College at Thatcher. Included
were engineering related courses in drafting and in electronics. One
conclusion was that students liked the ability to finish a selfpaced
modularized course early but disliked the lack of instructor pressure
in setting deadlines.
Student attitude is a matter of interest in applying the finding
of this dissertation. In the use of repeatable tests in college chem
istry, the previously reported research by Donovan (10) concluded that
students feel they learned in the process of repeating a test, that
pressure was relieved, and that cheating was reduced. Students did
not mind the extra work if an improved grade seemed to be almost a
certainty.
Student attitude concerning attendance at lectures was reported
by Nord (25), whose research has been previously mentioned. He stated
that students who had attended a noncompulsory attendance version of a
college course recommended use of noncompulsory attendance far more
than did students who had taken the compulsory attendance version.
Harris (17) in his previously reported comparison of two methods
of media supplements to engineering courses found that students pre
ferred to have both options available. One method involved an audio
tutorial approach and the other used printed transcripts.
The previously discussed research by Otten (26) measured attitude
among students proceeding in three versions of an engineering course.
20
No statistically significant difference in attitude attributed to the
method of instruction were shown to occur among the three groups of
students.
Various researchers have selected a multitude of variables that
offer promise of predicting academic achievement. Harding (16) used
multiple regression analysis to examine 38 endogenous (in school) and
exogenous (out of school) variables to determine their ability to pre
dict academic achievement of students at Illinois State University.
High school rank and ACT scores were the preadmission variables that
had predictive value. For students already in college, those who had
greater than average amounts of class cutting or participation in ath
letics and recreation or in television watching tended to have lower
grades. Students who spent greater than the average amount of time
with members of the administration tended to have lower grades, but
students who had more than the usual amount of out of class contact
with faculty members tended to have higher grades. This research
illustrates that student achievement is multidetermined.
Regression equations are used in the analysis of this disser
tation. In the use of regression equations to predict academic suc
cess of Tennessee community college transfer students, McCook (22)
found that a separate regression equation was necessary for the grad
uates of each community college. Similarly, Mouser (23) found pre
dictive capacity was improved when engineering transfer students from
two year and four year colleges were considered separately when using
regression procedure.
Summary
The literature reports many efforts to predict academic success
of individual students. Tests, accumulated gradepoint averages, and
noncognitive factors have been investigated. Various degrees of suc
cess have been obtained in various situations. A previously accumulated
gradepoint average, usually an overall GPA, has often been found to be
a valid predictor and the best predictor of future academic success.
Referring to upper division matriculation the literature suggests
that the differences between transfer and native students are small.
For college level courses little has been published about the effect of
nopenalty repeatable testing in selfpaced modularized courses. Inves
tigations of student attitude suggest that students prefer to have
alternative methods of learning available.
CHAPTER III
THE MODULARIZED INSTRUCTIONAL SYSTEM AND
THE TEST POPULATION
Introduction
This chapter outlines the instructional system which served as a
base for this study. The courses, their flexible packaging, the insti
tutional setting, and the student body are described. This chapter
outlines the general procedures used for selecting appropriate stu
dents to form a suitable base for statistical analysis.
The Modular Instruction System
The College of Engineering of the University of Florida has devel
oped a modularized curriculum for three junior level introductory engi
neering mechanics courses taken by most undergraduate engineering stu
dents. These engineering core courses are Statics (ESM 301), Dynamics
(ESM 302), and Mechanics of Materials (ESM 303). They are referred to
herein as the ESM courses.
The integrated modular instructional system was developed during
the 197273 academic year by Professor Martin A. Eisenberg, Ph.D., of
the Department of Engineering Sciences. Beginning in the 197374
academic year, the modularized system was offered to students in lieu
of the traditionally taught versions of the three courses. The modu
larized versions were the primary instructional mode but scheduled
examinations were available as an alternative to the endofmodule
tests. During its inception and initial use, the module system provided
for selfpacing and unlimited nopenalty retaking of module tests. As
the system evolved, various incentives were used to control the self
pacing feature in order to discourage procrastination. The nopenalty
retake feature remained essentially constant until the spring quarter
of the 197475 academic year. During the proceeding six quarters, the
instructional system for all three courses was in full operation with
unlimited use of the nopenalty retaking of a randomly selected version
of each module test. The data accumulated during these six quarters
were used for this research. The basic design philosophy and outline
of the system prior to April, 1975 is herein called "the instructional
system."
A detailed description of the instructional system was published
in the article "A Modular Instructional System For Introductory Courses
In Engineering Mechanics" by Martin A. Eisenberg, Ph.D., designer of
the system. This article was published in the December 1975 edition
of Engineering Education and is reproduced in part in Appendix A. A
brief description of the instructional system is presented in the fol
lowing paragraphs.
The instructional system during the period of data collection
employed a flexible combination of modular curriculum packaging, unlim
ited retaking of module tests, variable pacing, programmed learning
materials, and computer management of records.
There were 13 degree programs offered by the several departments
of the College of Engineering. The instructional system content was
designed to accommodate varying department objectives. Thus an elec
TABLE 1
MODULES OF THE VERSIONS FOR VARIOUS DEGREE PROGRAMS
Course
Degree version Module number
program 301 302 303 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
Aero.Eng. 10 22 33 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3
Agric.Eng. 10 21 32 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3
Chem.Eng. 10 20 30 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3
Civil.Eng. 10 20 30 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3
C.I.S. 12 24 1 1 1 1 2 1 2 1 1 1 1 2 2 2 2 2 2 1 1 1 2 2 2 2 2
Elec.Eng. 11 20 31 1 1 1 1 1 1 3 1 1 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3
Eng.Sci. 10 22 33 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3
Env.Eng. 12 24 1 1 1 1 2 1 2 1 1 1 1 2 2 2 2 2 2 1 1 1 2 2 2 2 2
Ind.Eng. 10 20 30 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3
Mech.Eng. 10 20 30 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3
Matls.Eng. 10 20 30 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3
Nuclr.Eng. 10 34 1 1 1 1 1 1 1 1 1 3 3 3 3 3 3 3 3 3 3
Syst.Eng. 10 20 30 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3
1 indicates module is part of ESM 301
2 indicates module is part os ESM 302
3 indicates module is part of ESM 303
Versions 10, 21, 22, 23, 31, 32, 33 and 34 are four credit hour courses
Versions 11, 20 and 30 are three quarter hour courses
Versions 12 and 24 are five quarter hour courses
25
trical engineering student and a civil engineering student did not take
the identical modules of ESM 301, Statics, nor did they receive the same
number of credit hours.
Table 1 shows the content of courses required for each of the under
graduate degree programs. As far as the University registrar was con
cerned, there were only three variable credit courses in which the stu
dent could enroll. Within each of the courses, however, students could
be enrolled in one of three to five different subcourses whose existence
was of no concern to the registrar. A study of the content of subcourses
30, 31, 32, 33, and 34 of ESM 303 shows similarity. Subcourse 33 taken
by aerospace engineering students differed from subcourse 30 only in
the addition of shear center and column buckling modules to the curric
ulum. Subcourse 34 taken by nuclear engineering students included
stress and deformations in thickwalled cylinders under thermal and
pressure loading, subcourse 31 taken by electrical engineering students
included damped vibrations of particles and rigid bodies. The differ
ences among most of these subcourses was small.
Individualization by student major caused a variation in the choice
and number of modules to be included in the three basic course packages.
Table 1 previously shown provides the details. A summary of the indi
vidualization by major is provided in Table 2. Versions 10, 20, and 30
were applicable to a greater number of students than were other versions.
Five professors taught the courses. A greater number of student
assistants staffed the learning laboratory and administered the tests.
The professors kept the records of individual student scores made on
the final examination of each course. These scores were the dependent
variable of this investigation. The student assistants recorded the
TABLE 2
NUMBER OF MODULES REQUIRED FOR INDIVIDUALIZATION
BY SUBDISCIPLINE
Course Version Number of Modules
301 10 9
11 8
12 12
302 20 9
21 11
22 11
24 11
303 30 9
31 9
32 11
33 11
34 10
results of individual module tests. Scores of 80% or above were
recorded numerically. Scores of below 80% were recorded as F for fail
ure. The module test number, the result and the date were the three
kinds of information recorded. The student record sheet used is shown
in Appendix B. The recording was in handwriting. Scores of units
passed were transferred to a computer. Record of units attempted but
failed appeared at no other place than on the student record sheet,
which was stored with the module test and work papers in individual stu
dent folders filed in the learning laboratory.
In addition to a standard textbook for the courses, students used
27
a set of programmed study guides. The study guide provided a learning
activity package. For each module there was a description of the con
tent and rationale for study of the prescribed material, a statement
of prerequisites, a list of behavioral objectives, a commentary and
guide to the text, and a sample proficiency test. An example of a
sample proficiency test is shown in Appendix C.
The endofmodule test taken by the student was similar in level
of difficulty to the sample test. During the time period of interest,
there were five to ten separate endofmodule tests for each of the
34 modules. When ready to take an endofmodule test, a student went
to the learning laboratory and was given a test selected randomly from
the several for that module. If the student made below 80%, he failed
and could then take another of the tests when he so desired. Upon
obtaining a grade of 80% of better, the student proceeded to the next
module. Only the final grade was counted in determining the letter
grade for the course. The algorithm for computer calculation of this
letter grade was changed from time to time. The letter grade is not
relevant to this research. All students simultaneously took an endof
course final examination.
The selfpaced feature of the instructional system was designed
to cater to the broad range of input student competencies. As origi
nally implemented, the system employed a significant element of flex
ible pacing strategy. Major traditional examinations were scheduled
during the fourth and eighth weeks of a tenweek quarter. Students
had the option, however, of demonstrating proficiency by passing the
related module tests prior to the dates of these two scheduled exam
inations. Students demonstrating A or B proficiency as indicated by
grades of 80% on the module tests were excused. This feature was to
discourage procrastination. The details of this feature varied some
what during the evolution period of the instructional system.
The learning laboratory staffed by student assistants provided
tutoring help in addition to administering the module tests.
The Test Population
The undergraduate engineering students who comprised the total
population had different types of educational institution backgrounds
prior to beginning the introductory courses of engineering mechanics.
The two main sources of students were the junior colleges and the
University College of the University of Florida. A few students came
from other universities. Thus the three general types of students
within the test population were native students and two types of
transfer students.
University College students begin as freshmen at the University
of Florida. At approximately the end of the sophomore years, students
transfer to the various upper division colleges such as the College of
Engineering. Some of these native students begin the junior level
engineering mechanics sequence of courses during their sophomore year.
Junior college graduates who transfer to the University as engi
neering students begin as juniors. Such students have completed their
general educational requirements and have been awarded the Associate
of Arts degree. Most of these students come from the 28 Florida pub
lic junior and community colleges. Some native and transfer students
do not initially register for the engineering mechanics course sequence
and therefore begin the sequence late.
Students from other universities who transfer to the University
of Florida as engineering students are sometimes well beyond comple
tion of the sophomore year and occasionally have some type of bacca
laurate degree.
The test population consisted of those students selected from
the total population. The total population consisted of all students
who took any of the three modularized engineering mechanics courses
from September 1973 through March 1976, in which unlimited nopenalty
retaking of module tests was permitted. Various types of native and
transfer students participated as described above. The following
paragraphs describe the selection of students who were included in the
data base of this investigation. Uniformity of situation and comple
teness of data were the two general considerations used to select those
students who were included in the test population.
Students who took few other courses concurrent with the modu
larized engineering mechanics courses accumulated too small a number
of credit hours to provide a reliable measure of their level of
achievement. The researcher set 20 usable hours as a minimum for pro
viding a reliable average, defined as the modified junior year grad
point average. Very few students were eliminated from the test popu
lation by this criterion.
Students participated in part or all of the three course engi
neering mechanics sequence during other times than their junior year.
Students who began in the last quarter of their sophomore year or who
completed by the end of the first quarter of their senior year were
considered by the researcher to have been in the standard time
sequence. Their data were combined with that of students who were
juniors at the time they took the courses. A considerable amount of
data would otherwise have been deleted. After review of the tran
script of each student, the researcher deleted from the total popu
lation those students whose matriculation pattern was further out of
time sequence or was otherwise nonstandard.
Further details about choices made in selecting students for
inclusion in the test population are given in Chapter IV.
Summary
Three engineering mechanics courses comprised the instructional
system. The system provided a modularized matriculation path indi
vidualized to each student's engineering subdiscipline. It also pro
vided a nonmodularized path utilizing periodic examinations. The modu
larized versions of the three courses consisted of nine modules for
most students. Unlimited nopenalty retaking of module proficiency
tests was permitted. All students took the same endofcourse final
examination. Several factors influenced selecting a relatively homo
geneous test population to provide a data base for analysis.
CHAPTER IV
METHODOLOGY, ANALYSIS OF DATA, AND FINDINGS
Introduction
This chapter explains the investigative nature of the study, states
the null hypotheses, discusses the data collection decisions, and pre
sents graphical representations of selected data for visual obser
vation. An initial chi square analysis and a revised analysis are then
summarized and reasons are stated for discontinuing use of the chi
square method of analysis. An alternative analysis using linear
regression is then described and applied to 10 ESM 301, 302, and 303
sections, and the findings are stated. The chapter then presents
adjunct findings related to selfpacing and to student attitude. A
summary concludes the chapter.
Nature of Study
This was an associational study which sought to determine if a
measurable association existed between the number of module test retakes
and the final examination score. This was an ex post facto scientific
inquiry examining educational variables in a real life setting. Sys
tematic controls of actions taken were lacking since no control group
was available for comparison. This study therefore sought to determine
an association and did not imply that a cause and effect relationship
necessarily existed. The "effect" or associational strength of a weak
32
variable was sought in the presence of many other variables that affect
student achievement.
The analysis proceeded heuristically, seeking to identify extra
neous variables and control their influence by the choices of data
selection and the designs of statistical analysis.
This was an exploratory field study seeking to detect a possible
association between the variables that would provide usable information
to the faculty administering the innovative instructional system and
that would lay the groundwork for those who may conduct a systematic
test of the primary hypothesis in an experimental setting.
List of Hypotheses
Primary Null Hypothesis
The primary focus of this research related to the retaking of
module completion tests as to whether the amount of retaking was
associated with student achievement.
The primary null hypothesis was
Hg: The number of times that students utilized the
provision for unlimited nopenalty retaking of
module proficiency tests during the ESM 301, 302,
or 303 course was not related to student achievement
.as indicated by score on the endofcourse final
examination when other reference or causal vari
ables were controlled.
Stated simply, this null hypothesis was
HO: The extent of retaking of module completion
tests was not related to student achievement.
Adjunct Null Hypotheses
Selfpacing and student attitude were two adjunct topics con
sidered by this research. Analyses of these topics are presented
briefly at the end of this chapter. The adjunct null hypotheses were
H0: The completion date of student following
the modularized version of ESM 301, 302 or 303
was not related to student achievement.
H0: The attitude of students concerning the
modularization of ESM 301, 302, and 303 was
not related to student achievement.
Rationale for Using "Modified Junior Year GPA"
The dependent variable or criterion was student achievement,
which was multidetermined. The effects of ability and environment had
to be statistically controlled. High school GPA, high school rank,
freshman GPA, cumulative GPA, SAT scores, and the Florida battery of
12th Grade test scores were examples of ability variables that could
have been used to predict a student's performance. Class load, resi
dence, car ownership, recreational and athletic participation, student
associations, family status, and parttime employment were examples of
environmental variables. Personality and work habit variables over
lapped both of the above categories. A control was needed for the
effect of these various ability and environmental variables. A spe
cially calculated gradepoint average was selected as a means of con
trolling for these variables.
Any gradepoint average is to some extent a possible predictor of
a given student's expected performance in some other academic situation.
Chapter II discusses this point. In general, research suggested that
the more current the GPA selected, the better it would serve as a pre
dictor. The plan of this research was, therefore, to use a specially
calculated concurrent GPA. The researcher proposed that a concurrent
GPA was a single measure reflecting the combined effort of the many
ability and environmental determinants summarized early in this section.
These variables affected a student's achievement in the modularized
courses. These same variables affected that student's achievement in
traditionally taught courses taken concurrently. These latter were
taken as a reference that reflected the effects of the many external
variables.
The researcher therefore established the term "modified junior
year GPA." Three quarters were included in order to correspond to the
three quarters in which a student took the three ESM junior level
courses. Many individual students were retained in the data pool even
though they began the series before their junior year, or completed
the series after their junior year, or did not take all three courses.
The researcher therefore had to select three representative quarters
for each student. The selected quarters were those in which the three
ESM courses were taken. For students not taking all three ESM courses,
junior year quarters were selected to provide enough data for a repre
sentative average.
In order not to compare something to itself, the ESM courses were
deleted from the record of the three selected quarters. The modularized
course thermodynamics, ME 360 was also deleted. The remaining courses
were therefore independent of the modularized courses being investigated
but concurrent with them. These selected courses were taken in approx
35
imately each student's junior year. The GPA of these courses was cal
culated for each student and the term "modified junior year gradepoint
average" was appropriate.
Analysis of the Uniformity of the Modules and Units
The several versions of the three courses of the modularized engi
neering mechanics instructional system were described in Chapter III.
Each module had several endofmodule tests.
After several boxes of student learning laboratory folders were
obtained, an analysis was made of the relative difficulty of the 34
modules. A tabulation was made of how many students failed a given
module the first time those students attempted one of its tests. The
results are shown in Appendix D. The number of tabulated initial mod
ule attempts varied because of course and version differences, drop
outs, legibility of student assistant handwriting, and completeness
of records.
The variation shown within the percentage column showed clearly
that the difficulty of the modules was nonuniform. This variation in
itself was not serious for purposes of this research because all com
parisons were made for students taking the same sequence of modules.
The total number of retakes was important, but where the retakes oc
curred was not.
Next an analysis was made of the relative difficulty of the end
ofmodule tests within each module. The number of different tests
available for random selection varied from module to module. A tab
ulation was made of the number of times students failed each of these
tests on the first attempt. The detailed results are shown in Appendix
E. The large variation of difficulty of the endofmodule tests with
in several of the modules constituted a problem for this research.
Students followed randomly occurring unequal paths as they progressed
through the sequence of modules. This condition was recognized and
accepted as a cause of dispersion that could not be statistically con
trolled.
Data Collection and Selection
Despite the researcher's intent to include all students of many
classes in the data base, the number of usable student histories
decreased considerably with the several types of preliminary tabulating
of the several components of the data. Many students were lost from
the test population because they did not follow or did not complete
the modular option. Students were also lost because they took the
courses unusually early or unusually late rather than in their junior
year, because they took an unusual version of a course, or because one
of the components of the data was missing. A discussion of the selec
tion process is given in Appendix F.
The resulting data pool consisted of selected students from 10
usable sections of ESM 301, 302, and 303. There were numbered 1
through 10 and are identified as shown in Table 3. A further pro
cessing of the data was made after the initial chi square analysis
described later in this chapter. Descriptive statistics concerning
the data are presented later in this chapter.
ESM SECTIONS
TABLE 3
USABLE FOR ANALYSIS
Section Designation Date Section Number
1 ESM 301 Fall '73 3743V
2 ESM 301 Fall '73 3744V
3 ESM 301 Winter '74 3548V
4 ESM 301 Spring '74 3411V
5 ESI 301 Fall '74 3762V
6 ESM 301 Fall '74 3765V
7 ESM 302 Summer '74 2136V
8 ESM 302 Fall '74 3767V
9 ESM 303 Spring '74 3414V
10 ESM 303 Fall '74 3768V
Characteristics of the Data
The data distribution was not bellshaped. The final examination
score distribution had an upper limit at 100 and was skewed. The mod
ified junior year GPA distribution has an upper limit at 4.00 and was
skewed. These deviations from the normal distribution were inherent
in the test instruments.
The number of retakes of module tests was a noncontinuous distri
bution. This was necessarily so because this was a count function
rather than a mearurement.
Graphical Presentation of Selected Data
Three types of raw data were collected as described in the pre
vious section. These were a GPA, the final examination score, and the
38
number of retakes of module tests. For purposes of chi square analysis,
the first two were combined into a measure of relative achievement.
Prior to that processing, this section presents selected graphs showing
the general nature of the selected data.
Graphs are presented in Appendix G for the data of Sections 1, 2,
3, and 9. Sections 1, 2, and 9 are the three sections having statis
tical significance in the multiple regression analysis described later
in this chapter. Section 3 is typical of most of the other sections,
which have high dispersion of data and for which no statistically
significant conclusions could be found. These data were graphed to
permit visual inspection of whatever patterns might be discernible in
the selected data. These graphs did not control for the effects of
aptitude and environment upon each student's final examination score.
This is to say that none of the graphs showed relative achievement and
hence none directly related to the primary null hypothesis of this
investigation. The graphs show that considerable dispersion existed
within the data but did not show any clustering worthy of special
investigation.
Summary of an Initial Chi Square Analysis
Three sections of ESM 301 were randomly selected for an initial
large scale evaluation of the method previously described as the
initial plan for chi square analysis. Ninetynine student histories
were used to permit their being divided into three equal categories
for the contingency table. The chi square statistic indicated no
statistical significance. This lack of significance indicated that
the number of retakes was indeed a subtle influence. If it existed, a
more precise method of analysis would be needed to reveal it.
Summary of a Revised Chi Square Analysis
Procedural changes were made before proceeding with a full scale
data analysis using the chi square method. In the initial analysis
the dividing line between students typically fell with a group of stu
dents all having the same number of retakes. Random numbers had been
used to determine how that group of students would be divided. The
use of equal size groups in the contingency table was discontinued to
increase the sensitivity of the method of analysis.
A chi square contingency table must have an expected values of at
least five in every cell. Otherwise the chi square statistic is not
valid because of the threat of instability. The problem of having
each individual cell expected frequency be equal to or greater than
five had originally appeared to be small because of apparent wealth of
large ESM sections available for analysis. The many deletions of stu
dents to provide homogeneity and reduce dispersions substantially
reduced the data pool as was described previously.
The original data tabulation used only Version 10 of ESM 301, a
nine module version. Version 11 was identical except that it did not
include Module 7. Electrical engineering students took Version 11.
They were a substantial group. The researcher retabulated all retake
data, disregarding Module 7 information on the student record sheets.
This adjustment made Version 10 become an eight module package iden
tical with Version 11. The total number of usable students per ESM
301 section was therefore increased.
Several chi square analyses were then made for different combi
nations of the revised data for the six ESM 301 sections. The nine
cell contingency table was compressed into a six cell table by grouping
the number of retakes into the two categories of few retakes and many
retakes. This was to insure that the expected frequency of each cell
would be at least five. Some of these data groupings produced an appar
ent statistical significance. Observation of the data points within the
contingency table led the researcher to conclude that the data pro
cessing method probably introduced a bias that contributed to the
apparent significance. The method of controlling for aptitude and
environment was to subtract a student's expected score from his actual
score after standardizing both types of data. The expected score was
determined from the modified junior year gradepoint average. This
process appeared to overcompensate and introduce a bias. For example
the researcher found that a high GPA student was almost precluded from
being categorized as achieving better than expected.
Because of threats to validity that appeared to have been intro
duced in preparing the raw data for chi square analysis and because
of the lack of sensitivity to minor distractions between observed data
points, the researcher abandoned further use of contingency tables and
chi square analysis.
Linear regression analysis was the method chosen for a reanalysis
of the data.
Multiple Linear Regression Analysis for All Ten Sections
The variables associated in this investigation were (1) GPA,
(2) number of retakes, and (3) score on the final examination. A high
41
GPA student would reasonably be expected to do well on the final exam
ination. A student requiring few module retake tests would seemingly
be having little difficulty and on this basis would be expected to do
well on the final examination. Stated conversely, a student requiring
many retakes would appear to be having difficulty in mastering the
subject matter and would reasonably be expected to have a resulting
low score on the final examination. These two expectations were com
bined into a single mathematical equation as follows:
Score = B0 + B1 x GPA B2 x Retakes
This typical linear probabilistic model was appropriate as a deter
ministic model for representing the best fit line through the scatter
diagram of the observed data. The method of least squares was used to
find this line of best fit to the empirical data. B1 and B2 were the
regression coefficients.
The above model was applicable only to students who chose to follow
the modular option to completion. Students who discontinued the modu
lar path could have taken the intracourse examinations or they could
have become dropouts. Students not completing the modular option were
a sizable group. Research concerning them was conducted by Dr. Eisenberg
and was described in his article which appears herein as Appendix A.
The data for each of the 10 ESM sections was transmitted to the
Northeast Regional Data Center located at the University of Florida at
Gainesville for processing by the IBM 370 Model 165 computer. A
remote terminal at the University of North Florida in Jacksonville was
used for data transmission. The procedures used were part of the Sta
tistical Analysis System (SAS) designed by Anthony James Barr and
James Howard Goodnight at the Department of Statistics at North Carolina
State University, Raleigh.
A confidence level of 95% was set as the criterion for judging
whether statistical significance existed. Stated in other terms, the
critical value chosen was at the .05 (5%) level of significance.
The appropriate Statistical Analysis System procedure was used to
determine the equation of the best fit line and analyze the dispersion
of the data points about this line to provide a numerical measure of
the statistical significance of the calculated B coefficients of the
multiple regression equations. Analysis of variance was the statistical
method used. Overall, if significance existed, at least one of the B1
and B2 coefficients was meaningful and further mention of these quan
tities was warranted.
A summary showing overall probability for the 10 sections is
shown in Table 4, an overall analysis of variance summary.
The level of significance, which was the probability of chance
occurrence, is shown at the right in the column labeled P for proba
bility. This was the probability of the occurrence being greater than
the F statistic calculated by analysis of variance. A probability of
.05 or less was significant.
The probability column shows values much larger and much smaller
than .05, the 5% value for adjudging significance. To seek an expla
nation the researcher prepared Table 5, summarizing pertinent infor
mation.
Table 5 shows the circumstances and the data that pertained to the
10 sections. The researcher looked for any clearcut explanation at the
great variation of probability value for the 10 sections. Variation in
difficulty of the final examination of each section was not an expla
nation. This uncontrolled random external variable caused the numerical
TABLE 4
ANALYSIS OF VARIANCE SUMMARY
Section Source of Var.
1 Regression
Error
Corrected Total
2 Regression
Error
Corrected Total
3 Regression
Error
Corrected Total
4 Regression
Error
Corrected Total
5 Regression
Error
Corrected Total
6 Regression
Error
Corrected Total
Sum of Squares
4760.4
6725.8
11486.3
4272.8
5657.3
9930.2
511.2
4672.7
5184.0
402.0
1703.1
2105.2
310.8
6161.1
6472.0
962.5
4649.2
5611.8
Mean Square
2380.2
203.8
2136.4
166.3
255.6
212.3
201.0
48.6
155.4
133.9
481.2
221.3
F Value
11.6
12.8
1.2
p
0.0003
0.0002
0.3195
4.1 0.0239
1.1 0.3227
2.1 0.1371
__
TABLE 4 continued
Section Source of Var.
7 Regression
Error
Corrected Total
8 Regression
Error
Corrected Total
9 Regression
Error
Corrected Total
10 Regression
Error
Corrected Total
Sum of Squares
2996.5
930.3
3926.8
371.9
3598.1
3970.1
1966.5
573.9
2540.4
2777.7
5141.1
7918.9
Mean Square
1498.2
155.0
185.9
133.2
983.2
40.9
1388.8
214.2
F Value
9.6
P
0.0139
1.3 0.2642
23.9
0.0001
6.4 0.0058
Source of Var. = Source of variation within the data
DF = Degrees of Freedom
P = Probability of chance occurrence. P = .05 or less indicates significance


TABLE 5
CONCOMITANT DATA FOR THE ANALYSIS OF VARIANCE SUMMARY
Section ESM Quarter N GPA Score Retakes P
Number Designation 1' o a p
1 301 Fall '73 36 2.96 .68 62.9 18.1 3.58 3.13 .0003*
2 301 Fall '73 27 3.09 .59 68.2 16.6 3.62 3.32 .0002*
3 301 Winter '74 25 3.27 .54 69.2 14.7 4.20 3.62 .3195
4 301 Spring '74 38 2.94 .53 86.6 7.54 3.58 2.60 .0239*
5 301 Fall '74 49 3.22 .57 68.6 11.6 2.35 1.94 .3227
6 301 Fall '74 24 3.17 .62 66.1 15.6 3.08 2.02 .1371
7 302 Summer '74 9 3.30 .48 43.1 22.2 2.89 2.03 .0139*
8 302 Fall '74 30 3.06 .52 52.8 11.7 4.47 2.71 .2642
9 303 Spring '74 17 3.14 .58 56.8 12.6 2.35 2.12 .0001*
10 303 Fall '74 27 3.00 .63 48.0 17.4 1.70 1.66 .0058*
N = Number of students in selected data
P = Probability from analysis of variance summary
= Significance
46
value of the regression coefficients to vary from section to section but
did not effect their reliability. Three of the 10 sections were taught
by Dr. Eisenberg, the designer of the modularized instructional system.
These were Sections 1, 9, and 10. An interesting observation is that
all three of these sections showed significance. This suggested that
closer control of these sections may have existed with the result that
the influences of external variables was reduced. On the other hand,
sections taught by the other two professors involved showed cases of
both high and low value for the same professor. This suggests that
some external variable other than the professor introduced dispersion
into the data. Correlation between two independent variables increases
the likelihood that a more basic determinant can affect both variables.
Erratic results from separate investigations can result from this cause.
(That correlation existed was obvious from the graphs of Appendix G.
Quantatively the correlation coefficients between GPA and Retakes for
the 10 sections were as follows: (1) .32; (2) .50; (3) .27; (4) .09;
(5) .34; (6) .39; (7) .08; (8) +.04; (9) .29; (10) .36.)
These sections having significance were selected for further exam
ination. Section 10 was included as essentially being significant with
its 5.8% value of probability. Six sections were thus selected. Table
6 shows the regression coefficients and their associated P values for
these six sections. In all six cases it was found that GPA was a sig
nificant predictor of the final examination score. The number of
retakes was not always significant at the 95% level. Table 6 was
therefore arranged into the two sections shown.
As shown in Table 6B, both GPA and Retakes were significant pre
dictors of final examination scores in Sections 1, 2, and 9. Unexpect
TABLE 6
MULTIPLE REGRESSION ANALYSIS RESULTS FOR SECTIONS SHOWING
OVERALL STATISTICAL SIGNIFICANCE
A. Summary data for sections in which only GPA was
individually significant
Section Source of Var. Regression Coef. P
4 Intercept 68.806 .0001
Retakes 0.110 .8050
GPA 6.178 .0076
7 Intercept 57.763 .1195
Retakes 4.360 .0923
GPA 34.420 .0061
10 Intercept 6.415 .6969
Retakes 1.438 .4448
GPA 14.684 .0061
B. Data for sections in which GPA and Retakes were
both individually significant
Section Sourve of Var. Regression Coef. Prob. Partial SS
1 Intercept
Retakes
GPA
Regression equation
2 Intercept
Retakes
GPA
Regression equation
9 Intercept
Retakes
GPA
Regression equation
30.438 .021
1.679 .047 866.38
13.000 .001 2464.59
is Score = 30.438+13.000(GPA)1.679(Retakes)
38.495 .013
1.722 .028 882.282
11.640 .009 1267.109
is Score = 38.495+11.640(GPA1.722(Retakes)
10.339 .313
1.963 .026 254.251
19.910 .0001 1954.818
is Score = 10.339+19.910(GPA)+1.963(Retakes)
A probability of .05 or less is significant
Partial SS = Partial sum of squares and indicates the amount of
variation attributed to one independent variable
when the effect of the other is controlled.
edly for Section 9 the retake coefficient was positive. This positive
coefficient indicated that students who had trouble in passing module
tests made higher than expected final examination scores. This finding
implied that students retaking the tests accomplished much learning and
surpassed the students initially appearing to be superior.
The Section 9 data were plotted to permit further investigation of
this unexpected result and are shown in Figure 10 of Appendix G. In
spection showed one apparent wild point representing the moderately
high raw score grade of 62 for a student having the markedly high num
ber of seven retakes. This one point in a leveraged position for a
small population appeared to be sufficient to cause the upward slope of
the data pattern and the corresponding positive value of the regression
coefficient. Removal of this one point would seemingly leave a data
pattern having a downward slope with increasing values of the retake
variable.
The wild point (Retake = 7, Score = 62) was removed from the data
base and the Section 9 data were reprocessed. The Retakes regression
coefficient (with GPA controlled) not only remained positive but decreased
only a trivial amount from 1.962 to 1.955. This indicated that when GPA
was controlled, the apparent wild point was "right on." The point was
very consistent with the regression equation when calculated from the
main body of data. The seemingly obvious explanation had failed to
explain the unexpected results for Section 9.
The researcher's next attempt to discredit or support the unex
pected findings for Section 9 involved use of additional calculations
provided by the SAS regression analysis.
Table 6B also shows the partial sum of squares, which was variation
attributable to one independent variable when the effect of the other
had been removed. Table 6B therefore indicates the relative effect of
the two independent variables. Specifically it shows how the concurrent
GPA predominated over Retakes in predicting final examination scores.
The proceeding does not refer to the size of the regression coefficients,
which were measured by different scales.
The appropriate SAS procedure was also used to perform regression
analysis for predicting scores from GPA alone and from the number of
retakes alone. These two analyses were performed for each of the 10
sections. Table 7 shows the probability that these regression analysis
outcomes could have occurred by chance.
TABLE 7
ANALYSIS OF VARIANCE RESULTS FOR REGRESSION ANALYSIS
PERFORMED FOR GPA ALONE AND RETAKES ALONE
Section No. Probability for Chance Occurrence
Retakes GPA
1 .0063* .0002*
2 .0004* .0001*
3 .4185 .1442
4 .6521 .0063*
5 .1487 .2911
6 .1328 .0753
7 .2110 .0136*
8 .8673 .1006
9 .7954 .0001*
10 .0973 .0016*
*Significant
Table 7 shows that except for Section 5 the probability for GPA
was much smaller than that for retakes. These smaller values for GPA
indicated that concurrent GPA was a more reliable predictor than was the
50
the number of retakes. This statement does not refer to the numerical
size of either regression coefficient. The size of the trend (the
slope of the regression line) was meaningless if there was little con
fidence in its existence because it likely occurred by chance. The
probability figure related to this confidence. The smaller the prob
ability figure, the less likely that the regression line, whether steep
or shallow, occurred by chance. It is worthy of note that only the six
sections showing significance of GPA used alone showed significance for
GPA and retakes used together.
The proceeding has shown that outcomes having statistical signi
ficance were found, that other outcomes were far from significant, that
correlation between the independent variables and varying environmental
conditions provides a possible explanation as to why these large differ
ences might have occurred among the sections, and that concurrent GPA
was a more reliable estimator (is more closely associated with) final
examination scores than was the number of retakes. For Sections 1 and 2
the final examination score appeared to decrease with increasing retakes.
For Section 9, the opposite appeared to be true.
At this point it could have reasonably been argued that the "effect"
of Retakes, if any, was so small that it was usually lost in the
residuals (the clutter remaining after the effect of GPA was removed)
and that occasional indications of significance which showed conflicting
results were random occurrences within the clutter. This reasonsing
would support a conclusion that any "effect" of the number of retakes
was too small to be detected by the methods of this investigation and
that the primary null hypothesis that no association existed could not
be rejected.
The researcher's final attempt to discredit or support the unex
pected finding for Section 9 involved still another effort to identify
an external causal variable. The researcher again reviewed the
learning laboratory folders and discovered a satisfactory explanation
for the contrasting findings of Sections 1 and 2 as compared to Sec
tion 9.
Sections 1 and 2 involved beginning engineering students who were
within the instructional system in Fall 1973. Section 9 involved stu
dents approximately beginning their senior year and who were within the
instructional system in Spring 1974. More mature students were involved,
and some of the instructional system procedures had changed. Unlimited
retaking of module tests was still permitted but delays between tests
were encouraged by procedural changes that had been introduced. The
results was clearcut. The researcher found that for Sections 1 and 2
students typically took two, three or even four tests of a module on
the same day. The particular unit finally passed was often one of those
shown by Appendix E to be unduly easy. The students had manifestly
abused the system and eventually passed a module without properly
learning the subject matter. On the other hand the researcher found
that students in Section 9 predominantly had at least one day between
each retake.
The dissimilarity of the two situation was pronounced. A rea
sonable explanation for the statistical results had been found and the
finding for Section 1, 2, and 9 can be supported rather than be attrib
uted to random occurrences within the clutter that remained after the
predominant effect of GPA had been removed.
Effect of Date of Completion of the Modular Path
As an adjunct study, the "effect" of the rate at which students
progressed through the modularized version was sought. As an overall
indicator of whether a student raced through to completion or whether
he procrastinated, the researcher extracted from the student record
sheet the date of completing the final module. Two sections were ran
domly selected. The date of the month was used as the numerical mea
surement. This constituted a third independent variable in addition
to GPA and Retakes and was called Date. Table 8 shows the results of
a multiple linear regression analysis. The high value of P associated
with Date indicated high likelihood that the regression coefficient
for date of completion occurred by chance. This limited investigation
therefore did not indicate that date of completion had a reliable
association with final examination scores.
TABLE 8
MULTIPLE LINEAR REGRESSION ANALYSIS
WITH DATE OF COMPLETION OF MODULES INCLUDED
Section Overall Date
P P
1 .0003* .7391
7 .0767 .3745
p = Probability of chance occurrence
* = Significant
Student Attitude
As asecond adjunct study, the association between student attitude
and final examination score was sought. Student attitude referred to
the feeling of students toward the modularizing of the engineering
mechanics courses and their taking of the courses in this form.
The researcher sought an attitudinal measuring instrument that
could be used at the time of the final examination. A quick response
instrument was needed in order to obtain the cooperation of professors
and students. Originally planned to suffice only to divide students
into broad categories for chi square analysis, a multiple choice ques
tionnaire was prepared.
To validate this instrument, a trial run was conducted using a
group of electrical engineering students who had already experienced
the ESM modularized sequence. Many students checked the most favorable
of the three choices offered for several of the questions. An inade
quate spread of results occurred. The researcher then prepared the
revised questionnaire of Appendix H, which had two choices of favorable
response such as "strongly prefer" and "prefer." This questionnaire
was presented to students in December 1974 during their final examination
period. The three sections of Fall ESM sections were involved. They
were Sections 5, 6, and 10. Table 9 shows the result of a multiple
linear regression analysis with attitude used as a third independent
variable in addition to GPA and Retakes. The high value of P associated
with attitude indicated that the regression coefficient for attitude
occurred by chance. This limited investigation did not indicate that
student attitude concerning the modularized ESM courses was reliably
associated with final examination scores.
TABLE 9
MULTIPLE LINEAR REGRESSION ANALYSIS
WITH ATTITUDE INCLUDED
Section Overall Attitude
P P
5 .1025 .1948
6 .2606 .5999
10 .0594* .8398
P = Probability
* = Significant (borderline case)
Summary
This investigation was an associational study seeking to relate
student utilization of nopenalty retaking of module tests with final
examination scores when other variables were controlled. This ex post
facto investigation proceeded heuristically, seeking paths of inquiry
that offered promise.
Careful selection of data was used as a major statistical control.
Much effort was devoted to establishing a large data base in which
elements had been screened for uniformity of situation so as to reduce
dispersion. Ten sections of engineering mechanics classes were usable.
Much effort was devoted to establishing broad categories for chi square
analysis. Modifications to the initial procedures were necessary.
Much data processing was performed to produce a numerical measure of
a student's achievement relative to his expected achievement. A modified
junior year gradepoint average was used as a reference for establishing
this relative achievement. The lack of a validated weighting factor
for applying this reference was the final reason found for questioning
the use of chi square analysis for the data of this investigation. A
finding of significance was disregarded because of threats to its
validity.
The investigation then proceeded by changing to regression anal
ysis. Final examination score was the dependent variable to be pre
dicted by concurrent gradepoint average and number of retakes as two
independent variables. Significance occurred in several cases. GPA
was more associated with final examination score than was the number of
retakes. Regression equations were stated for the three sections found
to have significance for both GPA and Retakes. An observation of much
apparent relevance was that students taking more than one proficiency
test of a module on the same day showed a decreasing final examination
score with increasing total number of retakes but that students not
retaking on the same day of an intiial failure showed an increasing
final examination score with increasing total number of retakes. Adjunct
studies concerned whether final examination score was associated with
the date of student completion of the modules or with student attitude
about the modularized courses. Unsophisticated measuring instruments
were used. No significant association with final examination score
was found for either adjunct variable.
CHAPTER V
SUMMARY
Introduction
This chapter presents a summary of the preceding four chapters.
The topics have been rearranged and combined. Details of unproductive
methods of inquiry have been minimized.
The Instructional System
Three introductory junior level courses of engineering mechanics
at the University of Florida were offered in an innovative modularized
format. The courses were Statics (ESM 301), Dynamics (ESM 302), and
Mechanics of Materials (ESM 303). Various options were available.
One feature was the taking of endofmodule proficiency tests selected
at random with provision for retesting of students until a grade of A
or B was obtained. Initial failure carried no weight in determining
a student's letter grade for the course. This feature was called the
unlimited nopenalty retaking of module tests.
Students could initially or later choose the option of taking
two scheduled intracourse examinations in lieu of completing the mod
ule tests. All students took the same final examination.
Individualization of the modular path was provided for students
of the several engineering specialties. This was accomplished by
having somewhat different versions of the three courses. The
57
prearranged versions consisted of slightly different grouping of the
total of 34 modules of the three course sequence. The different ver
sions did not always have the same number of modules.
Student assistants provided tutoring in a learning laboratory
and administered the module proficiency testing feature of the instruc
tional system.
Professors conducted regularly scheduled lectures for those stu
dents wishing to attend.
All students participated in this instructional system although
they did not have to follow the modularized path. There were no sep
arate traditional taught sections that could serve as control groups
for investigative comparisons.
Need for the Study
All students do not learn best in a given type of learning envi
ronment. A trend in American education has been to provide less rigid
course structuring. When an innovative system with options is intro
duced, a relevant question concerns how well the student population
masters the subject matter. This question concerns group attainment.
A control group would be needed to answer this question. Separate
questions could be asked concerning how various ability or personality
types achieve. Subgroup comparisons could be used to answer this ques
tion. When the modularized instructional system of this investigation
was implemented in the real life circumstances of an engineering col
lege, there was neither control groups nor knowledge of personality
parameters of individual students.
A remaining question concerned how student use of the unlimited
58
nopenalty retaking of module proficienty tests was related to student
performance on the final examination. The extent of retaking and also
the rate of progress along the modular path were observable character
istics. They could be of assistance to the faculty provided that
reliable meanings could be associated with these two characteristics.
A major question was whether student reliance upon the unlimited re
taking feature was associated with student achievement as measured by
the score on the final examination. (A meaningful answer required
that the effect of individual student ability and unique environmental
situation be eliminated.) Another question was whether a student's
rate of progress was associated with his achievement. Still another
question was whether an individual student's attitude toward the non
traditional instructional system was associated with his achievement.
Nature of the Study
Concerning the above questions, this investigation predominately
treated the one concerning the retaking of module tests. The primary
null hypothesis stated that the extent of student use of the unlimited
nopenalty retaking of module tests was not associated with student
score on the final examination.
This was an ex post facto field study. It examined the recorded
records of a functioning college of engineering in which experimental
controls designed to facilitate this investigation were lacking.
Appropriate selection of the data that was included was the method
used to establish a somewhat uniform test population in which the
effect of extraneous variables was reduced.
This was an associational study. Because no randomly assigned
control group was available, this investigation sought associations
rather than causal relationships between the variables.
Student performance on the final examination of a course is multi
determined. To provide a meaningful finding about the retaking vari
able alone, the strong effects of individual student ability and unique
environment needed to be eliminated. The methods of data reduction and
statistical analyses were used to accomplish this. One of the tech
niques used was to calculate a gradepoint average of selected courses
taken at the same period of time that a student participated in the
modularized sequence of engineering mechanics courses. It was assumed
that the many aspects of ability and environment were compared to this
GPA. The raw score on the final examination was compared to this GPA
in a way appropriate to provide a measure of relative achievement,
achievement relative to the expected or par performance of a student.
With other variables thus partially controlled, the existence of an
association between performance and extent of retaking of module tests
was then sought.
Review of the Literature
Modularized courses at the college level and provisions for no
penalty retaking of tests are seldom reported subjects. A computer
search of ERIC and of dissertation abstracts found few articles.
Manual searching was therefore used.
Examples of the extensive research in the use of gradepoint
averages to predict academic performance were reported. These examples
related to the use in this investigation of a GPA to estimate the
expected performance of individual students, which was then compared
to observed performance. The literature indicates that GPA was often
found to be one of the best predictors. A recent GPA seemed to be
better than an older one.
Research finding concerning selfpacing and comparison of teaching
methods were included in the review. The gist of these articles
revealed that student ability was the primary determinant of student
performance. The teaching method used was less important.
Uniformity of Difficulty of the Module Tests
Analysis of student records showed that some modules were harder
to pass than others. This condition did not necessarily mean that
individual students followed unequal paths. More serious for the pur
poses of this investigation was the finding that in some modules the
difficulty of different proficiency tests varied. The random selection
of unequal tests therefore created unequal paths for the students to
follow. This caused a dispersion within the observed data for which
there was no statistical control.
Data Collection and Selection
Data was collected from three sources. The special gradepoint
average was calculated from the Registrar's records. The number of
retakes was determined from student folders from the learning labora
tory. The final examination score was determined from the professors.
The data base used for statistical analysis consisted of carefully
selected histories of students taking and completing the same modular
path. The options and individualization that had existed caused the
records of many students to be unusable. Omissions of any part of
the data caused additional records to be unusable. The final data
base consisted of selected data from 10 sections that were taught
during approximately a twoyear period.
Results of Chi Square Analysis
The independent variable for the chi square analysis was the
number of times students retook module tests. This was count data
and was used to establish categories representing the extent of stu
dent utilization of the retake provision.
The dependent variable was the final examination score and was a
continuous variable. It was used to establish categories of achieve
ment. This measure of a student's achievement controlled for ability
and environment was the difference between his standardized final
examination score and his standardized modified junior year GPA.
These standardized numbers were standard deviations measured from the
mean translated to zero. Subtracting these quantities produced a
number assumed to be indicative of how well a student did on his final
examination as compared to his expected achievement. These differences
were placed in rank order and were then divided into suitable categories.
A contingency table and chi square test was used to compare the
observed and expected frequencies. The chi square analysis tests for
statistical significance of the divergence of observed data from
expected data.
The grouping of the data of this investigation into categories
required the arbitrarily establishing of boundaries within the data
distribution. Several groupings were tested.
An analysis of the data of the six ESM 301 sections showed greater
dissimilarities than had been expected. The groupings of data to pro
duce sufficiently large numbers was found to be less feasible than
anticipated.
One grouping involving compatible data from two sections produced
significance. Scrutiny of the steps of the data processing and of the
certain cell frequencies of the resulting contingency table suggested
that a threat to validity existed. The method of relating concurrent
GPA to raw score on the final examination to produce a quantitative
measure of a student's performance relative to his expected performance
appeared to overcorrect for GPA.
The several problems that had arisen in processing the raw data
into a form for use in the contingency table cast doubt on the validity
of whatever significance the chi square statistic might show. The
method of analysis using the chi square statistic was abandoned.
Results of Linear Regression Analysis
Ten ESM sections yielded usable data. This data was subjected to
linear regression analysis. The 95% confidence level was established
as the criterion for adjudging that statistical significance existed.
The dependent variable was raw score on the final examination.
The modified junior year gradepoint average represented a con
current GPA in other courses taken at the same time that the ESM
courses were taken. The concurrent GPA was the GPA used for all anal
yses. It was considered to represent the effects of both ability and
environment. The number of retakes referred to how many times a stu
dent attempted to pass a module proficiency test after failing an
initial attempt. Retakes was the other independent variable of interest.
Used alone as the single independent variable, GPA predicted final
examination score at a statistically significant level in six of the
10 sections. Used alone, retakes significantly predicted (was asso
ciated with) score only a single time. Thus it was shown that grade
point average was more reliable than retakes in predicting score.
The null hypothesis considered the effect of retakes when the
effect of GPA was controlled. A multiple regression analysis for the
10 sections was therefore performed using both GPA and retakes as inde
pendent variables predicting score.
Overall significance was obtained for the same six sections for
which GPA alone produced significance. Including retakes on a second
variable to account for part of the observed variation did not increase
the number of sections showing overall significance. This is a further
indication that gradepoint average is the dominant variable of the two.
The regression coefficients for GPA and score were calculated,
but the retake coefficient was not statistically significant for three
of the six sections showing overall significance. GPA alone had
accounted for enough of the variation to cause overall significance.
Thus, only three sections were found to have full statistical signifi
cance. Conclusions were then drawn for the analysis results of these
three sections.
For these three sections, the portion of the variation attributed
to GPA was much greater than that attributed to retakes. This is a
specific indication that GPA is the more powerful variable in pre
dicting the score on the final examination.
The existence of statistically significant nonzero regression
coefficients was the issue of vital interest to the hypothesis of this
investigation. The numerical values of these coefficients are
64
functions of the three scales used to measure the three variables (e.g.
Score was measured from 0 to 100) and are deliberately not repeated in
this summary. The details appear in Table 6 of Chapter IV.
The important finding concerns the nature of the regression coef
ficient for the retake variable in the three sections where it had
been isolated from the random clutter (had been found to have statistical
significance).
Two of the sections were ESM 301. The regression coefficient for
retake was negative indicating that an increasing number of retakes
was associated with a decreasing score on the final examination. The
remaining section was a small ESM 303 section. A positive coefficient
was found indicating that an increasing number of retakes was asso
ciated with an increasing score on the final examination. It was not
possible to attribute this unexpected finding to the effect of a "wild
point" in the data or to any other cause that would discredit the
finding.
Students of the two ESM 301 sections showed a strong tendency to
retake tests repeatedly on the same day that an initial attempt showed
that mastery of the subject matter had not been obtained. This was
not so for the ESM 303 section. This was the major environmental con
dition found that offers an explanation for the different character
istics of the regression coefficients for the retake variable.
Instances of statistical significance were found for an associ
ation between the number of retakes and the score on the final exami
nation. The lack of uniformity of the findings suggest that only
qualified conclusions should be drawn, although an explanation for the
lack of uniformity appeared to have been found. A more consistent
65
finding was that concurrent GPA is much more associated with final
examination score than is the extent of retaking of module tests.
CHAPTER VI
CONCLUSIONS AND RECOMMENDATIONS
Introduction
This chapter presents the researcher's conclusions, recommendations
for the faculty administering the instructional system studied, and
recommendations for further research.
Conclusions
The innovative instructional system investigated by this explor
atory field study was designed to provide options and alternatives to
the students. It was not designed for research purposes. Many causes
of dispersion could be only partially controlled.
A modified junior year gradepoint average was calculated for
traditionally taught courses taken concurrently with courses of the
modularized instructional system. This concurrent GPA was felt to be
a composite representation of individual student innate ability, per
sonality factors, and whatever external environmental conditions that
might have influenced individual students. Use of this concurrent
GPA as a predictor of student achievement was believed to control for
many of these basic influences.
The variation of difficulty from module to module was not felt to
cause a problem to this research. The variation of difficulty of pro
ficiency tests within some of the modules may have been a cause of
67
dispersion that could not be statistically controlled. This variation
is believed to have been a contributing cause of the negative regres
sion coefficient found for Sections 1 and 2 (ESM 301) for the retakes
variable. An easy test within a module allowed a student taking
repeat tests to have greater chance of receiving an easy test and
demonstrating an apparent proficiency that did not in fact exist.
The immediate retaking of a module test after an initial failure
did not occur in Section 9 (ESM 303). A further fact pertained. The
subject matter of ESM 301 was relatively easy. Students requiring
many retakes were likely to have been marginal students. Repeating
may not have produced genuine comprehension. The subject matter of
ESM 303 was difficult. The more mature students taking this course
may have profited from restudying and retaking the module tests.
Taking additional tests with restudying would constitute a greater
effort on the part of a student and probably provided increased
learning. The researcher believes that this was the cause contributing
to the positive regression coefficient for Section 9.
In summary, the extent of retaking of module tests was shown in
some statistically significant cases to have an association with stu
dent scores on the final examination. This association was less
reliably shown and was of lesser magnitude than that of concurrent
GPA. The nature of this association rather clearly appeared to depend
upon the conditions. An immediate retaking of a proficiency test after
an initial failure probably did little to enhance comprehension and was
likely to lead to an erroneous appearance of mastery. Restudying prior
to retesting probably caused increased learning and comprehension.
The researcher concludes that while aptitude and environmental
factors were major determinants of academic achievement, the extent
of retaking of endofmodule tests in the modularized courses was a
second order effect that was occasionally detected by the investigation.
The researcher interprets the findings to conclude that there was an
association between the extent of retaking of modules tests and final
examination score. The researcher concludes that the extent of
retaking per se was not the entity that matters. The conditions under
which the retaking occurred influenced the outcome.
The researcher does not feel that the negative regression coeffi
cient for Sections 1 and 2 conflicts with mastery learning theory.
Because of the several special features of the instructional
system investigated, the researcher suggests caution in generalizing
all observations and findings to other instructional systems. The
conclusion that the manner of retaking rather than the amount of
retaking is the entity that matters is felt to be a worthwhile con
clusion of general applicability.
Recommendations
The researcher recommends that the endofmodule proficiency tests
be periodically monitored for uniformity of difficulty. This recom
mendation is especially applicable if individual tests are replaced
from time to time to guard against the possibility of compromise of
the testing system. Concerning cheating, the researcher in reviewing
comments written in a space provided on the attitudinal questionnaire
did not find indications that the existence of cheating was a major
concern of the students.
There appears to be a discernible association between the extent
69
of module test retaking and the score on the course final examination.
The conclusion that the effect of retaking depends upon the conditions
is offered as a hypothesis to be investigated under controlled con
dition.
APPENDIX A
EXCERPTS FROM "A MODULAR INSTRUCTIONAL SYSTEM FOR INTRODUCTORY
COURSES IN ENGINEERING MECHANICS"
APPENDIX A
EXCERPTS FROM
A MODULAR INSTRUCTIONAL SYSTEM FOR INTRODUCTORY
COURSES IN ENGINEERING MECHANICS*
In designing an instructional system for the introductory courses
in engineering, one must contend with a number of problems associated
with the heterogeneity of the backgrounds, interests and abilities of
the students served. Typically, a single department is assigned the
responsibility of teaching a course which must meet the curricular
demands of perhaps a dozen distinct degree programs. In large state
universities it is not uncommon to find that students have completed
their prerequisite studies at many different community colleges as well
as at the university in which the course is offered. Consequently, the
input competency of the students is highly variable. "Open door" and
"affirmative action" policies tend to contribute to this variability.
If the introductory courses are to be anything more than a passive
sorting or classification system, then special attention must be given
to the problems they present. The introductory mechanics courses in
statics, dynamics and mechanics of materials are archetypical of ser
*The text of this article is reproduced herein without its acknowl
edgements, references, and footnotes. Only Tables 4, 5, and 6 are
reproduced herein. The full article by Martin A. Eisenberg, University
of Florida, Gainesville, appeared in the December 1975 issue of Engi
neering Education and is used herein with permission of the publisher.
vice courses which must meet the demands mentioned above.
At the University of Florida, an instructional system employing a
highly flexible combination of modular curriculum packaging, variable
pacing, programmed learning materials, computer managed instruction
and conventional lecture and examination methodologies has evolved.
In an earlier paper exploratory studies, the basic design philosophy,
and an outline of the system planned for the 197374 academic year
were described. In this paper the system as implemented and its evo
lution in response to operational experience and increasingly severe
budgetary constraints are described. The effect of alternate staffing
strategies (graduate vs. undergraduate teaching assistants) upon cost
and effectiveness is analyzed, as is the overall cost of operation in
comparison with conventional systems. The results of preliminary
studies on the effectiveness of the prescribed learning materials and
activities are also presented.
The Basic System
The system of instruction consists of a sequence of courses in
the subjects of Statics (ESM 301), Dynamics (ESM 302) and Mechanics
of Materials (ESM 303) which are completed during the junior year by
most students in the College of Engineering.
In the design of the system the following criteria were adopted:
Course Criteria
1) The system must accommodate variable student
and departmental content objectives.
2) Variable input competencies must be recognized
and accommodated.
3) Continuity of program effectiveness should be
assured.
4) Variable faculty teaching styles must be
respected.
5) Provision for continuous improvement and
modification must be possible.
6) Instructional costs must be kept at eco
nomical levels.
7) Sanctions sufficient to ensure reasonable
student progress should be imposed.
8) Objective measures of student performance
and program effectiveness should be provided.
9) A variety of learning modes should be
available to suit the student and subject
matter.
In response to the first criterion it became apparent that the sub
ject matter of these disciplines would have to be broken down into mod
ules, and a prerequisite hierarchy established. Figure 1 illustrates
a decomposition of the subject into the 34 modules upon which the sys
tem is based. The prerequisite hierarchy is indicated by the arrows.
For example, the study of unit 16 on Plane Motion Dynamics requires
direct prerequisites of units, 4, 5, 8, 12 and 15, which in turn imply
the prerequisite study of units 1, 2, 3, 6, 10, 11 and 14. Thia a la
carte menu was presented to the curriculum committees of each of the
client departments, and a course of study tailored to the needs of their
students was negotiated.
Table 1 shows the content of courses required for each of the
undergraduage degree programs. As far as the University registrar is
concerned there are only three variable credit courses in which the
student may enroll. Within each of the courses, however, students may
be enrolled in one of three to five different subcourses whose exis
tence is of no concern to the registrar. A careful study of the content
of subcourses 30, 31, 32, 33 and 34 of ESM 303 will reveal their
striking similarity. Subcourse 33 taken by aerospace engineering stu
dents differs from the standard course (subcourse 30) only in the addi
tion of shear center and column buckling units to the curriculum. The
nuclear engineers study the stress and deformations in thickwalled
cylinders under thermal and pressure loading, while electrical engineers,
for good but arcane reasons of a local nature, study units normally
part of other courses, e.g., the static analysis of beams and damped
vibrations of particles and rigid bodies.
The differences among most of the courses are small, but it is
these small differences that lead to major debates in college curric
ulum committees, proliferation of courses in "Mechanics for XYZ Engi
neering" and/or dissatisfied clients of service courses. By employing
the modular design it is possible to tailor the course to meet the dif
ferent requirements of the clients and thereby provide better service.
The modular structure has the additional advantage of allowing
one to tailor a course of instruction to meet special requirements.
For example, a student who has completed a sophomorejunior level
physics course in mechanics probably has a good background in particle
dynamics, no experience in the static analysis of simple structural
elements, and a weak background in rigid body dynamics. In such a
case one'may create an ad hoc course, administrable under the aegis of
the standard dynamics course, which will allow the student to complete
his study of statics and dynamics without leaving major gaps and with
out undue repetition.
Each of the courses employs a standard textbook, and a full
set of lectures is scheduled. The schedule of lectures is distributed
to the students at the beginning of the term so that they may use the
lecture as one of a number of learning resources. Our experience shows
that most students benefit from the lectures of most professors. In
any event, criterion no. 4 mandates the availability of a stage from
which the faculty member responsible for the course may deliver his
personal statement on the subject matter of the course.
In addition to the classical textbook a set of programmed study
guides has been developed for the course sequence. The study guides
are an adjunct and guide to the use of the standard texts. They are
designed to give the student sufficient guidance to permit him to
study the material independently. For each unit or module there cor
responds a learning activity package as part of the study guide. Each
of the modules contains:
1) A description of content and rationale for
study of the indicated material.
2) A statement of prerequisites.
3) A list of behavioral objectives which the
student is encouraged to use as a check list
of tasks to be accomplished and abilities to
be developed.
4) Commentary and guide to the text. This, the
heart of the study guide, is written in a
colloquial style but is carefully structured
to require active participation of and fre
quent feedback to the student. The student
is assigned tasks to read specific sections
of the text, to provide missing steps in
derivations, to identify implicit assump
tions, to solve specific problems, to per
form simple experiments, and to swear oaths
of allegiance to the use of free body dia
grams. In addition, the study guide provides
remedial and advanced material. In this
manner the student is asked to abandon his
customarily passive role as reader. The
technique is similar to that of standard
programmed texts, but the programming steps
are considerably larger than is usual. This
"macroprogramming" approach has the advantage
of being considerable less timeconsuming to
prepare than traditionally programmed mate
rial. It also places greater responsibility
upon the student and hopefully, by example,
encourages the student to read technical
material from a critical perspective.
5) Sample proficiency test. After the student
has worked through the unit and reread the
list of objectives, he is instructed to
respond to a quiz printed at the end of the
unit. Upon completing the quiz under the
indicated conditions (duration, open book,
closed book), he is instructed to read the
solution provided in the package and to
grade his own paper. The actual quiz in
which he will be asked to demonstrate com
petency will be of a similar level of dif
ficulty and offered under similar conditions.
The study guides were prepared and the modular course system im
plemented in stages during the academic year 197374. The statics,
dynamics and mechanics of materials guides were put into use in the
fall, winter and spring quarters, respectively. Since spring 1974,
modular courses have been offered to approximately 300 students each
quarter.
To provide additional assistance to the student, a learning lab
oratory has been established and staffed with graduate and undergrad
uate teaching assistants whose duties include tutorial services. The
laboratory has typically been staffed about 40 to 50 hours a week and
is heavily utilized by students.
All of the above features of the instructional system have been
in effect since the fall of 1973 and are expected to be maintained in
the foreseeable future. In other respects the system is undergoing a
continuing evolution.
Modularity, Programming and Selfpacing
Modular instructional systems, programmed instructional systems
and selfpaced instructional systems are not synonymous.
A modular system is one in which the complete system is substruc
tured into discrete components with specified functions and specified
interreactions with other components. It is designed to meet special
ized system criteria by selectively drawing upon a modular subset of
the total system. Modular instructional systems are individualized
systems in that they facilitate the construction of course syllabi to
meet individual needs.
Programmed instructional systems are characterized by: 1) the
explicit identification of changes (behavioral objectives) in demon
strable competencies and/or attitudes expected of the user of the sys
tem; and 2) by the adherence to psychological principles of behavior
modification through reinforcement (positive feedback), pioneered by
B. F. Skinner. These principles have been used to teach people to
assemble Ml rifles psychomotorr programming), to solve differential
equations (cognitive programming), and to develop value structures
(affective programming). Programmed instructional systems are indi
vidualized systems in that they are addressed to individuals rather
than a public. They demand individual participation. They are based
upon dialogues rather than soliloquies.
Selfpaced instructional systems are designed to cater to variable
student input competencies (criterion no. 2) and levels of effort.
They permit the system designer to demand minimum performance levels
of all students by permitting a variable time to meet these criteria.
The Keller or PSI methods are typical of such systems. Because self
paced systems are inherently subject to abuse by procrastination
(criterion no. 7), they are rarely implemented in pure form. Most so
called selfpaced systems are actually flexiblypaced systems. Self
paced systems are individualized systems of instruction in that they try
to accommodate individual differences in ability and are subject to a
significant extent to individual control.
Thus, modular, programmed and selfpaced systems are, in different
senses, individualized systems. Any given individualized system may
assume to varying degrees the characteristics of all or some of these
system types. The instructional system for engineering mechanics in
use at the University of Florida since 1973 has been a consistently
modular and programmed system with variable elements of flexible
pacing.
As originally implemented, the system employed a significant ele
ment of flexible pacing strategy. Major traditional examinations were
scheduled during the fourth and eighth weeks of a ten week quarter.
Students had the option, however, of demonstrating proficiency (AB
performance) on module quizzes which could be taken on demand and
repeated (different quizzes) as necessary without penalty. If a stu
dent demonstrated reasonable progress by the dates of the scheduled
major exams, he was excused from the examinations. All students were
required to take a comprehensive, traditional final examination. For
students who completed nearly all the quizzes the final grade computa
tion could be deferred until the end of the first week of the following
quarter, to permit completion of the remaining one or two modules at
the AB mastery level.
With this system the student could proceed at his own pace and
79
walk into the final examination room with a minimum average of B, pro
vided he made reasonable progress during the term in the completion of
his module requirements.
Although most students opted for this mode of completing the
course, not all students view such a system with favor. There are
those who would rather take a few rigidly scheduled exams on a tradi
tional sinkorswim basis. In accordance with criterion no. 9, such
provision was made. In fact, each student could elect a continuum of
options between the pure competencydemonstration selfpacing mode and
the traditional mode. With students registering for as many as five
different subcourse versions in the same class and with the possibility
of each student proceeding under some combination of selfpacing and
classical examination modes, a CMI routine was necessary. Such a com
puter code has been developed by the author and reduces the record
keeping requirements to easily manageable proportions.
The costeffectiveness of the system is analyzed in some detail
below. Such systems require moderate levels of funding for student
assistance. In particular, the updating and the maintenance of secu
rity on extensive quiz files and the logistics of individually assigning,
monitoring, grading and recording quizzes places a heavy but not unrea
sonable burden on such funds. In times of financial retrenchment, how
ever, it is not always possible to maintain reasonable levels of
funding. Such is the case now. In response to increasingly severe
budgetary restrictions which forced a cutback in student assistant
staffing, it was necessary to first curtail and eventually eliminate
the flexiblepaced aspects of the system. The modular unit quizzes
have been retained, but they are now scheduled on a weekly basis and
80
will not be repeatable. Thus, the system in adjusting to criteria no.
5 and no. 6 may lose some of its effectiveness in meeting criterion
no. 2.
Cost Effectiveness
The development of cost effectiveness data within a give institu
tional setting is always subject to charges of prejudice. To transfer
such data to other institutions is still more difficult. The data
(table 2) on the per quarter cost of alternative systems of instruction
is reasonably accurate for the University of Florida. During academic
year 197273 two course sequences in engineering mechanics were
offered (ESM 304, 305 (10 hrs.) and ESM 301, 302, 303 (12 hrs.)).
The data on faculty contact hours represent typical course scheduling
patterns. If any error has been made, it has been in an underestimation
of the AY 197273 student assistant expenditures. Data for the modular
system are based on actual experience and budgeted expenses. No amor
tization of development costs is shown for the modular system because
of the caprices to which such estimates are subject. If one assumes
that onethird of a faculty line item/quarter on a continuing basis
would be a reasonable allocation for the cost of development and mod
ification, then that would add about $2000 to the indicated cost
figures. The data for the austere traditional system would represent
a severe cutback in service offered, since it would require all depart
ments to take a common 12 hour course sequence. Such a development
would meet with severe resistance and lead to possible loss of cus
tomers and the duplication of courses. It would offer no advantages.
Teaching Assistant Effectiveness
The one salient feature of the cost data of table 2 is the great
increase in expense incurred by substituting graduate student assis
tants for undergraduate assistants.
At the end of the fall 1974 quarter the students were asked to
evaluate the effectiveness of each of the teaching assistants. This
was a quarter in which the students had extensive tutorial contact
with teaching assistants. They were asked to respond to the following
statements:
1) Had a good knowledge of the subject
matter for this course.
2) Graded quizzes fairly.
3) Helped you learn the material.
4) Treated students courteously.
5) Overall effectiveness.
The average scores of the three graduate assistants are compared with
those of the four undergraduate students in table 3. There is a con
sistent pattern of superior performance by the undergraduate students
in comparison with the graduate students.
The differences may be attributable to a number of facts which are
widely applicable to other institution. First, the graduate students
were average graduate students in the department. They were not vol
unteers, nor were they specially selected by the instructor. They were
available, were not working on reimbursable grants, and they were
assigned to the teaching lab as part of their assistantship respon
sibilities. The undergraduate students were among the very best stu
dents in the college. They were recruited and there were more appli
cants than positions. Moreover, if called upon to work longer hours
on occasion, they were pleased to do so since they were being paid by
82
the hour. The graduate students are paid a monthly stipend in return
for which they may be called upon to perform some duties. It would
be understandable if they felt that to some extent they were over
worked and underemployed.
The differences in performance are not large, and one should not
conclude that graduate students not be used for such assignments.
Rather, it is safe to say that no loss of effectiveness occurs if
undergraduate students are hired. It may be highly desirable to assign
graduate students to such duties as a means of supporting the graduate
program. Any increment in cost attributable to the use of graduate
students, however, should be considered as part of the cost of the
graduate program and not part of the cost of the undergraduate instruc
tional system.
Effectiveness of the Instructional System
It would be felicitous to report evidence that the modular, flex
ibly paced instructional system results in superior student mastery of
engineering mechanics in comparison with students educated by tradi
tional methods. Although there is reason to believe this to be so,
evidence for such a sweeping conclusion is difficult to produce. One
cannot trust comparisons between performance on nominally comparable
tests or even comparisons in performance on the same test graded by
different people, or on the same test graded by the same person on
two different occasions.
There are data to report, however, which tend to corroborate the
improvement in student performance attendant to the use of the system
described above. Table 4 shows the average grades on the final exams
for six randomly selected courses taught under the modular system.
TABLE 4
FINAL EXAM GRADES OF RANDOMLY SELECTED MODULAR COURSES
"A" "B" % Improvement
Term Course (07 quizzes (8 or more Sample ("B""A")xlOO
passed) quizzes size 100 "A"
passed)
F 73 ESM 301 47 63 52 30.2
W 74 ESM 302 54 68 23 30.4
S 74 ESM 303 53 56 49 6.38
F 74 ESM 301 84 91 34 43.8
F 74 ESM 302 48 47 27 1.9
F 74 ESM 303 46 54 52 14.8
Total sample size = 237 Weighted average % Improvement = 20.2%
The students have been split into two groups. In column 'A' the final
examination scores of students who completed 07 units prior to taking
the exam are listed. In column 'B' similar results are shown for stu
dents who completed 8 or more units. The average number of required
units for all courses is 9. The last column shows the percentage
improvement as 100 times the ratio of the difference in scores divided
by the maximum possible difference in scores. This figure of merit
provides a common basis on which examinations with different mean
scores may be compared.
During the spring 1975 quarter large numbers of students took the
major inclass examinations for the first time. Similar results for
these courses are tabulated in table 5.
TABLE 5
AVERAGE GRADE ON 4TH WEEK EXAM, SPRING 1975
No Some All % Improvement % Improvement
quizzes quizzes quizzes Sample Some quizzes All quizzes
Courses passed passed passed size passed passed
ESM 301 66 75 90 134 26.5% 70.6%
ESM 302 59 63 79 70 9.8% 48.8%
ESM 303 69 76 88 88 22.6% 61.3%
Total sample size = 292
Weighted average
% Improvement, some quizzes passed: 21.3%; all quizzes passed: 6.3%
Tables 4 and 5 show that participation in the competency demon
stration quiz program results in significantly improved performance on
the comprehensive final exams and on the comprehensive fourth week
exams, although the data indicate that the effect is less pronounced
in the final examination, which covers a larger amount of material
and longer time span.
There is a possibility that the data are misleading, in that the
students who complete the units may be better students and that this
factor may account for the observed improvement. To test this hypoth
esis the data for the large spring 1975 statics class were examined
in more detail. Table 6 shows the examination results for students
who passed varying numbers of quizzes prior to the examination. The
third column shows the average GPA for each of these groups of stu
dents. The consistent GPA trend suggests the possibility that the
TABLE 6
EXAM RESULTS FOR STUDENTS PASSING VARYING NUMBERS OR PRIOR QUIZZES
Number of Average University GPA corrected
quizzes grade on grade point examination
passed examination average grades
0 66 2.74/4.0 66
1 73 2.84/4.0 71
2 80 2.87/4.0 78
3 90 3.09/4.0 86
improvement in examination scores may be due to the fact that better
students pass more of the quizzes. However, when the GPA for each of
these groups was reduced to the base 2.74/4.0 level by randomly
deleting from the data set a sufficient number of students with high
GPAs, it was found that examination scores for the student with com
parable GPAs, but different numbers of quizzes passed, were markedly
different. Participation in the competency quiz program accounted
for 20 points of the 24 point grade spread observed.
Thus, one may conclude that students with comparable histories of
academic achievement performed better by participating fully in the
use of the procedures and materials in this instructional system. To
this conclusion a skeptic might respond what's new? All that has
been shown is that students who work hard do well. While to a degree
such criticism is justified, the data do indicate some useful informa
tion, e.g., the tasks assigned to the student are relevant to the
objectives and effective in meeting them. While one may intuitively
anticipate such a conclusion, it is nonetheless not trivial. Without
experimental verification, an engineering professor may be no more
certain that completion of certain exercises will contribute to the
mastery of a given subject than may a swimming coach be certain that
completion of a regimen of calisthenics will increase the speed of his
swimmers.
Conclusion
As a result of experience with the modular system during academic
years 197374 and 197475, its feasibility has been demonstrated. It
is competitive on a cost basis with conventional systems, and prelim
inary results indicate that student performance may be improved by the
use of the instructional materials and procedures of this system.
Course evaluation questionnaires and informal consultation with stu
dents and teaching assistants indicate positive affective response to
the system and a belief by the students that they are learning more
than they would have in more conventional systems. Attempts to make
a more definitive evaluation of system effectiveness are in progress.
However, it is clear now that by using the modular structure, the
departments served by the courses have more detailed knowledge and
control of the content of the material for which their students are
responsible.
APPENDIX B
STUDENT RECORD SHEET
APPENDIX B
STUDENT RECORD SHEET
NAME
SS#
Course: ESM 30 Credits
Major Department
U.F. G.P.A. Hours Registere
If working, how many hours
Lower Division Preparation at: UC
Highest
ESM 301
UNIT
Quarter
Subcourse Version
d This Quarter
Community
College (which)
Other
Math Completed PS 215 Grade
Grade When
SCORE
DATE
UNIT
SCORE
DATE
SCORE
EXAM
I
II
Final
APPENDIX C
SAMPLE MODULE PROFICIENCY TEST
