LEARNING RATES FOR ORAL READING AT INSTRUCTIONAL AND
FRUSTRATION LEVELS BY ELEMENTARY PUPILS WITH MILD LEARNING
JOHN SCOTT, JR.
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN
PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
John Scott, Jr.
I would like to thank Dr. William Wolking, Dr. Bob Algozzine, Dr. Vivian Correa,
Dr. Brian Iwata, and Dr. Cary Reichard for their support and guidance. Very special
thanks are in order for Bill Wolking who as my mentor, coach, and friend helped me
learn how to ask the right questions.
I wish to extend my thanks to those who assisted in the research reported in this study.
First, I want to thank Jolena Stoutimore for her assistance with tutor supervision and her
many helpful suggestions for improving the research. I am grateful for the efforts of the
students enrolled in EEX 3243L during the Spring 1988 term for not only for the data they
helped to generate but for the opportunity to be part of their teaching career. I would like to
thank the students at the P. K. Yonge Laboratory School for their cooperation and
enthusiasm in the after-school tutoring project. Also, I want to extend my gratitude to the
staff at the P. K. Yonge School, especially Jean Brown and Patti Rosenlund, for their
assistance and support.
My most heartfelt expression of gratitude is reserved for my wife, Wanda, for all her
loving support. Her patience, encouragement, and understanding, while I spent countless
hours developing materials, analyzing data, and writing, go well beyond the obligations of
TABLE OF CONTENTS
ACKNOWLEDGEMENTS ............................................. iii
ABSTRACT ........................................................ vi
I INTRODUCTION ............................................. 1
Statement of the Problem .................................... 1
Rationale ............................................... 2
Definition of Terms ....................................... 4
Delimitations ............................................ 5
Limitations .............................................. 5
II REVIEW OF RELATED LITERATURE ........................... 6
Placement and Informal Inventories .......................... 6
Attempts to Validate Reading Placement Criterion ............... 12
Suggestions for Alternative Placement Criterion ................ 16
Repeated Readings ........................................ 20
Summary ............................................... 22
III METHOD ................................................ 25
Experimental Questions ................................... 25
Subjects ................................................ 25
Setting .. .............................................. 26
Variables Under Investigation ............................. 27
Independent Variables ................................. 27
Dependent Variables .................................. 27
Measurement ........................................... 27
Independent Variables ................................. 27
Dependent Variables ................................. 28
Experimental Design .................................. 29
Experimental Procedures .................................. 30
Initial Assessment .................................... 30
Instruction .......................................... 30
Say Facts Comprehension Check........................ 32
Decision Making................................... 33
Materials ........................................... 33
Data Collection and Analysis .............................. 34
Data Collection ....................................... 34
Data Analysis ....................................... 35
Rate Data for Oral Reading ........................... 35
Comprehension Data ................................. 39
Other Measures ..................................... 40
Reliability .......................................... 41
Procedural Reliability ............................... 41
Interobserver Agreement............................. 42
IV RESULTS ............................................... 43
Question 1 .............................................. 43
Question 2 .............................................. 46
Question 3 .............................................. 49
Other Measures .......................................... 58
Summary of Results ...................................... 62
V DISCUSSION AND CONCLUSIONS ............................ 64
Methodological Progress in Reading Research .................. 64
Factors supporting Continued Use of the Betts' Criteria ............ 66
Potential Utility of High-Error Material in Individualized Instruction.67
A Simple and Effective Set of Instructional Procedures ............. 69
Limitation of the Study and Recommendations for Future Research... 69
Instructional Recommendations .............................. 71
REFERENCES ... ............................................. 74
A SAMPLE READING PASSAGE ........................... 80
B SAMPLE COMPREHENSION QUESTION SHEET ............ 82
C STUDENT DATA...................................... 85
D TUTOR MONITORING FORM ........................... 102
BIOGRAPHICAL SKETCH ....................................... 104
Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the Requirements for the Degree of
Doctor of Philosophy
LEARNING RATES FOR ORAL READING AT INSTRUCTIONAL AND FRUSTRATION
LEVELS BY ELEMENTARY PUPILS WITH MILD LEARNING PROBLEMS
John Scott, Jr.
Chairman: William D. Wolking
Major Department: Special Education
Learning rates for oral reading by elementary pupils with learning problems at
two levels of text difficulty were compared. Placement in an instructional level of
material, derived from the Betts' criterion of 95 to 98% initial oral reading accuracy, was
compared to reading progress of students on material with 80% or less initial accuracy
designated a frustration plus level. Subjects were 16 elementary school children with mild
learning problems. Individual tutoring, with daily assessment, was provided by special
education majors. Instruction consisted of daily timings, simple error correction, and a
repeated readings procedure in which the learner read aloud while listening to a paced
audiotape of the passage. An alternating treatments design was used. Performance data
were plotted on Standard Celeration Charts and rates of learning calculated.
Learning rates for corrects on the frustration plus level were superior or equal to
those of the instructional level for 100% of the students for experiment 1 and for 100% of the
students in experiment 2. Rates for errors on the frustration plus level were superior for
75% of the students in experiment 1 and 68% in experiment 2. Comprehension was
measured pretreatment and posttreatment and showed greater increases on the frustration
plus level for a say facts procedure and answers to comprehension questions.
The instructional reading level failed to offer students the most productive material
when a student was provided with instruction consistent with recommended individualized
instructional procedures. It is recommended that the use of the term frustration level be
reconsidered as it may be inaccurate and may have the effect of restricting teachers from
using potentially productive reading materials on which learners make numerous initial
errors. The methodological advantages of using oral reading as the dependent measure for
placement criterion research as well as the benefits of direct and frequent measurement are
discussed. The work of Lindsley, Powell, and others who have suggested that children
given appropriate instructional support are capable of making better progress on material
considerably more challenging than the material they have been asked to read was
supported by the results of this study.
This is a report of a study on the rates of learning by elementary students on reading
passages of widely different levels of difficulty. Decisions to place students in material at
specified grade levels are routinely made on the basis of a child's performance on an
informal reading inventory (IRI). The criterion for placement at an instructional level,
the level purported to offer the greatest opportunity for learning, is generally 95 to 98%
accuracy in word recognition on a IRI passage (Cook & Earlley, 1979; Hargis, 1982;
Johnson, Kress, & Pikulski, 1987; Mercer & Mercer, 1981; Savage & Mooney, 1979). This
criterion, as originally proposed by Betts (1946), has been widely and essentially
uncritically accepted as offering the best starting point for instruction. Special education
students, in the development of their individualized educational program (IEP), are
typically placed within a reading series based on this criterion. Reports of research
attempting to validate the efficacy of the instructional level based on learner performance
at various levels of difficulty are absent. Are children who are placed in reading materials
at the 95 to 98% accuracy level and provided with an effective and individualized reading
program capable of making the greatest gains in reading or is a placement determined by
some other criterion more effective in maximizing learning?
Statement of the Problem
Teachers use the Betts' criterion to place students in reading material. They do this
with the expectation that this placement provides the best possible starting point for
instruction and that the most effective learning takes place when children begin reading
at this level of difficulty. The problem investigated by this study was to compare learning
and performance of students in two levels of text: an easy level (Betts' criterion) and a
difficult (frustration plus) level. The objective of this study was to provide an empirical
test of both levels of text difficulty with mildly handicapped students in a tutorial setting by
answering the following question: Are children placed in reading materials at the 95%
accuracy level and provided with an effective and individualized reading program
capable of making the greatest gains in reading or is a placement determined by some
other criterion more effective in maximizing learning?
The following questions constitute components of the main question:
1. What are the learning rates for correct responding at each level of text difficulty?
2. What are the learning rates for error responding at each level of text difficulty?
3. How does comprehension compare across instructional and frustration plus levels
of text difficulty?
In spite of vast investments of money and effort, the academic performance of special
education students remains unimpressive. Calls for increased accountability on the part of
government and by parents have suggested reevaluation of existing practices (Will, 1986).
Special educators have been aware of the problems and have actively worked toward
solutions on several fronts (Algozzine & Maheady, 1986). Follow-up data gathered by
researchers from selected groups of former exceptional students suggest that they are
headed for low paying jobs with few prospects for advancement (Mithaugh, Horiuchi, &
Fanning, 1985). With the American economy heading ever closer to a two-tier society of
rich and poor (Thurow, 1987), those individuals with educational deficits are most likely to
wind up on the bottom of the socioeconomic heap.
No academic skill is more central than reading, yet it is an area in which a
remarkably poor job has been done. Poor quality teachers account for a great part of the
problem (Durkin, 1978-1979), while home and school organizational factors have
contributed as well (Samuels, 1986). For the child who does not learn to read or who reads
with great difficulty the impact on their school career and life adjustment can be
devastating (Perfetti, 1984). Reading instruction should be considered a number one
The rights guaranteed to handicapped learners and their families by Public Law 94-142
(Public Law 94-142, 1977) for improved special education instructional programs rely
heavily on accurate assessment (Bennett, 1983). Of the several purposes of assessment, it is
the educational planning role, assisting the teacher in determining what to teach and how to
teach, which was examined. It is this aspect of assessment most likely to be conducted with
informal instruments such as an informal reading inventory (IRI). The IRI, typically
used by classroom teachers, provides specific recommendations for placing the child in
instructional reading material. This technique, especially when based upon the actual
curricular materials used in the classroom, can be very valuable (Deno, 1985; Howell,
1986). The problem comes not from the format but from the various criteria employed for
placing children in appropriate reading materials. The widespread use of the placement
criteria developed by Betts (1941, 1946) calls for 95 to 98% word recognition accuracy with
75% or better comprehension on a graded reading passage. Children are routinely placed
in reading materials based on this recommendation. These arbitrarily selected criteria
have received "validation through use" (Shanahan, 1983, p. 581) but lack empirical
validation. This is at variance with the belief that tools used for assessment and the as-
sociated criteria for instructional placement should be technically adequate for assessing
the skills and abilities of handicapped children (Bennett, 1983).
There are several strong indications that the Betts' criterion does not lead to the most
appropriate placements of students in reading materials (Kragler, 1986; Neufeld &
Lindsley, 1980; Johnson, 1971; Powell & Dunkeld, 1971; Roberts, 1976; Stanley, 1986).
Validation of this criterion with special populations receiving individualized educational
services has not been reported in the literature. A question has emerged as to whether or not
this placement criterion leads to the most effective placement in reading materials or
whether some other placement criterion is associated with better learning. This hypothesis
was testable by the procedures followed in this study. If the Betts' criterion fails to predict the
most effective starting points for reading instruction, why continue to place special
children at these points? The use of reading grade level and placement consistent with the
instructional reading level recommended by the Betts' criterion form a metaphor that
powerfully shapes teaching practice (Cadenhead, 1987). Misplacement based upon a faulty
criterion may be interfering with a teacher's ability to provide the most effective reading
Definition of Terms
Accuracy. Accuracy is a measure of correct responses divided by the total of correct
and incorrect responses.
Celeration. Celeration is a basic unit of measurement of behavior change; change in
frequency per week (Lindsley, 1971).
Frequency. Frequency is a basic unit of behavioral measurement; the number of
movements per unit of time (minute).
Frustration plus reading level. This terms refers to the reading level in which the
learner initially reads 80% or fewer words correctly.
Learning. For the purposes of this study, learning refers to a change in performance
per unit time; also called celebration.
Learning rate. Learning rate is a measure of total academic change in behavior per
unit time; the celebration value for corrects multiplied by the celebration value for errors.
Informal reading inventory (IRI). An IRI is a reading assessment instrument based
on a series of passages with measured reading levels. Student responses are judged
against prescribed standards for oral reading and comprehension to determine placement
in reading material.
Instructional reading level. The reading grade level at which the learner, during
assessment, orally reads with 95 to 98% word recognition accuracy is the instructional
Readability level. The readability level is the degree of reading difficulty as computed
by the Fry Readability Formula (Fry, 1977).
Standard Celeration Chart. The Standard Celeration Chart is a standard, six-cycle
semi-logarithmic chart that measures frequency as movements/min and celebration as
movements/min/week. This chart is also called the Standard Behavior Chart (Lindsley,
Delimitations of the Study
This study was designed to examine the rates of learning for elementary children with
mild reading problems who are receiving individualized instruction. It would be
inappropriate to attempt to generalize the results to pupils in traditional, group-oriented
instructional settings. Caution should be exercised in relating the findings to learners
with severe reading disabilities and to those students experiencing no reading difficulty.
Extension of results to learners beyond the elementary grades would be unwise.
Limitations of the Study
The study was designed to investigate the learning rates for elementary school children
working at different levels of text difficulty. Subjects were selected from a university
laboratory school setting that did not use a traditional special education classification
system. A significant limitation concerns the nonclassified status of the subjects. As they
are not classified as special learners, they may not be truly representative of any
traditional exceptional student category. Additionally, extention of the results to students
beyond the Alachua county area may not be warranted.
REVIEW OF RELATED LITERATURE
There is a vast amount of literature addressing problems in reading instruction. This
is a selective review of studies closely related to the question selected for investigation. The
necessary selection criteria were broad and flexible. In order to develop the proper
historical context for this investigation, several works from the earlier part of this century
were reviewed. While the reading research is extensive, only a small portion directly
relates to this study.
First, an examination of the primary reading placement criterion in use with
handicapped and reading problem pupils is presented. The focus is on the origins of this
placement criterion and recent recommendations for use of such a criterion. Empirical
studies designed to test the efficacy of this placement criterion are given special attention.
Instructional implications of the high-accuracy, fluency-building criterion in wide use are
broadly explored as are criticisms of this placement tradition. Alternatives to this grade
level-dependent, high-accuracy placement criterion, primarily behavioral in orientation,
are reviewed next. Finally, the method of repeated readings is reviewed in detail as this
technique plays a key role in the intervention phase of the study.
Placement and Informal Reading Inventories
A useful perspective is provided by examining several historical trends in the de-
velopment of reading grade level and the concept of readability. Lively and Pressey (1923)
proposed "a method for measuring the vocabulary burden of textbooks" (p. 396). They
systematically sampled 1000 words from public school textbooks and by comparing the list
thus compiled to the entries in the Thomrndike Word Book derived measures of vocabulary
difficulty. Refinement of this time consuming method led to simpler procedures for
determining what is now generally referred to as readability and to the increased use of the
term "level" or reading level to designate the relative difficulty of a given reading passage
With methods for determining the grade level of a reading passage in place, the next
development of note was the use of techniques to assess and classify pupil performance at
varied grade levels. Betts (1946) in Foundations of Reading Instruction suggested the use
of four levels to describe pupil performance within a series of graded reading passages.
These levels were basal, instructional, frustration, and capacity (Betts, 1946). Betts credited
the work of Killgallon (1941) for the refinement of the procedures used to determine these
levels and, most importantly, for the performance criteria associated with each level.
Killgallon tested 41 fourth grade students with the Gates Survey (a standardized
reading test) and an informal reading inventory developed by Betts. While
acknowledging the importance of pupil interest and motivation, Killgallon concluded that
this IRI was the best single available index to reading ability at the elementary school level.
Killgallon provided average comprehension scores for instructional and frustration levels
as well as an analysis of tension or frustration behaviors.
Two levels arbitrarily defined by Killgallon and Betts are of special concern. The
basal reading level (comparable to the modern independent reading level) was determined
by the following criteria: (a) 90% or better comprehension, (b) fluent, error-less oral
reading, and (c) efficient silent reading with good comprehension and no signs of tension
behaviors. The "probable" instructional level (comparable to the current instructional
level) was the level at which (a) a minimum comprehension score of 50% (or if subjectively
rated dysfunctional reading behaviors were present, a 75% comprehension score was
required), and (b) no more than 7% word recognition errors or one error in 14 running
words. Killgallon described this level as the most appropriate for reading instruction as,
maximum development was to be expected when the reader was suitably challenged but not
so challenged as to be frustrated.
When these criteria were applied to the reading performances of the 41 fourth-grade
children, the average basal reading grade level was .86 (range, 0-5.0). The probable
instructional reading grade level was 3.16 (range, 0-9.0). Further, the informal inventory
placed students, on average, 1.06 grades lower than they would have been placed with the
Gates Reading Survey (Shanahan, 1983, p. 581).
These results should have strongly suggested to Killgallon, Betts, and others that the
proposed criteria were too stringent and would have the effect of placing children in reading
material that was below the level at which they had been working, presumably, suc-
cessfully. In any event, Killgallon proposed a set of arbitrary placement criteria that could
be easily adopted by teachers and reading specialists. A host of IRIs developed since that
time ( Ekwall, 1986; McCracken, 1966; Silvaroli, 1976; Woods & Moe, 1985 ) have accepted,
with only slight modifications, these basic placement criteria.
The essential circularity of Killgallon's work is highlighted by Shanahan in a 1983
critique. Shanahan identified the construct validity question that Killgallon appeared to be
addressing as, "Does providing instruction at the level which can be read with 95 to 98%
accuracy lead to demonstrably higher levels of achievement than does placement in
materials above or below that level?"(p. 583). The study designed to accomplish this, wrote
Shanahan, would have tested children, placed them at varied reading levels, and then, after
instruction, measured their progress. Instead of doing this, Killgallon determined the
mean percentage for errors of students whose performances fell within arbitrarily
predetermined ranges. Shanahan pointed out that Killgallon selected only those oral
reading performances which met his instructional level criteria and determined the mean
percentage of word recognition errors. It is the result of this computation, the average of all
pupil scores above 93.9% for word recognition with 50 or 75% correct comprehension, that
was used to determine the now ubiquitous 95% correct placement criterion. Shanahan noted
that the use of this reading level criterion has unwisely received "validation though use"
and that experimental validation, which Killgallon did not do, is the only way to arrive at
the "optimum criterion" (p. 581).
Teachers who use the highly structured basal series used in most reading programs rely
heavily on the notion that sentence and word length, vocabulary, and content should be
controlled to match the child's reading ability level. Reading instruction, to be most pro-
ductive, should match the child's reading level to materials of the same readability level.
Errors should be very few in number and a priority is placed on reducing them even
further. These general principles serve, in critical ways, as pillars of modern reading
instruction and are presented uncritically in textbooks aimed at both new and experienced
teachers (Cadenhead, 1987). Harris and Sipay (1980), for example, stated that, "the
instructional level indicates that the material is suitable for use under the teacher's
direction; serious errors are not numerous and comprehension is adequate. The material
presents some challenge so that new skills can be acquired" (p. 182). A passage from
Savage and Mooney (1979) indicated a similar viewpoint was recommended for the
exceptional child. They wrote, "the instructional level indicates the level of material that is
challenging enough to use in further developing the child's reading skills, yet is not so
difficult that it will be a source of frustration for the child" (p. 92). Savage and Mooney
provided a summary statement regarding the frustration level in which they concluded,
"material at this level is considered too difficult for the child to handle, even with support
and help from the teacher" (p. 92). Hargis (1982) wrote a reference work on reading
instruction for special children which provided an additional insight. Hargis adhered
closely to the Betts criterion in terms of both numerical criterion and the importance of a
very low error instruction for special children receiving instruction in an individualized
format. He recommended a 2 to 4% percent error rate for the instructional level while
classifying reading with more than an 8% errors the frustration level. A component of the
rationale Hargis offered for the superiority of the instructional level over the frustration
level was somewhat different from that given by other authors and may hint at a critical
factor in the durability of the Betts' placement criterion.
Any reading more difficult than the instructional level is considered at frustration
level. Not only does it produce frustration and failure in the child, but it requires an
inordinate amount of demand on teacher time to introduce, clarify, and support, if
the student is to be maintained on task. Demand on teacher time is often far too
great to permit using this material efficiently in reading instruction. (p. 109)
Demands on teacher time and issues of instructional convenience may be critical
elements in successful large group instructional arrangements. It would be, however,
inappropriate to suggest, as Hargis repeatedly does, that the instructional level yields the
most productive learning and then to offer as evidence not performance data but an
argument for traditional classroom practice. Perhaps what Hargis meant was that the
instructional level may be the most convenient level for reading instruction. The rationale
he offered for its use was that it was the most productive starting point for reading
Similar statements are found, without exception, in all of the major texts reviewed on
this topic (Cook & Earlley, 1979; Hargis, 1982; Harris & Sipay, 1980; Johnson, Kress, &
Pikulski, 1987; Mercer & Mercer, 1981; Savage & Mooney, 1979). A summary of several
placement criteria recommended for regular and special education students as well as the
authors' suggestions for the learning potential at the instructional and frustration levels
can be found in Table 1. The view consistently expressed was that the instructional level
was the more productive and most appropriate level on which to provide instruction while the
frustration level indicated misplacement in excessively difficult material with little
chance for the student to make any meaningful progress in reading. It was quite
conceivable that a less convenient instructional arrangement (in the sense of traditional
procedures) with a more challenging placement criterion may be far more capable of
I0 ti; ca 0
z 4t *-
4. j, dis"S *
- 8 2 0* 1 *0
*- 0 il sd
I-~l 0 s
0 ..~ '54.
r' .G *VZS
Cd I- *
) 0 a.c
4 a d
di 4 di
* .^ .
0 -0 0.b
'.3 La .
u i di *- 4.
w dc i 61
o* .S *
: o .d
C '- .2 -
producing high rates of student success in reading. Children consistently placed in easy
reading material may actually be disabled in functional reading skills. The student may
lack confidence to tackle unfamiliar material. The reader may consider material that
appears to contain a higher percentage of errors than they have become accustomed to, as
unreadable. He or she might fail to develop the persistence and ability to deal with
challenging reading material that is characteristic of mature readers. Authors, such as
Hargis, who cited classroom practice as justification for existing reading instructional
procedures should have fully considered the high rates of illiteracy and reading failure in
the United States when contrasted to other industrialized nations (Yarington, 1979). It may
well be that practices such as the emphasis on placing children in very low error material
have been, in large part, responsible for the failure of American schools to provide each
child with reading skills conmensurate with his or her ability.
Attempts to Validate Reading Placement Criteria
Several efforts to validate the placement criteria used in IRIs have been made with
conflicting results. Powell and Dunkeld (1971) attempted to more accurately describe the
instructional reading level. The investigators administered IRIs to 212 nonhandicapped
students in grades 2 through 6. They also administered standardized reading tests and
intelligence tests. After the initial testing, they examined the regular classroom
instructional materials used by these children, computing readability scores which were
then compared to the IRI derived instructional level recommendations. The readability
levels of the materials actually used by the children were matched to the readability level of
the IRI passages. The best predictor offered by the IRI in this analysis was not word
recognition errors but rather comprehension in the 70 to 75% range. At the end of the school
year, residual gain scores were computed for each child's reading achievement. These
adjusted gains were then compared to the levels of materials used for instruction. The
materials used by the children who had made the greatest gains then helped to identify new
criteria for instructional levels. Powell and Dunkeld recommended differentiated
criteria based on grade placement of the pupil and suggested the following:
1. second grade--85.5 to 95% word recognition accuracy,
2. third grade---89 to 97% word recognition accuracy, and
3. fourth, fifth, and sixth grades--comprehension scores of 60 to 90%.
While Powell and Dunkeld made a determined effort to validate the optimum
instructional level, their work has several significant shortcomings. First, they relied on
group averages, thereby losing the exactness of data showing how individuals interacted
with the learning task. Further, they excluded learning disabled, gifted, and other
exceptional learners, precisely the groups most likely to have important curricular
decisions made on the basis of such criteria. An additional limitation for the purposes of
this investigation was that these children were all receiving a traditional, group-oriented,
basal reading instructional program. Generalization to children receiving
individualized instruction, as was the case in special education practice, may not be at all
appropriate. Still, Powell and Dunkeld pointed to the need to move away from one criterion
for all age-ability levels and suggested that, for some children, less reliance on initial
accuracy may be positively correlated with reading gains.
A second phase of Powell's (1984) work relevant to this review concerned the
development of the mediated or "emergent reading level." The emergent reading level
suggested the level on which the child was likely to make the greatest gains in reading
when provided with mediation, or appropriate levels of teacher support on new vocabulary,
main idea, and story details. Powell suggested that the standard administration of an IRI
failed to provide a realistic assessment of the child's reading due to the absence of the
traditional supportive instruction provided in actual reading levels. When using the
procedures recommended by Powell's emergent placement criterion, the assessor provided
mediation or traditional forms of teacher assistance. The level at which the child can
achieve a high word recognition and high comprehension score with these procedures was
offered as the emergent reading level. Emergent reading levels tend to be significantly
higher than the instructional reading levels suggested by the Betts' criterion.
Two of Powell's students (Kragler, 1986; Stanley, 1986) have conducted investigations of
the emergent reading level germane to this review. Stanley (1986) compared traditional
IRI procedures to mediated IRI procedures to determine which more accurately predicted
reading level placement. Preinstruction and postinstruction comprehension and word
recognition measures were obtained for 21 second grade subjects. Placement determined
by the mediated procedure was two to three book levels higher than that of the traditional,
Betts-style, IRI procedure. Progress for all students placed according to the mediated
procedures was significantly greater than the traditional placement group. In this study,
more challenging material was associated with more successful learning while the easier,
traditional level of material difficulty was associated with less successful learning for
students receiving group reading instruction.
Kragler (1986) studied the silent reading and listening levels of 21 low -socioeconomic
reading underachievers when placement was guided by emergent reading level
procedures. One group of students received mediated vocabulary instruction while the
control group was exposed to a repeated readings and traditional instruction procedure.
The mediated group was able to maintain comprehension at the higher placement levels
suggested by the emergent placement criterion. Students working under the traditional
conditions could not maintain success at the higher placement level. Kragler suggested
that the higher placement levels recommended by the emergent conceptualization may be
appropriate when supported by sound instructional procedures which are sensitive to student
performance. Similarly, the higher placement level may not appropriately place children
when instruction is insufficiently responsive to student performance. It should be noted
that these studies by Kragler and Stanley were not focused on special students nor at
learners in individualized educational settings but rather those receiving essentially
traditional group-oriented instruction. Powell, Kragler, and Stanley challenged the
assumption that the Betts' criteria provided the most productive starting point for reading
instruction in regular, nonindividualized classroom settings.
Even if the placement criteria recommended by the many IRIs that have been used could
be validated as offering the instructional placement most likely to bring success in
reading, a host of other problems limited their usefulness. Allington (1979) measured the
ability of 57 certified teachers enrolled in a graduate reading course to accurately record
oral reading errors in an IRI format. The students were provided with extensive training
in using the coding system and completed numerous audio-taped practice scoring sessions.
Among the scored errors on a final test were failing to note an oral reading error and
scoring nonexistent errors. The average proportion of errors for all subjects was 28%,
leading Allington to conclude that neither training nor expertise had much impact on a
teacher's skill in recording oral reading errors. The author suggested that such poor
accuracy is unacceptable given the importance of the instructional decisions made based on
the analysis of the oral reading performance. He recommended tape recording the pupils
reading to provide multiple scoring opportunities in the hope of improving accuracy.
A recent study by Helgren-Lempesis and Mangrum (1986) examined the alternate-form
reliability of three of the most popular commercial IRIs. Four doctoral students with
specializations in reading administered the IRIs to 75 fourth-grade pupils and retested them
with alternate forms of the same inventory 10 days later. The same examiner retested the
same students. Reliability coefficients ranged from .60 to .78, considerably below the .90
level generally considered acceptable for two forms of a commercially available test used to
make important educational decisions about a child (Salvia & Ysseldyke, 1981). The
authors maintained that this level of reliability was better than some critics would have
suggested. This study clearly pointed to the inadequate reliability offered by these
instruments even when used by experts. When such weak instruments are used by
inadequately trained teachers, the quality of the instructional decisions made may be, at
best, suspect and, at worst ,may actively contribute to pupil learning problems. These
problems, when coupled with the concerns regarding the basic placement criterion, strongly
suggest the reading placement procedures may have been inadequate.
Suggestions for Alternative Placement Criteria
While the use of the Betts defined instructional placement criterion of 95 to 98% was
widespread, several important studies have raised fundamental questions that had not been
adequately addressed. Four of the most provocative studies will be briefly reviewed as they
each suggest the fragile nature of the Betts' criterion.
By far the most critical in its implications for the continued use of a high-accuracy
placement criterion is a 1978 study by Roberts. Roberts determined the word recognition
accuracy of 125 second and third grade pupils in six English schools. Applying protocols
used with a standard IRI, she scored students on their teacher assigned instructional
reading materials. Roberts found 23% of the students were reading in material that could
be considered at their instructional level while 57% were reading in the frustration level
according to the Betts' criteria. In one school, serving primarily low socioeconomic status
children, the rates were 18% instructional and 66% frustration. Reading attitudes, as
surveyed by the author, were very positive and student achievement in reading, as
measured by the standardized testing program, was well above average. Roberts credits a
positive, cohesive school climate, focusing on rewarding pupil progress in a non-
threatening atmosphere with much of the success of the reading program. The signs of
dislike for reading predicted by Betts when children were forced to read in the frustration
level material were totally absent. This simple study prompted Roberts to seriously
question the value of the Betts' criteria. If a true frustration or dysfunctional level of
difficulty did exist, asserted Roberts, it was surely not the 95% accuracy rate so closely
adhered to by many teachers.
Gates (1962) presented second grade pupils with 42 "new" words from basal readers at
the third and fourth grade levels. He found that 36% of the children could correctly say the
new words with 100% accuracy and 40% got 41 of 42 words correct while 47% got 40 or more
correct. The second graders had learned the vocabulary, either in the course of outside
readings or elsewhere, suggesting that tightly structured readers were unnecessary and
may hold back the better readers. Gates went on to make a convincing argument for
reducing the role of basal readers in instruction. In pointing out the extensive word
knowledge of pupils well beyond the limits of basal readers, Gates hinted at the reading
potential untapped by criteria which arbitrarily foster low-error reading in highly
A fundamental criticism of the Betts' criteria regarding the independent reading level
(98% or more correct) has been raised by Spache (1986). He maintained that children should
read independently in material based on interest, not a 98% or 99% accuracy standard. His
long experience with the materials actually chosen by children for independent reading
convinced him that even frustration level material posed little barrier to enjoyment when
the subject matter held interest for the reader. This position, while consistent with the
experience of most teachers, appeared to have had no significant impact on the practice of
placing children according to a high-accuracy criterion and recommending extremely
high-accuracy materials for independent practice.
Lindsley and his students (Johnson, 1971; Neufeld & Lindsley, 1980) have conducted
studies using precision teaching to monitor learner progress. These researchers
demonstrated the potential of precision teaching which permits the researcher to conduct
frequent and extremely fine-grained examinations of learner performance under actual
instructional conditions. Johnson (1971) used peer tutors and high-interest, high-difficulty
level materials to bring about reading improvements for 140 inner-city children. In a
series of investigations, Neufeld used performance data plotted on the Standard Celeration
Chart to address a question similar to that of this investigation. She asked, "is the
instructional level of the informal reading inventory or any of three other performance
levels a level that promotes highest speed and accuracy learning" (p. 10). A total of 49 fifth
graders from a rural midwestern school served as subjects. Each was assessed on an IRI,
and the performance within the 95 to 97% word recognition accuracy with 75 to 90%
comprehension accuracy was identified as the instructional level. Each pupil then received
a minimum of 10 days of instruction at their instructional level, one grade level below, and
two and six reading grade levels above their designated instructional level. Performance
data were recorded and charted daily. The instructional procedures included an untimed
silent reading on one of the four randomly selected stories followed by a 1 minute see-mark
timing on comprehension questions. These results were then checked and charted. Next,
came a 1 minute timing on oral reading of the same text. These results were similarly
checked and plotted. This procedure was then repeated for the remaining three levels. Both
statistical analysis of the raw data and visual analysis of the charted data showed no sig-
nificant differences in celebrations for corrects, celebrations for errors, or accuracy
improvement for word recognition or comprehension for any of the four levels examined.
This lack of significant difference is quite important as it suggested that progress on
material that is either slightly less difficult than traditionally recommended as well as on
material well into the frustration range of the pupil can produce comparable learning.
Neufeld and Lindsley concluded that neither the instructional level nor any of the other
levels they looked at were associated with maximum learning. If, they suggested, the
instructional level failed to promote highest learning, why should it be used at all? They
urged additional study of this question in an effort to further test the value of the high-
accuracy criterion in reading placement.
This study was noteworthy in the methods of data gathering and interpretation. This
was the first use of the Standard Celeration Chart found in the literature to test the Betts'
criterion. The instructional procedures, however, appeared to have been relatively
unrefined and casual, subtracting substantially from the generalizability of the
conclusions. Pupils read silently and could ask for help with new words. This procedure
suggested that routine, direct instruction was not offered and that the individual child's
tolerance for errors may have played a major role in whether they sought assistance or not.
Similarly, the daily emphasis on comprehension may have had the effect of slowing
students down, especially on the more challenging material. The pupils, as can be inferred
from the initial high reading rates (approximately 200 words per minute on the
instructional level passage), were familiar with the high response rates typical of
instruction provided by precision teachers. They were not, however, students with learning
problems. As such, they may have begun instruction with more highly developed reading
skills than other pupils. These researchers also failed to functionally define the reading
level relying instead on an arbitrary jump to higher grade levels. It was likely that a
student reading six grade levels above his instructional level was reading at the frustration
level, but this was not certain. A functional assessment procedure, in which learners were
placed in frustration material, would have alleviated this problem. It may be the lack of
instructional procedures consistent across instructional days and across passages that
constitutes the greatest limitation of this study in regard to reading placement criterion
questions. The authors have clearly used the most refined set of data gathering and data
analysis procedures available to test the Betts' criterion.
A method for placing learners with reading problems in the optimal materials based on
actual performance data was suggested by Lovitt and Hansen (1976). Again using
precision teaching techniques to measure performance, they presented a series of five
passages at each of eight reading levels (two sublevels for grade 2 and 3). The students had
several opportunities to read at each grade level and assessment continued until all 40
passages had been presented. The charted data for each student provided an excellent
source of information on actual learner performance at varied grade levels. On the lowest
grade level, the mean correct rates was 58.3 words per minute (wpm) with incorrects at 5.4
wpm (92% accuracy). At the sixth grade reading level, the highest level in this study,
corrects were at 31.3 with error at 6.9 wpm (82% accuracy). While the differences between
highest and lowest grade levels was large, differences between grade levels was small
averaging a loss of 5 wpm in corrects, the addition of one oral reading error and a 5%
decrease in comprehension accuracy. Based on theses findings, Lovitt and Hansen set
placement criteria at 45 to 65 wpm correct and 4 to 8 wpm incorrect (89 to 92% accuracy
range) with comprehension scores of 50 to 75% correct. The authors reported that all
learners made adequate progress with these placement criteria. A valuable opportunity to
empirically test the relative effectiveness of placement in high or low accuracy material
was lost, however, as the subjects were all placed according to the revised placement
criteria. Of considerable importance for the purposes of this study were the data showing a
gradual rather than an abrupt drop-off in performance with material of greater difficulty.
The fact that learners in a setting tolerant of high errors do not emit the frustration or ten-
sion behavior predicted by Betts (1946) and others (Gentile & McMillan, 1987) suggests that
performance gains expressed as learning rates may be the best determinant of placement
With the growth and refinement of individualized educational programing has come a
need not only for improved placement criteria but improved instructional techniques. The
repeated readings technique first described by Samuels (1979) is a procedure aimed at
individual rather than group instruction. With repeated readings, the learner re-reads the
same passage until a criterion rate is met or until a prespecified number of readings have
been attempted. With the automaticity theory, LaBerge and Samuels (1974) provided the
conceptual basis for this technique. LaBerge and Samuels postulated limitations in how
much information processing a reader could handle at one time. If the majority of reading
effort was devoted to decoding individual words, little processing capability remained for
comprehension. By repeatedly reading a single passage, processing capacity could then be
freed to deal with comprehension and other higher order tasks, according to LaBerge and
Samuels. While this theoretical position remained to be convincingly tested, the repeated
readings technique has proven to be of considerable practical value in reading instruction.
Numerous studies have reported favorable results for repeated reading techniques with
handicapped children (Moyer, 1982; O'Shea, Sindelar, & O'Shea, 1987; Smith, 1979) and as
a supplementary activity with nonhandicapped children (Allington, 1983; Chomsky, 1976;
Johns, 1986; Samuels, 1979). In a recent study, Dowhower (1987) suggested that repeated
readings may hold significant promise as a technique for helping an early elementary
student move from word-by-word reading to fluent decoding. One noteworthy exception to
this favorable picture was reported by Rashotte and Torgesen (1985). In this study, gains in
reading appeared only when stories shared many of the same words. Additionally, gains
did not surpass a control condition in which an equivalent amount of nonrepetetive reading
was programmed. It should be noted that Rashotte and Torgesen used only four repetitions
of the text, a number far lower than that recommended by other investigators.
The use of repeated readings is usually coupled with a high word per minute rate,
typically a rate approaching that of normal conversational speech (W. D. Wolking,
personal communication, August 3, 1988). Repeated readings may, in part be responsible
for a renewed interest in reading rate within the traditional reading community
(Allington, 1983; Anderson, 1981; Biemiller, 1978; Carver, 1982, 1983; Carver, 1987;
Fleisher, Jenkins, & Pany, 1979). Carver (1983) suggested the reading rate for mature
readers may be relatively fixed and not adapted to the nature of the reading task or level of
text difficulty as had been previously thought.
The use of repeated readings and other specialized techniques designed to maximize the
progress of individual learners casts further doubt over the ability of the Betts' criterion to
predict the most effective starting point for reading instruction-especially individualized
reading instruction programs.
The work of Betts (1946) and Killgallon (1941) helped to promote the widespread use of a
high-accuracy, 95 to 98% correct oral reading performance criterion for placing pupils in
reading material. Although lacking a sound empirical base to support the superiority of
placing a child in material with this low degree of difficulty, the Betts' criterion has
continued to exert a profound influence over reading placement decisions and has become
"validated through use" (Cadenhead, 1987). Attempts to validate the most effective
instructional level have, for the most part, failed to substantiate the relative superiority of
the Betts' criterion. Researchers have suggested that children can learn successfully in
frustration level material (Gates, 1962; Johnson, 1971; Neufeld & Lindsley, 1980; Roberts,
1976) or in material more challenging than suggested by the Betts' criterion (Kragler, 1986;
Powell, 1984; Stanley, 1986). This strongly suggests that the hypothesized frustration-
related behaviors may not be evident in otherwise well-structured instruction.
General problems with IRIs further decrease the likelihood that correct placement
decisions will be made based upon such instruments. Weaknesses in teacher scoring
(Allington, 1983), alternate-form reliability (Helgren-Lempesis & Mangrum, 1986) when
coupled with an instructional placement criterion of dubious value, make the IRI, as it has
been used, inappropriate for instructional decision making purposes. With new techniques
such as repeated readings (Samuels, 1979) designed to maximize the growth of the
individual learner, comes an additional need to reevaluate the best starting point for
remedial and individualized instruction. Researchers who have designed studies to assess
real learning in material at widely different levels strongly suggested that instruction in
the frustration level and beyond may be as or more productive than instruction beginning at
the traditional instructional level.
A study was needed to compare the Betts' criterion to a frustration level placement.
Such a study needed to use rates of learning for correct and error responding as the
dependent measure. Precision teaching provided a measurement system well suited to this
purpose as it was based upon direct, frequent, and sensitive measures of learning.
Comprehension has frequently served as the primary dependent measure in past
comparisons of the Betts' criterion to other criteria (Powell & Dunkeld, 1971; Stanley, 1986).
Precision teaching-based monitoring of oral reading was likely to prove a more sensitive
measure than pretest and posttest measures that are then statistically analyzed. The
previous emphasis on comprehension may have unnecessarily clouded the essential issue
which was basically, which placement criterion provided the best starting point for
instruction when progress in oral reading was the dependent measure? This study needed
to be conducted with students experiencing difficulty in reading and provided with an
individualized program as this was the group to which the findings were most logically
generalized. Neufeld and Lindsley (1980) may have come closest to providing a clear
answer to the criterion question for individualized instruction. Unfortunately, they relied
on a jump in grade levels to move a child into what they hoped would be frustration level
material. What was called for instead was a functional definition based on the percentage
of words correctly read. Further, the test of the frustration level should be markedly
different from the range included by various authors in the instructional reading level. A
study which defines the frustration level 10 or more percentage points below the minimum
criterion found for the instructional level would offer a suitable degree of difference. A
study incorporating these features might provide a convincing demonstration of the merits
of the Betts' criterion when compared to a criterion permitting a greater number of errors.
An assumption has been made by teachers using the Betts' criterion for placement of
children in reading materials that this criterion provides the best possible starting point
for instruction and that the most effective learning takes place when children begin
reading at the level of difficulty suggested by this criterion. The problem addressed by this
study was to compare learning and performance of students in two levels of text: an easy
level (Betts' criteron) and a difficult level (frustration plus). The objective of this study
was to provide an empirical test of this assumption with mildly handicapped students in a
tutorial setting by answering the following question: Will children placed by the Betts'
criterion (95 to 98% accuracy) or the frustration plus criterion (<80% accuracy) make the
greatest gains in oral reading? The following questions constitute components of the main
1. What are the learning rates for correct responding at each level of text difficulty?
2. What are the learning rates for error responding at each level of text difficulty?
3. How does comprehension compare across instructional and frustration plus levels of
The subjects for this study were 16 elementary school children from the P. K. Yonge
Laboratory School. The children were participants in an after-school tutoring program.
The median age of the children was 8 years and 6 months, with a range of from 7 through 10
years. The median grade level was second with a range of from first to fourth. None of the
students were in special education due to learning problems, although all had been
recommended for the tutoring program due to concerns about their failure to make
adequate academic progress. School policy did not permit the traditional special education
classifications to be used. These children could be seen as at-risk for special education
Tutors for this program were special education majors enrolled in an undergraduate
precision teaching course. Experimental procedures were integrated with the general
requirements for the practicum taken as a corequisite to the course. Students were not told
the specific purposes of the research. They were, however, told that a project involving two
distinctly different levels of reading difficulty was underway and that this provided them
with an opportunity to evaluate learner performance after placement under different
initial accuracy conditions. One tutor had a full-time specialist-teaching position at a
private school and worked with a subject in that setting. Except for the setting differences,
other aspects of this subject's instructional program did not differ significantly from that
of the students at the P. K. Yonge School.
Tutoring sessions took place at the P. K. Yonge Laboratory School in Gainesville,
Florida. There were 900 students enrolled in P. K. Yonge, representing all socioeconomic
and racial groups in the Alachua County, Florida school district. Tutoring took place in
an elementary classroom provided for the after-school program. A university supervisor
was on duty during each tutoring session to assist with data collection, coordinate student
efforts, and to help with problems of instruction or behavior management.
Data were gathered during the spring of 1988 over a total of 10 weeks. A school break of
1 week formed a natural break between experiment 1 and experiment 2.
Variables Under Investigation
The level of text difficulty, measured by the percentage of correct words, was the
primary independent variable. Graded reading passages from the Ginn Reading Series
(levels pre-primer through 8) and the Timed Reading Series (Spargo & Williston, 1980) by
Jamestown Publishers (for levels 9 through 12) consisting of 200 word passages for each
grade level were used. The reading level of passages for the Ginn readings and the
Jamestown readings were determined by the Fry Readability Formula (Fry, 1977). The
only modification made to any passage was the selection of a segment of 185 to 200 words.
The primary dependent variables under investigation were the number of words
correctly read per minute (wpm) and the number of error words per minute. Correct
responses were those true to the text. Counted as errors were omissions, substitutions,
repetitions, and mispronunciations. The frequency values for correct and error re-
sponding were charted on a Standard Celeration Chart (DC-9EN, Behavior Research
Company, Kansas City, KS).Learning lines were drawn and the rate of learning
measured (Lindsley, 1971).
Rate of reading performance improvement for correct and error responding was assessed
each instructional session by a 1 minute oral reading timing. The learner was asked to
read two passages. One was at the grade level suggested by the Betts criterion with 95 to 98%
word recognition accuracy. This was called the instructional level placement. The
second passage, designated the frustration plus level was determined by assessment as the
level on which the child made 20% or more word recognition errors. This level of correct
responding was a full 10 percentage points below the range for the frustration reading level
found in the review of the literature. This level was intended to provide a stringent test of
the relative merits of the frustration level when contrasted to the instructional level. A 185
to 200 word passage was provided at each grade level. All passages were transcribed into
identical horizontally oriented formats and then photocopied on 8.5 by 11 in. sheet of paper.
The learner was given one copy during instructional sessions and the tutor monitored the
reading and marked errors on a second, follow along copy (White & Haring, 1980).
Specific procedures for administering the readings are described in the instructional
procedures section of this chapter.
The number of words correctly read and the number of errors made per minute are the
primary dependent measures. The learner was timed on each passage during each
instructional session. Passages were presented in random order. The tutor presented the
child with the passage and gave him the following instructions:
"I want you to read this story as quickly and as accurately as you can. If you don't
know a word say 'skip' and go on to the next word and keep reading."
The tutor set a 1-minute count-down timer and instructed the child to begin reading. At
the end of 1-minute, the tutor recorded the learner's place in the text on the follow along
sheet and asked the child to finish reading the text. When the child finished reading the
passage the tutor recorded the rates of correct and error responding, for the first minute, for
later plotting on a Standard Celeration Chart.
Comprehension was measured after the first reading of a passage and again after the
final reading of the same passage. Comprehension was measured by both the free recall or
"say facts" technique described by Lovitt (1984) and by the "answer questions" technique.
In this latter procedure the tutor read a series of six written comprehension questions to the
child and recorded the child's responses.
An alternating treatments design (Barlow & Hayes, 1979; Johnston & Pennypacker,
1980; Sidman, 1960) was used in this study. This strategy involves the rapid alternation of
two or more treatments with a single subject. The subjects function as their own control,
thereby reducing threats to internal validity that may occur if subject history or maturation
effects are pronounced. The reliance on the single subject eliminates the problems of
intersubject variability, a potent source of difficulty for studies attempting to examine the
unique interaction of a single subject with various interventions.
Rapid alternation of conditions was provided for in the presentation of both levels of the
independent variable during each instructional session. The order of presentation was
randomized to control for any possible order effects. Multiple treatment interference
(Campbell & Stanley, 1963) can be a problem with some applications of this design. This
threat, in the context of this study, would concern the question of how learner performance
under the instructional reading condition in isolation from exposure to the frustration plus
level would differ from the alternating presentation of both conditions. Barlow and Hayes
(1979) minimize the threats to internal validity posed by multiple treatment interference
with this design.
Carryover (Ulman & Sulzer-Azaroff, 1975) is a type of multiple treatment interference
likely to be in operation in a study using an alternating treatments design. Randomized
order of presentation of each condition as well as the ability of students to quickly and
efficiently discriminate the relevant stimulus conditions associated with each condition
are factors which would tend to minimize carryover effects (Barlow & Hayes, 1979).
Further, Barlow and Hersen (1984) suggested that, in studies involving learning, subjects
could readily discriminate stimulus conditions without the addition of artificial signals.
A second experiment, designated experiment 2, permitted a direct replication of the
intervention (Sidman, 1960) with the same subject population. All experimental
conditions remained the same except for the use of new reading materials at levels
determined by a second round of assessment. The number of subjects participating in this
study allow for systematic replications with other change agents, other subject
characteristics, and a test of other levels of instructional difficulty (Barlow & Hersen,
1984; Sidman, 1960).
Prior to the beginning of the intervention tutors assessed each pupil's reading level.
Tutors selected readings from the graded passages used in the intervention phase of this
study. The learner was presented with a passage at their approximate grade level and
received the following instructions from the tutor: "I want you to read this story as quickly
and as accurately as you can. If you don't know a word say 'skip' and go on to the next
word and keep reading." Tutors counted, omissions, substitutions, mispronunciations,
and repetitions as errors by marking the words on an follow along sheet with a slash. Self-
corrections were not counted individually as errors but rather each self-correction episode
was counted as one repetition error. For example, if a child self-correct a four word phrase
only one self-correction, rather than four, was counted. Counting continued for 1 minute at
which time the tutor calculated the number of correct and error words and the accuracy
percentage. Tutors progressed though the passages until they determined both an
instructional and a frustration plus level. Individual assessment of these levels
determined the two levels of the independent variable for each child.
One half hour of instruction was provided three times each week for 10 weeks. The
tutors presented both levels of the independent variable according to a random schedule,
determined by the investigator, during each instructional session. The investigator used
the random number generating function of a Sharp, EL-506 P calculator to determine a
random series for each tutor. These series of random numbers were then used to
determine the order of presentation for each subject for each instructional session. Follow
along sheets were marked accordingly and the two sheets for each session were stapled in
the appropriate order to further prompt the tutors to follow the correct sequence. The
instructional procedures for both levels of difficulty were identical.
For each passage the following procedures were followed. The child was asked to read
the passage, receiving the same instructions used during assessment. They orally read for
1-minute while the tutor recorded all errors and noted the last word read for the timing on a
follow along sheet. The student was then instructed to continue and complete the oral
reading of the passage. The additional reading beyond the 1-minute timed segment
provided the child with exposure to the entire passage, exposure necessary for adequate
measurement of comprehension with the answer questions procedure. The tutor followed a
two-step error correction procedure. First, they pointed to each error word and said the
word, and asked the learner to repeat the word. The tutor repeated this process one
additional time. The second step involved the tutor saying the two words on either side of
the target word along with the target word and then asking the child to repeat this context
phrase. This procedure was repeated one additional time. No other instruction was
provided for this word and the tutor moved to the next error word. Only errors recorded
during this instructional session served as target words. This procedure was designed to
limit the variability in instructional procedures and reduce the chances that tutors might
engage in other forms of instruction such as phonetic or structural analysis of error words.
These procedures were derived from simple and effective techniques recommended for
students with learning problems (Haring, Lovitt, Eaton, & Hansen, 1978; Jenkins &
Larson, 1979; Lovitt, 1977; Lovitt & Hansen, 1976; Rose, McEntire & Dowdy, 1982).
The second major component of the instructional procedure involved repeated reading
and rate pacing (Berquist, 1984). Upon completion of the previous activities the tutor set up a
cassette tape recorder and played a prerecorded tape of the target passage being read
approximately 30 wpm faster than the child's current performance level. Recordings were
designed to provide a good model of oral reading by emphasizing fluency and inflection.
The learner was instructed to read aloud along with the tape. As the child's rate for correct
responding approached within 5 wpm of the recorded rate on the tape, the tutor prepared a
new recording of the passage paced an additional 30 wpm above the child's current rate.
The procedures differed for the first and last timing of each experiment. After the first
and last timing, the child was asked to provide a free-recall of facts from the story as a
comprehension check. This procedure was designated as the say-facts procedure. During
a 30-second timing, the tutor counted all meaningful facts according to the guidelines
provided during tutor training. These procedures are included in this section of the
chapter as the say-facts comprehension check. Additionally, the tutor asked a series of six
prepared comprehension questions. This was the answer questions procedure. These
questions were designed to assess recall of details, vocabulary knowledge, the main idea
of the story, and the child's ability to draw correct inferences from the reading. Tutors
presented each question orally and recorded the child's spoken response. An answer key
accompanied each set of questions. During timings in which comprehension was checked
the error correction and repeated reading procedures were not carried out.
Say-Facts Comprehension Check
Assessment of comprehension by free recall were described by Lovitt (1984). He offered
the following procedure:
Define the characteristics of the passages you want the child to describe. If, for
example, you want her to list facts about the story, then explain what a fact is. Here
is a detailed definition of a fact in this sense:
a. Any comment on the theme of the story counts as a fact (e.g., "Its a story about
b. Each noun plus an action verb counts as two facts (e.g., "The boy ran.").
c. Each adjective or adverb that describes a noun or verb is a fact(e.g., "The red
ball" or "She ran fast.").
d. Each preposition that describes a location is a fact (e.g.,'The ball was on the
The following are not counted as facts:
b. Value judgments about the story (e.g., "I liked the story.").
c. Nonspecific statements (e.g., "She was doing something.").
d. Information about the pictures.
Count the following as incorrect:
a. Statements that go beyond the facts of the story
b. Incorrect information about events or circumstances
c. Incorrect numbers, names, times. (p. 107)
This procedure was used to measure comprehension for the first and last reading of
each level of text for each of the two experiments.
An instructional aim of 200 wpm with one or fewer errors was used in this study. Two
experiments, each lasting three weeks and providing nine instructional days were
scheduled. If a learner reached aim on either reading level they completed the
comprehension assessments on both passages for that experiment. Instruction then
concluded for the passages used in experiment 1. Tutors next determined the reading
grade level of the passages to be used for experiment 2 by means of the same assessment
procedures used for experiment 1. New sets of reading passages, sets which the children
had not previously been exposed to, were used for both determining text levels and for
instruction in experiment 2. As most learners did not reach aim by the end of the first 3-
week experiment, instruction ended for the first set of passages on the ninth instructional
day. The new sets of passages were targeted for instruction beginning on the tenth
Tutors had, as part of their teaching supplies, a count-down timer and Standard
Celeration Charts for taking timings and charting data. They were provided with packets
of reading passages for the child's instructional reading level and for the frustration plus
level. Each passage was between 185 and 200 words in length and presented horizontally
on one sheet of 8.5 by 11 inch paper. A sample reading passage appears as Appendix A. The
structured comprehension questions were provided to the tutors for the first and last
timings of each experiment. Raw data sheets, in the form of scored reading passages, were
collected daily by the university supervisor.
Repeated reading and pacing tapes were prepared by the tutors under the direction of the
investigator. Tutors received instruction in developing their audio tapes from the
investigator. Instruction covered procedures for correct pacing of the reading, proper
inflection, and technical aspects of producing the tapes. The investigator monitored the
tapes periodically for accuracy to the text, proper inflection and voice quality, and reading
rate consistent with the experimental procedures. The investigator listened to each tape
and, guided by a monitoring sheet, examined the first minute of reading. The reading
rate was then determined by referring to the word counts on the reading passages. This
rate was contrasted to the learner's performance as assessed on the first day of instruction
for that passage. Taped readings that differed by more than five words per minute form the
child's first reading were flagged by the investigator for correction by the tutor. As a
child's rate increased to within 5 wpm of the tutors recorded rate the tutor retaped the
passage. The new target rate was an additional 30 wpm faster than the learner's most
recent performance. Each tutor was provided with an individual cassette tape and the
tapes were maintained by the investigator and made available at each instructional
session and for rerecording of reading passages. Eight General Electric 3-5016 cassette
player-recorders were used to record agreement samples and to play the rate pacing tapes.
Data Collection and Analysis
The tutors collected daily performance data for the number of words correctly and
incorrectly read during the 1 minute timings. These data were gathered each
instructional day for both levels of the independent variable. The tutors raw data sheets
included notation of word errors, the last word read, and rate correct and error rate. These
data sheets were collected daily. Tutors also plotted performance data on Standard
Celeration Charts and were encouraged to share this information with their student.
Comprehension data were gathered directly on a sheet of paper containing the
comprehension questions for a given passage. A summary of data gathering instructions
and space to record the say-facts data were also provided on each sheet. A sample
comprehension question sheet is shown as Appendix B.
Rate data for oral reading. Daily performance data for each level of the independent vari-
able were plotted on Standard Celeration Charts by the investigator. Charted data for
reading rates and raw data for rates and comprehension measures are presented as
Appendix C. Celeration lines for corrects and errors were drawn using the quarter-
intersect method described by White and Haring (1980). The celebration values or learning
rates were computed for these lines (Pennypacker, Koenig, & Lindsley, 1972). Graphic
analysis (Parsonson & Baer, 1986) of these data yielded conclusions as to the rate of
learning under each condition of the independent variable
Guidelines for determining the quarter-intersect line were derived from White and
Haring (1980). These steps permit different assessors to draw nearly identical progress
lines from the same set of data plotted on Standard Celeration Charts. Lines are drawn
separately for each condition for corrects and errors.
1. Find the middle data point of the scores (rates) plotted during a phase. To find
the middle data point, do the following: Count the number of rates plotted on the
chart in the phase and draw a vertical line which divides the left half from the
2. Draw another vertical line through the middle data point for the first half of the
data, and a third vertical line through the middle data point for the second half of
the data. In other words, divide each of the two halves in half again, to make a
total of four segments, each with the same number of data points.
3. Find the middle rate (mid-rate) for the first half of the data and draw a
horizontal line through that mid-rate. To find the mid-rate for a set of data, do the
following; Count the number of data plots in the quarter-section and draw a
horizontal line to divide the top half from the bottom half. If the total number of
rates is an odd number, the mid-rate will fall on a rate. If the total number of rates
is an even number, the mid-rate will fall between two rates. If more than one rate
falls on the same horizontal line (i.e., the two rates are of equal value), count each
4. Now, find the mid-rate in the second half of the data. Be sure to draw a
horizontal line through the mid-rate in each half. Make sure your horizontal
lines are level (i.e., parallel to the horizontal lines on the chart).
5. In each half, find where the vertical line which you drew in Step 2 crosses the
horizontal in which you drew in Steps 3 and 4. Draw a little circle around the spot
where the two lines intersect each other. Do this for both halves of your data.
6. Now, draw a line to connect the two intersections of mid-dates and mid-rates
(marked by the little circles you drew in Step 5). At this point, we have just
completed drawing a quarter-intersect line. (pp. 331-335)
Determining the celebration value was accomplished by laying a plastic finder (RD
8546, Experimental Education Unit, University of Washington) on the chart. The finder
was placed parallel to the day lines with the indicator point on the left portion of the quarter-
intersect line. The celebration line was then read at the right edge of the finder at the
intersection of the quarter-intersect line and the values inscribed on the finder.
Increasing or accelerating values were designated by an "X" symbol, while decreasing or
decelerating values were represented by a "" sign.
Celeration values for the performance on the instructional level were compared to
performance on the corresponding frustration plus level for each subject. The results of
these comparisons were then assigned to one of three categories. If the celebration for the
instructional level was greater than that of the frustration plus level, the result was
assigned to an "instructional superior" category. If the frustration plus level had a higher
celebration value the result was assigned to the "frustration plus superior" category. If the
celebration values did not differ by more than 10% (an X .1 celebration the results were
assigned to the "same" category.
The selection of 10% as the amount needed to establish a significant difference was
based on two considerations. First, Beck (1979), in summarizing the work of the Great
r F. ^
31niNIiW I3d iNAOO
S *k C4
eq 8 a 9
c: *.' '0 '0 t *~
~ ) ) 4) 4 4'* ) -'
cc :3 * "i g
Falls Precision Teaching Project, has suggested regular classroom teachers (non-
precision teachers) are capable of producing a 10% change in academic behavior per week.
Lindsley (1985) was in substantial agreement with this figure. Thus, a difference of more
than 10% attributed primarily to placement in reading material of an easier or more
challenging level represents an instructional variable as large as the contribution of the
teacher. The second reason for the selection of the 10% difference concerns the amount of
progress a learner makes over a traditional unit of public school time, the 9-week grading
period. A comparison of learning rates at a X 1.1 and a X 1.2 celebration is presented as
Figure 1. A pupil learning at a rate of X 1.2 (20% per week increase) will have
accomplished more than twice as much (a factor of 2.19) for a targeted skill over the course
of nine weeks when contrasted to learning at the rate of X 1.1 (10% per week). Such a dif-
ference would be seen as a highly meaningful amount of behavior change in the context of
public school instruction. If both students started at 40 words per minute, the 10% per week
student would finish with 86 wpm and the 20% per week student would finish with 206 wpm
at the end of 9 weeks. In addition to the greater amount of behavior change indicated by
such values, the higher celebration values make it feasible for more instruction to be
delivered at higher frequencies. High frequencies have been shown to be positively
correlated with subsequent skill acquisition (Evans, Mercer, & Evans, 1983) and skill
mastery (Evans & Evans, 1985).
Each of the 16 subjects contributed one celebration comparison for corrects and one for
errors for each experiment for a total of 64 comparisons. The results of the comparisons for
experiment 1 and experiment 2 were separately analyzed for correct and error celebrations.
Question 1, regarding the highest rates for correct responding was answered by means of
16 comparisons in experiment 1 and 16 comparisons in experiment 2. The total number of
celebration values assigned to each of the three categories were determined and expressed
as percentages. The same procedures were followed for experiment 2.
Question 2, regarding the celebrations for error responding, was addressed in the same
manner. Each mispronunciation, substitution, omission, skipped word, teacher aided
word, and repetition was counted as an error. The change in error rates over time was
determined according to the same procedures as described for correct responding. A
reduction in errors, the expected direction, was assigned a "7" sign, while an accelerating
trend was given a "X" sign. A "?" directly beneath the record floor of the Standard
Celeration Chart indicates that no errors were observed during the behavior sample for that
Comprehension Data. The third question posed by this investigator concerned the status of
comprehension at the two levels of text difficulty. The say facts procedure yielded
measures of comprehension after the first instructional reading of the text and then again
after the last reading. A rate of facts correct per minute and errors per minute were
determined for both reading levels. The pretest results were compared to posttest results to
determine the level on which the greatest amount of progress had been made. This was
done by dividing the posttest values by the pretest values and multiplying 100 and then
subtracting 100 from the product to obtain the percentage of increase.
The three conditions used with the rate data, instructional superior, frustration plus
superior, and same were used to categorize the results. The results of these comparisons
were then tabulated, the number in each category determined, and the sum for each
category expressed as a percentage of the total number of comparisons. These percentage
values were used to make statements about the comprehension progress under either level
of the independent variable. The ending rates for correct facts, while not comparisons,
were treated in a similar manner.
The answer questions data were treated in essentially the same manner as the say
facts data. Pretest and posttest measures for each subject's performance on each level of
difficulty were compared. Percentages of increase were determined and placed into one of
the three categories. Ending percentage values for comprehension (a value ranging
between 0 and 100%) were compared and the values assigned to one of the three categories
used for the previous classifications.
Other measures. Three sets of values were determined which help to provide additional
understanding of subject performances. Values for median learning rates for corrects
and errors for all learners, median differences between the reading grade level of each
child's instructional and frustration level passage, and last, the typical values for the
move to higher or lower levels of reading passages in the transition between experiment 1
and experiment 2.
The median celebration rates for each level of text difficulty for both errors and corrects
were determined for experiment 1 and experiment 2. These values provided a measure of
the general instructional effectiveness of each level of text difficulty. These values also
permitted statements to be made regarding the overall instructional effectiveness of the
intervention package. The reading grade level difference between the instructional and
frustration plus level passages were determined for each subject for both experiments.
These values were seen as important in determining the stability of the assessment
procedures and in supplying summary data that may be helpful to future researchers and
teachers should they seek to replicate this investigation.
The change in reading level between experiment 1 and experiment 2 for each level of
the independent variable were also determined. The number and percentage of subjects
who decreased in level, remained the same, and who increased in level were calculated for
instructional and frustration conditions. These values were deemed critical as exposure
to frustration level readings should have, based on the prevailing conceptualizations
guiding placement decisions, a deleterious affect on the motivation and degree of reading
effort put forth by children. A learner reading instructionally at the third grade level and
placed in the sixth grade level for the frustration plus condition could be expected to
experience negative attitudes toward reading and when presented with a new set of
assessment passages, perform with less enthusiasm and skill. A significant decrease at
either level between experiment 1 and experiment 2 would suggest that this frustration
effect was in operation. Alternatively, reading levels staying the same or increasing
would provide evidence that frustration was not a significant factor in the instructional
Determining the reliability of the data was accomplished in two ways. The degree to
which the independent variables were consistently and accurately implemented was
assessed and will be discussed in the procedural reliability section. The degree of
agreement between the tutors' measurement of the dependent variable and the same
measurements by an independent observer are reported for each tutor in the interobserver
Procedural Reliability. Procedural reliability is a measure of the degree to which the in-
structional conditions maintained by an investigator matched the conditions described in
the procedures section of an investigation (Peterson, Homer, & Wonderlich, 1982). In
order to determine the extent to which the tutors consistently and accurately carried out the
experimental conditions a monitoring system was implemented. A checklist (appears as
Appendix D) was used to routinely monitor the implementation of the instructional
conditions for each tutor. Deviations from the prescribed instructional program were
brought to the attention of the tutor and corrective action suggested. Additional monitoring,
in the form of daily supervision of each tutor's work, further insured compliance with the
The data collection procedures provided an additional check of procedural reliability.
Tutors provided the primary investigator or a practicum supervisor with the raw data
sheets immediately after each tutoring session. Each data sheet contained the passage
reading and included all of the observer notations of the student's performance. Data
sheets turned in incorrectly or incompletely prompted a rapid request for corrective action
from the investigator or supervisor.
The raw data sheets were used by the investigator as a vehicle which permitted further
assurance of procedural reliability. Tutors were provided with a set of readings for their
learner. The order of daily readings was randomized and the sets were labeled to reflect
the random presentation. Students were asked to follow the order of presentation
determined by the data sheets. Compliance with this requirement was easily monitored by
the on-site supervisors.
Interobserver Agreement. Interobserver agreement was calculated for each tutor. The
raw data sheets contained notations indicating that selected readings should be audio-
taped by the tutor. These audio-taped readings were then scored by an independent
observer who was trained to score the reading passages. This observer recorded the total
number of words correctly read and the total number of words read for the 1 min sample.
Tutor-scorer agreement was calculated for each sample by dividing the smaller number
by the larger and multiplying the quotient by 100. A total of 29 passages were scored to
determine interobserver agreement. The median agreement for all tutors for words read
correctly was 98% (range 91 to 100%). Median agreement for the total number of words
read was 99% (range 87 to 100%). Barlow and Hersen (1984) suggested independent
observers achieve at least 80% agreement. Cooper, Heron and Heward (1987) suggested
permanent product generated data should exhibit higher agreement than simultaneous
observational data. These authors suggested that adequate agreement measures for
permanent products should be in the high 90% range. In this investigation, the tutor's
observation constituted a simultaneous observation while the second observer was able to
rely upon a type of permanent product, in this case the audio tape of the reading. The
agreement values obtained appeared to be adequate for the purposes of this investigation.
The major purpose of this study was to investigate learning rates obtained by
elementary students reading at two levels of text difficulty. Learners received similar
treatments on an Instructional reading level and on a frustration plus level passage in an
alternating treatments design. Progress for correct and error responding was calculated
for each subject for both levels for experiment 1 and for a replication designated
experiment 2. Comprehension was assessed after the first instructional session and again
after the last reading of each passage. Two procedures were used to check comprehension,
a say facts procedure and an answer questions procedure. The researcher sought to explore
the validity of the commonly used reading placement procedure which guides teachers in
placing students in text in which they read 95 to 98% of the words correctly.
The results of the three major questions posed in the study are given first. Next, other
measures of learner performance and progress in reading materials are provided.
Question 1. What are the Learning Rates for Correct Responding
at Different Levels of Text Difficulty?
Celeration values were used to measure the rate of learning for correct responding. A
comparison of celebration values between the instructional and frustration plus levels
permitted placement of results in one of three categories. When the celebration value for the
instructional reading level was higher than that of the frustration plus reading level the
subjects results were assigned to the "instructional superior" category. When the value for
frustration plus reading level was higher, a subject's results were assigned to the
"frustration plus superior" category. When the celebrations did not differ by more than 10%
per week (e. g.,. X1.1 vs. XI1.2), the two values were assigned to the "same" category.
A summary of the comparisons of celebration values for correct responding for
experiment 1 and for experiment 2 appears as Table 2 while a listing of correct celebration
values and the categorization of the comparisons appears as Table 3. For the first
experiment, the instructional reading level did not yield a celebration higher than the
frustration plus reading level for any subject (0%). The frustration plus level was
associated with a higher celebration value for 13 cases or 81% of the total. Three subjects had
celebration values that were essentially the same under either condition (19%). The
instructional reading level was not associated with the most rapid progress for any of the
subjects while the frustration plus reading level, was equal to or better than the
instructional reading level for 100% of the learners.
Comparisons of 16 Celeration Values for Correct Responding at the Instructional Reading
Level and at the Frustration Plus Reading Level
Experiment 1 Experiment 2
Number Percentage Number Percentage
of Cases of Cases
Instructional Superior 0 0.0% 0 0.0%
Frustration Plus 13 81% 13 81%
Same Category 3 19% 3 19%
(difference less than
10% per week)
.flnmnnriqnns nf CePleratlon Values for Conrrect Resnonding at the Instructional Reading
flVC lht LI A U f~ qIt. W .T.? d,- T,~ a.
Experiment 1 -Experiment 2
Subject Cel INS Cel FR+ Result of Cel INS Cel FR+ Result of
Cel INS = Celeration Value for Instructional Reading Level
Cel FR+ = Celeration Value for Frustration Plus Reading Level
Note: The reading level with the higher celebration value is designated by its abbreviation.
Same indicates a difference of less than 10% per week growth in correct responding.
The results for experiment 2 are similar. In no case did a subject achieve a higher rate
of learning in the instructional reading level condition when compared to the frustration
plus level. Thirteen subjects (81%) had celebration values for the frustration plus condition
that were higher than that of the instructional reading level. For three subjects (19%) there
was no difference in rate of learning on the two levels of text. For experiment 1 and
experiment 2, all subjects made gains in reading at the frustration plus level equal to or
greater than those made in the instructional reading level.
Question 2. What are the Learning Rates for Error Resoonding
at Different Levels of Text Difficulty?
Table 4 provides comparisons of the celebration values for subjects at the instructional
and frustration plus levels of passage difficulty. Table 5 provides a summary of
comparisons of learning rates on the two levels of the independent variable. The results
for experiment 1 show that four subjects (25%) had more rapid reductions for errors in the
instructional reading level condition when contrasted to the frustration plus reading
level. Eleven subjects or, 69% of the total, achieved faster reductions in errors, as
measured by their celebration values, when working with the frustration plus materials.
One subject (6%) had deceleration values that were essentially the same under both
conditions. In total, 75% of the subjects had celebration values indicating either a more
rapid or equal reduction of errors when working with the frustration plus condition when
contrasted to the instructional reading level condition. In experiment 2 five subjects (31%)
had more rapid reduction of errors in the instructional reading level condition, while nine
(56%) achieved more rapid reductions in the frustration plus condition. Two subjects (13%)
did equally well in reducing errors under both conditions. For experiment 2, 68% of the
subjects had celebration values for error reduction that equalled or surpassed the celebrations
of the instructional condition. The results of neither experiment support the contention that
the instructional reading level provides a starting point for reading instruction likely to be
Comparisons of Celeration Values for Error Responding at the Instructional Reading
Level and the Frustration Plus Reading Level
Experiment 1 Experiment 2
Subject Cel INS Cel FR+ Result of Cel INS Cel FR+ Result of
Cel INS = Celeration Value for Instructional Reading Level
Cel FR+ = Celeration Value for Frustration Plus Reading Level
Note: Note: The reading level with the higher celebration value is designated by its
abbreviation. Same indicates a difference of less than 10% per week in error reduction.
the most productive in terms of error reduction. Rather, the frustration plus condition was,
for the majority of subjects, the level on which they achieved the greatest error reduction
Comparisons of 16 Celeration Values for Error Resnonding at the Instructional Reading
Level and at the Frustration Plus Reading Level
Experiment 1 Experiment 2
Number Percentage Number Percentage
of Cases of Cases
Instructional Superior 4 25% 5 31%
Frustration Plus 11 69% 9 56%
Same Category 1 6% 2 13%
(difference less than
10% per week)
While a majority of the cases were classified in the frustration plus superior category,
the results are not as definitive as were the data for the correct celebrations. Several factors
would appear to account for this difference. First, learners were prompted to read as
quickly and accurately as possible. In the frustration plus condition, with the much higher
initial error rates (by definition at least 20% of the words on the assessment trial), the child
reading faster was also likely to make more errors. As both the raw and charted data
show, the rate at which the learner read the plus passage increased steadily throughout the
experiment for most students. While the accuracy ratio (words correct to error words)
might have remained the same, the number of errors would increase along with the in-
creased rate for corrects. Typically the ratio of errors to corrects remained constant for the
early portions of the experimental phase (e. g., 15 corrects to 1 error), increasing as the
learner became more fluent with the newly learned words (e. g., 77 corrects to 1 error). A
feature of the experimental design appears to have influenced the data in this regard.
Upon reaching aim in one reading level, instruction for both levels was concluded. The
yoked design prevented the collection for performance data of the level which had not yet
reached aim, in all cases the frustration plus level. The substantial reductions in errors
found after 5 or 6 days of instruction did not have an opportunity to come into play for the
frustration plus level. This occurred in three cases cases in experiment 2. In each of these
cases the celebration for errors was higher for the instructional reading level.
Question 3. How Does Comprehension Compare Across
Widely Different Levels of Text Difficulty?
An ancillary question addressed by this investigation was a comparison of
comprehension on the instructional and frustration plus levels of difficulty.
Comprehension was assessed twice for each passage; once after the first instructional
reading of the text and then after the final reading of the passage. Two measures of com-
prehension were used for each passage. First, the say facts procedure was used followed by
six, tutor-read comprehension questions comprising the answer questions procedure. It
should be noted that no systematic instruction in comprehension was provided for any
passage on either level of text difficulty.
The results of say facts gains for both levels of text difficulty are presented in Table 6
and a summary of these comparisons appears as Table 7. In experiment 1, 11 subjects
(79%) had greater percentage increases in facts correctly stated on the frustration plus
level while only three subjects (21%) had greater increases on the instructional reading
level. The median number of facts gained was eight for the frustration plus level (range -6
to 18) and two for the instructional reading level (range -4 to 18). For experiment 2, 13
-I2 + + +3 + + +++ I -+
U. U) . . .. . .
Z W Z 19 E9E4Ef P4 P4& 9 P4 0 0 V
r.4 cq r4 -4
++ + + +.+ + + + + UO + CO 0
4. 44 4 0 4 r4 O 4 44 44 44 4 0 -4 4 "4 0 .4
|jC g L 9 g 9a eC? i 61 - j- -
13 S ~ ^^ -V kor.4
* ecv e^
I) 0 ^i oM
Q oW E- =E40 E4M E- Z E-4
~c i +
,p g W tp E g
o' 00 M 1
subjects had higher increases on the frustration plus level (87%), while 2 (13%) had higher
increases on the instructional level. The median increase for the instructional reading
level was 2 facts (range -6 to 26) and the median increase for the frustration plus level was
10 (range 0-30). The higher percentage increases are associated with the frustration plus
condition. The absolute number of correct facts per minute was higher in the instructional
reading level for all but four cases. Table 8 presents the comparisons of ending facts per
minute values(final say facts comprehension rates) for each level of text difficulty while
Table 9 shows a summary of the comparisons.
Comparisons of Comprehension Say Facts Percentage of Improvement Values for
Instructional Reading Level and Frustration Plus Reading Level for Experiment 1
and Experiment 2
Experiment 1 Experiment 2
Number of Cases Percentage Number of Cases Percentage
Instructional Superior 3 21% 2 13%
Frustration Plus 11 79% 13 87%
Same Category 0 0.0% 0 0.0%
The say facts error rates for both levels of text difficulty in experiment 1 and
experiment 2 were extremely low. A median value of 0 fact errors per minute was
calculated with the range for all subjects being 0 to 6.
9) C40 M +U2 mm mm + +m )3
0-4 rX4 0-- o t" 2 -4
+ + + C12 CI0J03
gp552zS 5zzo 5zzz
------ P- .- .- .-r )"P -
W-O E-- = r ;: 950 Q Z M E- -( !Z; 9E--
QoE--4 WE-iqE.4 D W w6-4 z SUQ!3
While the learners were able to generate fewer fact statements from the passage at the
frustration plus reading level when contrasted to the instructional reading level they were
likely to have shown a greater amount of improvement on the more difficult material.
Clearly, the initial complexity of the frustration plus passage, as inferred from the high
word recognition error rates, did not prevent learners from being able to make rapid gains
in their ability to generate factual statements about the text.
Results of Comparisons of Comprehension Say Facts Ending Performance Values for
Instructional Reading Level and Frustration Plus Reading Level for Experiment 1 and
Experiment 1 Experiment 2
Number Percentage Number Percentage
of Cases of Cases
Instructional Superior 12 75% 11 69%
Frustration Plus 3 19% 3 19%
Same Category 1 6% 2 13%
Comprehension was also assessed by presenting students with six comprehension
questions. One subject was excluded from this portion of the experiment. Her reading
rates on the lowest level passages were so low as to rule out routine reading of the full 185-
200 word passage as would be required for comprehension assessment. Percentage of
improvement scores were calculated for each subject for both experiments and appear as
a) + 03) ) 4) . . . . .
1CE z E Ef;~feO;0O
.-4~ - OC OO O< DLOktO O
o mz t-i t- m t-
+ +*n + +
04 g~t C- 4P
r.. fi ET -E. .
^ ^D ^4 4 ^ z m F4 Z g 0 4
E-1 E. W W 1- rz4 X Q00 c)
Table 10. Table 11 provides a summary of the results for this analysis. For experiment 1,
three subjects (21%) made greater percentage gains on the instructional reading level
passage while eight subjects (57%) made greater gains on the frustration plus level pas-
sage. Three students (21%) made equal gains on both passages. In experiment 2, 1 student
(7%) made greater gains on the instructional reading level while 11 (73%) made greater
improvement on the frustration plus passage. Three students (20%) did equally well on
Comparisons of Comnrehension Questions Percentage of Improvement Values at the
Instructional Reading Level and at the Frustration Plus Reading Level
Experiment 1 Experiment 2
Number Percentage Number Percentage
of Cases of Cases
Instructional Superior 3 21% 1 7%
Frustration Plus 8 57% 11 73%
Same Category 3 21% 3 20%
Table 12 shows a comparison of ending comprehension values for both levels of text
difficulty for experiment 1 and experiment 2. Table 13 provides a summary of the
comparisons. Nine students (60%) had higher ending comprehension values on the
instructional level material while only 1 (7%) had a higher ending value at the frustration
M3 SM0M0MCOQ + c) w + co
z gz^zzzzz5z z
0i- P 0.44 co 0-4 T -
20- -. 0
UOUW E-Q= 29E4ZCME- Z!
4: (7E4 WH E, WW wH rZ4;: ^ U::)
0 r2 0
0 ~- CI
-4> 03 '+ 0S
plus level. Five students (33%) had the same ending comprehension scores on both levels.
Similar findings were obtained for experiment 2 with 10 students (67%) scoring higher on
the instructional passage, 2 (13%) scoring higher on the frustration passage, and 3 (20%)
achieving equally well on either level.
While the learners' abilities to comprehend material, as measured by standard
comprehension questions, was inferior at the frustration plus level passage, the potential
for progress and learning
Comparisons of Comprehension Questions Ending Percentage Values at the Instructional
Reading Level and at the Frustration Plus Reading Level
Experiment 1 Experiment 2
Number Percentage Number Percentage
of Cases of Cases
Instructional Superior 9 60% 10 67%
Frustration Plus 1 7% 2 13%
Same Category 5 33% 3 20%
was as great or greater than that offered by the instructional reading level passage. Again,
caution should be exercised in examining these results as no systematic instruction was
provided to enable the learner to improve comprehension. As a possible consequence,
comprehension was not complete ( students did not answer all comprehension questions
correctly) after as many as 18 readings of a passage. That learners reading at the
frustration plus level of difficulty did not lose ground and, in fact, made greater gains in
their ability to answer comprehension questions when compared to the instructional
reading level material, suggested that very challenging material did not prevent the
learner from developing comprehension, even when no instruction was provided for
Two additional measures readily available from the data may prove helpful for a full
understanding of the main findings and may be supportive of conclusions made as a
result of this investigation. Included among the additional results are the differences
between the instructional and frustration plus reading grade levels between experiment 1
and experiment 2 and an analysis of the reading grade level selected during assessment
by each of the tutors.
The differences between the reading grade for the instructional level passage and the
frustration Plus passage are presented in Table 14. For experiment 1, the median
difference between the reading levels was two reading grade levels (range one to seven
grades). For experiment 2 the median difference was 2.5 grades (range one to eight
Closely related to these results are comparisons of the reading grade level within each
condition between experiment 1 and experiment 2. For example, a student working at the
instructional level on a second grade passage for experiment 1 and a fourth grade passage
for the instructional level for experiment 2 would have a difference of two reading grade
levels. The reading grades levels were determined by assessment just prior to instruction
in experiment 1 and again prior to instruction in experiment 2. The status of these levels
can be placed in one of three categories. First, the child could have stayed at the same level
for both experiments, they could have increased in level from experiment 1 to experiment 2
or they could have similarly decreased. These comparisons appear as Table 15. The
c 4f c4> ci 4- vif P4 o6 c4c ooIN c~i 6- 4n*
C OO C OOOOOOOOO C R RO RO RO R COOO
4 v V I m ul e t- q 1- V4
P W (YE-aW E4 E4 W WE-4 r, 20::
cl C6 14; 00
4) 4) a)
U2 g 2
Q) 4) 4)
to -I N Lo v 1 m m Dw m
UOOWM- :29OQZPME-4Z E6-'
Q (7 1- W 1-4E-4 :D W WE-4ON21) ;
UC4 I CC
P. 0., 0. a. "I C'
CL4eQ 4 Q o c
0 0 0 0 V V 0-4 0N M V-4 r-4N' Ol
results of these comparisons (appearing as Table 16) show that, for the instructional
reading level six subjects worked at the same level (38%), while 9 (56%) increased to higher
reading grade level passages. One subject (6%) worked at a lower level and would be
classified as decreasing. For the frustration plus level 2 subjects worked a the same level
(13%), 1 decreased (7%), and 13 increased (81%).
The general trend toward a higher level can be explained, in part, by the fact that the
students had greater familiarity with the materials, the instructional format, and their
tutors. Initial nervousness may have caused some children to perform less well than they
would have otherwise. Of relevance to the purposes of this study was the finding that
motivation and effort, as measured by the reading performances on the second
assessment, were not impaired. The students' reading
Comparisons of Reading Grade Placements Within the Same Instructional Condition
Between Experiment 1 and Experiment 2
Instructional Frustration Plus
Reading Level Reading Level
Number Percentage Number Percentage
of Cases of Cases
Reading Grade Level 9 56% 13 81%
Reading Grade Level 1 6% 1 6%
Reading Grade Level 6 38% 2 13%
Remains the Same
showed no evidence of being less skillful or less motivated. Exposure to the frustration
plus level readings, material with a level of difficulty hypothesized to be far too great to
permit success, did not produce a reduction in effort or precipitate a loss of motivation on
the part of the students that might have resulted in poor performance.
Summary of Results
The answers to the three major questions of this study as well as additional measures
have been detailed in this chapter. The first question was: What are the learning rates for
correct responding at different levels of text difficulty? The answer was that the
frustration plus level of difficulty was associated with higher rates of learning or with
rates that equalled that of the instructional reading level for 100% of the students in
experiment 1 and for 100% of the students in experiment 2. The second question was: What
are the learning rates for error responding at different levels of text difficulty? The
answer to this question again favors the frustration plus reading level. In experiment 1,
75% of the students had error learning rates on the frustration plus passages that either
equalled or surpassed those of the instructional reading level. For experiment 2 similar
results were obtained with 68% of the learners doing better or as well on the frustration plus
passage when compared to the instructional passage.
The third question was: How does comprehension compare across widely different
levels of text difficulty? In both experiment 1 and experiment 2 increases in
comprehension were greatest for the frustration plus reading level. Comprehension gains,
as measured by both the say facts procedure and an answer questions procedure, favored
the more difficult reading condition. Both measurement procedures demonstrated,
however, that comprehension, for either the first reading of the story or the last, was clearly
superior, in absolute terms, for the instructional reading level.
Additional summary data were presented on the reading grade levels for both
conditions and for both experiments. The median difference between the instructional
reading level and the frustration plus reading level was two grades for experiment 1 and
2.5 grades for experiment 2. In experiment 2, 56% of the subjects worked at a higher grade
level of material in the instructional level condition. For the frustration plus condition,
88% of the children worked on more difficult material. These data are offered as evidence
that motivation and effort were not deleteriously effected by exposure to the frustration plus
level reading material. It was quite possible that the challenge presented by the frustration
plus level materials actually motivated students to put forth a greater effort.
In general, the more difficult reading material was associated with higher rates of
learning for corrects, errors, and for growth in comprehension. Initial performance on
the second experiment and the high rates of learning for both levels of text in experiment 1
and experiment 2 suggested that motivation and effort were not harmed by exposure to the
more challenging reading material. All of these results seem to be relatively stable as
demonstrated by the similarities between the results of experiment 1 and experiment 2.
DISCUSSION AND CONCLUSIONS
In this chapter the results of the study are considered in light of reading placement and
instructional practices. Conclusions based on the data are offered. These conclusions
then served as the foundation for specific recommendations. Four recommendations for
improving the effectiveness of reading instruction are given. While most of the
recommendations related directly to instructional arrangements for special students or
learners receiving individualized instruction, they may have applicability to a wide
range of students receiving reading instruction. As mentioned previously, the Betts'
placement criterion was believed to promote maximal learning for the largest number of
students. The findings of this study failed to support this contention for students with mild
learning problems who are receiving individualized reading instruction.
Methodological Progress in Reading Research
As results of this study provided evidence contradicting previous findings, several
beliefs about the ability of learners to improve their reading performance under various
instructional conditions should be questioned. A traditional reliance on group statistical
research may have interfered with efforts to assess individual responses to large
differences in the difficulty of reading material. The early works of Betts (1946) and
Killgallon (1941) are examples of the reliance on indirect measures of reading ability to
determine a placement criterion. Killgallon, as Betts' student, administered IRIs to a
group of pupils and then determined their performance on the comprehension portion of the
IRI. Killgallon next applied an arbitrary comprehension criterion of 75% correct to the
data and selected only the corresponding reading scores meeting the criterion. The mean
of those scores was 93.9% which was later rounded off to yield the widely used 95%
placement criterion. Progress in oral reading was not a consideration. With this early
placement criterion research, as with more recent researchers efforts to determine the best
starting point for reading instruction (Powell, 1984; Kragler, 1986; Stanley, 1986), the
suggested placement guidelines were based not on progress in oral reading but rather on
predictions as to which level of oral reading accuracy predicted what level of
comprehension. It is possible to accurately measure progress in oral reading by means (e.
g., precision teaching) not available to early researchers (Starlin, 1971). An historical in-
ability to separate oral reading performance from comprehension skill may have
unnecessarily complicated research efforts seeking to determine the most fruitful starting
points for reading instruction (Starlin, 1971). While comprehension is a critically
important component of reading, it may not be the most reliable dependent measure in
experimental efforts to find the best starting point for instruction ( Killgallon, 1941;
Kragler, 1986; Powell, 1984; Stanley, 1986). The few published studies that used rate for
correct and error responding, most notably Johnson, (1971) and Neufeld and Lindsley
(1980), have arrived at conclusions similar to those offered by this investigator.
The first and clearest conclusion derived from this investigation was that the
instructional reading level, with initial word recognition accuracy of from 95 to 98%
correct, was not the most productive starting point for instruction for most learners studied.
In experiment 1, none of the learners made most rapid progress for correct responding
when this placement criterion was applied. The results of experiment 2, in which no
subject had better growth on the instructional level, provided additional confirmation.
Error reduction data lead the researcher to suggest a similar conclusion. The data failed
to support the efficacy of the Betts' instructional level for students with mild learning
problems receiving individualized reading instruction.
Factors Suopporting Continued Use of the Betts' Criterion
Why, then, is this traditional placement criterion so widely used and recommended
even for special learners receiving individualized instruction (Cook & Earlley, 1979;
Hargis, 1982; Mercer & Mercer, 1981; Savage & Mooney, 1979). Speculation on this point
would be premature except to point out that by definition, as the findings of this investiga-
tion verify, low error rates on an assessment passage were clearly associated with low
error rates during subsequent instruction on the same passage. If low errors, either
initially, concurrently, or terminally are to be used as the sole measure of instructional
effectiveness, then placement of a learner by the traditional instructional criterion
assures success. Reading teachers using traditional techniques do not appear to have had
a measurement technology capable of determining improvement or learning (Starlin,
1971). Therefore, low-error performances may have substituted for a measure of progress
or learning for some teachers. Unfortunately, progress on reading and the development of
capable readers was not necessarily fostered by such an unchallenging level of material
Perhaps the key feature in answer to why the instruction level placement continues to
enjoy such currency relates directly to its origins. Betts, Killgallon and other investi-
gators of that era were concerned almost exclusively with large group reading instruction.
Large group instruction with one teacher may call for reading materials that children can
read with a minimal level of assistance. Low errors permitted the teacher to manage the
instruction most readily. This may have been a legitimate use by the teacher of a
placement criterion and a tradition of assigning children to reading materials. But, the
assumption that this placement level offers the greatest growth potential has been seriously
challenged by the results of this investigation.
Large group instruction continued to give way to more sophisticated, smaller group ar-
rangements. Special learners served under the provisions of Public Law 94-142 were
guaranteed an individualized educational program and, for the majority of these
children, reading instruction is to be delivered as part of an individualized educational
program. These types of arrangements call for reading instruction to be individualized so
as to permit maximum progress based on the assessed needs of the child. A placement
criterion designed to facilitate large group instruction must surely be reconsidered and, in
light of these findings, challenged.
Potential Utility of High-Error Reading Material in Individualized
A second issue raised by this investigation concerned the value of the frustration level.
This level, typically 90% or less initial word recognition accuracy on an assessment
passage, was considered to be so difficult and so likely to produce negative emotional reac-
tions that teachers were cautioned not to place children in frustration level readings. It is
important to recall that the frustration level used in this study provided for at least 20%
errors. Interestingly, while the instructional and independent levels both have names
suggesting instructional arrangements the frustration plus level alone bears a name
referring to a hypothetical mental state. The label itself tends to bias teachers against
placing students in more difficult reading material. Frustration reading level has
become, as Cadenhead (1987) has suggested regarding the general use of the term "reading
level," a powerful metaphor that shapes practice.
The results of this study quite clearly demonstrate that, when provided with an
instructional program similar to those recommended in the literature as appropriate for
special education students, the frustration plus level may be the more productive starting
point for instruction. In both experiment 1 and 2, 100% of the learners made greater or
equal progress on the frustration plus level when contrasted to the instructional Level. The
results for error responding on both experiments yielded similar, though not as conclusive,
findings. The frustration plus level was associated with higher rates of learning for
corrects and errors. While the absolute level of comprehension is certainly worse under
the frustration plus condition, the rate of growth for comprehension as measured by either a
say facts or answer questions procedure was better under the frustration plus level. While
no systematic instruction in comprehension was provided during this study, the findings
suggest to the investigator that a systematic program of comprehension instruction at the
frustration plus level would produce comprehension gains surpassing those made at the
instructional level. A child's comprehension may have started much lower on the
frustration plus level, but it typically grew more rapidly and the learner reached terminal
proficiencies similar to th )se obtained on the instructional level material.
Comprehension learning was not adversely affected under the frustration plus condition
and this was a finding of some importance.
In addition to the superior rates of learning under the frustration plus condition, the
overall lack of the inferred frustration reactions was significant. It was noted by the tutors
that some learners balked at the frustration plus reading task during the first few days of
experiment 1. Most of these incidents were no greater in intensity than had been
reluctance to engage in other, nonpreferred instructional activities such as spelling or
handwriting. Many of the tutors reported that their learner preferred the more challenging
material and looked forward to "beating" the rate of the audio-taped reading passage.
Significant noncompliance was not noted for experiment 2. No unusual behaviors that
would have indicated educationally relevant frustration were noted. The high rates of
learner progress for both levels of the independent variables are consistent with the
learning accomplished by effectively motivated students (Starlin, 1971) and not at all
characteristic of learners who are frustrated. Similarly, the trend observed in which
students moved to higher reading levels for both instructional and frustration plus levels
for experiment 2 suggest that the learners were eager and motivated to improve their
performance using even more challenging material. The findings reported by this
investigator contradicted the long-held but poorly supported assumptions about reading
behavior in challenging material.
A Simple and Effective Set of Instructional Procedures
While not one of the formal questions posed by this investigation, the success of a
relatively simple package of instructional procedures was demonstrated. The median
rate of learning for correct responding was X 1.53 and an error value of/1.78 was
obtained. This represented a 53% improvement per week for corrects and a 78% reduction
for errors across all students and all levels of reading passages. Public school learning, to
put these values in perspective, was estimated by Beck (1979) to yield a 10% average change
per week. The use of two simple error correction procedures and allowing the child to read
along with a tutor-prepared audio tape paced 30 words per minute faster than the learner's
performance constituted the entire instructional sequence. These procedures were
intentionally kept simple for the purposes of experimental uniformity but their potential
utility in classrooms appears to be high. These procedures may derive a great deal of their
effectiveness from the emphasis on frequent, high-rate responding by the learner.
Limitations of the Study and Recommendations for Future Research
A limitation described in the results section of this study was imposed by the nature of
the yoked design. As a learner reached aim on any level of text difficulty, instruction
stopped for both levels. This avoided the problem of having the child prompted to read at
potentially unfunctionally high rates on one passage. This yoking failed to capture the
anticipated performance improvements at the reading level which did not reach the
performance aim, in all cases the frustration plus level. Allowing the tutors to continue
instruction on the frustration plus level passage until the end of the prescribed nine
instructional day period could have alleviated this limitation and may have made the
results of this study even more clearly in favor of the frustration plus strategy.
Another limitation was more methodological and less procedural. Limits on response
opportunities in the form of both ceiling and floor effects (Haughton, 1972) must be taken
into consideration. A child who was making very few errors had less opportunity to
achieve a substantial reduction in errors. Similarly, a reader at initially high correct
rates may not have been able to accelerate their performance as dramatically as the
learner who began at a much lower rate. Fortunately, an analysis of these data suggested
that even those learners who began instruction at higher correct rates made gains
comparable to lower rate learners on similar levels of passage difficulty. Error rates
posed a special problem in that the floor effect was more likely to have been encountered by
the learner on the instructional level passage. The charted data provided an excellent
graphic display of this potential problem. Typically error rates for the instructional level
were not reduced to '0' (record floor) until, if at all, the last few instructional days of the
experimental phase. With no opportunity to achieve further reductions in errors the
celebration values would flatten out. Had such a floor been reached earlier, as might be the
case with sophisticated readers or learners placed in very easy material, the resulting
celebration comparisons may not have been appropriate.
The use of an abbreviated version of the Betts' criterion was also a limitation. This
study used only the word recognition accuracy portion of the criterion and excluded the
comprehension component. This was done to facilitate placement in materials and to
avoid time-consuming comprehension assessment of all relevant reading levels. The
summary data on comprehension indicates that an approximation of the Betts'
comprehension criterion was incidentally accomplished for the instructional as well as
the traditional frustration level. A replication of this study using both components of the
Betts' criterion appears to be warranted although it seems that the addition of the
comprehension criterion would have exaggerated the present findings and have more
clearly demonstrated the potential of the frustration level material. Using the
comprehension component of the Betts' criterion would have moved some students to lower
error (easier) readings at the instructional level while having had no effect on placement
for the frustration plus level.
Further research is warranted into the questions posed and raised by this study. A
question not addressed was the best initial accuracy placement criterion. Parametric
studies comparing learner progress at various accuracies, such as 75, 85, or 90% would help
to clarify this issue. Also deserving further study is the question of whether individual
learner characteristics are better addressed by a higher or lower level of material
difficulty. This question, to be answered with any validity, would have to be carefully
structured to prevent the learner's subjective feelings about the material from determining
the course of the instructional program. What learners may say about the work would have
to be far less important than how they performed.
The results and conclusions of this investigation lead to a series of recommendations
for improving reading instruction for elementary school age children taught on a one-to-
one basis. These four recommendations are presented in order of priority.
1. The practices associated with using the frustration level as a designation for
reading material with 90% or less initial word recognition accuracy should be scrutinized.
This researcher has shown that children permitted to read material much more
challenging than the 90% accuracy level can, for most learners studied, hope to enjoy
greater progress than that which would occur with materials guided by the instructional
level criterion. The combination of challenging materials and effective instruction
appeared to offer the greatest likelihood of rapid progress in reading. The entrenched use
of the term "frustration level" by its very nature implies an undesirable, unteachable, and
unprofessional range of difficulty in which to provide instruction.
The term 'frustration' should be replaced when referring to reading material with a
high potential for instructional success. Powell has suggested the "emergent level" as the
region in which, with appropriate mediation and instructional support, the child will make
greatest progress. Powell's criterion, however, is too closely tied to comprehension and is
not based on progress-sensitive, performance data. A term specifying the necessary
instructional arrangements would be most desirable and would be consistent with the
nomenclature for the independent and instructional levels. The "tutorial level" or
perhaps the "individualized reading level" are each worth consideration. Alternatively, a
designation specifying the instructional objective of the reading plan is recommended.
Terms such as "error reduction level" or "fluency building level" are offered for
consideration. A clear statement that the challenge range found to be most productive in
this investigation may be an appropriate and, in fact, a desirable range in which to provide
instruction is called for.
With a change in terminology, a change in both placement and instructional practices
should be considered. Rather than placing learners in material in which they seldom
make errors, the students could, based on the data-guided judgment of the teacher, be placed
in material with far greater potential for progress. Progress would be determined by
increases in correct responding and decreases in error responding over a given period of
2. Instructional programs designed to provide instruction consistent with the view that
the 95 to 98% initial accuracy level, the instructional level, yields the greatest potential for
reading growth should be reassessed in light of the findings of this and other studies
(Johnson, 1971; Kragler, 1986; Neufeld & Lindsley, 1980; Stanley, 1986). Such
reassessment would appear to be especially necessary for special education programs and
other instructional arrangements in which the teacher is obligated to offer individualized
educational programs. To continue to maintain that teaching and placement practices
consistent with the Betts' criterion or some low-error variant are likely to yield the greatest
learning, in light of these findings, seems unrealistic. At worst, such continued support
for the Betts-type criterion by authors of reading textbooks and by special education
researchers suggests a disregard for pupil progress and a mechanical adherence to
3. An instructional program similar to that used in this study warrants consideration
for more widespread adoption for use in tutoring, special education resource rooms, and in
general education classrooms. The essential components of the instructional program
including repeated readings with paced audio tapes, simple error correction procedures in
which the tutor tells the learner how to say error words, and then placing the error word in
context for two practice trials, are easily replicated with little cost or effort. This set of
procedures avoids possibly unnecessary drill in structural or phonetic analysis and
appears to maintain effective instructional momentum. As Lovitt (1977) has suggested, it
is best to use the simplest instruction such as telling the child the correct word, before
resorting to more complicated strategies.
4. The last recommendation is that reading instructional programs as well as reading
research should begin to make use of direct and continuous measures of learner
performance. This would permit more accurate appraisal of individual differences in
learning and allow the performance of learners to suggest to teachers the most appropriate
instructional decisions. In regard to research, past reliance on reading comprehension as
the dependent measure for reading program effectiveness appears to have clouded the
nature of individual performance gains in reading. Direct and repeated measures of oral
reading performance may provide the best measure of gains in oral reading skill.
Algozzine, B., & Maheady, L. (1986). When all else fails, teach! Exceptional Children. 92.,
Allington, R. L. (1979). Teacher ability in recording oral reading performances.
Academic Therapy, 14, 187-192.
Allington, R. L. (1983). Fluency: The neglected goal. The Reading Teacher. 3, 556-561.
Anderson, B. (1981). The missing ingredient: Fluent oral reading. The Elementary
School Journal. 81,173-177.
Barlow, D. H., & Hayes, S. C. (1979). Alternating treatments design: One strategy for
comparing the effects of two treatments in a single subject. Journal of Applied
Behavior Analysis, 12,199-210.
Barlow, D. H., & Hersen, M. (1984). Single case experimental designs: Strategies for
studying behavior change. New York: Pergamon.
Beck, R. (1979). Precision teaching project: Sacaiawea plan II. Great Falls, Montana:
Great Falls Public Schools.
Bennett, R. E. (1983). Research and evaluation for special education assessment.
Exceptional Children. 50, 110-117.
Bergquist, L. (1984). Rapid silent reading: Techniques for improving rate in
intermediate grades. The Reading Teacher, 38, 50-53.
Betts, E.A. (1941). Reading problems at the intermediate grade level. Elementary School
Journal, 49, 737-746.
Betts, E. A. (1946). Foundations of reading instruction. New York: American Book.
Biemiller, A. (1978). Relationships between oral reading rates for letters, words, and
simple text in the development of reading achievement. Reading Research
Quarterly, 13, 223-253.
Cadenhead, K. (1987). Reading level: A metaphor that shapes practice. Phi Delta Kappan,
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs
for research. In N. L. Gage (Ed.), Handbook of research on teaching (pp. 171-246).
Carver, R. P. (1982). Optimal rate of reading prose. Reading Research Quarterly. 28, 56-
Carver, R. P. (1983). Is reading rate constant or flexible? Reading Research Quarterly,
Carver, R. P. (1987). Teaching rapid reading in the intermediate grades: Helpful or
harmful? Reading Research and Instruction, 26, 65-76.
Chomsky, C. (1976). After decoding: What? Language Arts, 53, 288-296.
Cook, J. E., & Earlley, E. C. (1979). Remediating reading disabilities. Germantown,
MD: Aspen Systems.
Cooper, J. C., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis.
Columbus, OH: Charles E. Merrill.
Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative.
Exceptional Children, 2, 219-232.
Dowhower, S. L. (1987). Effects of repeated reading on second grade transitional reader
fluency and comprehension. Reading Research Quarterly, 22, 389-406.
Durkin, D. D. (1978-1979). What classroom observation reveals about reading
comprehension. Reading Research Quarterly, 14, 481-533.
Ekwall, E. K. (1986). Ekwall reading inventory (2nd ed.). Boston: Allyn and Bacon.
Evans, S., & Evans, W. (1983). Frequencies that ensure competency. Journal of Precision
Teaching, 6, 25-30.
Evans, S., Mercer, C., & Evans, W. (1983). The relationship of frequency to subsequent
skill acquisition. Journal of Precision Teaching, 4, 28-35.
Fleisher, L. S., Jenkins, J. R., & Pany, D. (1979). Effects on poor readers' comprehension
of training in rapid decoding. Reading Research Quarterly, 15, 30-48.
Fry, E. (1977). Elementary reading instruction. New York: McGraw-Hill.
Gates, A. I. (1962). The word recognition ability and the reading vocabulary of second and
third grade children. Reading Teacher, 15, 443-448.
Gentile, L. M., & McMillan, M. M. (1987). Stress and reading difficulties. Newark, DE:
International Reading Association.
Ginn Reading Program. (1984-1982). Lexington, MA: Ginn.
Hargis, C. H. (1982). Teaching reading to handicapped children. Denver, CO: Love.
Haring, N. G., Levitt, T. C., Eaton, M. D., & Hansen, C. L. (1978). The fourth R:
Research in the classroom. Columbus, OH: Charles E. Merrill.
Harris, A. J., & Sipay, E. R. (1980). How to increase reading ability (7th ed.). New York:
Haughton, E. (1972). Aims-growing and sharing. In J. B. Jordon & L. S. Robbins (Eds.),
Let's try doing something else kind of thing (pp. 20-39). Arlington, VA: The
council for Exceptional Children.
Helgren-Lempesis, V. A., & Mangrum, C. T. (1986). An analysis of alternate form
reliability of three commercially-prepared informal reading inventories.
Reading Research Quarterly, 21, 209-215.
Howell, K. W. (1986). Direct assessment of academic performance. School Psychology
Jenkins, J. R., & Larson, K. (1979). Evaluating error-correction procedures for oral
reading. The Journal of Special Education. 13, 145-156.
Johns, J. L. (1986). Handbook for remediation of reading difficulties. Englewood Cliffs,
Johnson, M. J., Kress, R. A., & Pilulski, J. J. (1987). Informal reading inventories (2nd
ed.). Newark, DE: International Reading Association.
Johnson, N. J. (1971). Acceleration of inner-city elementary school pupils' reading
performance (Doctoral dissertation, University of Kansas, 1971). Dissertation
Abstracts International, 32, 6250A.
Johnston, J. M. & Pennypacker, H. S. (1980). Strategies and tactics of human behavior
research. Hillsdale, NJ: Lawrence Erlbaum.
Killgallon, P. A. (1941). A study of the relationships among certain pupil adjustments in
language situations. Unpublished doctoral dissertation, Pennsylvania State
University, State College.
Kragler, S. K. (1986). Static versus dynamic assessment: The impact of reading
placement on reading underachievers (Doctoral dissertation, University of
Florida, 1986). Dissertation Abstracts International. 47, 1673A.
LaBerge, D., & Samuels, S. J. (1974). Toward a theory of automatic information
processing in reading. Cognitive Psychology, 6, 293-323.
Lindsley, 0. R. (1971). Precision teaching in perspective. Teaching Exceptional
Children, 3, 114-119.
Lindsley, 0. R. (1985). Quantified trends in the results of behavior analysis. Presidential
address at the Eleventh Annual Convention for Behavior Analysis, Columbus, OH.
Lively, B. A., & Pressey, S.L. (1923). A method for measuring the "vocabulary burden" of
textbooks. Educational Administration and Supervision. 9, 389-398.
Lovitt, T. C. (1977). In spite of my resistance. I've learned from children. Columbus, OH:
Charles E. Merrill.
Lovitt, T. C. (1984). Tactics for teaching. Columbus, OH: Charles E. Merrill.
Lovitt, T. C., & Hansen, C.L. (1976). Round one--Placing the child in the right reader.
Journal of Learning Disabilities, 9 (6), 18-24.
Mercer, C. D., & Mercer, A. R. (1981). Teaching students with learning problems.
Columbus, OH: Charles E. Merrill.
McCracken, R. A. (1966). Standard reading inventory. Kalmath, OR: Kalmath Printing.
Mithaugh, D. E., Horiuchi, C. C., & Fanning, P. N. (1985). A report of the Colorado
statewide follow-up study of special education students. Exceptional Children, 51,
Moyer, S. B. (1982). Repeated reading. Journal of Learning Disabilities. 15, 619-623.
Neufeld, K. A., & Lindsley, 0. R. (1980). Charting to compare children's learning at four
different reading performance levels. Journal of Precision Teaching, 1 (1), 9-17.
O'Shea, L. J., Sindelar, P. T., & O'Shea, D. J. (1987). The effects of repeated readings and
attentional cues on the reading fluency and comprehension of learning disabled
readers. Learning Disabilities Research, 2, 103-109.
Parsonson, B. S., & Baer, D. M. (1986). The graphic analysis of data. In A. Poling and R.
W. Fuqua (Eds.), Research methods in applied behavior analysis: Issues and
advances (pp. 157-186). New York: Plenum Press.
Pennypacker, H. S., Koening, C. H., & Lindsley, 0. R. (1972). Handbook of the Standard
Behavior Chart. Kansas City, KS: Precision Media.
Perfetti, C. A. (1984). Some reflections on learning and not learning to read. RASE, 5 (3),
Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent
variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477-492.
Powell, W. R. (1984). Mediated (emergent) reading levels: A construct. In J. A. Niles
(Ed.), Thirty-third yearbook of the National Reading Conference (pp. 117-123).
Rochester, NY: National Reading Conference.
Powell, W. R., & Dunkeld, C. G. (1971). Validity of the IRI reading levels. Elementary
English, 48, 637-642.
Public Law 94-142. (1977, August 23) Federal Register.
Rashotte, C. A., & Torgesen, J. K. (1985). Repeated reading and reading fluency in
learning disabled children. Reading Research Quarterly, 2U, 180-188.
Resolutions Passed by Delegate Assembly, (1981). Reading Research Quarterly fi, 16,615.
Roberts, T. (1976). "Frustration level" reading in the infant school. Educational
Research, 19I (1) 41-44.
Rose, T. L., McEntire, E., & Dowdy, C. (1982). Effects of two error-correction procedures
on oral reading. Learning Disabilities Quarterly, 5, 100-105.
Salvia, J., & Ysseldyke, J. (1981). Assessment in special and remedial education (2nd
Ed.). Boston: Houghton Mifflin.
Samuels, S. J. (1979). The method of repeated readings. The Reading Teacher, 32, 403-408.
Samuels, S. J. (1986). Why children fail to learn and what to do about it. Exceptional
Children, 53, 7-16.
Savage, J. F., & Mooney, J. F. (1979). Teaching reading to children with special needs.
Boston: Allyn and Bacon.
Shanahan, T. (1983). A critique of P. A. Killgallon's study: A study of the relationships
among certain pupil adjustments in reading situations. In L. Gentile, M. Kamil
& J. Blanchard, (Eds.) Reading research revisited (pp. 577-582). Columbus, OH:
Charles E. Merrill.
Sidman, M. (1960). Tactics of scientific research. New York: Basic Books.
Silvaroli, N. J. (1976). Classroom reading inventory (3rd ed.). Dubuque, IA: William C.
Smith, D. D. (1979). The improvement of children's oral reading through the use of teacher
modeling. Journal of Learning Disabilities, 12(3), 39-42.
Spache, G. D. (1981). Diagnosing and correcting reading disabilities (2nd ed.). Boston:
Allyn and Bacon.
Spargo, E., & Williston, G. R. (1980). Timed readings series. Providence, RI:
Stanley, N. V. (1986). A concurrent validity study of the emergent reading level (Doctoral
dissertation, University of Florida, 1986). Dissertation Abstracts International, 47,
Starlin, C. (1971). Evaluating progress toward reading proficiency. In B. Bateman (Ed.),
Learning disorders: Volume 4. Reading (pp. 389-465). Seattle, WA:. Special Child.
Thurow, L. C. (1987). A surge in inequity. Scientific American, 256 (5), 30-37.
Ulman, J. D., & Sulzer-Araroff, B. (1975). Multielement baseline design in educational
research. In E. Ramp & G. Semb (eds.), Behavior analysis: Areas of research and
application (pp. 371-391). Englewood Cliffs, NJ: Prentice-Hall.
White, 0. R., & Haring, N. G. (1980). Exceptional teaching (2nd ed.). Columbus, OH:
Charles E. Merrill.
Will, M. C. (1986). Educating children with learning problems: A shared responsibility.
Exceptional Children. 52, 411-415.
Wolking, W. D. (1987). Handbook of high performance instruction. Gainesville, FL:
Woods, M. L., & Moe, A. J. (1985). Analytical reading inventory (3rd ed.). Columbus,
OH: Charles E. Merrill.
Yarington, D. J. (1979). The great American reading machine. Rochelle Park, NJ:
SAMPLE READING PASSAGE
Vq 0 Lo ONO (N 00 CO r- N > D N- (N CO "D (N
^-l (N 'Ir LO lZ o0 Dr' VH (N V Lo L0 r- r- co C
TH V-i IH T-H T-H %H TH r-1 (N
4 C4) 0.
1 4, b b0 w
4-' 0 0t 5 o 0..
+ W* +, .- -o LO
a) 'U rjS ^
H -- 0
'- a) a) -^ '; ^
0 a) ^ a U) 0 0
a) ) (- U
.> 1 0 a)
i > y Q ( 5 U)
U bOo 0
4- *g 0
(t5 M W Z a)
o 4- ^ r "5- a
0T -~ >^ *4- > -^ a) b
l- >. a) "5 0 -
H ) C 4-1 -
0 0) q:1 -0'J
05 af) > g s bO
.- a )O ~ W' U)
C .0 US
| ^ | ,-IkI
0 0 0 4) 0
+-~ ~-' ^ bO *4 jS ('5 -4-S .^ 0 S -
bO - 0 .0
U 0 'I- I.~ i' 0
4- UO a) T U) bO
u bO 4- "5 .0 -0 U)
ro ~> 05 1wI^ : ^
C ^ (- .5 ^ .i ^ O -4-' _. )
10 0 ^ I U)cX:
LV c0~*. ~ : 4-' )U
-~~~ U)) U b 5
>- 0 > 0 w >1
-" .0 .& 0. a)O -w w 0 5 J "5
0~~~ )4 4' .
U)~ '4- 14- $-
S w 0 >' '- l4- 0 a) U ('5If ^
('5 4 S a S
C'3 > 0) U .0 a) 0 .a 0
SAMPLE COMPREHENSION QUESTION SHEET
Level 4 THE MYSTERY OF THE ROLLTOP DESK
Tutor's name_____ Date_____
Note to tutors: Use the comprehension checking procedure only twice for each reading
passage. Once after the first full reading of the text and again on the last day of
instruction for the passage.
Do the Say-facts before the structured questions. If you do the structured questions first
you unfairly suggest a response format to the child and you give an advantage on the
Turn this sheet in to Jack or Jolena on the day you gather these data.
SAY-FACTS Do a 30 second timing and be sure to multiply your count by 2 to get the count
Correct facts per minute ____ Errors per minute___
Tallies can go here
Tutors: Please read these questions in the order given and do not allow the learner to go
back and correct or change an answer. (Questions that follow may provide answers to
(mi) 1. What strange thing is happening in this story?
(A strange man has followed Mother home.)
(f) 2. What was the color of Mrs. Marsh's pick-up truck?
(f) 3. Who did Mrs. Marsh buy the rolltop desk from?
(v) 4. What does the word "insisted" mean?
(ask forcefully, ask repeatedly, kept asking, etc)
(inf) 5. What should Mrs. Marsh do about the man in the green car?
(Call the police, or find out what he wants, get help, etc.)
(inf) 6. How was Mrs. Marsh acting when she arrived at her home?
(Nervous, afraid, scared.)
S1OOIJ O0183d DNIINnOo-.-
4 -- ________ - - . -- ~- I -
o I, -m
I Ba W1
S ~N io a O
0 0 0 0
I I I I I-
N ~~ oo o M n
I Z F4 a 00S 82 ON p pr
go to t~
........L.~ I ~4 -. u-.-- -
_____ _____ kIL] _
______________ LL I I i.
3IrlNIW 83d INnoo
I leljlii Cie
t 0 as
0 00 4)
9 + + +Go A ++
S ~~ *-4 CO
a t.I -J
I I I
- Z ;
,, ,, ,, I
SHOOiI 0O013d DNIINnoo-J-o-
*^- p i^rr %pp
'* " | U H
- N to
0 0 0 0 0a
1 o o 1o
.3 3 3 .3 .3 1
MM + T +4 -o5v +
5 2 z 0SAS ASSv
D0 0 t-
30NI 13d LNnO
UinNiw 83d INnoo
Ii I 5
-- ---- OIL-
SuOOiJ a0113d 3NIlNn3-im-
i ad O & A i o
| !w g *
|~~~~~~r a S
0 (0 0
0 0 0 00
0 000 0 0
,t 2 A L_ o
a 0 t..- cli e4faie4S%
Ouoa A 11 z zwol o
S54 Sg s g .SAw ZS" 4 0S. 0
WD fa t~
I I I I I
0 1 ~-oo oo
- 0 0NO
31nNllWt Ud lNno3
[E IT~ I =
*^^^|= -: 0 ^^ ^^
3 3 a nO> ?
0 0 0
C I N to
1 1 f 1 -T -
Inl 1 1
SH001J O0183d DNIIN0oo-,-
| a oe 15 mat
== ,0 Lq ."
N tii!I I 1
.. . . 1= ^ i 1 | ii :" |iiii | i i : T .
~0 o 0
3lnNiP a38 lN-oo
3.1nNIWN H:d .LNnOD
Ig i I +
C4 00 .4 S C SB a 1 C4
rig es rrcaB
18 A (B r
I I I I I' I 19
S1OOIJ 001I3d 9NIIlNno03-.-
,,, WoO oo8
(' ID Cl
00 0 0
N to N 10 != C4
80 ~~6 9o
" = . .
** = y .I .
S".9.1" .. r
I "a 0
%Pk O> bO91ob z t I
I I II
hJ ^ -^4=-
airiiw ~d Nfloo
E ;9 0 1 "B0a
- --r- --
_ W _
_ ,t-f- r-,
S1OOIJ OOlO3d 9NIlNn03.-.-
J 0Q 9
z z ++
- 4 u _,2
3 C &BOI
* I 4 I I
| e g.- -, U
d' *^ l 0
Q0a Q04 9L
I~ ~ s is jSl~
ca M3 0C-
SI. I 4
3UInNIIA a3d INnoo
-s -*^ E
=' .J I I I I | !
0 10 2 0
a a9 : N 0o oo 2
so01u aoiH3d WNI1Nn=O-e- I I I I I I I
g WOOS2 : 4 0 0$
L Ii I .
ii_ &| o= ol e | 8ooo o o o
,1 e .* 0. v4aa 3
i N i u N I''1
'*' "l tF !l y l y S i "
!S <^ ^S g~ag g |ij
3inflNIW 3d INnO3O
SOOOIJ 001o3d SNIiNnoo--',-
5- N 10
i i ~ i i
EM I I I
0 0 0o
N4 W. C4
Illllll-lli' ll r lt-Iffr il llll 111i i ii ll~l
+ t w T ~ T C8 GoTVT +tJ +
0-1 C 4 W 10 .
0:.. _ -
I I I
I III I I
00 l0 00
f0 o fLOo
UinNiW~ 83d INnoo
0 0 VI
C3 * *
- -r- --a'
61 e l,
-8 Se8 o
SVIOOIJ M0U3d SNIINOI3'--
16,1 6 P
2^0 j 0O
~I S "e I S
T .i B
s gO4 gIcg
I 4 iN ( 10 I 2
I l I I l
.!10 u Pfl
Ic o"o o+a a-: a a +:
Z z z
.2 4 S .SS .
S I I I
0 C .
u F F
31NNINA H3d INnOO
SU : | "1^ 1 l
Uio '^ ~~ r'
- sEEE ^ d^E^
- U- - r t-- -- -. '1