Learning rates for oral reading at instructional and frustration levels by elementary pupils with mild learning problems

MISSING IMAGE

Material Information

Title:
Learning rates for oral reading at instructional and frustration levels by elementary pupils with mild learning problems
Physical Description:
vii, 104 leaves : ill. ; 28 cm.
Language:
English
Creator:
Scott, Johnny, 1948-
Publication Date:

Subjects

Subjects / Keywords:
Oral reading   ( lcsh )
Reading (Elementary)   ( lcsh )
Learning ability   ( lcsh )
Children -- Books and reading   ( lcsh )
Special Education thesis Ph.D
Dissertations, Academic -- Special Education -- UF
Genre:
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Thesis:
Thesis (Ph. D.)--University of Florida, 1988.
Bibliography:
Includes bibliographical references.
Additional Physical Form:
Also available online.
Statement of Responsibility:
by John Scott.
General Note:
Typescript.
General Note:
Vita.

Record Information

Source Institution:
University of Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 024623974
oclc - 19935919
System ID:
AA00025748:00001

Full Text











LEARNING RATES FOR ORAL READING AT INSTRUCTIONAL AND
FRUSTRATION LEVELS BY ELEMENTARY PUPILS WITH MILD LEARNING
PROBLEMS











BY

JOHN SCOTT, JR.


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN
PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
DEGREE OF DOCTOR OF PHILOSOPHY



UNIVERSITY OF FLORIDA


1988























Copyright 1988

by

John Scott, Jr.















ACKNOWLEDGMENTS


I would like to thank Dr. William Wolking, Dr. Bob Algozzine, Dr. Vivian Correa,

Dr. Brian Iwata, and Dr. Cary Reichard for their support and guidance. Very special

thanks are in order for Bill Wolking who as my mentor, coach, and friend helped me

learn how to ask the right questions.

I wish to extend my thanks to those who assisted in the research reported in this study.

First, I want to thank Jolena Stoutimore for her assistance with tutor supervision and her

many helpful suggestions for improving the research. I am grateful for the efforts of the

students enrolled in EEX 3243L during the Spring 1988 term for not only for the data they

helped to generate but for the opportunity to be part of their teaching career. I would like to

thank the students at the P. K. Yonge Laboratory School for their cooperation and

enthusiasm in the after-school tutoring project. Also, I want to extend my gratitude to the

staff at the P. K. Yonge School, especially Jean Brown and Patti Rosenlund, for their

assistance and support.

My most heartfelt expression of gratitude is reserved for my wife, Wanda, for all her

loving support. Her patience, encouragement, and understanding, while I spent countless

hours developing materials, analyzing data, and writing, go well beyond the obligations of

matrimonial duty.















TABLE OF CONTENTS


ACKNOWLEDGEMENTS ............................................. iii

ABSTRACT ........................................................ vi

CHAPTERS

I INTRODUCTION ............................................. 1

Statement of the Problem .................................... 1
Rationale ............................................... 2
Definition of Terms ....................................... 4
Delimitations ............................................ 5
Limitations .............................................. 5

II REVIEW OF RELATED LITERATURE ........................... 6

Placement and Informal Inventories .......................... 6
Attempts to Validate Reading Placement Criterion ............... 12
Suggestions for Alternative Placement Criterion ................ 16
Repeated Readings ........................................ 20
Summary ............................................... 22

III METHOD ................................................ 25

Experimental Questions ................................... 25
Subjects ................................................ 25
Setting .. .............................................. 26
Variables Under Investigation ............................. 27
Independent Variables ................................. 27
Dependent Variables .................................. 27
Measurement ........................................... 27
Independent Variables ................................. 27
Dependent Variables ................................. 28
Experimental Design .................................. 29
Experimental Procedures .................................. 30
Initial Assessment .................................... 30
Instruction .......................................... 30
Say Facts Comprehension Check........................ 32
Decision Making................................... 33
Materials ........................................... 33
Data Collection and Analysis .............................. 34
Data Collection ....................................... 34
Data Analysis ....................................... 35
Rate Data for Oral Reading ........................... 35









Comprehension Data ................................. 39
Other Measures ..................................... 40
Reliability .......................................... 41
Procedural Reliability ............................... 41
Interobserver Agreement............................. 42

IV RESULTS ............................................... 43

Question 1 .............................................. 43
Question 2 .............................................. 46
Question 3 .............................................. 49
Other Measures .......................................... 58
Summary of Results ...................................... 62

V DISCUSSION AND CONCLUSIONS ............................ 64

Methodological Progress in Reading Research .................. 64
Factors supporting Continued Use of the Betts' Criteria ............ 66
Potential Utility of High-Error Material in Individualized Instruction.67
A Simple and Effective Set of Instructional Procedures ............. 69
Limitation of the Study and Recommendations for Future Research... 69
Instructional Recommendations .............................. 71

REFERENCES ... ............................................. 74

APPENDICES

A SAMPLE READING PASSAGE ........................... 80

B SAMPLE COMPREHENSION QUESTION SHEET ............ 82

C STUDENT DATA...................................... 85

D TUTOR MONITORING FORM ........................... 102

BIOGRAPHICAL SKETCH ....................................... 104
















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the Requirements for the Degree of
Doctor of Philosophy



LEARNING RATES FOR ORAL READING AT INSTRUCTIONAL AND FRUSTRATION
LEVELS BY ELEMENTARY PUPILS WITH MILD LEARNING PROBLEMS


By

John Scott, Jr.

December, 1988

Chairman: William D. Wolking
Major Department: Special Education

Learning rates for oral reading by elementary pupils with learning problems at

two levels of text difficulty were compared. Placement in an instructional level of

material, derived from the Betts' criterion of 95 to 98% initial oral reading accuracy, was

compared to reading progress of students on material with 80% or less initial accuracy

designated a frustration plus level. Subjects were 16 elementary school children with mild

learning problems. Individual tutoring, with daily assessment, was provided by special

education majors. Instruction consisted of daily timings, simple error correction, and a

repeated readings procedure in which the learner read aloud while listening to a paced

audiotape of the passage. An alternating treatments design was used. Performance data

were plotted on Standard Celeration Charts and rates of learning calculated.

Learning rates for corrects on the frustration plus level were superior or equal to

those of the instructional level for 100% of the students for experiment 1 and for 100% of the

students in experiment 2. Rates for errors on the frustration plus level were superior for

75% of the students in experiment 1 and 68% in experiment 2. Comprehension was










measured pretreatment and posttreatment and showed greater increases on the frustration

plus level for a say facts procedure and answers to comprehension questions.

The instructional reading level failed to offer students the most productive material

when a student was provided with instruction consistent with recommended individualized

instructional procedures. It is recommended that the use of the term frustration level be

reconsidered as it may be inaccurate and may have the effect of restricting teachers from

using potentially productive reading materials on which learners make numerous initial

errors. The methodological advantages of using oral reading as the dependent measure for

placement criterion research as well as the benefits of direct and frequent measurement are

discussed. The work of Lindsley, Powell, and others who have suggested that children

given appropriate instructional support are capable of making better progress on material

considerably more challenging than the material they have been asked to read was

supported by the results of this study.















CHAPTER I
INTRODUCTION


This is a report of a study on the rates of learning by elementary students on reading

passages of widely different levels of difficulty. Decisions to place students in material at

specified grade levels are routinely made on the basis of a child's performance on an

informal reading inventory (IRI). The criterion for placement at an instructional level,

the level purported to offer the greatest opportunity for learning, is generally 95 to 98%

accuracy in word recognition on a IRI passage (Cook & Earlley, 1979; Hargis, 1982;

Johnson, Kress, & Pikulski, 1987; Mercer & Mercer, 1981; Savage & Mooney, 1979). This

criterion, as originally proposed by Betts (1946), has been widely and essentially

uncritically accepted as offering the best starting point for instruction. Special education

students, in the development of their individualized educational program (IEP), are

typically placed within a reading series based on this criterion. Reports of research

attempting to validate the efficacy of the instructional level based on learner performance

at various levels of difficulty are absent. Are children who are placed in reading materials

at the 95 to 98% accuracy level and provided with an effective and individualized reading

program capable of making the greatest gains in reading or is a placement determined by

some other criterion more effective in maximizing learning?

Statement of the Problem

Teachers use the Betts' criterion to place students in reading material. They do this

with the expectation that this placement provides the best possible starting point for

instruction and that the most effective learning takes place when children begin reading

at this level of difficulty. The problem investigated by this study was to compare learning









and performance of students in two levels of text: an easy level (Betts' criterion) and a

difficult (frustration plus) level. The objective of this study was to provide an empirical

test of both levels of text difficulty with mildly handicapped students in a tutorial setting by

answering the following question: Are children placed in reading materials at the 95%

accuracy level and provided with an effective and individualized reading program

capable of making the greatest gains in reading or is a placement determined by some

other criterion more effective in maximizing learning?

The following questions constitute components of the main question:

1. What are the learning rates for correct responding at each level of text difficulty?

2. What are the learning rates for error responding at each level of text difficulty?

3. How does comprehension compare across instructional and frustration plus levels

of text difficulty?

Rationale

In spite of vast investments of money and effort, the academic performance of special

education students remains unimpressive. Calls for increased accountability on the part of

government and by parents have suggested reevaluation of existing practices (Will, 1986).

Special educators have been aware of the problems and have actively worked toward

solutions on several fronts (Algozzine & Maheady, 1986). Follow-up data gathered by

researchers from selected groups of former exceptional students suggest that they are

headed for low paying jobs with few prospects for advancement (Mithaugh, Horiuchi, &

Fanning, 1985). With the American economy heading ever closer to a two-tier society of

rich and poor (Thurow, 1987), those individuals with educational deficits are most likely to

wind up on the bottom of the socioeconomic heap.

No academic skill is more central than reading, yet it is an area in which a

remarkably poor job has been done. Poor quality teachers account for a great part of the

problem (Durkin, 1978-1979), while home and school organizational factors have










contributed as well (Samuels, 1986). For the child who does not learn to read or who reads

with great difficulty the impact on their school career and life adjustment can be

devastating (Perfetti, 1984). Reading instruction should be considered a number one

priority.

The rights guaranteed to handicapped learners and their families by Public Law 94-142

(Public Law 94-142, 1977) for improved special education instructional programs rely

heavily on accurate assessment (Bennett, 1983). Of the several purposes of assessment, it is

the educational planning role, assisting the teacher in determining what to teach and how to

teach, which was examined. It is this aspect of assessment most likely to be conducted with

informal instruments such as an informal reading inventory (IRI). The IRI, typically

used by classroom teachers, provides specific recommendations for placing the child in

instructional reading material. This technique, especially when based upon the actual

curricular materials used in the classroom, can be very valuable (Deno, 1985; Howell,

1986). The problem comes not from the format but from the various criteria employed for

placing children in appropriate reading materials. The widespread use of the placement

criteria developed by Betts (1941, 1946) calls for 95 to 98% word recognition accuracy with

75% or better comprehension on a graded reading passage. Children are routinely placed

in reading materials based on this recommendation. These arbitrarily selected criteria

have received "validation through use" (Shanahan, 1983, p. 581) but lack empirical

validation. This is at variance with the belief that tools used for assessment and the as-

sociated criteria for instructional placement should be technically adequate for assessing

the skills and abilities of handicapped children (Bennett, 1983).

There are several strong indications that the Betts' criterion does not lead to the most

appropriate placements of students in reading materials (Kragler, 1986; Neufeld &

Lindsley, 1980; Johnson, 1971; Powell & Dunkeld, 1971; Roberts, 1976; Stanley, 1986).

Validation of this criterion with special populations receiving individualized educational










services has not been reported in the literature. A question has emerged as to whether or not

this placement criterion leads to the most effective placement in reading materials or

whether some other placement criterion is associated with better learning. This hypothesis

was testable by the procedures followed in this study. If the Betts' criterion fails to predict the

most effective starting points for reading instruction, why continue to place special

children at these points? The use of reading grade level and placement consistent with the

instructional reading level recommended by the Betts' criterion form a metaphor that

powerfully shapes teaching practice (Cadenhead, 1987). Misplacement based upon a faulty

criterion may be interfering with a teacher's ability to provide the most effective reading

instruction possible.



Definition of Terms

Accuracy. Accuracy is a measure of correct responses divided by the total of correct

and incorrect responses.

Celeration. Celeration is a basic unit of measurement of behavior change; change in

frequency per week (Lindsley, 1971).

Frequency. Frequency is a basic unit of behavioral measurement; the number of

movements per unit of time (minute).

Frustration plus reading level. This terms refers to the reading level in which the

learner initially reads 80% or fewer words correctly.

Learning. For the purposes of this study, learning refers to a change in performance

per unit time; also called celebration.

Learning rate. Learning rate is a measure of total academic change in behavior per

unit time; the celebration value for corrects multiplied by the celebration value for errors.

Informal reading inventory (IRI). An IRI is a reading assessment instrument based

on a series of passages with measured reading levels. Student responses are judged










against prescribed standards for oral reading and comprehension to determine placement

in reading material.

Instructional reading level. The reading grade level at which the learner, during

assessment, orally reads with 95 to 98% word recognition accuracy is the instructional

reading level.

Readability level. The readability level is the degree of reading difficulty as computed

by the Fry Readability Formula (Fry, 1977).

Standard Celeration Chart. The Standard Celeration Chart is a standard, six-cycle

semi-logarithmic chart that measures frequency as movements/min and celebration as

movements/min/week. This chart is also called the Standard Behavior Chart (Lindsley,

1971).

Delimitations of the Study

This study was designed to examine the rates of learning for elementary children with

mild reading problems who are receiving individualized instruction. It would be

inappropriate to attempt to generalize the results to pupils in traditional, group-oriented

instructional settings. Caution should be exercised in relating the findings to learners

with severe reading disabilities and to those students experiencing no reading difficulty.

Extension of results to learners beyond the elementary grades would be unwise.

Limitations of the Study

The study was designed to investigate the learning rates for elementary school children

working at different levels of text difficulty. Subjects were selected from a university

laboratory school setting that did not use a traditional special education classification

system. A significant limitation concerns the nonclassified status of the subjects. As they

are not classified as special learners, they may not be truly representative of any

traditional exceptional student category. Additionally, extention of the results to students

beyond the Alachua county area may not be warranted.














CHAPTER II
REVIEW OF RELATED LITERATURE


There is a vast amount of literature addressing problems in reading instruction. This

is a selective review of studies closely related to the question selected for investigation. The

necessary selection criteria were broad and flexible. In order to develop the proper

historical context for this investigation, several works from the earlier part of this century

were reviewed. While the reading research is extensive, only a small portion directly

relates to this study.

First, an examination of the primary reading placement criterion in use with

handicapped and reading problem pupils is presented. The focus is on the origins of this

placement criterion and recent recommendations for use of such a criterion. Empirical

studies designed to test the efficacy of this placement criterion are given special attention.

Instructional implications of the high-accuracy, fluency-building criterion in wide use are

broadly explored as are criticisms of this placement tradition. Alternatives to this grade

level-dependent, high-accuracy placement criterion, primarily behavioral in orientation,

are reviewed next. Finally, the method of repeated readings is reviewed in detail as this

technique plays a key role in the intervention phase of the study.

Placement and Informal Reading Inventories

A useful perspective is provided by examining several historical trends in the de-

velopment of reading grade level and the concept of readability. Lively and Pressey (1923)

proposed "a method for measuring the vocabulary burden of textbooks" (p. 396). They

systematically sampled 1000 words from public school textbooks and by comparing the list

thus compiled to the entries in the Thomrndike Word Book derived measures of vocabulary










difficulty. Refinement of this time consuming method led to simpler procedures for

determining what is now generally referred to as readability and to the increased use of the

term "level" or reading level to designate the relative difficulty of a given reading passage

(Cadenhead, 1987).

With methods for determining the grade level of a reading passage in place, the next

development of note was the use of techniques to assess and classify pupil performance at

varied grade levels. Betts (1946) in Foundations of Reading Instruction suggested the use

of four levels to describe pupil performance within a series of graded reading passages.

These levels were basal, instructional, frustration, and capacity (Betts, 1946). Betts credited

the work of Killgallon (1941) for the refinement of the procedures used to determine these

levels and, most importantly, for the performance criteria associated with each level.

Killgallon tested 41 fourth grade students with the Gates Survey (a standardized

reading test) and an informal reading inventory developed by Betts. While

acknowledging the importance of pupil interest and motivation, Killgallon concluded that

this IRI was the best single available index to reading ability at the elementary school level.

Killgallon provided average comprehension scores for instructional and frustration levels

as well as an analysis of tension or frustration behaviors.

Two levels arbitrarily defined by Killgallon and Betts are of special concern. The

basal reading level (comparable to the modern independent reading level) was determined

by the following criteria: (a) 90% or better comprehension, (b) fluent, error-less oral

reading, and (c) efficient silent reading with good comprehension and no signs of tension

behaviors. The "probable" instructional level (comparable to the current instructional

level) was the level at which (a) a minimum comprehension score of 50% (or if subjectively

rated dysfunctional reading behaviors were present, a 75% comprehension score was

required), and (b) no more than 7% word recognition errors or one error in 14 running

words. Killgallon described this level as the most appropriate for reading instruction as,









maximum development was to be expected when the reader was suitably challenged but not

so challenged as to be frustrated.

When these criteria were applied to the reading performances of the 41 fourth-grade

children, the average basal reading grade level was .86 (range, 0-5.0). The probable

instructional reading grade level was 3.16 (range, 0-9.0). Further, the informal inventory

placed students, on average, 1.06 grades lower than they would have been placed with the

Gates Reading Survey (Shanahan, 1983, p. 581).

These results should have strongly suggested to Killgallon, Betts, and others that the

proposed criteria were too stringent and would have the effect of placing children in reading

material that was below the level at which they had been working, presumably, suc-

cessfully. In any event, Killgallon proposed a set of arbitrary placement criteria that could

be easily adopted by teachers and reading specialists. A host of IRIs developed since that

time ( Ekwall, 1986; McCracken, 1966; Silvaroli, 1976; Woods & Moe, 1985 ) have accepted,

with only slight modifications, these basic placement criteria.

The essential circularity of Killgallon's work is highlighted by Shanahan in a 1983

critique. Shanahan identified the construct validity question that Killgallon appeared to be

addressing as, "Does providing instruction at the level which can be read with 95 to 98%

accuracy lead to demonstrably higher levels of achievement than does placement in

materials above or below that level?"(p. 583). The study designed to accomplish this, wrote

Shanahan, would have tested children, placed them at varied reading levels, and then, after

instruction, measured their progress. Instead of doing this, Killgallon determined the

mean percentage for errors of students whose performances fell within arbitrarily

predetermined ranges. Shanahan pointed out that Killgallon selected only those oral

reading performances which met his instructional level criteria and determined the mean

percentage of word recognition errors. It is the result of this computation, the average of all

pupil scores above 93.9% for word recognition with 50 or 75% correct comprehension, that










was used to determine the now ubiquitous 95% correct placement criterion. Shanahan noted

that the use of this reading level criterion has unwisely received "validation though use"

and that experimental validation, which Killgallon did not do, is the only way to arrive at

the "optimum criterion" (p. 581).

Teachers who use the highly structured basal series used in most reading programs rely

heavily on the notion that sentence and word length, vocabulary, and content should be

controlled to match the child's reading ability level. Reading instruction, to be most pro-

ductive, should match the child's reading level to materials of the same readability level.

Errors should be very few in number and a priority is placed on reducing them even

further. These general principles serve, in critical ways, as pillars of modern reading

instruction and are presented uncritically in textbooks aimed at both new and experienced

teachers (Cadenhead, 1987). Harris and Sipay (1980), for example, stated that, "the

instructional level indicates that the material is suitable for use under the teacher's

direction; serious errors are not numerous and comprehension is adequate. The material

presents some challenge so that new skills can be acquired" (p. 182). A passage from

Savage and Mooney (1979) indicated a similar viewpoint was recommended for the

exceptional child. They wrote, "the instructional level indicates the level of material that is

challenging enough to use in further developing the child's reading skills, yet is not so

difficult that it will be a source of frustration for the child" (p. 92). Savage and Mooney

provided a summary statement regarding the frustration level in which they concluded,
"material at this level is considered too difficult for the child to handle, even with support

and help from the teacher" (p. 92). Hargis (1982) wrote a reference work on reading

instruction for special children which provided an additional insight. Hargis adhered

closely to the Betts criterion in terms of both numerical criterion and the importance of a

very low error instruction for special children receiving instruction in an individualized

format. He recommended a 2 to 4% percent error rate for the instructional level while










classifying reading with more than an 8% errors the frustration level. A component of the

rationale Hargis offered for the superiority of the instructional level over the frustration

level was somewhat different from that given by other authors and may hint at a critical

factor in the durability of the Betts' placement criterion.

Any reading more difficult than the instructional level is considered at frustration
level. Not only does it produce frustration and failure in the child, but it requires an
inordinate amount of demand on teacher time to introduce, clarify, and support, if
the student is to be maintained on task. Demand on teacher time is often far too
great to permit using this material efficiently in reading instruction. (p. 109)

Demands on teacher time and issues of instructional convenience may be critical

elements in successful large group instructional arrangements. It would be, however,

inappropriate to suggest, as Hargis repeatedly does, that the instructional level yields the

most productive learning and then to offer as evidence not performance data but an

argument for traditional classroom practice. Perhaps what Hargis meant was that the

instructional level may be the most convenient level for reading instruction. The rationale

he offered for its use was that it was the most productive starting point for reading

instruction.

Similar statements are found, without exception, in all of the major texts reviewed on

this topic (Cook & Earlley, 1979; Hargis, 1982; Harris & Sipay, 1980; Johnson, Kress, &

Pikulski, 1987; Mercer & Mercer, 1981; Savage & Mooney, 1979). A summary of several

placement criteria recommended for regular and special education students as well as the

authors' suggestions for the learning potential at the instructional and frustration levels

can be found in Table 1. The view consistently expressed was that the instructional level

was the more productive and most appropriate level on which to provide instruction while the

frustration level indicated misplacement in excessively difficult material with little

chance for the student to make any meaningful progress in reading. It was quite

conceivable that a less convenient instructional arrangement (in the sense of traditional

procedures) with a more challenging placement criterion may be far more capable of

















*-






I0 ti; ca 0

00 0
z 4t *-
4. j, dis"S *



- 8 2 0* 1 *0
dIdI S|J-

bOOpl
02* oS'
*- 0 il sd
I-~l 0 s
0 ..~ '54.
r' .G *VZS


o 'c
*Odi
0 e



0su



(0






h0

C0







Io-i
l0
hfi 0
BO.r
0


Boo


.3
CA,
d|
10


Cd I- *
q-.




|.64
ri l




-0 03
r."!
) 0 a.c


S0 $.di
*t-






4.



03



4 a d
d i

oo .





0 0













h M
*d11

'5 :2'a
.Udi




di 4 di








* .^ .






I-




U -
I. ^
dI 0


0^-



4) 4.

c S2







o 0
di'5













di ;a0




0 -0 0.b
>






4.
>
'.3 La .
idi




u i di *- 4.



w dc i 61









dill
0l i

o* .S *
di0.

|ia?



0*


>C 0
-o

0.

v.4


to
-a






1i~


r.11











0 0
i:S g









biD







4.-
: o .d











03 0
>


C '- .2 -















C-
|-s









0o








0a








producing high rates of student success in reading. Children consistently placed in easy

reading material may actually be disabled in functional reading skills. The student may

lack confidence to tackle unfamiliar material. The reader may consider material that

appears to contain a higher percentage of errors than they have become accustomed to, as

unreadable. He or she might fail to develop the persistence and ability to deal with

challenging reading material that is characteristic of mature readers. Authors, such as

Hargis, who cited classroom practice as justification for existing reading instructional

procedures should have fully considered the high rates of illiteracy and reading failure in

the United States when contrasted to other industrialized nations (Yarington, 1979). It may

well be that practices such as the emphasis on placing children in very low error material

have been, in large part, responsible for the failure of American schools to provide each

child with reading skills conmensurate with his or her ability.

Attempts to Validate Reading Placement Criteria

Several efforts to validate the placement criteria used in IRIs have been made with

conflicting results. Powell and Dunkeld (1971) attempted to more accurately describe the

instructional reading level. The investigators administered IRIs to 212 nonhandicapped

students in grades 2 through 6. They also administered standardized reading tests and

intelligence tests. After the initial testing, they examined the regular classroom

instructional materials used by these children, computing readability scores which were

then compared to the IRI derived instructional level recommendations. The readability

levels of the materials actually used by the children were matched to the readability level of

the IRI passages. The best predictor offered by the IRI in this analysis was not word

recognition errors but rather comprehension in the 70 to 75% range. At the end of the school

year, residual gain scores were computed for each child's reading achievement. These

adjusted gains were then compared to the levels of materials used for instruction. The

materials used by the children who had made the greatest gains then helped to identify new










criteria for instructional levels. Powell and Dunkeld recommended differentiated

criteria based on grade placement of the pupil and suggested the following:

1. second grade--85.5 to 95% word recognition accuracy,

2. third grade---89 to 97% word recognition accuracy, and

3. fourth, fifth, and sixth grades--comprehension scores of 60 to 90%.

While Powell and Dunkeld made a determined effort to validate the optimum

instructional level, their work has several significant shortcomings. First, they relied on

group averages, thereby losing the exactness of data showing how individuals interacted

with the learning task. Further, they excluded learning disabled, gifted, and other

exceptional learners, precisely the groups most likely to have important curricular

decisions made on the basis of such criteria. An additional limitation for the purposes of

this investigation was that these children were all receiving a traditional, group-oriented,

basal reading instructional program. Generalization to children receiving

individualized instruction, as was the case in special education practice, may not be at all

appropriate. Still, Powell and Dunkeld pointed to the need to move away from one criterion

for all age-ability levels and suggested that, for some children, less reliance on initial

accuracy may be positively correlated with reading gains.

A second phase of Powell's (1984) work relevant to this review concerned the

development of the mediated or "emergent reading level." The emergent reading level

suggested the level on which the child was likely to make the greatest gains in reading

when provided with mediation, or appropriate levels of teacher support on new vocabulary,

main idea, and story details. Powell suggested that the standard administration of an IRI

failed to provide a realistic assessment of the child's reading due to the absence of the

traditional supportive instruction provided in actual reading levels. When using the

procedures recommended by Powell's emergent placement criterion, the assessor provided

mediation or traditional forms of teacher assistance. The level at which the child can










achieve a high word recognition and high comprehension score with these procedures was

offered as the emergent reading level. Emergent reading levels tend to be significantly

higher than the instructional reading levels suggested by the Betts' criterion.

Two of Powell's students (Kragler, 1986; Stanley, 1986) have conducted investigations of

the emergent reading level germane to this review. Stanley (1986) compared traditional

IRI procedures to mediated IRI procedures to determine which more accurately predicted

reading level placement. Preinstruction and postinstruction comprehension and word

recognition measures were obtained for 21 second grade subjects. Placement determined

by the mediated procedure was two to three book levels higher than that of the traditional,

Betts-style, IRI procedure. Progress for all students placed according to the mediated

procedures was significantly greater than the traditional placement group. In this study,

more challenging material was associated with more successful learning while the easier,

traditional level of material difficulty was associated with less successful learning for

students receiving group reading instruction.

Kragler (1986) studied the silent reading and listening levels of 21 low -socioeconomic

reading underachievers when placement was guided by emergent reading level

procedures. One group of students received mediated vocabulary instruction while the

control group was exposed to a repeated readings and traditional instruction procedure.

The mediated group was able to maintain comprehension at the higher placement levels

suggested by the emergent placement criterion. Students working under the traditional

conditions could not maintain success at the higher placement level. Kragler suggested

that the higher placement levels recommended by the emergent conceptualization may be

appropriate when supported by sound instructional procedures which are sensitive to student

performance. Similarly, the higher placement level may not appropriately place children

when instruction is insufficiently responsive to student performance. It should be noted

that these studies by Kragler and Stanley were not focused on special students nor at










learners in individualized educational settings but rather those receiving essentially

traditional group-oriented instruction. Powell, Kragler, and Stanley challenged the

assumption that the Betts' criteria provided the most productive starting point for reading

instruction in regular, nonindividualized classroom settings.

Even if the placement criteria recommended by the many IRIs that have been used could

be validated as offering the instructional placement most likely to bring success in

reading, a host of other problems limited their usefulness. Allington (1979) measured the

ability of 57 certified teachers enrolled in a graduate reading course to accurately record

oral reading errors in an IRI format. The students were provided with extensive training

in using the coding system and completed numerous audio-taped practice scoring sessions.

Among the scored errors on a final test were failing to note an oral reading error and

scoring nonexistent errors. The average proportion of errors for all subjects was 28%,

leading Allington to conclude that neither training nor expertise had much impact on a

teacher's skill in recording oral reading errors. The author suggested that such poor

accuracy is unacceptable given the importance of the instructional decisions made based on

the analysis of the oral reading performance. He recommended tape recording the pupils

reading to provide multiple scoring opportunities in the hope of improving accuracy.

A recent study by Helgren-Lempesis and Mangrum (1986) examined the alternate-form

reliability of three of the most popular commercial IRIs. Four doctoral students with

specializations in reading administered the IRIs to 75 fourth-grade pupils and retested them

with alternate forms of the same inventory 10 days later. The same examiner retested the

same students. Reliability coefficients ranged from .60 to .78, considerably below the .90

level generally considered acceptable for two forms of a commercially available test used to

make important educational decisions about a child (Salvia & Ysseldyke, 1981). The

authors maintained that this level of reliability was better than some critics would have

suggested. This study clearly pointed to the inadequate reliability offered by these










instruments even when used by experts. When such weak instruments are used by

inadequately trained teachers, the quality of the instructional decisions made may be, at

best, suspect and, at worst ,may actively contribute to pupil learning problems. These

problems, when coupled with the concerns regarding the basic placement criterion, strongly

suggest the reading placement procedures may have been inadequate.

Suggestions for Alternative Placement Criteria

While the use of the Betts defined instructional placement criterion of 95 to 98% was

widespread, several important studies have raised fundamental questions that had not been

adequately addressed. Four of the most provocative studies will be briefly reviewed as they

each suggest the fragile nature of the Betts' criterion.

By far the most critical in its implications for the continued use of a high-accuracy

placement criterion is a 1978 study by Roberts. Roberts determined the word recognition

accuracy of 125 second and third grade pupils in six English schools. Applying protocols

used with a standard IRI, she scored students on their teacher assigned instructional

reading materials. Roberts found 23% of the students were reading in material that could

be considered at their instructional level while 57% were reading in the frustration level

according to the Betts' criteria. In one school, serving primarily low socioeconomic status

children, the rates were 18% instructional and 66% frustration. Reading attitudes, as

surveyed by the author, were very positive and student achievement in reading, as

measured by the standardized testing program, was well above average. Roberts credits a

positive, cohesive school climate, focusing on rewarding pupil progress in a non-

threatening atmosphere with much of the success of the reading program. The signs of

dislike for reading predicted by Betts when children were forced to read in the frustration

level material were totally absent. This simple study prompted Roberts to seriously

question the value of the Betts' criteria. If a true frustration or dysfunctional level of










difficulty did exist, asserted Roberts, it was surely not the 95% accuracy rate so closely

adhered to by many teachers.

Gates (1962) presented second grade pupils with 42 "new" words from basal readers at

the third and fourth grade levels. He found that 36% of the children could correctly say the

new words with 100% accuracy and 40% got 41 of 42 words correct while 47% got 40 or more

correct. The second graders had learned the vocabulary, either in the course of outside

readings or elsewhere, suggesting that tightly structured readers were unnecessary and

may hold back the better readers. Gates went on to make a convincing argument for

reducing the role of basal readers in instruction. In pointing out the extensive word

knowledge of pupils well beyond the limits of basal readers, Gates hinted at the reading

potential untapped by criteria which arbitrarily foster low-error reading in highly

structured material.

A fundamental criticism of the Betts' criteria regarding the independent reading level

(98% or more correct) has been raised by Spache (1986). He maintained that children should

read independently in material based on interest, not a 98% or 99% accuracy standard. His

long experience with the materials actually chosen by children for independent reading

convinced him that even frustration level material posed little barrier to enjoyment when

the subject matter held interest for the reader. This position, while consistent with the

experience of most teachers, appeared to have had no significant impact on the practice of

placing children according to a high-accuracy criterion and recommending extremely

high-accuracy materials for independent practice.

Lindsley and his students (Johnson, 1971; Neufeld & Lindsley, 1980) have conducted

studies using precision teaching to monitor learner progress. These researchers

demonstrated the potential of precision teaching which permits the researcher to conduct

frequent and extremely fine-grained examinations of learner performance under actual

instructional conditions. Johnson (1971) used peer tutors and high-interest, high-difficulty










level materials to bring about reading improvements for 140 inner-city children. In a

series of investigations, Neufeld used performance data plotted on the Standard Celeration

Chart to address a question similar to that of this investigation. She asked, "is the

instructional level of the informal reading inventory or any of three other performance

levels a level that promotes highest speed and accuracy learning" (p. 10). A total of 49 fifth

graders from a rural midwestern school served as subjects. Each was assessed on an IRI,

and the performance within the 95 to 97% word recognition accuracy with 75 to 90%

comprehension accuracy was identified as the instructional level. Each pupil then received

a minimum of 10 days of instruction at their instructional level, one grade level below, and

two and six reading grade levels above their designated instructional level. Performance

data were recorded and charted daily. The instructional procedures included an untimed

silent reading on one of the four randomly selected stories followed by a 1 minute see-mark

timing on comprehension questions. These results were then checked and charted. Next,

came a 1 minute timing on oral reading of the same text. These results were similarly

checked and plotted. This procedure was then repeated for the remaining three levels. Both

statistical analysis of the raw data and visual analysis of the charted data showed no sig-

nificant differences in celebrations for corrects, celebrations for errors, or accuracy

improvement for word recognition or comprehension for any of the four levels examined.

This lack of significant difference is quite important as it suggested that progress on

material that is either slightly less difficult than traditionally recommended as well as on

material well into the frustration range of the pupil can produce comparable learning.

Neufeld and Lindsley concluded that neither the instructional level nor any of the other

levels they looked at were associated with maximum learning. If, they suggested, the

instructional level failed to promote highest learning, why should it be used at all? They

urged additional study of this question in an effort to further test the value of the high-

accuracy criterion in reading placement.










This study was noteworthy in the methods of data gathering and interpretation. This

was the first use of the Standard Celeration Chart found in the literature to test the Betts'

criterion. The instructional procedures, however, appeared to have been relatively

unrefined and casual, subtracting substantially from the generalizability of the

conclusions. Pupils read silently and could ask for help with new words. This procedure

suggested that routine, direct instruction was not offered and that the individual child's

tolerance for errors may have played a major role in whether they sought assistance or not.

Similarly, the daily emphasis on comprehension may have had the effect of slowing

students down, especially on the more challenging material. The pupils, as can be inferred

from the initial high reading rates (approximately 200 words per minute on the

instructional level passage), were familiar with the high response rates typical of

instruction provided by precision teachers. They were not, however, students with learning

problems. As such, they may have begun instruction with more highly developed reading

skills than other pupils. These researchers also failed to functionally define the reading

level relying instead on an arbitrary jump to higher grade levels. It was likely that a

student reading six grade levels above his instructional level was reading at the frustration

level, but this was not certain. A functional assessment procedure, in which learners were

placed in frustration material, would have alleviated this problem. It may be the lack of

instructional procedures consistent across instructional days and across passages that

constitutes the greatest limitation of this study in regard to reading placement criterion

questions. The authors have clearly used the most refined set of data gathering and data

analysis procedures available to test the Betts' criterion.

A method for placing learners with reading problems in the optimal materials based on

actual performance data was suggested by Lovitt and Hansen (1976). Again using

precision teaching techniques to measure performance, they presented a series of five

passages at each of eight reading levels (two sublevels for grade 2 and 3). The students had










several opportunities to read at each grade level and assessment continued until all 40

passages had been presented. The charted data for each student provided an excellent

source of information on actual learner performance at varied grade levels. On the lowest

grade level, the mean correct rates was 58.3 words per minute (wpm) with incorrects at 5.4

wpm (92% accuracy). At the sixth grade reading level, the highest level in this study,

corrects were at 31.3 with error at 6.9 wpm (82% accuracy). While the differences between

highest and lowest grade levels was large, differences between grade levels was small

averaging a loss of 5 wpm in corrects, the addition of one oral reading error and a 5%

decrease in comprehension accuracy. Based on theses findings, Lovitt and Hansen set

placement criteria at 45 to 65 wpm correct and 4 to 8 wpm incorrect (89 to 92% accuracy

range) with comprehension scores of 50 to 75% correct. The authors reported that all

learners made adequate progress with these placement criteria. A valuable opportunity to

empirically test the relative effectiveness of placement in high or low accuracy material

was lost, however, as the subjects were all placed according to the revised placement

criteria. Of considerable importance for the purposes of this study were the data showing a

gradual rather than an abrupt drop-off in performance with material of greater difficulty.

The fact that learners in a setting tolerant of high errors do not emit the frustration or ten-

sion behavior predicted by Betts (1946) and others (Gentile & McMillan, 1987) suggests that

performance gains expressed as learning rates may be the best determinant of placement

criteria effectiveness.

Repeated Readings

With the growth and refinement of individualized educational programing has come a

need not only for improved placement criteria but improved instructional techniques. The

repeated readings technique first described by Samuels (1979) is a procedure aimed at

individual rather than group instruction. With repeated readings, the learner re-reads the

same passage until a criterion rate is met or until a prespecified number of readings have










been attempted. With the automaticity theory, LaBerge and Samuels (1974) provided the

conceptual basis for this technique. LaBerge and Samuels postulated limitations in how

much information processing a reader could handle at one time. If the majority of reading

effort was devoted to decoding individual words, little processing capability remained for

comprehension. By repeatedly reading a single passage, processing capacity could then be

freed to deal with comprehension and other higher order tasks, according to LaBerge and

Samuels. While this theoretical position remained to be convincingly tested, the repeated

readings technique has proven to be of considerable practical value in reading instruction.

Numerous studies have reported favorable results for repeated reading techniques with

handicapped children (Moyer, 1982; O'Shea, Sindelar, & O'Shea, 1987; Smith, 1979) and as

a supplementary activity with nonhandicapped children (Allington, 1983; Chomsky, 1976;

Johns, 1986; Samuels, 1979). In a recent study, Dowhower (1987) suggested that repeated

readings may hold significant promise as a technique for helping an early elementary

student move from word-by-word reading to fluent decoding. One noteworthy exception to

this favorable picture was reported by Rashotte and Torgesen (1985). In this study, gains in

reading appeared only when stories shared many of the same words. Additionally, gains

did not surpass a control condition in which an equivalent amount of nonrepetetive reading

was programmed. It should be noted that Rashotte and Torgesen used only four repetitions

of the text, a number far lower than that recommended by other investigators.

The use of repeated readings is usually coupled with a high word per minute rate,

typically a rate approaching that of normal conversational speech (W. D. Wolking,

personal communication, August 3, 1988). Repeated readings may, in part be responsible

for a renewed interest in reading rate within the traditional reading community

(Allington, 1983; Anderson, 1981; Biemiller, 1978; Carver, 1982, 1983; Carver, 1987;

Fleisher, Jenkins, & Pany, 1979). Carver (1983) suggested the reading rate for mature










readers may be relatively fixed and not adapted to the nature of the reading task or level of

text difficulty as had been previously thought.

The use of repeated readings and other specialized techniques designed to maximize the

progress of individual learners casts further doubt over the ability of the Betts' criterion to

predict the most effective starting point for reading instruction-especially individualized

reading instruction programs.

Summary

The work of Betts (1946) and Killgallon (1941) helped to promote the widespread use of a

high-accuracy, 95 to 98% correct oral reading performance criterion for placing pupils in

reading material. Although lacking a sound empirical base to support the superiority of

placing a child in material with this low degree of difficulty, the Betts' criterion has

continued to exert a profound influence over reading placement decisions and has become
"validated through use" (Cadenhead, 1987). Attempts to validate the most effective

instructional level have, for the most part, failed to substantiate the relative superiority of

the Betts' criterion. Researchers have suggested that children can learn successfully in

frustration level material (Gates, 1962; Johnson, 1971; Neufeld & Lindsley, 1980; Roberts,

1976) or in material more challenging than suggested by the Betts' criterion (Kragler, 1986;

Powell, 1984; Stanley, 1986). This strongly suggests that the hypothesized frustration-

related behaviors may not be evident in otherwise well-structured instruction.

General problems with IRIs further decrease the likelihood that correct placement

decisions will be made based upon such instruments. Weaknesses in teacher scoring

(Allington, 1983), alternate-form reliability (Helgren-Lempesis & Mangrum, 1986) when

coupled with an instructional placement criterion of dubious value, make the IRI, as it has

been used, inappropriate for instructional decision making purposes. With new techniques

such as repeated readings (Samuels, 1979) designed to maximize the growth of the

individual learner, comes an additional need to reevaluate the best starting point for










remedial and individualized instruction. Researchers who have designed studies to assess

real learning in material at widely different levels strongly suggested that instruction in

the frustration level and beyond may be as or more productive than instruction beginning at

the traditional instructional level.

A study was needed to compare the Betts' criterion to a frustration level placement.

Such a study needed to use rates of learning for correct and error responding as the

dependent measure. Precision teaching provided a measurement system well suited to this

purpose as it was based upon direct, frequent, and sensitive measures of learning.

Comprehension has frequently served as the primary dependent measure in past

comparisons of the Betts' criterion to other criteria (Powell & Dunkeld, 1971; Stanley, 1986).

Precision teaching-based monitoring of oral reading was likely to prove a more sensitive

measure than pretest and posttest measures that are then statistically analyzed. The

previous emphasis on comprehension may have unnecessarily clouded the essential issue

which was basically, which placement criterion provided the best starting point for

instruction when progress in oral reading was the dependent measure? This study needed

to be conducted with students experiencing difficulty in reading and provided with an

individualized program as this was the group to which the findings were most logically

generalized. Neufeld and Lindsley (1980) may have come closest to providing a clear

answer to the criterion question for individualized instruction. Unfortunately, they relied

on a jump in grade levels to move a child into what they hoped would be frustration level

material. What was called for instead was a functional definition based on the percentage

of words correctly read. Further, the test of the frustration level should be markedly

different from the range included by various authors in the instructional reading level. A

study which defines the frustration level 10 or more percentage points below the minimum

criterion found for the instructional level would offer a suitable degree of difference. A






24



study incorporating these features might provide a convincing demonstration of the merits

of the Betts' criterion when compared to a criterion permitting a greater number of errors.















CHAPTER III
METHOD


Experimental Questions

An assumption has been made by teachers using the Betts' criterion for placement of

children in reading materials that this criterion provides the best possible starting point

for instruction and that the most effective learning takes place when children begin

reading at the level of difficulty suggested by this criterion. The problem addressed by this

study was to compare learning and performance of students in two levels of text: an easy

level (Betts' criteron) and a difficult level (frustration plus). The objective of this study

was to provide an empirical test of this assumption with mildly handicapped students in a

tutorial setting by answering the following question: Will children placed by the Betts'

criterion (95 to 98% accuracy) or the frustration plus criterion (<80% accuracy) make the

greatest gains in oral reading? The following questions constitute components of the main

question:

1. What are the learning rates for correct responding at each level of text difficulty?

2. What are the learning rates for error responding at each level of text difficulty?

3. How does comprehension compare across instructional and frustration plus levels of

text difficulty?

Subjects

The subjects for this study were 16 elementary school children from the P. K. Yonge

Laboratory School. The children were participants in an after-school tutoring program.

The median age of the children was 8 years and 6 months, with a range of from 7 through 10

years. The median grade level was second with a range of from first to fourth. None of the










students were in special education due to learning problems, although all had been

recommended for the tutoring program due to concerns about their failure to make

adequate academic progress. School policy did not permit the traditional special education

classifications to be used. These children could be seen as at-risk for special education

placement.

Tutors for this program were special education majors enrolled in an undergraduate

precision teaching course. Experimental procedures were integrated with the general

requirements for the practicum taken as a corequisite to the course. Students were not told

the specific purposes of the research. They were, however, told that a project involving two

distinctly different levels of reading difficulty was underway and that this provided them

with an opportunity to evaluate learner performance after placement under different

initial accuracy conditions. One tutor had a full-time specialist-teaching position at a

private school and worked with a subject in that setting. Except for the setting differences,

other aspects of this subject's instructional program did not differ significantly from that

of the students at the P. K. Yonge School.

Setting

Tutoring sessions took place at the P. K. Yonge Laboratory School in Gainesville,

Florida. There were 900 students enrolled in P. K. Yonge, representing all socioeconomic

and racial groups in the Alachua County, Florida school district. Tutoring took place in

an elementary classroom provided for the after-school program. A university supervisor

was on duty during each tutoring session to assist with data collection, coordinate student

efforts, and to help with problems of instruction or behavior management.

Data were gathered during the spring of 1988 over a total of 10 weeks. A school break of

1 week formed a natural break between experiment 1 and experiment 2.










Variables Under Investigation

Independent Variables

The level of text difficulty, measured by the percentage of correct words, was the

primary independent variable. Graded reading passages from the Ginn Reading Series

(levels pre-primer through 8) and the Timed Reading Series (Spargo & Williston, 1980) by

Jamestown Publishers (for levels 9 through 12) consisting of 200 word passages for each

grade level were used. The reading level of passages for the Ginn readings and the

Jamestown readings were determined by the Fry Readability Formula (Fry, 1977). The

only modification made to any passage was the selection of a segment of 185 to 200 words.

Dependent Variables

The primary dependent variables under investigation were the number of words

correctly read per minute (wpm) and the number of error words per minute. Correct

responses were those true to the text. Counted as errors were omissions, substitutions,

repetitions, and mispronunciations. The frequency values for correct and error re-

sponding were charted on a Standard Celeration Chart (DC-9EN, Behavior Research

Company, Kansas City, KS).Learning lines were drawn and the rate of learning

measured (Lindsley, 1971).

Measurement

Independent Variables

Rate of reading performance improvement for correct and error responding was assessed

each instructional session by a 1 minute oral reading timing. The learner was asked to

read two passages. One was at the grade level suggested by the Betts criterion with 95 to 98%

word recognition accuracy. This was called the instructional level placement. The

second passage, designated the frustration plus level was determined by assessment as the

level on which the child made 20% or more word recognition errors. This level of correct

responding was a full 10 percentage points below the range for the frustration reading level










found in the review of the literature. This level was intended to provide a stringent test of

the relative merits of the frustration level when contrasted to the instructional level. A 185

to 200 word passage was provided at each grade level. All passages were transcribed into

identical horizontally oriented formats and then photocopied on 8.5 by 11 in. sheet of paper.

The learner was given one copy during instructional sessions and the tutor monitored the

reading and marked errors on a second, follow along copy (White & Haring, 1980).

Specific procedures for administering the readings are described in the instructional

procedures section of this chapter.

Dependent Variables

The number of words correctly read and the number of errors made per minute are the

primary dependent measures. The learner was timed on each passage during each

instructional session. Passages were presented in random order. The tutor presented the

child with the passage and gave him the following instructions:

"I want you to read this story as quickly and as accurately as you can. If you don't

know a word say 'skip' and go on to the next word and keep reading."

The tutor set a 1-minute count-down timer and instructed the child to begin reading. At

the end of 1-minute, the tutor recorded the learner's place in the text on the follow along

sheet and asked the child to finish reading the text. When the child finished reading the

passage the tutor recorded the rates of correct and error responding, for the first minute, for

later plotting on a Standard Celeration Chart.

Comprehension was measured after the first reading of a passage and again after the

final reading of the same passage. Comprehension was measured by both the free recall or

"say facts" technique described by Lovitt (1984) and by the "answer questions" technique.

In this latter procedure the tutor read a series of six written comprehension questions to the

child and recorded the child's responses.










Experimental Design

An alternating treatments design (Barlow & Hayes, 1979; Johnston & Pennypacker,

1980; Sidman, 1960) was used in this study. This strategy involves the rapid alternation of

two or more treatments with a single subject. The subjects function as their own control,

thereby reducing threats to internal validity that may occur if subject history or maturation

effects are pronounced. The reliance on the single subject eliminates the problems of

intersubject variability, a potent source of difficulty for studies attempting to examine the

unique interaction of a single subject with various interventions.

Rapid alternation of conditions was provided for in the presentation of both levels of the

independent variable during each instructional session. The order of presentation was

randomized to control for any possible order effects. Multiple treatment interference

(Campbell & Stanley, 1963) can be a problem with some applications of this design. This

threat, in the context of this study, would concern the question of how learner performance

under the instructional reading condition in isolation from exposure to the frustration plus

level would differ from the alternating presentation of both conditions. Barlow and Hayes

(1979) minimize the threats to internal validity posed by multiple treatment interference

with this design.

Carryover (Ulman & Sulzer-Azaroff, 1975) is a type of multiple treatment interference

likely to be in operation in a study using an alternating treatments design. Randomized

order of presentation of each condition as well as the ability of students to quickly and

efficiently discriminate the relevant stimulus conditions associated with each condition

are factors which would tend to minimize carryover effects (Barlow & Hayes, 1979).

Further, Barlow and Hersen (1984) suggested that, in studies involving learning, subjects

could readily discriminate stimulus conditions without the addition of artificial signals.

A second experiment, designated experiment 2, permitted a direct replication of the

intervention (Sidman, 1960) with the same subject population. All experimental










conditions remained the same except for the use of new reading materials at levels

determined by a second round of assessment. The number of subjects participating in this

study allow for systematic replications with other change agents, other subject

characteristics, and a test of other levels of instructional difficulty (Barlow & Hersen,

1984; Sidman, 1960).

Experimental Procedures

Initial Assessment

Prior to the beginning of the intervention tutors assessed each pupil's reading level.

Tutors selected readings from the graded passages used in the intervention phase of this

study. The learner was presented with a passage at their approximate grade level and

received the following instructions from the tutor: "I want you to read this story as quickly

and as accurately as you can. If you don't know a word say 'skip' and go on to the next

word and keep reading." Tutors counted, omissions, substitutions, mispronunciations,

and repetitions as errors by marking the words on an follow along sheet with a slash. Self-

corrections were not counted individually as errors but rather each self-correction episode

was counted as one repetition error. For example, if a child self-correct a four word phrase

only one self-correction, rather than four, was counted. Counting continued for 1 minute at

which time the tutor calculated the number of correct and error words and the accuracy

percentage. Tutors progressed though the passages until they determined both an

instructional and a frustration plus level. Individual assessment of these levels

determined the two levels of the independent variable for each child.

Instruction

One half hour of instruction was provided three times each week for 10 weeks. The

tutors presented both levels of the independent variable according to a random schedule,

determined by the investigator, during each instructional session. The investigator used

the random number generating function of a Sharp, EL-506 P calculator to determine a










random series for each tutor. These series of random numbers were then used to

determine the order of presentation for each subject for each instructional session. Follow

along sheets were marked accordingly and the two sheets for each session were stapled in

the appropriate order to further prompt the tutors to follow the correct sequence. The

instructional procedures for both levels of difficulty were identical.

For each passage the following procedures were followed. The child was asked to read

the passage, receiving the same instructions used during assessment. They orally read for

1-minute while the tutor recorded all errors and noted the last word read for the timing on a

follow along sheet. The student was then instructed to continue and complete the oral

reading of the passage. The additional reading beyond the 1-minute timed segment

provided the child with exposure to the entire passage, exposure necessary for adequate

measurement of comprehension with the answer questions procedure. The tutor followed a

two-step error correction procedure. First, they pointed to each error word and said the

word, and asked the learner to repeat the word. The tutor repeated this process one

additional time. The second step involved the tutor saying the two words on either side of

the target word along with the target word and then asking the child to repeat this context

phrase. This procedure was repeated one additional time. No other instruction was

provided for this word and the tutor moved to the next error word. Only errors recorded

during this instructional session served as target words. This procedure was designed to

limit the variability in instructional procedures and reduce the chances that tutors might

engage in other forms of instruction such as phonetic or structural analysis of error words.

These procedures were derived from simple and effective techniques recommended for

students with learning problems (Haring, Lovitt, Eaton, & Hansen, 1978; Jenkins &

Larson, 1979; Lovitt, 1977; Lovitt & Hansen, 1976; Rose, McEntire & Dowdy, 1982).

The second major component of the instructional procedure involved repeated reading

and rate pacing (Berquist, 1984). Upon completion of the previous activities the tutor set up a











cassette tape recorder and played a prerecorded tape of the target passage being read

approximately 30 wpm faster than the child's current performance level. Recordings were

designed to provide a good model of oral reading by emphasizing fluency and inflection.

The learner was instructed to read aloud along with the tape. As the child's rate for correct

responding approached within 5 wpm of the recorded rate on the tape, the tutor prepared a

new recording of the passage paced an additional 30 wpm above the child's current rate.

The procedures differed for the first and last timing of each experiment. After the first

and last timing, the child was asked to provide a free-recall of facts from the story as a

comprehension check. This procedure was designated as the say-facts procedure. During

a 30-second timing, the tutor counted all meaningful facts according to the guidelines

provided during tutor training. These procedures are included in this section of the

chapter as the say-facts comprehension check. Additionally, the tutor asked a series of six

prepared comprehension questions. This was the answer questions procedure. These

questions were designed to assess recall of details, vocabulary knowledge, the main idea

of the story, and the child's ability to draw correct inferences from the reading. Tutors

presented each question orally and recorded the child's spoken response. An answer key

accompanied each set of questions. During timings in which comprehension was checked

the error correction and repeated reading procedures were not carried out.

Say-Facts Comprehension Check

Assessment of comprehension by free recall were described by Lovitt (1984). He offered

the following procedure:

Define the characteristics of the passages you want the child to describe. If, for
example, you want her to list facts about the story, then explain what a fact is. Here
is a detailed definition of a fact in this sense:
a. Any comment on the theme of the story counts as a fact (e.g., "Its a story about
lungs.").
b. Each noun plus an action verb counts as two facts (e.g., "The boy ran.").
c. Each adjective or adverb that describes a noun or verb is a fact(e.g., "The red
ball" or "She ran fast.").









d. Each preposition that describes a location is a fact (e.g.,'The ball was on the
table.").
The following are not counted as facts:
a. Repetitions
b. Value judgments about the story (e.g., "I liked the story.").
c. Nonspecific statements (e.g., "She was doing something.").
d. Information about the pictures.
Count the following as incorrect:
a. Statements that go beyond the facts of the story
b. Incorrect information about events or circumstances
c. Incorrect numbers, names, times. (p. 107)

This procedure was used to measure comprehension for the first and last reading of

each level of text for each of the two experiments.

Decision Making

An instructional aim of 200 wpm with one or fewer errors was used in this study. Two

experiments, each lasting three weeks and providing nine instructional days were

scheduled. If a learner reached aim on either reading level they completed the

comprehension assessments on both passages for that experiment. Instruction then

concluded for the passages used in experiment 1. Tutors next determined the reading

grade level of the passages to be used for experiment 2 by means of the same assessment

procedures used for experiment 1. New sets of reading passages, sets which the children

had not previously been exposed to, were used for both determining text levels and for

instruction in experiment 2. As most learners did not reach aim by the end of the first 3-

week experiment, instruction ended for the first set of passages on the ninth instructional

day. The new sets of passages were targeted for instruction beginning on the tenth

instructional day.

Materials

Tutors had, as part of their teaching supplies, a count-down timer and Standard

Celeration Charts for taking timings and charting data. They were provided with packets

of reading passages for the child's instructional reading level and for the frustration plus

level. Each passage was between 185 and 200 words in length and presented horizontally










on one sheet of 8.5 by 11 inch paper. A sample reading passage appears as Appendix A. The

structured comprehension questions were provided to the tutors for the first and last

timings of each experiment. Raw data sheets, in the form of scored reading passages, were

collected daily by the university supervisor.

Repeated reading and pacing tapes were prepared by the tutors under the direction of the

investigator. Tutors received instruction in developing their audio tapes from the

investigator. Instruction covered procedures for correct pacing of the reading, proper

inflection, and technical aspects of producing the tapes. The investigator monitored the

tapes periodically for accuracy to the text, proper inflection and voice quality, and reading

rate consistent with the experimental procedures. The investigator listened to each tape

and, guided by a monitoring sheet, examined the first minute of reading. The reading

rate was then determined by referring to the word counts on the reading passages. This

rate was contrasted to the learner's performance as assessed on the first day of instruction

for that passage. Taped readings that differed by more than five words per minute form the

child's first reading were flagged by the investigator for correction by the tutor. As a

child's rate increased to within 5 wpm of the tutors recorded rate the tutor retaped the

passage. The new target rate was an additional 30 wpm faster than the learner's most

recent performance. Each tutor was provided with an individual cassette tape and the

tapes were maintained by the investigator and made available at each instructional

session and for rerecording of reading passages. Eight General Electric 3-5016 cassette

player-recorders were used to record agreement samples and to play the rate pacing tapes.



Data Collection and Analysis

Data Collection

The tutors collected daily performance data for the number of words correctly and

incorrectly read during the 1 minute timings. These data were gathered each










instructional day for both levels of the independent variable. The tutors raw data sheets

included notation of word errors, the last word read, and rate correct and error rate. These

data sheets were collected daily. Tutors also plotted performance data on Standard

Celeration Charts and were encouraged to share this information with their student.

Comprehension data were gathered directly on a sheet of paper containing the

comprehension questions for a given passage. A summary of data gathering instructions

and space to record the say-facts data were also provided on each sheet. A sample

comprehension question sheet is shown as Appendix B.

Data Analysis

Rate data for oral reading. Daily performance data for each level of the independent vari-

able were plotted on Standard Celeration Charts by the investigator. Charted data for

reading rates and raw data for rates and comprehension measures are presented as

Appendix C. Celeration lines for corrects and errors were drawn using the quarter-

intersect method described by White and Haring (1980). The celebration values or learning

rates were computed for these lines (Pennypacker, Koenig, & Lindsley, 1972). Graphic

analysis (Parsonson & Baer, 1986) of these data yielded conclusions as to the rate of

learning under each condition of the independent variable

Guidelines for determining the quarter-intersect line were derived from White and

Haring (1980). These steps permit different assessors to draw nearly identical progress

lines from the same set of data plotted on Standard Celeration Charts. Lines are drawn

separately for each condition for corrects and errors.

1. Find the middle data point of the scores (rates) plotted during a phase. To find
the middle data point, do the following: Count the number of rates plotted on the
chart in the phase and draw a vertical line which divides the left half from the
right half.
2. Draw another vertical line through the middle data point for the first half of the
data, and a third vertical line through the middle data point for the second half of
the data. In other words, divide each of the two halves in half again, to make a
total of four segments, each with the same number of data points.









3. Find the middle rate (mid-rate) for the first half of the data and draw a
horizontal line through that mid-rate. To find the mid-rate for a set of data, do the
following; Count the number of data plots in the quarter-section and draw a
horizontal line to divide the top half from the bottom half. If the total number of
rates is an odd number, the mid-rate will fall on a rate. If the total number of rates
is an even number, the mid-rate will fall between two rates. If more than one rate
falls on the same horizontal line (i.e., the two rates are of equal value), count each
separately nevertheless.
4. Now, find the mid-rate in the second half of the data. Be sure to draw a
horizontal line through the mid-rate in each half. Make sure your horizontal
lines are level (i.e., parallel to the horizontal lines on the chart).
5. In each half, find where the vertical line which you drew in Step 2 crosses the
horizontal in which you drew in Steps 3 and 4. Draw a little circle around the spot
where the two lines intersect each other. Do this for both halves of your data.
6. Now, draw a line to connect the two intersections of mid-dates and mid-rates
(marked by the little circles you drew in Step 5). At this point, we have just
completed drawing a quarter-intersect line. (pp. 331-335)

Determining the celebration value was accomplished by laying a plastic finder (RD

8546, Experimental Education Unit, University of Washington) on the chart. The finder

was placed parallel to the day lines with the indicator point on the left portion of the quarter-

intersect line. The celebration line was then read at the right edge of the finder at the

intersection of the quarter-intersect line and the values inscribed on the finder.

Increasing or accelerating values were designated by an "X" symbol, while decreasing or

decelerating values were represented by a "" sign.

Celeration values for the performance on the instructional level were compared to

performance on the corresponding frustration plus level for each subject. The results of

these comparisons were then assigned to one of three categories. If the celebration for the

instructional level was greater than that of the frustration plus level, the result was

assigned to an "instructional superior" category. If the frustration plus level had a higher

celebration value the result was assigned to the "frustration plus superior" category. If the

celebration values did not differ by more than 10% (an X .1 celebration the results were

assigned to the "same" category.

The selection of 10% as the amount needed to establish a significant difference was

based on two considerations. First, Beck (1979), in summarizing the work of the Great
















1 I
I- 0


;. x

4



>*S


ri.

2x

r F. ^


4I


0
(00


0
('4


.0





)


31niNIiW I3d iNAOO


E)Y




S *k C4
eq 8 a 9
c: *.' '0 '0 t *~
~ ) ) 4) 4 4'* ) -'









( 8





6.^ FA
cc :3 * "i g


04
*S




4)


I



'8-



*8



' r4

i4










Falls Precision Teaching Project, has suggested regular classroom teachers (non-

precision teachers) are capable of producing a 10% change in academic behavior per week.

Lindsley (1985) was in substantial agreement with this figure. Thus, a difference of more

than 10% attributed primarily to placement in reading material of an easier or more

challenging level represents an instructional variable as large as the contribution of the

teacher. The second reason for the selection of the 10% difference concerns the amount of

progress a learner makes over a traditional unit of public school time, the 9-week grading

period. A comparison of learning rates at a X 1.1 and a X 1.2 celebration is presented as

Figure 1. A pupil learning at a rate of X 1.2 (20% per week increase) will have

accomplished more than twice as much (a factor of 2.19) for a targeted skill over the course

of nine weeks when contrasted to learning at the rate of X 1.1 (10% per week). Such a dif-

ference would be seen as a highly meaningful amount of behavior change in the context of

public school instruction. If both students started at 40 words per minute, the 10% per week

student would finish with 86 wpm and the 20% per week student would finish with 206 wpm

at the end of 9 weeks. In addition to the greater amount of behavior change indicated by

such values, the higher celebration values make it feasible for more instruction to be

delivered at higher frequencies. High frequencies have been shown to be positively

correlated with subsequent skill acquisition (Evans, Mercer, & Evans, 1983) and skill

mastery (Evans & Evans, 1985).

Each of the 16 subjects contributed one celebration comparison for corrects and one for

errors for each experiment for a total of 64 comparisons. The results of the comparisons for

experiment 1 and experiment 2 were separately analyzed for correct and error celebrations.

Question 1, regarding the highest rates for correct responding was answered by means of

16 comparisons in experiment 1 and 16 comparisons in experiment 2. The total number of

celebration values assigned to each of the three categories were determined and expressed

as percentages. The same procedures were followed for experiment 2.










Question 2, regarding the celebrations for error responding, was addressed in the same

manner. Each mispronunciation, substitution, omission, skipped word, teacher aided

word, and repetition was counted as an error. The change in error rates over time was

determined according to the same procedures as described for correct responding. A

reduction in errors, the expected direction, was assigned a "7" sign, while an accelerating

trend was given a "X" sign. A "?" directly beneath the record floor of the Standard

Celeration Chart indicates that no errors were observed during the behavior sample for that

day.

Comprehension Data. The third question posed by this investigator concerned the status of

comprehension at the two levels of text difficulty. The say facts procedure yielded

measures of comprehension after the first instructional reading of the text and then again

after the last reading. A rate of facts correct per minute and errors per minute were

determined for both reading levels. The pretest results were compared to posttest results to

determine the level on which the greatest amount of progress had been made. This was

done by dividing the posttest values by the pretest values and multiplying 100 and then

subtracting 100 from the product to obtain the percentage of increase.

The three conditions used with the rate data, instructional superior, frustration plus

superior, and same were used to categorize the results. The results of these comparisons

were then tabulated, the number in each category determined, and the sum for each

category expressed as a percentage of the total number of comparisons. These percentage

values were used to make statements about the comprehension progress under either level

of the independent variable. The ending rates for correct facts, while not comparisons,

were treated in a similar manner.

The answer questions data were treated in essentially the same manner as the say

facts data. Pretest and posttest measures for each subject's performance on each level of

difficulty were compared. Percentages of increase were determined and placed into one of










the three categories. Ending percentage values for comprehension (a value ranging

between 0 and 100%) were compared and the values assigned to one of the three categories

used for the previous classifications.

Other measures. Three sets of values were determined which help to provide additional

understanding of subject performances. Values for median learning rates for corrects

and errors for all learners, median differences between the reading grade level of each

child's instructional and frustration level passage, and last, the typical values for the

move to higher or lower levels of reading passages in the transition between experiment 1

and experiment 2.

The median celebration rates for each level of text difficulty for both errors and corrects

were determined for experiment 1 and experiment 2. These values provided a measure of

the general instructional effectiveness of each level of text difficulty. These values also

permitted statements to be made regarding the overall instructional effectiveness of the

intervention package. The reading grade level difference between the instructional and

frustration plus level passages were determined for each subject for both experiments.

These values were seen as important in determining the stability of the assessment

procedures and in supplying summary data that may be helpful to future researchers and

teachers should they seek to replicate this investigation.

The change in reading level between experiment 1 and experiment 2 for each level of

the independent variable were also determined. The number and percentage of subjects

who decreased in level, remained the same, and who increased in level were calculated for

instructional and frustration conditions. These values were deemed critical as exposure

to frustration level readings should have, based on the prevailing conceptualizations

guiding placement decisions, a deleterious affect on the motivation and degree of reading

effort put forth by children. A learner reading instructionally at the third grade level and

placed in the sixth grade level for the frustration plus condition could be expected to










experience negative attitudes toward reading and when presented with a new set of

assessment passages, perform with less enthusiasm and skill. A significant decrease at

either level between experiment 1 and experiment 2 would suggest that this frustration

effect was in operation. Alternatively, reading levels staying the same or increasing

would provide evidence that frustration was not a significant factor in the instructional

program.

Reliability

Determining the reliability of the data was accomplished in two ways. The degree to

which the independent variables were consistently and accurately implemented was

assessed and will be discussed in the procedural reliability section. The degree of

agreement between the tutors' measurement of the dependent variable and the same

measurements by an independent observer are reported for each tutor in the interobserver

agreement section.

Procedural Reliability. Procedural reliability is a measure of the degree to which the in-

structional conditions maintained by an investigator matched the conditions described in

the procedures section of an investigation (Peterson, Homer, & Wonderlich, 1982). In

order to determine the extent to which the tutors consistently and accurately carried out the

experimental conditions a monitoring system was implemented. A checklist (appears as

Appendix D) was used to routinely monitor the implementation of the instructional

conditions for each tutor. Deviations from the prescribed instructional program were

brought to the attention of the tutor and corrective action suggested. Additional monitoring,

in the form of daily supervision of each tutor's work, further insured compliance with the

instructional program.

The data collection procedures provided an additional check of procedural reliability.

Tutors provided the primary investigator or a practicum supervisor with the raw data

sheets immediately after each tutoring session. Each data sheet contained the passage










reading and included all of the observer notations of the student's performance. Data

sheets turned in incorrectly or incompletely prompted a rapid request for corrective action

from the investigator or supervisor.

The raw data sheets were used by the investigator as a vehicle which permitted further

assurance of procedural reliability. Tutors were provided with a set of readings for their

learner. The order of daily readings was randomized and the sets were labeled to reflect

the random presentation. Students were asked to follow the order of presentation

determined by the data sheets. Compliance with this requirement was easily monitored by

the on-site supervisors.

Interobserver Agreement. Interobserver agreement was calculated for each tutor. The

raw data sheets contained notations indicating that selected readings should be audio-

taped by the tutor. These audio-taped readings were then scored by an independent

observer who was trained to score the reading passages. This observer recorded the total

number of words correctly read and the total number of words read for the 1 min sample.

Tutor-scorer agreement was calculated for each sample by dividing the smaller number

by the larger and multiplying the quotient by 100. A total of 29 passages were scored to

determine interobserver agreement. The median agreement for all tutors for words read

correctly was 98% (range 91 to 100%). Median agreement for the total number of words

read was 99% (range 87 to 100%). Barlow and Hersen (1984) suggested independent

observers achieve at least 80% agreement. Cooper, Heron and Heward (1987) suggested

permanent product generated data should exhibit higher agreement than simultaneous

observational data. These authors suggested that adequate agreement measures for

permanent products should be in the high 90% range. In this investigation, the tutor's

observation constituted a simultaneous observation while the second observer was able to

rely upon a type of permanent product, in this case the audio tape of the reading. The

agreement values obtained appeared to be adequate for the purposes of this investigation.















CHAPTER IV
RESULTS


The major purpose of this study was to investigate learning rates obtained by

elementary students reading at two levels of text difficulty. Learners received similar

treatments on an Instructional reading level and on a frustration plus level passage in an

alternating treatments design. Progress for correct and error responding was calculated

for each subject for both levels for experiment 1 and for a replication designated

experiment 2. Comprehension was assessed after the first instructional session and again

after the last reading of each passage. Two procedures were used to check comprehension,

a say facts procedure and an answer questions procedure. The researcher sought to explore

the validity of the commonly used reading placement procedure which guides teachers in

placing students in text in which they read 95 to 98% of the words correctly.

The results of the three major questions posed in the study are given first. Next, other

measures of learner performance and progress in reading materials are provided.

Question 1. What are the Learning Rates for Correct Responding
at Different Levels of Text Difficulty?

Celeration values were used to measure the rate of learning for correct responding. A

comparison of celebration values between the instructional and frustration plus levels

permitted placement of results in one of three categories. When the celebration value for the

instructional reading level was higher than that of the frustration plus reading level the

subjects results were assigned to the "instructional superior" category. When the value for

frustration plus reading level was higher, a subject's results were assigned to the

"frustration plus superior" category. When the celebrations did not differ by more than 10%

per week (e. g.,. X1.1 vs. XI1.2), the two values were assigned to the "same" category.










A summary of the comparisons of celebration values for correct responding for

experiment 1 and for experiment 2 appears as Table 2 while a listing of correct celebration

values and the categorization of the comparisons appears as Table 3. For the first

experiment, the instructional reading level did not yield a celebration higher than the

frustration plus reading level for any subject (0%). The frustration plus level was

associated with a higher celebration value for 13 cases or 81% of the total. Three subjects had

celebration values that were essentially the same under either condition (19%). The

instructional reading level was not associated with the most rapid progress for any of the

subjects while the frustration plus reading level, was equal to or better than the

instructional reading level for 100% of the learners.



Table 2

Comparisons of 16 Celeration Values for Correct Responding at the Instructional Reading
Level and at the Frustration Plus Reading Level



Experiment 1 Experiment 2
Number Percentage Number Percentage
of Cases of Cases



Instructional Superior 0 0.0% 0 0.0%
Category

Frustration Plus 13 81% 13 81%
Superior Category

Same Category 3 19% 3 19%
(difference less than
10% per week)










Table 3

.flnmnnriqnns nf CePleratlon Values for Conrrect Resnonding at the Instructional Reading


PTaAmnor T4.O'70,1


flVC lht LI A U f~ qIt. W .T.? d,- T,~ a.


Experiment 1 -Experiment 2

Subject Cel INS Cel FR+ Result of Cel INS Cel FR+ Result of
Comparison* Comparison


DC

SG

QC

TE

ET

TH

TM

UR

EO

ED

EN

TB

FT

MN

CR

UT


X1 .45

X1.55

X1.40

X1.70

X1.55

X1 .25

X1.75

X1.15

X1 .25

X1.40

X1.15

X1 .25

X1.15

X1 .35

Xl .85

X1 .20


Xl.90

Xl.75

X2.50

X2.45

X1.65

Xl .50

X2.55

X1.60

X1.25

X1.75

Xl.70

X1.50

X2.00

Xl.60

X1.75

X1.33


FR+

FR+

FR+

FR+

Same

FR+

FR+

FR+

Same

FR+

FR+

FR+

FR+

FR+

Same

FR+


X1 .60

X1.23

X1 .50

X1.50

Xl1.23

X1.25

Xl.23

X1.30

Xl .25

Xl .45

X1.40

X1.05

Xl.35

X1.40

Xl.20

Xl.35


X1.65

X2.00

X1.95

X1.75

X1.28

X1 .20

X1.40

X1.55

X1.55

X1.70

XI1.90

X1 .45

X1.65

X1.95

X1.95

X1.75


Same

FR+

FR+

FR+

Same

Same

FR+

FR+

FR+

FR+

FR+

FR+

FR+

FR+

FR+

FR+


Cel INS = Celeration Value for Instructional Reading Level


Cel FR+ = Celeration Value for Frustration Plus Reading Level

Note: The reading level with the higher celebration value is designated by its abbreviation.

Same indicates a difference of less than 10% per week growth in correct responding.


&










The results for experiment 2 are similar. In no case did a subject achieve a higher rate

of learning in the instructional reading level condition when compared to the frustration

plus level. Thirteen subjects (81%) had celebration values for the frustration plus condition

that were higher than that of the instructional reading level. For three subjects (19%) there

was no difference in rate of learning on the two levels of text. For experiment 1 and

experiment 2, all subjects made gains in reading at the frustration plus level equal to or

greater than those made in the instructional reading level.

Question 2. What are the Learning Rates for Error Resoonding
at Different Levels of Text Difficulty?

Table 4 provides comparisons of the celebration values for subjects at the instructional

and frustration plus levels of passage difficulty. Table 5 provides a summary of

comparisons of learning rates on the two levels of the independent variable. The results

for experiment 1 show that four subjects (25%) had more rapid reductions for errors in the

instructional reading level condition when contrasted to the frustration plus reading

level. Eleven subjects or, 69% of the total, achieved faster reductions in errors, as

measured by their celebration values, when working with the frustration plus materials.

One subject (6%) had deceleration values that were essentially the same under both

conditions. In total, 75% of the subjects had celebration values indicating either a more

rapid or equal reduction of errors when working with the frustration plus condition when

contrasted to the instructional reading level condition. In experiment 2 five subjects (31%)

had more rapid reduction of errors in the instructional reading level condition, while nine

(56%) achieved more rapid reductions in the frustration plus condition. Two subjects (13%)

did equally well in reducing errors under both conditions. For experiment 2, 68% of the

subjects had celebration values for error reduction that equalled or surpassed the celebrations

of the instructional condition. The results of neither experiment support the contention that

the instructional reading level provides a starting point for reading instruction likely to be










Table 4

Comparisons of Celeration Values for Error Responding at the Instructional Reading
Level and the Frustration Plus Reading Level


Experiment 1 Experiment 2

Subject Cel INS Cel FR+ Result of Cel INS Cel FR+ Result of
Comparison* Comparison


DC

SG

QC

TE

ET

TH

TM

UR

EO

ED

EN

TB

FT

MN

CR

UT


/1.45

/3.20

/1.85

/1.90

/1.25

/4.20

/1.70

/1.15

/1.20

/3.0

X1.0

/1.35

/1.40

/1.07

/8.0

/1.13


/2.90

/5.30

1.70

X1.45

/1.65

/2.75

/2.70

/7.50

/1.18

/4.70

/1.80

/1.90

/1.80

/2.05

/4.40

/2.70


FR+

FR+

INS

INS

FR+

INS

FR+

FR+

Same

FR+

FR+

FR+

FR+

FR+

INS

FR+


/1.55

/2.20

/1.40

X1.20

/3.00

/1.65

/1.65

/2.10

X1.25

/1.70

/1.30

/1.45

/3.00

/1.10

/1.25

/2.45


/1.45

/2.45

/1.25

/1.33

/1.85

1.15

/1.80

/4.60

/2.70

/3.30

/1.45

/2.70

/2.50

/1.85

/1.25

/1.95


Same

FR+

INS

FR+

INS

INS

FR+

FR+

FR+

FR+

FR+

FR+

INS

FR+

Same

INS


Cel INS = Celeration Value for Instructional Reading Level
Cel FR+ = Celeration Value for Frustration Plus Reading Level

Note: Note: The reading level with the higher celebration value is designated by its

abbreviation. Same indicates a difference of less than 10% per week in error reduction.










the most productive in terms of error reduction. Rather, the frustration plus condition was,

for the majority of subjects, the level on which they achieved the greatest error reduction

rates.


Table 5

Comparisons of 16 Celeration Values for Error Resnonding at the Instructional Reading
Level and at the Frustration Plus Reading Level



Experiment 1 Experiment 2

Number Percentage Number Percentage
of Cases of Cases



Instructional Superior 4 25% 5 31%
Category

Frustration Plus 11 69% 9 56%
Superior Category

Same Category 1 6% 2 13%
(difference less than
10% per week)


While a majority of the cases were classified in the frustration plus superior category,

the results are not as definitive as were the data for the correct celebrations. Several factors

would appear to account for this difference. First, learners were prompted to read as

quickly and accurately as possible. In the frustration plus condition, with the much higher

initial error rates (by definition at least 20% of the words on the assessment trial), the child

reading faster was also likely to make more errors. As both the raw and charted data

show, the rate at which the learner read the plus passage increased steadily throughout the

experiment for most students. While the accuracy ratio (words correct to error words)

might have remained the same, the number of errors would increase along with the in-










creased rate for corrects. Typically the ratio of errors to corrects remained constant for the

early portions of the experimental phase (e. g., 15 corrects to 1 error), increasing as the

learner became more fluent with the newly learned words (e. g., 77 corrects to 1 error). A

feature of the experimental design appears to have influenced the data in this regard.

Upon reaching aim in one reading level, instruction for both levels was concluded. The

yoked design prevented the collection for performance data of the level which had not yet

reached aim, in all cases the frustration plus level. The substantial reductions in errors

found after 5 or 6 days of instruction did not have an opportunity to come into play for the

frustration plus level. This occurred in three cases cases in experiment 2. In each of these

cases the celebration for errors was higher for the instructional reading level.

Question 3. How Does Comprehension Compare Across
Widely Different Levels of Text Difficulty?

An ancillary question addressed by this investigation was a comparison of

comprehension on the instructional and frustration plus levels of difficulty.

Comprehension was assessed twice for each passage; once after the first instructional

reading of the text and then after the final reading of the passage. Two measures of com-

prehension were used for each passage. First, the say facts procedure was used followed by

six, tutor-read comprehension questions comprising the answer questions procedure. It

should be noted that no systematic instruction in comprehension was provided for any

passage on either level of text difficulty.

The results of say facts gains for both levels of text difficulty are presented in Table 6

and a summary of these comparisons appears as Table 7. In experiment 1, 11 subjects

(79%) had greater percentage increases in facts correctly stated on the frustration plus

level while only three subjects (21%) had greater increases on the instructional reading

level. The median number of facts gained was eight for the frustration plus level (range -6

to 18) and two for the instructional reading level (range -4 to 18). For experiment 2, 13























-I2 + + +3 + + +++ I -+
U. U) . . .. . .
Z W Z 19 E9E4Ef P4 P4& 9 P4 0 0 V


r.4 cq r4 -4


++ + + +.+ + + + + UO + CO 0
4. 44 4 0 4 r4 O 4 44 44 44 4 0 -4 4 "4 0 .4



|jC g L 9 g 9a eC? i 61 - j- -



13 S ~ ^^ -V kor.4


* ecv e^
I) 0 ^i oM


Q oW E- =E40 E4M E- Z E-4


0










C3
'-o
Co


u,
'0

w


1C









>, a








o
C. 05


II I


~c i +


v-4


,p g W tp E g
o' 00 M 1










subjects had higher increases on the frustration plus level (87%), while 2 (13%) had higher

increases on the instructional level. The median increase for the instructional reading

level was 2 facts (range -6 to 26) and the median increase for the frustration plus level was

10 (range 0-30). The higher percentage increases are associated with the frustration plus

condition. The absolute number of correct facts per minute was higher in the instructional

reading level for all but four cases. Table 8 presents the comparisons of ending facts per

minute values(final say facts comprehension rates) for each level of text difficulty while

Table 9 shows a summary of the comparisons.


Table 7

Comparisons of Comprehension Say Facts Percentage of Improvement Values for
Instructional Reading Level and Frustration Plus Reading Level for Experiment 1
and Experiment 2


Experiment 1 Experiment 2

Number of Cases Percentage Number of Cases Percentage



Instructional Superior 3 21% 2 13%
Category


Frustration Plus 11 79% 13 87%
Superior Category


Same Category 0 0.0% 0 0.0%
(no difference)





The say facts error rates for both levels of text difficulty in experiment 1 and

experiment 2 were extremely low. A median value of 0 fact errors per minute was

calculated with the range for all subjects being 0 to 6.























9) C40 M +U2 mm mm + +m )3
zz ~z~zzzz9~
0-4 rX4 0-- o t" 2 -4


+ + + C12 CI0J03
gp552zS 5zzo 5zzz
------ P- .- .- .-r )"P -


HT C3,
A i2












411





0







x4c


CO







0
0







c!
0
0

0





C*







4r.2
02



0
0
z














0>









0
,0























CL)
0














Z
0
'-0
0)~

0I.





o*-t




II


W-O E-- = r ;: 950 Q Z M E- -( !Z; 9E--
QoE--4 WE-iqE.4 D W w6-4 z SUQ!3










While the learners were able to generate fewer fact statements from the passage at the

frustration plus reading level when contrasted to the instructional reading level they were

likely to have shown a greater amount of improvement on the more difficult material.

Clearly, the initial complexity of the frustration plus passage, as inferred from the high

word recognition error rates, did not prevent learners from being able to make rapid gains

in their ability to generate factual statements about the text.


Table 9

Results of Comparisons of Comprehension Say Facts Ending Performance Values for
Instructional Reading Level and Frustration Plus Reading Level for Experiment 1 and
Experiment 2


Experiment 1 Experiment 2

Number Percentage Number Percentage
of Cases of Cases



Instructional Superior 12 75% 11 69%
Category


Frustration Plus 3 19% 3 19%
Superior Category


Same Category 1 6% 2 13%
(no difference)





Comprehension was also assessed by presenting students with six comprehension

questions. One subject was excluded from this portion of the experiment. Her reading

rates on the lowest level passages were so low as to rule out routine reading of the full 185-

200 word passage as would be required for comprehension assessment. Percentage of

improvement scores were calculated for each subject for both experiments and appear as



























a) + 03) ) 4) . . . . .
1CE z E Ef;~feO;0O







.-4~ - OC OO O< DLOktO O






o mz t-i t- m t-


+ +*n + +
04 g~t C- 4P
r.. fi ET -E. .


- 1
12 Cf)


^ ^D ^4 4 ^ z m F4 Z g 0 4







E-1 E. W W 1- rz4 X Q00 c)










QQOy~aEZ&aH-'ZESU


0)

>0
4.E
~bJg



00

o
Ca2




II II
Z +
.> Ak










Table 10. Table 11 provides a summary of the results for this analysis. For experiment 1,

three subjects (21%) made greater percentage gains on the instructional reading level

passage while eight subjects (57%) made greater gains on the frustration plus level pas-

sage. Three students (21%) made equal gains on both passages. In experiment 2, 1 student

(7%) made greater gains on the instructional reading level while 11 (73%) made greater

improvement on the frustration plus passage. Three students (20%) did equally well on

either passage.


Table 11

Comparisons of Comnrehension Questions Percentage of Improvement Values at the
Instructional Reading Level and at the Frustration Plus Reading Level


Experiment 1 Experiment 2

Number Percentage Number Percentage
of Cases of Cases



Instructional Superior 3 21% 1 7%
Category


Frustration Plus 8 57% 11 73%
Superior Category


Same Category 3 21% 3 20%
(No Difference)


Table 12 shows a comparison of ending comprehension values for both levels of text

difficulty for experiment 1 and experiment 2. Table 13 provides a summary of the

comparisons. Nine students (60%) had higher ending comprehension values on the

instructional level material while only 1 (7%) had a higher ending value at the frustration





























M3 SM0M0MCOQ + c) w + co
z gz^zzzzz5z z
0i- P 0.44 co 0-4 T -


20- -. 0
czzz Cl
I0-030-0303 |


UOUW E-Q= 29E4ZCME- Z!
4: (7E4 WH E, WW wH rZ4;: ^ U::)


C/2


0 r2 0
z zgI


z
0







4-4.
0









-o
0


















> w
0)

I-.


















is
0

0
z



















to0
o- a
.0


01

*-o






bi1



I-
-0)


*fr


bDe.


0 ~- CI
0:1

>00






wg >i;
1o .-Z

>S .2
-4> 03 '+ 0S
0


r-I










plus level. Five students (33%) had the same ending comprehension scores on both levels.

Similar findings were obtained for experiment 2 with 10 students (67%) scoring higher on

the instructional passage, 2 (13%) scoring higher on the frustration passage, and 3 (20%)

achieving equally well on either level.

While the learners' abilities to comprehend material, as measured by standard

comprehension questions, was inferior at the frustration plus level passage, the potential

for progress and learning


Table 13

Comparisons of Comprehension Questions Ending Percentage Values at the Instructional
Reading Level and at the Frustration Plus Reading Level


Experiment 1 Experiment 2

Number Percentage Number Percentage
of Cases of Cases



Instructional Superior 9 60% 10 67%
Category


Frustration Plus 1 7% 2 13%
Superior Category


Same Category 5 33% 3 20%
(No Difference)


was as great or greater than that offered by the instructional reading level passage. Again,

caution should be exercised in examining these results as no systematic instruction was

provided to enable the learner to improve comprehension. As a possible consequence,

comprehension was not complete ( students did not answer all comprehension questions










correctly) after as many as 18 readings of a passage. That learners reading at the

frustration plus level of difficulty did not lose ground and, in fact, made greater gains in

their ability to answer comprehension questions when compared to the instructional

reading level material, suggested that very challenging material did not prevent the

learner from developing comprehension, even when no instruction was provided for

comprehension development.

Other Measures

Two additional measures readily available from the data may prove helpful for a full

understanding of the main findings and may be supportive of conclusions made as a

result of this investigation. Included among the additional results are the differences

between the instructional and frustration plus reading grade levels between experiment 1

and experiment 2 and an analysis of the reading grade level selected during assessment

by each of the tutors.

The differences between the reading grade for the instructional level passage and the

frustration Plus passage are presented in Table 14. For experiment 1, the median

difference between the reading levels was two reading grade levels (range one to seven

grades). For experiment 2 the median difference was 2.5 grades (range one to eight

grades).

Closely related to these results are comparisons of the reading grade level within each

condition between experiment 1 and experiment 2. For example, a student working at the

instructional level on a second grade passage for experiment 1 and a fourth grade passage

for the instructional level for experiment 2 would have a difference of two reading grade

levels. The reading grades levels were determined by assessment just prior to instruction

in experiment 1 and again prior to instruction in experiment 2. The status of these levels

can be placed in one of three categories. First, the child could have stayed at the same level

for both experiments, they could have increased in level from experiment 1 to experiment 2

or they could have similarly decreased. These comparisons appear as Table 15. The
























c 4f c4> ci 4- vif P4 o6 c4c ooIN c~i 6- 4n*





C OO C OOOOOOOOO C R RO RO RO R COOO






4 v V I m ul e t- q 1- V4










0000000000000000


QOUWE-iZOPZME-iZ9E-
P W (YE-aW E4 E4 W WE-4 r, 20::


+G)


+0)
^Z4


cl C6 14; 00
























-


II
0 II
U



00s


0)





0










Z -4


b4 F





















4) 4) a)
U2 g 2
Q) 4) 4)
V C
P4 P-.


to -I N Lo v 1 m m Dw m


UOOWM- :29OQZPME-4Z E6-'
Q (7 1- W 1-4E-4 :D W WE-4ON21) ;


ci~


p~Ci2


UC4 I CC


4) 4)

-

P. 0., 0. a. "I C'
CL4eQ 4 Q o c


t 9i








r. .

a4
CO4 C9











co ci




cq
w5 W


0 0 0 0 V V 0-4 0N M V-4 r-4N' Ol










results of these comparisons (appearing as Table 16) show that, for the instructional

reading level six subjects worked at the same level (38%), while 9 (56%) increased to higher

reading grade level passages. One subject (6%) worked at a lower level and would be

classified as decreasing. For the frustration plus level 2 subjects worked a the same level

(13%), 1 decreased (7%), and 13 increased (81%).

The general trend toward a higher level can be explained, in part, by the fact that the

students had greater familiarity with the materials, the instructional format, and their

tutors. Initial nervousness may have caused some children to perform less well than they

would have otherwise. Of relevance to the purposes of this study was the finding that

motivation and effort, as measured by the reading performances on the second

assessment, were not impaired. The students' reading

Table 16

Comparisons of Reading Grade Placements Within the Same Instructional Condition
Between Experiment 1 and Experiment 2


Instructional Frustration Plus
Reading Level Reading Level

Number Percentage Number Percentage
of Cases of Cases



Reading Grade Level 9 56% 13 81%
Increases


Reading Grade Level 1 6% 1 6%
Decreases


Reading Grade Level 6 38% 2 13%
Remains the Same












showed no evidence of being less skillful or less motivated. Exposure to the frustration

plus level readings, material with a level of difficulty hypothesized to be far too great to

permit success, did not produce a reduction in effort or precipitate a loss of motivation on

the part of the students that might have resulted in poor performance.

Summary of Results

The answers to the three major questions of this study as well as additional measures

have been detailed in this chapter. The first question was: What are the learning rates for

correct responding at different levels of text difficulty? The answer was that the

frustration plus level of difficulty was associated with higher rates of learning or with

rates that equalled that of the instructional reading level for 100% of the students in

experiment 1 and for 100% of the students in experiment 2. The second question was: What

are the learning rates for error responding at different levels of text difficulty? The

answer to this question again favors the frustration plus reading level. In experiment 1,

75% of the students had error learning rates on the frustration plus passages that either

equalled or surpassed those of the instructional reading level. For experiment 2 similar

results were obtained with 68% of the learners doing better or as well on the frustration plus

passage when compared to the instructional passage.

The third question was: How does comprehension compare across widely different

levels of text difficulty? In both experiment 1 and experiment 2 increases in

comprehension were greatest for the frustration plus reading level. Comprehension gains,

as measured by both the say facts procedure and an answer questions procedure, favored

the more difficult reading condition. Both measurement procedures demonstrated,

however, that comprehension, for either the first reading of the story or the last, was clearly

superior, in absolute terms, for the instructional reading level.










Additional summary data were presented on the reading grade levels for both

conditions and for both experiments. The median difference between the instructional

reading level and the frustration plus reading level was two grades for experiment 1 and

2.5 grades for experiment 2. In experiment 2, 56% of the subjects worked at a higher grade

level of material in the instructional level condition. For the frustration plus condition,

88% of the children worked on more difficult material. These data are offered as evidence

that motivation and effort were not deleteriously effected by exposure to the frustration plus

level reading material. It was quite possible that the challenge presented by the frustration

plus level materials actually motivated students to put forth a greater effort.

In general, the more difficult reading material was associated with higher rates of

learning for corrects, errors, and for growth in comprehension. Initial performance on

the second experiment and the high rates of learning for both levels of text in experiment 1

and experiment 2 suggested that motivation and effort were not harmed by exposure to the

more challenging reading material. All of these results seem to be relatively stable as

demonstrated by the similarities between the results of experiment 1 and experiment 2.














CHAPTER V
DISCUSSION AND CONCLUSIONS

In this chapter the results of the study are considered in light of reading placement and

instructional practices. Conclusions based on the data are offered. These conclusions

then served as the foundation for specific recommendations. Four recommendations for

improving the effectiveness of reading instruction are given. While most of the

recommendations related directly to instructional arrangements for special students or

learners receiving individualized instruction, they may have applicability to a wide

range of students receiving reading instruction. As mentioned previously, the Betts'

placement criterion was believed to promote maximal learning for the largest number of

students. The findings of this study failed to support this contention for students with mild

learning problems who are receiving individualized reading instruction.

Methodological Progress in Reading Research

As results of this study provided evidence contradicting previous findings, several

beliefs about the ability of learners to improve their reading performance under various

instructional conditions should be questioned. A traditional reliance on group statistical

research may have interfered with efforts to assess individual responses to large

differences in the difficulty of reading material. The early works of Betts (1946) and

Killgallon (1941) are examples of the reliance on indirect measures of reading ability to

determine a placement criterion. Killgallon, as Betts' student, administered IRIs to a

group of pupils and then determined their performance on the comprehension portion of the

IRI. Killgallon next applied an arbitrary comprehension criterion of 75% correct to the

data and selected only the corresponding reading scores meeting the criterion. The mean

of those scores was 93.9% which was later rounded off to yield the widely used 95%










placement criterion. Progress in oral reading was not a consideration. With this early

placement criterion research, as with more recent researchers efforts to determine the best

starting point for reading instruction (Powell, 1984; Kragler, 1986; Stanley, 1986), the

suggested placement guidelines were based not on progress in oral reading but rather on

predictions as to which level of oral reading accuracy predicted what level of

comprehension. It is possible to accurately measure progress in oral reading by means (e.

g., precision teaching) not available to early researchers (Starlin, 1971). An historical in-

ability to separate oral reading performance from comprehension skill may have

unnecessarily complicated research efforts seeking to determine the most fruitful starting

points for reading instruction (Starlin, 1971). While comprehension is a critically

important component of reading, it may not be the most reliable dependent measure in

experimental efforts to find the best starting point for instruction ( Killgallon, 1941;

Kragler, 1986; Powell, 1984; Stanley, 1986). The few published studies that used rate for

correct and error responding, most notably Johnson, (1971) and Neufeld and Lindsley

(1980), have arrived at conclusions similar to those offered by this investigator.

The first and clearest conclusion derived from this investigation was that the

instructional reading level, with initial word recognition accuracy of from 95 to 98%

correct, was not the most productive starting point for instruction for most learners studied.

In experiment 1, none of the learners made most rapid progress for correct responding

when this placement criterion was applied. The results of experiment 2, in which no

subject had better growth on the instructional level, provided additional confirmation.

Error reduction data lead the researcher to suggest a similar conclusion. The data failed

to support the efficacy of the Betts' instructional level for students with mild learning

problems receiving individualized reading instruction.










Factors Suopporting Continued Use of the Betts' Criterion

Why, then, is this traditional placement criterion so widely used and recommended

even for special learners receiving individualized instruction (Cook & Earlley, 1979;

Hargis, 1982; Mercer & Mercer, 1981; Savage & Mooney, 1979). Speculation on this point

would be premature except to point out that by definition, as the findings of this investiga-

tion verify, low error rates on an assessment passage were clearly associated with low

error rates during subsequent instruction on the same passage. If low errors, either

initially, concurrently, or terminally are to be used as the sole measure of instructional

effectiveness, then placement of a learner by the traditional instructional criterion

assures success. Reading teachers using traditional techniques do not appear to have had

a measurement technology capable of determining improvement or learning (Starlin,

1971). Therefore, low-error performances may have substituted for a measure of progress

or learning for some teachers. Unfortunately, progress on reading and the development of

capable readers was not necessarily fostered by such an unchallenging level of material

difficulty.

Perhaps the key feature in answer to why the instruction level placement continues to

enjoy such currency relates directly to its origins. Betts, Killgallon and other investi-

gators of that era were concerned almost exclusively with large group reading instruction.

Large group instruction with one teacher may call for reading materials that children can

read with a minimal level of assistance. Low errors permitted the teacher to manage the

instruction most readily. This may have been a legitimate use by the teacher of a

placement criterion and a tradition of assigning children to reading materials. But, the

assumption that this placement level offers the greatest growth potential has been seriously

challenged by the results of this investigation.

Large group instruction continued to give way to more sophisticated, smaller group ar-

rangements. Special learners served under the provisions of Public Law 94-142 were










guaranteed an individualized educational program and, for the majority of these

children, reading instruction is to be delivered as part of an individualized educational

program. These types of arrangements call for reading instruction to be individualized so

as to permit maximum progress based on the assessed needs of the child. A placement

criterion designed to facilitate large group instruction must surely be reconsidered and, in

light of these findings, challenged.

Potential Utility of High-Error Reading Material in Individualized
Reading Programs

A second issue raised by this investigation concerned the value of the frustration level.

This level, typically 90% or less initial word recognition accuracy on an assessment

passage, was considered to be so difficult and so likely to produce negative emotional reac-

tions that teachers were cautioned not to place children in frustration level readings. It is

important to recall that the frustration level used in this study provided for at least 20%

errors. Interestingly, while the instructional and independent levels both have names

suggesting instructional arrangements the frustration plus level alone bears a name

referring to a hypothetical mental state. The label itself tends to bias teachers against

placing students in more difficult reading material. Frustration reading level has

become, as Cadenhead (1987) has suggested regarding the general use of the term "reading

level," a powerful metaphor that shapes practice.

The results of this study quite clearly demonstrate that, when provided with an

instructional program similar to those recommended in the literature as appropriate for

special education students, the frustration plus level may be the more productive starting

point for instruction. In both experiment 1 and 2, 100% of the learners made greater or

equal progress on the frustration plus level when contrasted to the instructional Level. The

results for error responding on both experiments yielded similar, though not as conclusive,

findings. The frustration plus level was associated with higher rates of learning for










corrects and errors. While the absolute level of comprehension is certainly worse under

the frustration plus condition, the rate of growth for comprehension as measured by either a

say facts or answer questions procedure was better under the frustration plus level. While

no systematic instruction in comprehension was provided during this study, the findings

suggest to the investigator that a systematic program of comprehension instruction at the

frustration plus level would produce comprehension gains surpassing those made at the

instructional level. A child's comprehension may have started much lower on the

frustration plus level, but it typically grew more rapidly and the learner reached terminal

proficiencies similar to th )se obtained on the instructional level material.

Comprehension learning was not adversely affected under the frustration plus condition

and this was a finding of some importance.

In addition to the superior rates of learning under the frustration plus condition, the

overall lack of the inferred frustration reactions was significant. It was noted by the tutors

that some learners balked at the frustration plus reading task during the first few days of

experiment 1. Most of these incidents were no greater in intensity than had been

reluctance to engage in other, nonpreferred instructional activities such as spelling or

handwriting. Many of the tutors reported that their learner preferred the more challenging

material and looked forward to "beating" the rate of the audio-taped reading passage.

Significant noncompliance was not noted for experiment 2. No unusual behaviors that

would have indicated educationally relevant frustration were noted. The high rates of

learner progress for both levels of the independent variables are consistent with the

learning accomplished by effectively motivated students (Starlin, 1971) and not at all

characteristic of learners who are frustrated. Similarly, the trend observed in which

students moved to higher reading levels for both instructional and frustration plus levels

for experiment 2 suggest that the learners were eager and motivated to improve their

performance using even more challenging material. The findings reported by this










investigator contradicted the long-held but poorly supported assumptions about reading

behavior in challenging material.

A Simple and Effective Set of Instructional Procedures

While not one of the formal questions posed by this investigation, the success of a

relatively simple package of instructional procedures was demonstrated. The median

rate of learning for correct responding was X 1.53 and an error value of/1.78 was

obtained. This represented a 53% improvement per week for corrects and a 78% reduction

for errors across all students and all levels of reading passages. Public school learning, to

put these values in perspective, was estimated by Beck (1979) to yield a 10% average change

per week. The use of two simple error correction procedures and allowing the child to read

along with a tutor-prepared audio tape paced 30 words per minute faster than the learner's

performance constituted the entire instructional sequence. These procedures were

intentionally kept simple for the purposes of experimental uniformity but their potential

utility in classrooms appears to be high. These procedures may derive a great deal of their

effectiveness from the emphasis on frequent, high-rate responding by the learner.

Limitations of the Study and Recommendations for Future Research

A limitation described in the results section of this study was imposed by the nature of

the yoked design. As a learner reached aim on any level of text difficulty, instruction

stopped for both levels. This avoided the problem of having the child prompted to read at

potentially unfunctionally high rates on one passage. This yoking failed to capture the

anticipated performance improvements at the reading level which did not reach the

performance aim, in all cases the frustration plus level. Allowing the tutors to continue

instruction on the frustration plus level passage until the end of the prescribed nine

instructional day period could have alleviated this limitation and may have made the

results of this study even more clearly in favor of the frustration plus strategy.










Another limitation was more methodological and less procedural. Limits on response

opportunities in the form of both ceiling and floor effects (Haughton, 1972) must be taken

into consideration. A child who was making very few errors had less opportunity to

achieve a substantial reduction in errors. Similarly, a reader at initially high correct

rates may not have been able to accelerate their performance as dramatically as the

learner who began at a much lower rate. Fortunately, an analysis of these data suggested

that even those learners who began instruction at higher correct rates made gains

comparable to lower rate learners on similar levels of passage difficulty. Error rates

posed a special problem in that the floor effect was more likely to have been encountered by

the learner on the instructional level passage. The charted data provided an excellent

graphic display of this potential problem. Typically error rates for the instructional level

were not reduced to '0' (record floor) until, if at all, the last few instructional days of the

experimental phase. With no opportunity to achieve further reductions in errors the

celebration values would flatten out. Had such a floor been reached earlier, as might be the

case with sophisticated readers or learners placed in very easy material, the resulting

celebration comparisons may not have been appropriate.

The use of an abbreviated version of the Betts' criterion was also a limitation. This

study used only the word recognition accuracy portion of the criterion and excluded the

comprehension component. This was done to facilitate placement in materials and to

avoid time-consuming comprehension assessment of all relevant reading levels. The

summary data on comprehension indicates that an approximation of the Betts'

comprehension criterion was incidentally accomplished for the instructional as well as

the traditional frustration level. A replication of this study using both components of the

Betts' criterion appears to be warranted although it seems that the addition of the

comprehension criterion would have exaggerated the present findings and have more

clearly demonstrated the potential of the frustration level material. Using the










comprehension component of the Betts' criterion would have moved some students to lower

error (easier) readings at the instructional level while having had no effect on placement

for the frustration plus level.

Further research is warranted into the questions posed and raised by this study. A

question not addressed was the best initial accuracy placement criterion. Parametric

studies comparing learner progress at various accuracies, such as 75, 85, or 90% would help

to clarify this issue. Also deserving further study is the question of whether individual

learner characteristics are better addressed by a higher or lower level of material

difficulty. This question, to be answered with any validity, would have to be carefully

structured to prevent the learner's subjective feelings about the material from determining

the course of the instructional program. What learners may say about the work would have

to be far less important than how they performed.

Instructional Recommendations

The results and conclusions of this investigation lead to a series of recommendations

for improving reading instruction for elementary school age children taught on a one-to-

one basis. These four recommendations are presented in order of priority.

1. The practices associated with using the frustration level as a designation for

reading material with 90% or less initial word recognition accuracy should be scrutinized.

This researcher has shown that children permitted to read material much more

challenging than the 90% accuracy level can, for most learners studied, hope to enjoy

greater progress than that which would occur with materials guided by the instructional

level criterion. The combination of challenging materials and effective instruction

appeared to offer the greatest likelihood of rapid progress in reading. The entrenched use

of the term "frustration level" by its very nature implies an undesirable, unteachable, and

unprofessional range of difficulty in which to provide instruction.










The term 'frustration' should be replaced when referring to reading material with a

high potential for instructional success. Powell has suggested the "emergent level" as the

region in which, with appropriate mediation and instructional support, the child will make

greatest progress. Powell's criterion, however, is too closely tied to comprehension and is

not based on progress-sensitive, performance data. A term specifying the necessary

instructional arrangements would be most desirable and would be consistent with the

nomenclature for the independent and instructional levels. The "tutorial level" or

perhaps the "individualized reading level" are each worth consideration. Alternatively, a

designation specifying the instructional objective of the reading plan is recommended.

Terms such as "error reduction level" or "fluency building level" are offered for

consideration. A clear statement that the challenge range found to be most productive in

this investigation may be an appropriate and, in fact, a desirable range in which to provide

instruction is called for.

With a change in terminology, a change in both placement and instructional practices

should be considered. Rather than placing learners in material in which they seldom

make errors, the students could, based on the data-guided judgment of the teacher, be placed

in material with far greater potential for progress. Progress would be determined by

increases in correct responding and decreases in error responding over a given period of

time.

2. Instructional programs designed to provide instruction consistent with the view that

the 95 to 98% initial accuracy level, the instructional level, yields the greatest potential for

reading growth should be reassessed in light of the findings of this and other studies

(Johnson, 1971; Kragler, 1986; Neufeld & Lindsley, 1980; Stanley, 1986). Such

reassessment would appear to be especially necessary for special education programs and

other instructional arrangements in which the teacher is obligated to offer individualized

educational programs. To continue to maintain that teaching and placement practices










consistent with the Betts' criterion or some low-error variant are likely to yield the greatest

learning, in light of these findings, seems unrealistic. At worst, such continued support

for the Betts-type criterion by authors of reading textbooks and by special education

researchers suggests a disregard for pupil progress and a mechanical adherence to

teacher-convenience issues.

3. An instructional program similar to that used in this study warrants consideration

for more widespread adoption for use in tutoring, special education resource rooms, and in

general education classrooms. The essential components of the instructional program

including repeated readings with paced audio tapes, simple error correction procedures in

which the tutor tells the learner how to say error words, and then placing the error word in

context for two practice trials, are easily replicated with little cost or effort. This set of

procedures avoids possibly unnecessary drill in structural or phonetic analysis and

appears to maintain effective instructional momentum. As Lovitt (1977) has suggested, it

is best to use the simplest instruction such as telling the child the correct word, before

resorting to more complicated strategies.

4. The last recommendation is that reading instructional programs as well as reading

research should begin to make use of direct and continuous measures of learner

performance. This would permit more accurate appraisal of individual differences in

learning and allow the performance of learners to suggest to teachers the most appropriate

instructional decisions. In regard to research, past reliance on reading comprehension as

the dependent measure for reading program effectiveness appears to have clouded the

nature of individual performance gains in reading. Direct and repeated measures of oral

reading performance may provide the best measure of gains in oral reading skill.















REFERENCES


Algozzine, B., & Maheady, L. (1986). When all else fails, teach! Exceptional Children. 92.,
487-488.

Allington, R. L. (1979). Teacher ability in recording oral reading performances.
Academic Therapy, 14, 187-192.

Allington, R. L. (1983). Fluency: The neglected goal. The Reading Teacher. 3, 556-561.

Anderson, B. (1981). The missing ingredient: Fluent oral reading. The Elementary
School Journal. 81,173-177.

Barlow, D. H., & Hayes, S. C. (1979). Alternating treatments design: One strategy for
comparing the effects of two treatments in a single subject. Journal of Applied
Behavior Analysis, 12,199-210.

Barlow, D. H., & Hersen, M. (1984). Single case experimental designs: Strategies for
studying behavior change. New York: Pergamon.

Beck, R. (1979). Precision teaching project: Sacaiawea plan II. Great Falls, Montana:
Great Falls Public Schools.

Bennett, R. E. (1983). Research and evaluation for special education assessment.
Exceptional Children. 50, 110-117.

Bergquist, L. (1984). Rapid silent reading: Techniques for improving rate in
intermediate grades. The Reading Teacher, 38, 50-53.

Betts, E.A. (1941). Reading problems at the intermediate grade level. Elementary School
Journal, 49, 737-746.

Betts, E. A. (1946). Foundations of reading instruction. New York: American Book.

Biemiller, A. (1978). Relationships between oral reading rates for letters, words, and
simple text in the development of reading achievement. Reading Research
Quarterly, 13, 223-253.

Cadenhead, K. (1987). Reading level: A metaphor that shapes practice. Phi Delta Kappan,
68,436-441.

Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs
for research. In N. L. Gage (Ed.), Handbook of research on teaching (pp. 171-246).
Chicago: Rand-McNally.

Carver, R. P. (1982). Optimal rate of reading prose. Reading Research Quarterly. 28, 56-
88.










Carver, R. P. (1983). Is reading rate constant or flexible? Reading Research Quarterly,
28, 191-215.

Carver, R. P. (1987). Teaching rapid reading in the intermediate grades: Helpful or
harmful? Reading Research and Instruction, 26, 65-76.

Chomsky, C. (1976). After decoding: What? Language Arts, 53, 288-296.

Cook, J. E., & Earlley, E. C. (1979). Remediating reading disabilities. Germantown,
MD: Aspen Systems.

Cooper, J. C., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis.
Columbus, OH: Charles E. Merrill.

Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative.
Exceptional Children, 2, 219-232.

Dowhower, S. L. (1987). Effects of repeated reading on second grade transitional reader
fluency and comprehension. Reading Research Quarterly, 22, 389-406.

Durkin, D. D. (1978-1979). What classroom observation reveals about reading
comprehension. Reading Research Quarterly, 14, 481-533.

Ekwall, E. K. (1986). Ekwall reading inventory (2nd ed.). Boston: Allyn and Bacon.

Evans, S., & Evans, W. (1983). Frequencies that ensure competency. Journal of Precision
Teaching, 6, 25-30.

Evans, S., Mercer, C., & Evans, W. (1983). The relationship of frequency to subsequent
skill acquisition. Journal of Precision Teaching, 4, 28-35.

Fleisher, L. S., Jenkins, J. R., & Pany, D. (1979). Effects on poor readers' comprehension
of training in rapid decoding. Reading Research Quarterly, 15, 30-48.

Fry, E. (1977). Elementary reading instruction. New York: McGraw-Hill.

Gates, A. I. (1962). The word recognition ability and the reading vocabulary of second and
third grade children. Reading Teacher, 15, 443-448.

Gentile, L. M., & McMillan, M. M. (1987). Stress and reading difficulties. Newark, DE:
International Reading Association.

Ginn Reading Program. (1984-1982). Lexington, MA: Ginn.

Hargis, C. H. (1982). Teaching reading to handicapped children. Denver, CO: Love.

Haring, N. G., Levitt, T. C., Eaton, M. D., & Hansen, C. L. (1978). The fourth R:
Research in the classroom. Columbus, OH: Charles E. Merrill.

Harris, A. J., & Sipay, E. R. (1980). How to increase reading ability (7th ed.). New York:
Longman.









Haughton, E. (1972). Aims-growing and sharing. In J. B. Jordon & L. S. Robbins (Eds.),
Let's try doing something else kind of thing (pp. 20-39). Arlington, VA: The
council for Exceptional Children.

Helgren-Lempesis, V. A., & Mangrum, C. T. (1986). An analysis of alternate form
reliability of three commercially-prepared informal reading inventories.
Reading Research Quarterly, 21, 209-215.

Howell, K. W. (1986). Direct assessment of academic performance. School Psychology
Review, 15,324-335.

Jenkins, J. R., & Larson, K. (1979). Evaluating error-correction procedures for oral
reading. The Journal of Special Education. 13, 145-156.

Johns, J. L. (1986). Handbook for remediation of reading difficulties. Englewood Cliffs,
NJ: Prentice-Hall.

Johnson, M. J., Kress, R. A., & Pilulski, J. J. (1987). Informal reading inventories (2nd
ed.). Newark, DE: International Reading Association.

Johnson, N. J. (1971). Acceleration of inner-city elementary school pupils' reading
performance (Doctoral dissertation, University of Kansas, 1971). Dissertation
Abstracts International, 32, 6250A.

Johnston, J. M. & Pennypacker, H. S. (1980). Strategies and tactics of human behavior
research. Hillsdale, NJ: Lawrence Erlbaum.

Killgallon, P. A. (1941). A study of the relationships among certain pupil adjustments in
language situations. Unpublished doctoral dissertation, Pennsylvania State
University, State College.

Kragler, S. K. (1986). Static versus dynamic assessment: The impact of reading
placement on reading underachievers (Doctoral dissertation, University of
Florida, 1986). Dissertation Abstracts International. 47, 1673A.

LaBerge, D., & Samuels, S. J. (1974). Toward a theory of automatic information
processing in reading. Cognitive Psychology, 6, 293-323.

Lindsley, 0. R. (1971). Precision teaching in perspective. Teaching Exceptional
Children, 3, 114-119.

Lindsley, 0. R. (1985). Quantified trends in the results of behavior analysis. Presidential
address at the Eleventh Annual Convention for Behavior Analysis, Columbus, OH.

Lively, B. A., & Pressey, S.L. (1923). A method for measuring the "vocabulary burden" of
textbooks. Educational Administration and Supervision. 9, 389-398.

Lovitt, T. C. (1977). In spite of my resistance. I've learned from children. Columbus, OH:
Charles E. Merrill.

Lovitt, T. C. (1984). Tactics for teaching. Columbus, OH: Charles E. Merrill.

Lovitt, T. C., & Hansen, C.L. (1976). Round one--Placing the child in the right reader.
Journal of Learning Disabilities, 9 (6), 18-24.









Mercer, C. D., & Mercer, A. R. (1981). Teaching students with learning problems.
Columbus, OH: Charles E. Merrill.

McCracken, R. A. (1966). Standard reading inventory. Kalmath, OR: Kalmath Printing.

Mithaugh, D. E., Horiuchi, C. C., & Fanning, P. N. (1985). A report of the Colorado
statewide follow-up study of special education students. Exceptional Children, 51,
397-404.

Moyer, S. B. (1982). Repeated reading. Journal of Learning Disabilities. 15, 619-623.

Neufeld, K. A., & Lindsley, 0. R. (1980). Charting to compare children's learning at four
different reading performance levels. Journal of Precision Teaching, 1 (1), 9-17.

O'Shea, L. J., Sindelar, P. T., & O'Shea, D. J. (1987). The effects of repeated readings and
attentional cues on the reading fluency and comprehension of learning disabled
readers. Learning Disabilities Research, 2, 103-109.

Parsonson, B. S., & Baer, D. M. (1986). The graphic analysis of data. In A. Poling and R.
W. Fuqua (Eds.), Research methods in applied behavior analysis: Issues and
advances (pp. 157-186). New York: Plenum Press.

Pennypacker, H. S., Koening, C. H., & Lindsley, 0. R. (1972). Handbook of the Standard
Behavior Chart. Kansas City, KS: Precision Media.

Perfetti, C. A. (1984). Some reflections on learning and not learning to read. RASE, 5 (3),
34-38.

Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent
variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477-492.

Powell, W. R. (1984). Mediated (emergent) reading levels: A construct. In J. A. Niles
(Ed.), Thirty-third yearbook of the National Reading Conference (pp. 117-123).
Rochester, NY: National Reading Conference.

Powell, W. R., & Dunkeld, C. G. (1971). Validity of the IRI reading levels. Elementary
English, 48, 637-642.

Public Law 94-142. (1977, August 23) Federal Register.

Rashotte, C. A., & Torgesen, J. K. (1985). Repeated reading and reading fluency in
learning disabled children. Reading Research Quarterly, 2U, 180-188.

Resolutions Passed by Delegate Assembly, (1981). Reading Research Quarterly fi, 16,615.

Roberts, T. (1976). "Frustration level" reading in the infant school. Educational
Research, 19I (1) 41-44.

Rose, T. L., McEntire, E., & Dowdy, C. (1982). Effects of two error-correction procedures
on oral reading. Learning Disabilities Quarterly, 5, 100-105.

Salvia, J., & Ysseldyke, J. (1981). Assessment in special and remedial education (2nd
Ed.). Boston: Houghton Mifflin.

Samuels, S. J. (1979). The method of repeated readings. The Reading Teacher, 32, 403-408.










Samuels, S. J. (1986). Why children fail to learn and what to do about it. Exceptional
Children, 53, 7-16.

Savage, J. F., & Mooney, J. F. (1979). Teaching reading to children with special needs.
Boston: Allyn and Bacon.

Shanahan, T. (1983). A critique of P. A. Killgallon's study: A study of the relationships
among certain pupil adjustments in reading situations. In L. Gentile, M. Kamil
& J. Blanchard, (Eds.) Reading research revisited (pp. 577-582). Columbus, OH:
Charles E. Merrill.

Sidman, M. (1960). Tactics of scientific research. New York: Basic Books.

Silvaroli, N. J. (1976). Classroom reading inventory (3rd ed.). Dubuque, IA: William C.
Brown.

Smith, D. D. (1979). The improvement of children's oral reading through the use of teacher
modeling. Journal of Learning Disabilities, 12(3), 39-42.

Spache, G. D. (1981). Diagnosing and correcting reading disabilities (2nd ed.). Boston:
Allyn and Bacon.

Spargo, E., & Williston, G. R. (1980). Timed readings series. Providence, RI:
Jamestown.

Stanley, N. V. (1986). A concurrent validity study of the emergent reading level (Doctoral
dissertation, University of Florida, 1986). Dissertation Abstracts International, 47,
1675A.

Starlin, C. (1971). Evaluating progress toward reading proficiency. In B. Bateman (Ed.),
Learning disorders: Volume 4. Reading (pp. 389-465). Seattle, WA:. Special Child.

Thurow, L. C. (1987). A surge in inequity. Scientific American, 256 (5), 30-37.

Ulman, J. D., & Sulzer-Araroff, B. (1975). Multielement baseline design in educational
research. In E. Ramp & G. Semb (eds.), Behavior analysis: Areas of research and
application (pp. 371-391). Englewood Cliffs, NJ: Prentice-Hall.

White, 0. R., & Haring, N. G. (1980). Exceptional teaching (2nd ed.). Columbus, OH:
Charles E. Merrill.

Will, M. C. (1986). Educating children with learning problems: A shared responsibility.
Exceptional Children. 52, 411-415.

Wolking, W. D. (1987). Handbook of high performance instruction. Gainesville, FL:
Performance Data.

Woods, M. L., & Moe, A. J. (1985). Analytical reading inventory (3rd ed.). Columbus,
OH: Charles E. Merrill.

Yarington, D. J. (1979). The great American reading machine. Rochelle Park, NJ:
Hayden.





















APPENDIX A

SAMPLE READING PASSAGE










Vq 0 Lo ONO (N 00 CO r- N > D N- (N CO "D (N
^-l (N 'Ir LO lZ o0 Dr' VH (N V Lo L0 r- r- co C
TH V-i IH T-H T-H %H TH r-1 (N


4-1- 0
4 C4) 0.
1 4, b b0 w
4-' 0 0t 5 o 0..
+ W* +, .- -o LO
a) 'U rjS ^
H -- 0
'- a) a) -^ '; ^
0 a) ^ a U) 0 0
a) ) (- U
.> 1 0 a)
i > y Q ( 5 U)

U bOo 0
4- *g 0
(t5 M W Z a)

o 4- ^ r "5- a
0T -~ >^ *4- > -^ a) b
l- >. a) "5 0 -
H ) C 4-1 -
0 0) q:1 -0'J
0- 4-J

05 af) > g s bO
.- a )O ~ W' U)
4-) 4-


C .0 US




| ^ | ,-IkI
0 0 0 4) 0




+-~ ~-' ^ bO *4 jS ('5 -4-S .^ 0 S -
bO - 0 .0

>1 0
U 0 'I- I.~ i' 0
4- UO a) T U) bO
u bO 4- "5 .0 -0 U)



ro ~> 05 1wI^ : ^



C ^ (- .5 ^ .i ^ O -4-' _. )
10 0 ^ I U)cX:
LV c0~*. ~ : 4-' )U
-~~~ U)) U b 5





>- 0 > 0 w >1


-" .0 .& 0. a)O -w w 0 5 J "5
0~~~ )4 4' .
U)~ '4- 14- $-

0- -






S w 0 >' '- l4- 0 a) U ('5If ^
('5 4 S a S
C'3 > 0) U .0 a) 0 .a 0





80





















APPENDIX B

SAMPLE COMPREHENSION QUESTION SHEET













Level 4 THE MYSTERY OF THE ROLLTOP DESK

Tutor's name_____ Date_____

READING COMPREHENSION

Note to tutors: Use the comprehension checking procedure only twice for each reading

passage. Once after the first full reading of the text and again on the last day of

instruction for the passage.

Do the Say-facts before the structured questions. If you do the structured questions first

you unfairly suggest a response format to the child and you give an advantage on the

say-facts.

Turn this sheet in to Jack or Jolena on the day you gather these data.

SAY-FACTS Do a 30 second timing and be sure to multiply your count by 2 to get the count

per minute.

Correct facts per minute ____ Errors per minute___

Tallies can go here





Tutors: Please read these questions in the order given and do not allow the learner to go

back and correct or change an answer. (Questions that follow may provide answers to

earlier questions.)

(mi) 1. What strange thing is happening in this story?

(A strange man has followed Mother home.)

answer:



(f) 2. What was the color of Mrs. Marsh's pick-up truck?

(Red)

answer:










(f) 3. Who did Mrs. Marsh buy the rolltop desk from?

(Mr. Sloan)

answer:_______________



(v) 4. What does the word "insisted" mean?

(ask forcefully, ask repeatedly, kept asking, etc)

answer:_______________



(inf) 5. What should Mrs. Marsh do about the man in the green car?

(Call the police, or find out what he wants, get help, etc.)

answer: _______________



(inf) 6. How was Mrs. Marsh acting when she arrived at her home?

(Nervous, afraid, scared.)

answer: _______________





















APPENDIX C

STUDENT DATA












S1OOIJ O0183d DNIINnOo-.-


0 (4
c


4 -- ________ - - . -- ~- I -


o I, -m
11296 .
I Ba W1


0M9
'o8


\
0.







N
CM


, 0

C j
a*05<


S ~N io a O
0 0 0 0
I I I I I-
N ~~ oo o M n
So 1-o



0 p

IOU


U2+ +CIX~~
I Z F4 a 00S 82 ON p pr
Il.ill.ll.
Oilillll
ssSZsggs2


go to t~


h..


[' I
........L.~ I ~4 -. u-.-- -
-I -


gi i
go. ,


U
~IIi I
_____ _____ kIL] _
______________ LL I I i.


0 LO


0 00
-o Qo
0


3IrlNIW 83d INnoo


I leljlii Cie
t 0 as
0 00 4)
9 + + +Go A ++
94z
S ~~ *-4 CO


0


610


-J|
*0 0
a t.I -J
tm

(3
AO

>


II
I


hId


VP'


'i~iI

ii~Eu


0


w


04



-4
tI.




II
U


0
0

0 '5
o I-
I
I4(

>
V2 a


~~~~~1


oo~

Oi:~


I


I


I I I


d ....


- Z ;


a


r


,, ,, ,, I


X o


8 00













SHOOiI 0O013d DNIINnoo-J-o-



l|9o3 R~om
I1


0 ~0
*^- p i^rr %pp
'* " | U H


D0.

















m
(-


- N to


0 0 0 0 0a

0
1 o o 1o


I. I


.3 3 3 .3 .3 1


MM + T +4 -o5v +
5 2 z 0SAS ASSv


* M


C4 f


W


D0 0 t-


+1-4~


:1.
4-4 ~-
~ ___


I 0
0 0


0 0
30NI 13d LNnO
0n 8
UinNiw 83d INnoo


;


1L0
0

C0,

0







IJaii o



co
Ii I 5


0




4; 0







LiO



0
0


. .


U.
.4
Ga


0\
a
0


*z
Ua


wi










I-0
z

w
-J



C.D

u>
C-


I






'U
Ci
I'


I


r--j -"


-- ---- OIL-














SuOOiJ a0113d 3NIlNn3-im-

SI I



Q+

i ad O & A i o
| !w g *
|~~~~~~r a S


0 (0 0
0 0 0 00
0 000 0 0
,t 2 A L_ o


a 0 t..- cli e4faie4S%


.3 .3


Ouoa A 11 z zwol o
S54 Sg s g .SAw ZS" 4 0S. 0


CM C(


00

C







C:
a -
V4 -
0








N

P,4





0 O
SI-.
t a-
k.


I^ I
0
0


WD fa t~


UMM


I I I I I
0 1 ~-oo oo
- 0 0NO
0 LO
31nNllWt Ud lNno3


-0to

0


6
o ^


4

S


[E IT~ I =


6I
.10
I-
u~z
oc.~



-J
1-4 ~

1~. (2


z
w



-aJ



C.,J


CI
(2


K








"I










U,


I .













a

C0.














(NI


I0
0


^ 0^^^E~s^~
*^^^|= -: 0 ^^ ^^


3 3 a nO> ?

0 0 0
C I N to
1 1 f 1 -T -
Inl 1 1


SH001J O0183d DNIIN0oo-,-
2


| a oe 15 mat
S-

a*






== ,0 Lq ." Q FA j 04.-










> M
1 0



N tii!I I 1


i~i


.. . . 1= ^ i 1 | ii :" |iiii | i i : T .
~0 o 0
3lnNiP a38 lN-oo
3.1nNIWN H:d .LNnOD


Ig i I +
.





111


C4 00 .4 S C SB a 1 C4





rig es rrcaB



18 A (B r


%-W'.

C .

u



l Ta
4)






-'4 -
0


o 4W


*6
I-2
,i,









Z
- A
uJ


* h













0--
"*












.4j




=1
Ea4














w










i
UJ













CL.
' 4



la a
0;
LU0
_1
-.
0-


I I I I I' I 19












S1OOIJ 001I3d 9NIIlNno03-.-
z
U


44994499 .


,
* -


04 co


01


,,, WoO oo8
(' ID Cl
0 0
00 0 0
N to N 10 != C4

80 ~~6 9o
aEe a~
" = . .
..


** = y .I .
S".9.1" .. r


I "a 0

A'MsAwi.Me iA
SA555
%Pk O> bO91ob z t I


-Cl
I I II


'U
0
*0
~0~~~


I


~~Iii
11ii'7I


<.db &1,*


..~. LiL~1Att4t


6I
hJ ^ -^4=-


IO -o
0 t
10 8t1
Lo n
airiiw ~d Nfloo


e-


CD.


E ;9 0 1 "B0a
- --r- --


U
~4 .g


3


0


C
4,',
0
0


4-. t.


~f~z


o U





C14
0





00
U=





0
o I


0 -


_ W _


m0


_ ,t-f- r-,


0
- 0














S1OOIJ OOlO3d 9NIlNn03.-.-


Zoos


J 0Q 9
Sm&


gcqe0


Qea


z z ++
Z
SMC


- 4 u _,2


3 C &BOI


* I 4 I I

| e g.- -, U



I EEL
d' *^ l 0

Q0a Q04 9L


I r


I~ ~ s is jSl~



+ ++
ssggssgcord


ca M3 0C-


I.e -
~~zz ;~
14


SI. I 4


0
0
Ifi


I0
8 0


3UInNIIA a3d INnoo


-






-







A1
0
1


-I'
UQ
s6
140
-P w

6gid
-9^a





,j w


z
-AJ

C-,
wi
C.,


ac-
z-


0 1
-4


04
-s -*^ E


I


=' .J I I I I | !


\n/-


0 0
0 10 2 0


I


Q0
0


o 1


6 Lo













a a9 : N 0o oo 2
so01u aoiH3d WNI1Nn=O-e- I I I I I I I
C0 0
g WOOS2 : 4 0 0$



L Ii I .
ii_ &| o= ol e | 8ooo o o o



,1 e .* 0. v4aa 3









t OMk

a C*R
i N i u N I''1

'*' "l tF !l y l y S i "
!S <^ ^S g~ag g |ij







II 0


,0
0
;.CO


C\.

,cs

C


3inflNIW 3d INnO3O


*0J
SW


W






00
< .
x





z
LAJ
-J
w

4C,
IL,
LAJ




;C,








0;>
U0
aM




u jd











z
SOOOIJ 001o3d SNIiNnoo--',-
5- N 10
i i ~ i i


EM I I I
0 0
0 0 0o
N4 W. C4


0





0.







C~-



3-








I-,

U^-



'.4
4'
o -o


,o' if
00 1


0 0
0 0
4 I'


C4

V-4


Illllll-lli' ll r lt-Iffr il llll 111i i ii ll~l
+ t w T ~ T C8 GoTVT +tJ +
a z
0-1 C 4 W 10 .
0


I8
800D


0:.. _ -


I I I


I III I I
00 l0 00
f0 o fLOo

UinNiW~ 83d INnoo


I 0
o (


- !










I
RUll


l I
0 -2
(Y)
2

0 0 VI
o .





0
A >lo
CL u
m ;
cm 01


06




C3 * *


- -r- --a'


61 e l,
-8 Se8 o

SeImazamBSas


0
00


w '.

4 Io
@5 Z






4s!
ccsj
cM cJ-











('4
I I


SVIOOIJ M0U3d SNIINOI3'--


16,1 6 P
2^0 j 0O

~I S "e I S


T .i B
s gO4 gIcg
id
4 CO
S


'4


I F
0 LO


1l


0 0
I 4 iN ( 10 I 2



I l I I l


.!10 u Pfl
IC


Ic o"o o+a a-: a a +:
Ie l.tiiogois
Z z z
.2 4 S .SS .


la to


I-
5= S"=


'.4
'4-,-.-
S3
f~e~


00


OU)


S I I I
0 0
0n 0
0 C .


0


I
ff1
041.111


~jpi*I

~I~i ~~!i

-u
u F F
8


\.


00



z
w
-J
4

w
LJ
u >
C5


0~
4


J'IM
OK(f
U4
4|


31NNINA H3d INnOO


ila
5"

*o uI
1.ls
U l
0 BL.-
4 *

Ia o


*~ r~it


SU : | "1^ 1 l
I '4
Uio '^ ~~ r'
- sEEE ^ d^E^
-'4 ^^^^
- U- - r t-- -- -. '1


'px
11
a


0
0


o 3s
000
,0


Os


%t- a


\-,Ic.


0