THE INFLUENCE OF PICTURES, CONTEXT AND DIFFICULTY
ON BEGINNING READING
PEGGY RUTH BROOKS
A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL
OF THE UNIVERSITY OF FLORIDA
I' PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
The author expresses her sincerest gratitude to
all members of her Supervisory Committee for their
participation in her training and in the development
of her career as a psychologist. Nathan X. Perry has
contributed so much to the direction and guidance of
the author, from her undergraduate honors research
to the present study. Suzanne Bennett Johnson deserves
the greatest thanks for her active supervision and
enthusiastic support of this dissertation. ,,. eith
Berg and Wiley C. Rasbury have also contributed greatly
to the author's development as a scientist, throughout
her graduate education. Appreciation is also extended
to Donald S. Childers for participating as the "outside"
committee member on both the qualifying exam and the
TABLE OF CONTENTS
ACKNOW'LEDE 7 :TS . . . . .
ABSTRACT. . . . . . . .
INTRODUCT.IC ... . . . .
PILOT STUDY . . . . . .
PreDaration of Test Yaterials.
METHOD . . . . . . .
Dependent Variables. . . .
Reliability . . . .
Data Analysis. . . . .
RESULTS . . . . . . . . .
Experimenter, Sex and Race Effects .
The Rezression Model . . . .
The Effect of Pictures on Total Errors
The Effect of Context on Word
RecoZnition. . . . .. .. . .
The Effect of Trials on Word
Recognition . . . . . . .
The Effect of Difficulty and
Level on Word Recognition. . .. .
Models for other Dependent Measures,
The Effects of Pictures on other
Measures . . . . . . . .
. . . iii
. . . 1
. . . 14
. . . 21
. . . 25
TABLE OF CONTENTS (cont.)
Comprehension . . ., ., . .2
Repetitions, Reversals, and
Insertions . . . . . . . .
Stem Errors. . . . .. . . . ,
Affix Errors . . . . . . . .
Substitutions. . . . . . . .
Nonsense Errors. . . ..... . .
Omissions . . . ... . . . .7
Self-Corrections . . . . . . .
Repeated Errors.. . . , ... . .
DISCUSSION. . . . . . . . . . 51
Pictures . . . . . . . . .1
Context. . . . . . . . .. .
Difficulty . . . . . . . . 60
Learning . . . . . . . .. 61
Types of Story Category Errors ...., . 62
CONCLUSION. .. .. . . . . . 66
REFERENCES. .... .. . . . . . .6
APPENDIX . . . . . . . . .. 72
BIOGRAPHICAL SKETCH . . . . . . . 105
Abstract of Dissertation Presented to the Graduate Council
of the University of Florida in Partial Fulfillment of
the Requirements for the Degree of Doctor of Philosophy
THE INFLUENCE OF PICTURES, CONTEXT AND DIFFICULTY
ON 7BEINNINI READI;
Peg-vy uth Brooks
Chairman: Nathan W. Perry
Major Department: Clinical Psychology
Eighty-three children (39 females, 44 males) who had
completed first grade participated as subjects in this
study to determine the effects of picture, context and
difficulty on reading. Superior and below average readers
were grouped into picture, no picture and control con-
ditions (level and picture were between subjects factors).
The experimental groups received easy and difficult word
lists and stories (trials and difficulty were within
subjects factors) while the control group received only
the word lists. All groups received aid on a missed word
during the second presentation of the stories (or word
lists in the case of the control group). Total error
frequencies, comprehension scores, and story category types
of error frequencies served as dependent variables. 5our
factor, repeated measures analyses of variance revealed
significant three-way interactions for all variables and
subsequent analyses were necessary to assess main effects
of one variable while two others were held constant.
The effect of pictures was such that below average
readers who received stories accompanied by Dictures
made more errors than any other rrout. Superior readers
did net appear to be affected by the presence of pic-:res,
Pictures also did not significantly alter conprehension
scores or particular story error ty-es for an -1rou.
Results on the effect of context were similar: selow
average readers made more errors on stories (in whicn
contextual cues exist) than on word lists (in which :nly
graphic cues exist) while superior readers made equivalent
numbers of errors on both stories and word lists.
Difficulty of material was significant for all dependent
measures. All groups made significant improvements in
word recognition on difficult material over trials.
Difficult material produced poorer performance Dy toch
superior and below average readers on comprehension and
story error types, and in some cases, produced different
error patterns for both groups. Experimenter aid also
bettered performance of poorer readers. Effects of these
variables on superior reading of easy material may have
been obscured by basal effects in which superior readers
made so few errors initially, that there was not much
room for improvement over trials,
Results of this study clearly support the hypothesis
that pictures and context interfere with poorer readers'
word-decoding performance and suzsesn that trese readers
rely on inappropriate strategies such as searching for
meaning and contextual cues to learn to read. Superior
readers appear to use both graphic and contextual infor-
mation efficiently when reading easy material out te-d to
rely more on graphic cues on difficult material. Sug-
gestions for more appropriate reading material for each
group are made, based on these findings. These resul-s
are also discussed in light of developmental, atten-ional
and psycholinruistic theories of reading ability.
The abundant use of bright colorful illustrations
in children's reading texts certainly leads one to believe
that pictures accompanying printed reading material are
facilitative in the acquisition of reading skill. Yet
there exist very little experimental research data to
support this notion. generally, pictures have simply been
assumed to foster reading skill, mostly through their
attractive stimulus properties. This is a conclusion,
which as Bourisseau (1965) put it, ". has persisted in
the absence of solid research evidence and has achieved a
prominent and seldom-disputed position (p.250)". Indeed,
in her interview of twenty-five proponents of different
beginning reading approaches, Chall (1967) noted that none
of them seriously questioned the need for pictures. Miller
(1938) reported that teachers believe children learn to
read better with pictures for several reasons: (1) pictures
introduce characters in a story; (2) they arouse and sustain
interest; (3) they clarify unfamiliar concepts appearing
in print; and (4) pictures furnish clues to word recognition.
However, Miller (1938) found that when duplicate copies of
first grade texts with no illustrations were given to half
of his subjects (n=600 first through third graders), the
nonpicture group made 12 out of 18 possible statistically
significant reading gains during the year while the picture
group made only ten out of 18. While not arguing to remove
pictures from all textbooks, Miller (1938) stated the absence
of pictures did not cause children to read with less compre-
hension or interest. He concluded, ". . it is probable
that many illustrations leave much to be desired in furnish-
ing clues to the reading material which they accompany.
Anyone who has watched beginning readers at work has seen
them shift their eyes from a printed word to the picture,
trying to get a clue to the word from the picture. Such
shifting of attention is considered by some persons to be
an interference with reading (p. 676)".
Yet, because of the direct sensory appeal of pictures,
it has been assumed that they facilitate learning through
sensory and cognitive stimulation. Because of the popularity
of pictures and belief in their instructional utility, very
little systematic research has been carried out to determine
the influence they have on children's reading. A recent
review article by Concannon (1975) cites fewer than ten
studies on the effects of pictures on reading, yet virtually
every study indicated that pictures do not contribute sig-
nificantly to the word-decoding ability of the child.
In the studies involving the influence of pictures on
word recognition, a common paradigm has been one in which
children are presented with a small number of words in no-
picture and picture conditions, with training and test trials.
Samuels (1967) utilized this paradigm, with 30 pre-first
graders in a preliminary word recognition task, and showed
that both a simple and complex picture provided more clues
for recognition and produced more correct responses than
the no-picture condition. However, in the test trials when
no pictures were present, the subjects trained in the no-
picture condition excelled. This study alternated the
acquisition and test trials purposefully so that the subjects
in the picture-plus-word conditions would be aware that the
words were important stimuli. Still, these subjects tended
to rely on pictures when they were present instead of graphic
detail of the words as cues. In a second experiment in the
same study, 52 students with seven months of first grade
experience were divided into picture and no-picture conditions
in which a story made up of 50 separate words was used in
reading instruction. The pre- and posttest consisted of
presenting the subjects with the individual words for recog-
nition. For the better readers, Samuels found no significant
difference in reading acquisition between the picture and
no-picture groups. However, the poorer readers learned to
read significantly more words in the no-picture condition
than in the picture condition. Samuels interpreted this
finding as support for the notion that the poorer students
were more distracted by the picture stimulus which interfered
with learning to read the words. Samuels cites 3.J. Underwood's
principle of least effort (1963) in summarizing his research
findings the principle of least effort suggests that when
both word and picture are presented together, the picture
most readily produces the desired response because at first
the picture can elicit a response more quickly than the
printed stimulus. So, given the two stimuli, producing a
response to the picture requires less effort; however, in-
stead of focusing on the word, the subject then attends to
the picture, failing to give the necessary attention to the
Two other studies utilizing versions of Samuels'
paradigm reveal similar findings. Duell (1968) found in
teaching sight vocabulary, pictures were less effective as
cues than prompter training without visual pictorial stimuli.
A more recent study by Singer, Samuels and Spiroff (1974)
utilized four training treatments with 164 first and second
graders to determine the effect of pictures and contextual
conditions on learning printed words. The four training
treatments were: word-no picture; word-picturei sentence-no
picture; and sentence-picture. In the test trials, only
words were shown to the subjects. On both number of trials
to criterion and number of correct responses, subjects in the
word-no picture treatment scored significantly better than
any other treatment group. In the report of their study,
Singer et al. state their findings are in support of a focal
attention hypothesis of Samuels (1967). Central to this
hypothesis is the notion that focusing on printed words is
crucial for reading acquisition and that pictures (and other
contextual cues) distract children from focusing attention
on the necessary orthographic detail of the words themselves.
It is indeed difficult to find experimental evidence to
support the other side of the picture vs. no picture controversy.
While many authors have faithfully expounded on and cited
anecdotal evidence in favor of pictures in reading texts
(e.g., Whipple, 1953; and Schonell, 1961), solid data to
document this belief is difficult to find. In fact, the
only study not reporting evidence against the use of pictures
was conducted by Hartley (1970). Hartley could not find
data to support a generalization about the relative effective-
ness of any one of three conditions words alone; words
with a picture; or words presented with oral context.
One quite recent report (Wardle, 1977) which suggests
that illustrations in textbooks may improve reading compre-
hension test performance of below median reading level students
also must be interpreted with caution. In that study,
Wardle looked at reading comprehension for science textbook
material presented in one of five ways to 191 above and below
median reading level seventh-graders. Her five test con-
ditions were (1) text plus an illustration that answered
a high number of test questions; (2) text plus an illus-
tration answering a medium number of test questions; (3) text
plus an illustration that answered a low number of questions;
(4) an unillustrated text; and (5) a test without text or
illustration. The results of her experiment yielded no
improved reading comprehension with an illustrated or unillus-
trated text for the total pupil group or the above median
pupil group. The below median group did answer significantly
more test questions with an illustrated text, but there were
no differences in improvement as a function of the amount of
relevant information in the illustration. Thus, the im-
provement seen for below median readers could not be
attributed to the abstraction of test information from
the illustration. The author suggested that perhaps the
illustrations served only to interest the student in
spending more time studying the text.
Two earlier studies by Vernon (1953t1954) also with
older subjects (12 and 17 year olds), indicated pictures
were found to occasionally help a subject remember a
particular fact about a story. In general, though, an
illustrated story was remembered no better than an unillus-
trated one. In the 1953 study, Vernon also noted that the
younger or less intelligent child paid more attention to
pictures than the older, more sophisticated readers.
In his dissertation, Weintraub (1960) was also inter-
ested in the effect of pictures on comprehension of the
main ideas and events in a story. He compared several
groups of second graders on a multiple choice test of
comprehension. These groups were: boys and girls sepa-
rately, children with high intelligence but low reading
achievement levels, good readers and poor readers. For all
groups, comprehension was greatest when text only (without
pictures) was seen. Boys and girls did not differ signifi-
cantly. Good readers appeared to do well regardless of
the picture or no picture condition, but poor readers scored
better with the text only than with either text and pictures
or pictures alone.
From these studies on the effect of pictures on reading
comprehension, no definitive statement can be made. While
several authors feel that picturesmay be motivating variables
(Whipple, 1953; Schonell, 1961), nothing can yet be said
for the instructional value of pictures for reading compre-
hension. The fact that publishers of certain basal readers
wholeheartedly endorse the use of pictures for reading compre-
hension development was illustrated by Chall (1967) in her
analysis of two series of basal readers. Chall found, on
the average, that preprimers introduce more new pictures
per story than new words. In one series, she found more
comprehension questions about the pictures than about the
text Even in third grade readers, 30 per cent of the
questions could be answered without reading the words at
The studies by Vernon (195311954), Wardle (1977) and
Weintraub (1960) allow some comparison between good and poor
readers and make differential statements about the results
for the two groups. Besides the age differences which may
account for the discrepancies between Wardle's and Weintraub's
results, in no study was the difficulty level of reading
material comparable for both groups. No attempt was made,
for example, to give the poorer readers material containing
easier vocabulary than the material for the superior readers.
This was true for the previously cited studies on word recog-
nition, as well, and may be an important variable for de-
termining poor readers' utilization of pictures as clues
for comprehension, e.g., if the level of material is
exceedingly more difficult for poor readers than for good
readers, the poor readers may be more likely to use the
principle of least effort (Underwood, 1963) in searching
the illustration for clues to the story than the good
readers. Clearly, when comparing good and poor readers,
one important variable may be the difficulty level of
reading material for both groups.
The entire picture controversy has also been seen
as part of a larger controversy, namely, the role any kind
of contextual information plays in the facilitation of
word identification. A major proponent of the contextual
or linguistic hypothesis, Goodman (1965) found that 100
readers in first, second and third grades recognized words
in a story with better accuracy than when the same words
appeared on lists, Goodman argued that on the word list,
the child had only the cues within the words to use for
recognition, while in the story, the presence of additional
contextual cues in the flow of language allowed the child
more information with which to recognize the words. Children
in his study were given a word list to determine their level
of reading and presented a corresponding story containing
the words to read, thereby equating difficulty level for
the subjects, However, Goodman did not provide a control
group which received only words and thus the improvement he
reported for word recognition in stories possibly may have
been due to a practice effect or exposure to the words in
the previous word test condition. He also did not report
whatever differences in performance may have existed for
the groups of readers he tested.
Singer, Samuels and Spiroff (1974) found results
contrary to Goodman's (1965) in their comparison of word
alone, word plus picture, word plus sentence and word plus
sentence plus picture conditions. As reported above,
subjects in their study scored best when presented with
the words alone. Thus, the addition of the syntactic and
semantic contextual cues in sentences did not produce
superior results. However, Singer et al. (1974) did not
use different words for children of different reading levels
or address the relative differences between above and below
average readers in these conditions. Hence their results
might have obscured the differences which may exist between
above and below average readers in their use of contextual
An alternative view may be seen in the work of 3iemiller
(1970) on children's development of the use of graphic and
contextual information. Siemiller studied two first grade
classes' reading acquisition throughout a school year and
found an overall developmental progression from reliance on
contextual cues through analysis of graphic detail of words
to an effective combination of both strategies. Biemiller
defined these three phases on the basis of the frequency of
a particular type of reading error which he termed a non-
response error. A non-response error was defined as the
occasion on which the child stopped reading just before a
word it was assumed she/he did not know. Biemiller termed
the first phase a "pre-Nonresponding" phase in which readers
seemed to rely primarily on contextual cues for reading.
The second stage was called the "Nonresponding" phase
because of the high frequency of non-response errors found.
Biemiller hypothesized that readers in this phase began to
grapple with the graphic features of words. The third
phase Biemiller described was the "post-Nonresponding"
stage, in which readers appeared to use both context and
graphic information efficiently.
He reported poorer readers tended to remain in the
first phase longer than better readers. However, the poor
readers who did reach the second phase did not make signifi-
cantly more graphic substitutions. This led 3iemiller to
admit that perhaps the pattern of error types in the differ-
ent phases might only reflect overall ability differences.
While Biemiller's view suggested some kind of developmental
factor may operate in children's use of contextual infor-
mation, it also appeared from his study that poor readers
and superior readers may simply differ in the extent -o which
they typically utilize graphic and contextual information,
regardless of their place on his developmental continuum.
At any rate, Biemiller stood in opposition to the encourage-
ment of the early use of contextual and picture cues and
argues this: ". . the child's first task in learning to
read is mastery of the use of graphic information, and
possibly, of the notion that one specific spoken word corre-
sponds to one written word. The child's early use of
contextual information does not appear to greatly facilitate
progress in acquiring reading skill (1970, p.95)".
It does appear, then, that the presence of pictures
and contextual information may not facilitate acquisition
of early reading skill, especially in poorer readers. If
poor readers tend to ignore the orthographic detail of
words, are they distracted by the presence of pictures
and contextual information or is the task so difficult
for them that they become frustrated and turn to extra-
graphical information for cues? Might not a child of
even average or superior ability also turn to extragraphical
cues when faced with equally difficult reading material?
The possibility exists that all beginning readers may re-
spond to difficult stories similarly, i.e., by relying more
on extragraphical information.
That distractibility is a factor involved in reading
disabilities is supported by a review of several studies
by Tarver and Hallahan (1974) which indicated that dyslexic
children may be unable to filter out extraneous and irrele-
vant information or focus selectively on a learning task.
Results from a study by Elkind, Larson and Van Doorninck
(1965) supported this idea. These investigators employed
an embedded figures task in their study of slow and average
readers and found that slow readers perceived fewer hidden
figures than normal readers. They hypothesized that skilled
reading is related to the Piagetian (1958) concept of
decentration in which freedom from distractibility within
an embedded context, or the ability to decenter from domi-
nating field effects, is required.
Sabatino and Ysseldyke (1972) also found differences
between readers and nonreaders on Bender Visual-Motor
Gestalt tests in which stimulus designs were embedded in
extraneous backgrounds, but found no differences on the
standard Bender test or a Bender memory test.
These studies offer at least some support for the
notion that skilled reading requires the ability to seek
out relevant and ignore irrelevant information. The fact
that Sabatino and Ysseldyke (1972) found nonreaders performing
most poorly on the clearly more difficult task of an embedded
figures design led Tarver and Hallahan (1974) to speculate
that when the discrepancy between relevant and irrelevant
stimuli is of sufficient magnitude, children with reading
and other learning disabilities may De no more highly
distracted than normal controls.
Being a bit more specific about the differences between
below and above average readers, might not the difficulty
level of the reading material be analogous to Tarver and
Hallahan's (1974) hypothesis about relevancy of the stimuli?
If poor readers are presented with reading material that is
less difficult for them, i.e., in terms of vocabulary word
knowledge, would they be no more distracted than better
readers with the level of material that is typically used in
these studies? Conversely, would superior readers be equally
as "distracted," i.e., make as many errors, in a condition
in which they received material that was very difficult for
them? Virtually no study cited here has addressed this
problem. Only in one study (Goodman. 1965) were the
difficulty levels of reading material adjusted for readers'
word knowledge and in the report of that study no compara-
tive data were presented for performance differences between
groups on the different levels of material, e.g., poorer
vs. better readers' performance, In short, a very important
part of the puzzle is missing if below and above average
readers' performance differences cannot be separated from
the difficulty level factor.
The present study was designed to address three major
questions concerning beginning reading. These were: (1)
To what extent do pictures influence and/or interfere with
the performance of superior and below average readers? (2)
To what extent does the presence of contextual cues influence
beginning readers' (both superior and below average) word
recognition ability? (3) To what extent does difficulty
level of reading material influence oral reading performance
of below and above average reading? An additional objective
of the present study was to assess the effect of the presen-
tation of the word lists and stories in terms of learning
over trials, with aid given on missed words in one condition
so that all subjects would have had exposure to all words.
To measure these effects on oral reading performance, the
following dependent variables were of interest, (1) total
number of errors; (2) story comprehension; (3) ten types of
errors n stories, including a frequency count of self-
corrections; and (4) a measure of graphic similarity of errors
to story text. These measures were felt adequate to meet the
objectives of the present study.
Because a major objective of the present study was to
attempt to adjust difficulty of reading material for both
superior and below average readers, a pilot study was
designed and carried through to provide a list of easy,
intermediate and difficult words from which the testing
stories and word lists would be composed.
A total of 84 children from three first grade classes
participated in the pilot study. Participation in the pilot
study excluded participation in the formal study itself.
These children comprised entire classes and so included all
levels of reading proficiency.
One hundred, sixteen words were chosen from pre-primer
to fifth grade level texts and printed in lower case letters
on 4 x 6 cards, one word per card. Children were tested
individually in a quiet place on their sight vocabulary of
these 116 words. The children were allowed ten seconds to
respond to each card. All children were shown all words.
A frequency count was then made of the number of correct
responses to the words, and each child's score on the Reading
subtest of the Metropolitan Achievement Test was compared
with performance on the pilot vocabulary list.
Preparation of Test Materials
Based on the frequency counts and reading subtest scores,
three separate lists of words were derived, of easy,
intermediate and difficult levels. Words included in the
easy list were recognized by 88 to 100 percent of the pilot
subjects. Words of intermediate difficulty were recognized
by 66.6 to 79.7 percent of the children in the pilot study
and the difficult words were recognized by 9.5 to 48.8 percent
of the pilot subjects. The easy material was therefore easy
even for poor readers, the intermediate material was difficult
for poor readers and easy for better readers, and the
difficult material was difficult even for better readers.
From these three word lists, three stories were composed.
Since the stories had to be composed from the tested words,
content of the stories was not highly predictable from a
semantic or plot development standpoint. Appendix contains
copies of each of the stories used. There were 27 different
words on the easy list and story (there were repetitions of
some words, especially words such as the, a, and, etc., in
all stories); 25 different words on the intermediate list
and story; and 32 different words on the difficult list and
story. Pictures were drawn to match each story. The pictures
included details relevant to the stories as well as some
aspects not directly referred to in the stories, e.g., the
number of boys or girls was obvious from the picture but was
not referred to in the story itself. Pictures were drawn
in black ink. Amounts of information available from picture
content was not assessed; however, since each picture was
drawn to match each story, similar amounts of information
were contained in each. Thus, the amount of story-relevant
picture content was not considered as a variable in the
Subjects. Eighty-three children who had completed
first grade in the Alachua County School system of 3aines-
ville, Florida, participated as subjects in this study.
The 39 females and 44 males came from the following schools,
in order of greatest number from each Glen Springs, Stephen
Foster, J.J. Finley, Williams, and Metcalfe elementary
schools. All subjects were tested during the summer of
1976, from June through the middle of August.
Children were selected for the study based on their
total reading scores from the Metropolitan Achievement Test
given in their respective schools in the spring of the year.
For the purpose of this study, children who scored above
the 80th percentile were considered superior readers and
those who scored below the 50th percentile were considered
below average readers. This higher cut-off point for the
poorer readers was necessary since children who scored at
the 20th and 30th percentiles were mostly nonreaders, i.e.,
could not read at all. Additionally, children with an Otis-
Lennon Intelligence Quotient (IQ) below 85 were excluded
from the sample. Superior and Below Average subjects were
then randomly grouped into one of three conditions A
Picture condition, a No Picture condition, or a Word List
Letters describing the study were sent home from
school with the children selected (See Appendix 3 for a
sample letter), and parents were contacted by a subsequent
phone call. The parents were informed in greater detail
of the study's goals (though the subject of pictures was
not mentioned prior to the testing, in order to control
for any bias the child might have before testing) and
invited to have their child participate in the study.
Parents were asked to bring the child into the laboratory
in the Psychology Department for testing) if this was
impossible, testing was done at the child's home.
A total of 117 children were tested. Of these, the
data from 31 had to be discarded because of a subject's
either limited or perfect score on the word list vocabulary.
A perfect score meant that the subject recognized 100 percent
of the words on the list and could not be included because
no words could be classified as "difficult" for that subject.
Likewise, a limited score on the word list meant that the
subject recognized so few words (less than 63 percent of
the "easy words") that no material would be "easy" enough
for that subject, i.e., all words would be "difficult" for
that subject. In the case of three other subjects, experi-
menter or equipment error required data from these subjects
to be discarded.
Experimenters. Four experimenters were trained to
administer and score the reading test. Three were females
two Caucasian (one of whom was the author) and one Blacki
one was a Caucasian male. The procedure for administration
and scoring was based on the procedure outlined in Goodman
and Burke's (1972, Chap. 4) Manual for the Reading Miscue
Inventory. Training lasted for one week until all experi-
menters had discussed and reached agreement on their inde-
pendent scoring of six practice tapes. Experimenter
instructions are included in Appendix C .
Apparatus. A quiet place with no distracting stimuli
was required for testing. A cassette tape recorder was
used to record the entire session with each subject.
Word lists, stories and pictures were laminated for
use with the children. Copies of the word lists and stories
were used by the Es to score reading errors. Samples of
the word lists, stories and pictures appear in Appendix
D. A small selection of inexpensive toys and candy bars
was maintained, from which a child could choose one item
at the close of the session, with parental permission.
This reward was not mentioned at any time prior to or during
the testing session.
The subject was seated in a chair in front of a table
or desk on which the testing materials were placed. The
experimenter explained the procedure to the subject in one
of three ways, depending upon the condition. For the
Picture condition, subjects were asked to read a list of
words and two stories with a picture that went with each
of the stories always in the same order. For the No Picture
condition, subjects were asked to read a list of words and
two stories, also in the same order. For the Word List
control group, subjects were asked to read a word list
In both the Picture (P) and No Picture (N) conditions,
subjects were presented with the word list (which comprised
an easy and more difficult list of words, each word in
order of difficulty), the easier story (with or without
an accompanying picture) and the more difficult story,
all without help. Then the subject was asked to again read
the stories in the same orders however, on this second
reading, the subject was told that aid would be given on
a word which was missed (omitted or mispronounced), and
help was given in this trial only. In the Word List (WL)
control condition, help was offered on a missed word
during the second presentation only. finally all subjects
were presented with the same word list without help, on
the last trial.
The Comprehension portion of the testing comprised
three questions to the subjects after each reading of a
story, The subjects was asked (1) to tell the experi-
menter everything she or he remembered about the story
(recall); (2) who was in the story and (3) what happened
in the story. The experimenter was permitted to ask the
additional questions "Who else was in the story?" and
"What else happened in the story?" once each if the subject
gave less than a complete answer to the initial questions.
The three word lists which were compiled on the
basis of the pilot research were each of a different
difficulty level. The word list presented to the subject
as mentioned above was actually made up of two of the
three word lists designed to be easy and difficult for the
subject from the pilot study. Below Average (BA) readers
received the easiest and intermediate lists and Superior
(S) readers received the intermediate and most difficult
lists. In addition, the BA readers received the stories
(made from the word lists) of easiest and intermediate
difficulty, while the S readers received the intermediate
and most difficult stories, again designed to be easy
and difficult for them. Since both BA and S readers
received an identical word list and story, that of inter-
mediate difficulty, this comparison matched the design of
previous studies in which all subjects received the same
word list or story. However, the present study also
allowed for the evaluation of the relative comparisons
of an easy and a difficult word list and story for both
BA and S readers.
Experimenters scored the subject's errors on the
word lists and stories during the session, following along
on a separate copy. The tape recording of the child's
oral reading was also played back afterward to complete
and check the scoring.
In summary, the P and N conditions first received the
word list, then the stories without aid, then the stories
with aid, and finally the original word list. On each
reading of a story, the comprehension questions were asked.
The WL control group was presented with the word list
three times, receiving aid on a missed word during the
second presentation only. The duration of the entire
session was never longer than one-half hour. Sources of
data were the protocols scored by the experimenters and
the tape recordings of each subject's oral reading.
Error Frequencies. Total number of errors on word
lists and stories, before and after aid, were totaled in
order to make comparisons between the groups with regard
to the effect of pictures, context of the word, effect of
training (aid and practice) and level of difficulty. In
the trials with aid, an error was counted and scored before
correction. Comparisons between word list errors and
story errors were made possible by subtracting the number
of repeated errors on stories from the total number of
errors on stories.
Story Error Categories. Because it is believed that
oral reading provides a convenient and objective method
for studying central processes occurring during reading
(Fairbanks, 1937) and that oral reading errors may reflect
the cognitive strategies used by readers to make sense of
the reading material (Biemiller, 19701 Weber, 1968), the
present study employed several categories of oral reading
errors in hopes of providing important clues about the
differences between poor and superior readers. The study
was designed to investigate these categories of oral
reading errors as a function of the independent variables:
reading level, pictures, difficulty and training.
In an article which reviews previous error classi-
fication systems and provides reliability data for a
particular classification system, Hood (1976) advocates
the use of a standardized and reliable system. The error
classification scheme used in the present study was
modeled closely after Hood's and also owes much to the
system in the Reading Miscue Inventory (Burke and Goodman,
Errors were counted when the subject's response oor
lack of response) did not match the printed text material
in its spoken form. Dialect differences were ignored.
Changes in word order, repetition errors and insertions
were counted as one error (each time they occurred) even
though they might have involved more than one word.
The types of errors identified were: (1) repetition,
a repetition of a word or phrase (not mispronounciation
of the word or phrase)---these repetitions were often
anticipatory responses to a following difficult word or
phrase or changes in the intonation of the word or phrase;
(2) orders changes in the order of the words, e.g., sun
up was/sun was upl1 (3) reversal: substitution of a word
containing the same letters as the text word but in a
different sequence, e.g., form/froms (4) stem: substi-
tution of a word containing the same stem as the text
word, e.g., looking/looked; (5) affixt substitution of
a word containing the same affix as the text word, e.g.,
picked/looked, dissatisfied/disappeared; (6; substitution:
substitution of a meaningful word for a text word if it
may not be previously categorized as a reversal, stem
or affix error; (7) nonsense: substitution of part of
a word or a nonsense word if it may not be previously
categorized as a reversal, stem, or affix error. It
should be noted that that reversal, stem, affix, substi-
tution and nonsense errors are all kinds of a more general
substitution for text words. In addition, (6) insertion
an insertion of one or more words between two text words;
and (9) omission a word omitted, either inadvertently,
or as an indication that it was not known. In addition,
it was considered desirable to know whether a child spon-
taneously self-corrected his/herself, whether the error
was a repeated one, e.g., the child produced an error on
the same word later in the story, and whether or not the
error produced was graphically similar to the text.
Graphic similarity was considered important as a measure
IThe error is italicized and precedes diagonal line; text
of the child's use of graphic cues. Graphic similarity
was determined from a modification of Cohen's (1975)
criteria. An error was considered graphically similar
to the text if, (1) the error word and the text word
shared a common first, last or both first and last letter-
sound, e.g., lighted/looked; or (2) the error word and
the text word have at least half of the letter sounds in
common, e.g., eel/leaf, terrific/traffic. Common last
letters were included as a criterion after a literature
review found that both Weber (1970) and Marchbank and Levin
(1965) contend that beginning readers exploit the letters
in the end positions of words as "salient cues yielding
high information (Weber, 1970, p. 156)".
Comprehension. For the experimental groups, a
story comprehension measure was obtained from the subject's
responses to the comprehension questions mentioned above.
Scoring was based on assigned points for character recall
and development, events and general plot or theme of the
stories. Since the stories were relatively short and
simple, these answers were typically brief. Detailed
guidelines for scoring comprehension were composed (See
Appendix E ) and a total of 25 points were possible for
In studies where data points are subject to varia-
bility due to differences among experimenters, a check
on data reliability is necessary before statistical analy-
sis proper can be undertaken. In Hood's (1976) study,
she not only isolated the most useful error categories;
in addition, she obtained measures of inter-judge
reliability on scoring of total error, meaning loss
errors, repetitions, proportions of graphically similar
errors and errors self corrected, and scores for contextual
appropriateness. Her judges were five female college
graduates who were trained for one week. Cronbach's
alpha reliability coefficient (1951) was used and relia-
bilities which would hold under various numbers of judges
were estimated. For most errors, Hood found that no
appreciable increase in reliability occurred when more
than two judges were used. Support for this finding also
comes from Y. Goodman (1971) who recommends two judges be
employed for recording and counting errors. In the present
study then, two judges per task were employed.
In the present study, four female college under-
graduates as well as the author were judges for reliability
scores on word list errors, story error categories, total
errors on stories and comprehension. Training of the
judges required approximately five hours. A random sample
of data from 30 subjects (with equal numbers from each of
the six treatment groups) was scored for reliability.
Percentage of agreement between two judges was used as
the measure of reliability. Percentage agreement was
defined as the number of agreements divided by the sum
of the number of agreements plus the number of disagreements,
multiplied by 100. The percentage agreement for total
errors on word lists was 99.28; for total errors on
stories, 98.651 for story error categories, 87.79: and
for total comprehension scores, 86.45. In the case of a
disagreement, the rater and the author discussed the
differences. In most cases, the data as scored by one of
the original experimenters were used for the analyses.
The data required a four factor repeated measures
analysis of variance; the two Between subjects factors
were pictures and reading level and the two Within subjects
factors were difficulty level and trials (the four repeated
measurements). A Statistical Analysis System (SAS: Barr,
Goodnight, Sall and Helwig, 1976) general linear models
procedure was utilized to accommodate the data. The
general linear models (glm) procedure was chosen because
it provides tests of hypotheses for the effects of a
linear model regardless of the number of missing cells
or the degree of confounding in the model. SAS also
allows the user to specify one or more of four types of
sums of squares and their estimable functions. Type II SS
were suggested for use in the present analysis because they
provide the reduction in SS due to a particular effect,
given all other effects, i.e., only the parameters associ-
ated with the effect are involved.
Initial analyses were performed to determine any
main effects due to Experimenter or Sex and subsequent
analyses for main effects due to each of the four factors
and their interactions were performed for each of the
Experimenter, Sex and Race Effects
Four separate analyses were employed to test for
an experimenter effect, each using the glm procedure.
The first was a model in which the effect of experimenter,
picture and level (of the subject's reading ability)
were tested for main effects and interactions on a sum
of errors across all trials. There was no significant
main effect for experimenter (F = 1.79, p <(.15) or
for any two-way or three-way interactions with picture
and level (See Table 1 ). Three subsequent analyses were
run using the same model, on the following combinations
of total errors on the repeated measures (trials): the
word lists before the stories, the stories, and the word
lists following the stories. None of these analyses was
significant either (See Table 1 ) despite the fact that
these extra analyses might increase the chance of finding
a significant effect, and so the variable experimenter
was dropped from the model.
To test for significant effects due to sex, a model
which included sex, picture and level was employed. There
was no significant main effect of sex (F 7 .69, p <.19)
nor any significant two-way interactions involving the
Source df SS p
For Summed Errors
Experimenter x Pictures
Experimenter x Level
Experimenter x Pictures x
For Word List Before
Experimenter x Pictures
Experimenter x Level
Experimenter x Pictures x
Table 1 (cont.)
Source df SS F
Experimenter 3 842.973 1.51 -.s.
Experimenter x Pictures 6 468.419 0.42 n.s.
Experimenter x Level 3 1221.357 2.19 n.s.
Experimenter x Pictures x
Level 7 2522.404 1.94 n.s.
For Word List After
Experimenter 3 283.658 1.84 n.s.
Experimenter x Pictures 6 57.925 0.19 n.s.
Experimenter x Level 3 78.971 0.51 n.s.
Experimenter x Pictures x
Level 7 196.415 0.55 n.s.
Error 60 3079.331
independent variables picture and level (See Table 2 ).
There was, however, a significant three-way interaction
involving sex, picture and level (F = 2.89. p <.03).
Upon examination of the six picture-level treatment
combinations, it was determined that this effect may
have been the result of the particular below average-
no picture (BAN) grouping,l in which there were ten
males and three females. The fact that there was no
significant sex x level interaction was most likely due
to the occurrence in the BAWL group of an oppositely-
loaded imbalance of nine females and five males. At any
rate, this three-way interactions was not considered
important enough to leave the variable sex in the model
and so it was also dropped.
The effect due to race of the subject was not
included in the analysis. There were a total of 13 Black
children included in the study, one of whom was in a
superior reading group. Since no group contained an
equal number of Black and white children, it was felt that
the contribution of race to the differences between groups
would e a random effect. Consequently, due to the few
numbers of Black children, the variable race was not
included in the subsequent models for data analysis.
1Hereafter each of the six treatment groups will be
indicated by combinations of their letter abbreviations
for picture and level conditions, e.g., BAN=BA for below
average level plus N for the no picture condition.
Source df SS p
Sex 1 345.657 1.69 n.s.
Sex x Pictures 2 284.040 C.07 n.s.
Sex x Level 1 84.347 C.-i n.s.
Sex x Pictures x Level 4 2359.781 2.;9 <0.0282
Error 71 14497.874
The Regression Model
Once the variables experimenter and sex were removed
from the model, a complete four-factor model was written
which included pictures, level, trials, (repeated measures)
and difficulty. The dependent variable was a corrected
total error score, i.e., for the stories, repeated errors
were subtracted from total errors to allow for fair
comparisons across trials. The design for this model was
based on a multifactor repeated measures plan found in
Winer (1962, p. 350). The model used the SAS glm pro-
cedure. A summary of this analysis of variance appears
in Table 3. There were main effects for all four factors
(Picturest F2-7=.85, p e_ .0001; Level: F 17 5.94,
p <-- .0001; Trialst F = 100.24, p< .0001; Difficulty:
P = 824.20, p .0001) and all two-way interactions
were also significant. There was also two three-way
interactions (pictures x level x trials and pictures x
level x difficulty) which were significant. Because these
significant three-way interactions involved both Between
and Within subjects factors and because each of the Between
and Within factors was crossed with the other, it was
necessary to perform a post-hoc analysis on all possible
pairs of the smallest cells (each four-way combination).
For this purpose, a Duncan's multiple comparison procedure
Four Factor ANOVA
Pictures x Level
Errori Subject (Pic-
tures x Level)
Pictures x Trials
Level x Trials
Pictures x Level
Pictures x Difficulty
Level x Difficulty
Pictures x Level x
Trials x Difficulty
Pictures x Trials x
Level x Trials x Diffi-
Pictures x Level x
Trials x Difficulty
6 12.683 0.30 n.s.
3 19.132 0.91 n.s.
was utilized. Although Duncan's multiple comparison test
was not specifically designed for multiply classified
data, it is acceptable to use it for this purpose and
the User's Guide to SAS (Barr, Goodnight, Sall and
Helwig, 1976) specifies a procedure which allows for this
function. This procedure compared the 48 cells (four
of which were redundant for the control group since
control subjects received only three trials compared :o
four trials for the experimental groups) for significant
differences. Many of these comparisons were meaningless
and will not be reported here.
The Effect of Pictures on Total Errors
The BAP treatment group differed from all other
groups on the most difficult story (x=21.93 errors).
Similarly, the BAP group differed significantly from
the BAN group on the easy story (x=12,00 vs. x=9.076,
respectively). In the other story condition (stories
with aid), again the effect of pictures on the two BA
groups was such that the group which received pictures
made significantly more errors on the difficult story
(x=16.64) than the group which received no pictures
(x-14.384). However, for the easy story with aid, the
two BA groups did not differ significantly ( BA=9.285,
For the superior groups, pictures did not produce
any significant differences on the difficult story or
the easy story. Nor did pictures significantly influence
differences between the two superior groups on the subse-
quent stories with aid.
The Effect of Context on Word Recognition
The control groups (SWL and BAWL) were included in
these comparisons to rule out possible explanations of
differences based on practice effects over trials. Or
the story conditions, it must be remembered that a fair
comparison dependent measure was obtained by subtracting
repeated errors from total errors.
The BA readers were clearly influenced by the
presence of context effects in stories. For the difficult
word list and story, the BAP group made significantly
more errors on stories (xWL=15.642 vs. XSTRY=21.928).
Again, for the easy word list and story, the effect of
context for this group was such that significantly fewer
errors were made on the word list (x=6.928 vs. x=12.000).
For the BAN group, results were parallels more errors
were made on stories than word lists, regardless of
difficulty of material (x=4.846 vs. x=9.O76 for the easy
material, x=12.769 vs. 16.000 for the difficult material).
None of the BA groups significantly differed from each
other on errors on the easy word list (XBAp=6.928,
XBAWL'4.857, XBAN4.846). The control group (BAWL) did
not differ significantly from the first presentation of
the word lists to the second, for either easy or difficult
material, thus ruling out a practice effect on the first
The superior groups, on the other hand, did not
differ significantly on errors made on word lists vs.
stories regardless of difficulty of the material. The
SP and SN groups also did not differ from each other
significantly on these conditions, nor did the SP group
differ from the SWL group significantly. The means for
these groups appear in Table 4.
The Effect of Trials on Word Recognition
Comparisons were made between word lists given before
and after stories (or repeated presentations of the word
list in the case of the control groups) to assess for
For difficult material, every group made significant
gains over trials (See Table 4). Performance improvements
ranged from 4.643 to 2.571 words. The BAWL group made
the largest decrease in errors, an absolute difference
of 4.643, over trials.
However, this significant improvement in every group's
performance was not always the case for easy material
(See Table 4). There were no significant differences in
Error Means on Word Lists
SP 8.214 4.500
BAP 15.643 11.357
SN 10.357 6.000
BAN 12.769 9.154
SWL 6.286 3.714
BAWL 10.286 5.643
SP 2.286 1.571
BAP 6.928 4.143
SN 2.786 2.000
BAN 4.846 2.692
SWL 1.643 1.071
BAWL 4.857 2.071
the performance of superior readers across trials, on
easy material, which may have been due to small differ-
ences between basal and ceiling measures for these
readers. In general, superior readers did not make
as many errors in the first trials as below average
readers, thus there may have been little room for signifi-
cant improvement for the superior readers.
The Effect of Difficulty and Level on Word Recognition
Easy and difficult materials were clearly dis-
tinguished by number of errors made on each. In every
condition (WL, STRY, STRY with AID, and WL) there were
significant differences between easy and difficult
material for every group (SP, BAP, SN, BAN, SWL and BAWL).
In some cases, BA readers and S readers performed
equally well on material that had been designed to be
similarly easy and difficult for both groups. For
example, there were no significant differences between
BAWL, SN and SP groups or the BAN, BAWL and SN groups on
the easy word list. However, in most cases S readers made
fewer errors than BA readers on easy and difficult
material, e.g., on the difficult story, S readers made
significantly fewer errors than BA readers in the same
conditions (x s9.571, xB l21.928 x N=11.642, x N-16.000i
SP BAP SN BAN
x =6.857, x =9.857).
A multiple comparison test does not allow an effect
to be analyzed holding all other effects constant, so
that effects of difficulty and level on word recognition
have been reported elsewhere under the effects of
pictures and trials, since they are factors crossed with
these other factors. In addition, difficulty and context
effects are inseparable as reported above, though one may
still isolate one or the other factor.
Models for other Dependent Measures
For all other dependent measures, the data was
reorganized to look at only those groups and trials on
which these measures were recorded. An overall analysis
using a total model which included the independent varia-
bles picture, level, aid (formerly trials, the new varia-
bles included only two levels --- stories with aid and
stories without aid) and difficulty. A glm analysis of
variance was performed with this model on all dependent
variables. The story error category labeled "order" had
only two readings differing from zero and so was dropped
from the analyses. Also, the dependent variable affix
took a zero value in two of the subsequent analyses and
consequently an additional analysis excluding the variable
affix was performed for these two cases. A KYANOVA was
performed in the glm procedure using the full model to
test for main effects and interactions for all other
dependent variables including comprehension and story
The Effect of Pictures on Other Measures
There was no significant main effect for pictures
for any of the other dependent variables which included
comprehension and story error types. A MANOVA test
using the Motelling-Lawley Trace also did not reject
the hypothesis of no overall pictures effect (12,40=0.74,
p 4.71). Similarly, there were no significant two-way,
three-way or four-way interactions involving the Dictures
The Effect of Aid, Level and Difficulty
on Story Error Categories
The MANOVA tests for main effects due to aid, level,
and difficulty yielded significant ratios. Since there
were also significant interactions among these factors,
the effects of each were studied by observing one varia-
ble at fixed levels of the other two. The results of
this procedure are presented in the subsequent sections
for the remaining dependent variables.
A general note in regard to the effects of the
independent variable aid is in order here. Since aid on
stories was always given in the same order, i.e., no
aid, then aid, there was no way to rule out an effect
of order when scores were higher due to aid. Thus, the
effects of aid and practice are confounded for all varia-
bles. This factor may alternatively or more appropriately
be seen as the effect of trials or practice, as it was in
the previous analysis for effects on total errors.
The dependent variable comprehension was considered
an important variable in this experiment, particularly
with regard to the effect of pictures on comprehension.
As presented above in Table AL the effect of pictures
on comprehension was nonsignificant.
The effect of difficulty on comprehension scores
was significant for both S and BA subjects on the first
stories (without aid). Superior readers had a mean
comprehension score of 3.928 on the difficult story vs.
a mean score of 5.000 on the easy story. Likewise, the
BA readers had a mean score of 1.222 on the difficult story
and a mean score of 3.555 on the easy story.
The effect of aid on comprehension was significant
for S and BA reader for difficult stories only. Superior
subjects had higher mean comprehension scores when aid
was given than when it was not (x=5.500 vs. x=3.928).
Similarly, BA subjects scored higher on the comprehension
measure when aid was given than when it was not
(x=4.296 vs. x=3.555).
The effect of level on comprehension was significant
in two cases: on the easy story with aid, superior
subjects had a mean score of 5.678 while below average
subjects had a mean score of 4.296; on the difficult
story without aid, superior subjects had a mean score
of 3.928 while BA subjects had a mean score of 1.222.
These significant results may not be easily interpreted
in the former case because of the effect of aid/practice,
but it is interesting to note that comprehension scores
of BA and S readers did not differ for easy stories
before aid, or for difficult stories with aid. The
results of these independent variables' effects on compre-
hension are summarized in Table of the Appendix.
Repetitions, Reversals, and Insertions
The results of the analyses of the effects on
repetitions, reversals and insertion errors are grouped
together because the analyses revealed no significant:
differences between groups for any combination of the
independent variables. The summary table for these results
appear in Table A2 of the Appendix.
There was only one condition in which significant.
differences occurred in stem errors. This was for the
variable aid on BA reading of difficult stories. These
subjects made more stem errors (x=0.852) on stories before
aid than on stories with aid (x=0.370). Again, this
effect must be interpreted with caution since a practice
effect cannot be ruled out. The summary of results
appears in Table A3 of the Appendix.
In two instances, errors of the affix type could
not be included in the overall model because the variable
affix took on only the value of zero which produced
singular error matrices. These were the analyses labeled
aid for BA subjects on easy material and level on easy
stories with aid. Additional analyses removing the
variable affix from the model were performed for the
other variables in these instances and are included in
the appropriate sections.
The effect of difficulty on affix errors was
significant only for the S readers, at both levels of
aid. The means for easy and difficult stories without
aid, respectively, were 0.071 and 1.607. Likewise, the
means for easy and difficult stories, respectively, with
aid were 0.000 and 1.107.
The effect of aid was nonsignificant at every combi-
nation of level and difficulty, except the one that could
not be computed due to zero values in both aid and no aid
The effect of level was significant for difficult
stories with and without aid. There were no significant
effects for easy stories. On difficult stories without
aid, S readers made significantly more affix errors than
BA readers (x=1.607 vs. x=0.111). On difficult stories
with aid, S readers again made significantly more affix
errors (x=1.107) than BA readers (x=0.074). Results of
these analyses are summarized in Table A of the Appendix.
The effect of difficulty on substitution errors was
significant in every case except one. In general, both
BA and S readers made more substitutions on difficult
material. Superior readers made significantly different
mean substitutions 1.571 and 2.857 times, respectively,
on easy and difficult stories without aid; they obtained
significantly different mean substitution error scores
of 0.964 and 2.357, respectively, on easy and difficult
stories with aid. BA readers also obtained significantly
different mean substitution error scores of 3.852 and
5.815, respectively, on easy and difficult stories with
aid. Below average readers did not make significantly
different mean numbers of substitutions on easy or
difficult stories without aid.
The effect of aid or practice did not produce any
significant differences on substitution errors for three
out of four combinations of level and difficulty. How-
ever, for BA readers on easy material, aid was associated
with fewer substitution errors (x=3.852) than no aid
Readers' level contributed significantly to substi-
tutions on every combination of aid and difficulty. In
every case, S readers made fewer substitutions than BA
readers. The means were 1.571 and 0.964, respectively,
vs. 6.185 and 3.852, respectively, for superior vs.
below average readers on easy stories with and without
aid. For difficult stories with and without aid, re-
spectively, the mean scores were 2.857 and 2.357 for S
readers vs. 6.296 and 5.815 for 3A readers. Results of
the analyses for substitutions appear in Table Ac of
Difficulty of the reading material caused signifi-
cant differences in nonsense errors for superior readers
only, in both aid and no aid conditions. In the no aid
conditions, these subjects made more nonsense errors on
difficult material (x=2.893) than on easy material
(x=0.500). In the aid trials, S readers again made
significantly more errors on difficult material (x=1.250)
than on easy material (x=0.286).
Aid was a significant variable for S readers on
difficult stories (XNo Aid=2.893 vs' XAid=1250) and
for BA readers on easy stories (xNo Aid=1.888 vs.
xAid=1.148), in both cases aid or practice producing
fewer nonsense errors.
Level was significant for the difficult stories,
with and without aid. For stories without aid, S readers
made significantly more nonsense errors (x=2.893) than
BA readers (x=2.111). For stories with aid, however,
BA readers made significantly more errors (x=1.333) than
S readers (x=1.250). Table A6 of the Appendix summarizes
For every analysis studying difficulty at four
combinations of level and aid, significant differences
existed. Superior readers made more omissions on diffi-
cult than easy material, regardless of aid. The means
were 4.571 and 0.714 errors, respectively, on difficult
and easy stories without aid. The means on difficult
and easy stories with aid were 4.250 and 0.786 errors,
respectively. Likewise, BA readers made significantly
more omissions on difficult than on easy material, re-
gardless of aid. On stories without aid, BA mean error
scores were 19.11 and 6.074, respectively, for difficult
and easy material. Similarly, on stories with aid, the
means were 12.630 errors vs. 3.630 errors, respectively,
for difficult vs. easy material.
The effect of aid or practice was significant in
two instances for BA readers only. These subjects made
fewer omissions with aid on both easy and difficult
stories. For the easy stories, the mean errors were
3.630 with aid and 6.074 without aid. For the difficult
stories, mean errors were 12.630 with aid and 19.111
Level of subjects' reading ability was also an
important factor in omission errors, BA readers making
more omissions than S readers in every case (x3A=6.074
vs. xS=0.714 for easy stories without aid; XBA=19.111
vs. xs=4.571 for difficult stories without aid; XBA=3.630
vs. xS=0.786 for easy stories with aid; and xBA=12.630
vs. xS=4.250 for difficult stories with aid). Summa--r
of these results appears in Table A7 of the Appendix.
Difficulty did not affect mean numbers of self-
corrections to a significant extent, for any combination
of level and aid.
Aid or practice produced significant differences in
three instances: for both S and BA readers on easy
material and for S readers on difficult material. Mean
number of self-corrections were lower with aid or practice
than without it in these conditions. For S readers, the
mean scores were 1.428 and 1.857, respectively, for easy
and difficult stories without aid, and 0.571 and 0.893,
respectively, for easy and difficult stories with aid.
Below average mean scores for easy stories were 2.852
without aid and 1.555 with aid.
Level was a significant factor in self-corrections
in two cases, both on easy stories. On easy stories
without aid, S readers made fewer self-corrections
(x=1,428) than BA readers (x=2.852). On easy stories
with aid, S readers also made fewer self-corrections
than BA readers (x=0.571 vs. x=1.555). Again, since S
readers made so few errors, they could hardly self-
correct very much. Results are summarized in Table AS
of the Appendix.
In every case, repeated errors were more frequent
on difficult than easy material (x=3.107 vs. x=0.500
for S readers on stories without aid; x=0.893 vs. x=0.286
for S subjects on stories with aid; x=11.888 vs. x=5.592
for BA subjects on stories without aid; and x=6.074 vs.
x=1.481 for BA subjects on stories with aid).
Also, in every case, S readers made fewer repeated
errors than BA readers (see mean scores above).
The effect of aid or practice was significant in
three out of four combinations. Aid did not produce
significant differences in S readers' repeated errors on
easy stories without aid. In all other cases, though,
aid or practice produced fewer repeated errors than no
aid (see mean scores above). Results of these analyses
are summarized in Table A9 of the Appendix.
The results from the analyses of the effect of
pictures on word recognition and types of errors provide
substantial support for the previous findings of Singer,
Samuels and Spiroff (1974) and Samuels (1967). Superior
readers did not differ on number of errors in picture
and no picture conditions, while below average readers
in the picture condition made significantly greater
numbers of errors than those in the no picture condition.
Thus it appears that pictures do indeed influence be-
ginning reading of below average ability. However, when
aid for a missed word was given on the easy stories,
pictures did not significantly influence differences
between the two below average groups. Thus, if a below
average reader has had practice with words and the words
are of an easier level, pictures may not interfere.
Pictures did not significantly influence compre-
hension or types of errors made. The result for compre-
hension is in line with earlier results of Vernon's (1954)
and supports Chall's (1967, p. 259) contention that while
full-color illustrations are the most expensive features
of reading texts, it has not been demonstrated that they
help children either to recognize words or comprehend
text. Given the finding that pictures interfere with
word recognition on stories for below average readers,
one might expect that as a result of that process, compre-
hension of story material should also be impaired. While
pictures did not significantly contribute to comprehension
scores, they also did not impair them in the present study.
It may be that better comprehension measures than the
crude one in the present study or the one in Vernon's
study (number of facts remembered about a story) need to
be developed to adequately assess the effect of pictures
on this measure of reading. A very detailed and complex
measure has very recently been developed by Drum (1977)
which may prove helpful in this regard. Another explanation
may be that even while pictures interfered with word recog-
nition for poorer readers, these readers may have used
the pictures or contextual cues in the stories to bring
them up to the level of the other readers on comprehension.
Still another alternative may be that superior readers were
able to recognize or sound out correctly more words, but
were not reading for meaning or understanding, since the
below average and superior groups did not differ from
each other in two cases of comprehension scores.
An additional original question was whether pictures
might not produce certain types of errors more frequently,
e.g., more types of errors which were nonsensical or not
graphically similar to the text, since the child might be
distracted by the pictures. The fact that pictures in
this study did not account for a significant amount of
variance between groups or particular types of errors
did not support this hypothesis. In the present study,
the number and distinction between types of errors may
have been too many or may have obscured the effect, but
the graphic similarity measure has at least partially
ruled out that explanation. A more precise measure of
graphic similarity, using a mathematical formula (Weber,
1970) may be a future task to undertake in this regard.
Yet the interpretation may simply be that while pictures
interfere with poorer readers' word recognition, these
children may use a combination of cues to produce their
wrong answers. This interpretation might concur with
Biemiller's (1970) notion of developmental phases in the
use of contextual and graphic information in learning to
read. Perhaps the poorer readers in the present study
would be in his "pre-nonresponding" phase in which there
is more reliance upon contextual cues. However, one
might expect to see better performance on stories than
word lists if this is the case. In the present study,
this result was not obtained for below average readers.
Still, the poorer readers may indeed have been relying
more on contextual cues which did not bring them success
on the stories.
These results imply readers of below average ability
may require early reading material without pictures and
with aid on focusing attention to graphic detail when
vocabulary is especially difficult for them. The fact
that the poorer readers performed better with aid is not
surprising since the experimenter's help on a given word
would most likely direct the reader's attention to the
word and thus force the reader to focus on the graphic
detail or configuration of the word. For easy material,
the below average readers may have mastered the words
enough to be able to use orthographical cues efficiently
without the frustration of difficult material which
might have forced them to turn to extragraphical infor-
mation. Pictures may serve this function for the below
average reader on difficult material, i.e., they may offer
the child an alternative method for decoding words which
is not successful since a picture cannot reliably direct
the child to an abstract symbolization or picture-word
association. There are two methods of beginning reading
instruction containing no pictures which do exist, based
on the author's beliefs that the child learning to read
should not rely on them. One method, the Bloomfield
system (1963) argues that the first task of learning
to read is breaking the code, or the alphabetic princi-
ple. Oral reading is stressed over silent reading at
first, and use of context cues is also discouraged.
The second method, the Carden Reading Method (Carden,
1967) also excludes pictures and emphasizes comprehension
and literary appreciation as well as phonic skills.
Though this study did not address the issue, pictures
may also inhibit a child's development of creative
imagery and storytelling techniques. In Chall's interview
(1967) of 25 reading specialists, one author said, ".
in first grade, pictures are a big part of our word-
analysis program, . but there is no question that
there may be a delimiting effect of pictures on concepts
and creativity. . (p. 70)". In addition, Chall suggests
that having children rely on pictures to learn to read
may rob them of early intellectual growth. She states,
"Pointing to and naming or writing a letter at an early
age is quite different from pointing to or drawing a
picture of a cat, truck or tree. The child who can
identify. .. a letter engages in symbolic representation.
When the child engages in symbolic representation, he is
already practicing a higher form of intellectual behavior
(1967, p. 159)". The author cites no research to support
these beliefs but it may well be another important issue
The possibility of a developmental trend over age
groups would be a further avenue to pursue in regard to
the picture question. Wardles' (1977) data may suggest
that older readers of below average ability may be
helped by the presence of pictures, on certain types
of material. Because of what appears to be the publishing
companies' insistence on more and more full-color illus-
trations in texts, do the reading series themselves set
up a learning paradigm in which children are shaped to
look at the pictures for clues because the answers to
the comprehension questions can more easily be found in
the pictures? Or, as a child grows older, do pictures
really begin to take on relevant meaning for text
material, especially in some content areas such as science?
The question of what is considered relevant for
picture content also arises. The present study did not
attempt to modulate relevancy, color or location of
pictures. The pictures used were relatively modest in
comparison to those in most readers; they were black and
white, story-related and were placed at the left side
of the story. Still they distracted below average
readers from the stories. The fact that the pictures
used in the present study did not contribute significantly
to comprehension scores or other types of errors may
reflect this simplicity of design. However, there is at
least one reading program, the Gibson-Richards program,
or Language through Pictures Series (1963) which uses
only black and white stick figures and argues for their
success in serving as useful clues to sentences. Ricnards
(1968) suggests the use of pictures of "hieroglyphic
simplicity --- not as cues merely to word meanings but
as an accompanying pictorial language, pruned of dis-
tractive possibilities (p. 362)". The position of
pictures as well as their stimulus content may be another
potential area of study. For example, if the pictures
are seen after the story, as in the Distar method (SEnel-
mann and Bruner, 1974) and not with the story, might
they be less likely to interfere with attention to
graphic detail, and more likely to reinforce the learning
of new words? This appears to be an exciting avenue to
In summary, the results of the present study indicate
that pictures may interfere with beginning reading when
readers are of below average level. When the same words
have been used repeatedly, individual aid is given, and
the words are of an easy difficulty level, pictures may
no longer distract the poor reader. Pictures do not
appear to help or hinder superior beginning reading.
Also, the pictures used in this study did not contribute
significantly to comprehension scores or to particular
types of errors.
The results of this study also support and extend
the focal attention hypothesis (Samuels, 1967) with
regard to the issue of contextual cues in reading.
Again, the poorer readers made more errors on stories
than on word lists, while those who received only word
lists did not differ across the two trials, -hus ruling
out a practice effect. While Samuels reported the
effect of pictures on poor reading, neither ne nor his
colleagues, Singer et al. (1974) reported the deleterious
contextual effect specifically for poor readers, They
did note overall reduced performance with context, but
did not report differences due to level of the readers.
This finding is in opposition to that of Goodman's (1965)
in which word recognition in stories was higher than that
on word lists, regardless of a subject's reading ability.
The fact that superior readers did not differ on
word recognition from word list to story may be interpreted
as support for a hypothesis such as that of 3iemiller's
(1970) in which readers progress to a stage in which they
utilize both graphic detail of individual words and
contextual information present in stories. The pattern
of errors revealed for superior readers on difficult
material would suggest that even superior readers,
though, may "regress" to an earlier developmental stage
when faced with difficult material. This error pattern
is discussed further in a later section.
Again, the results of the present study would
indicate that poorer readers do not learn words as
efficiently when they are presented in stories as when
they are presented in isolation on a word list.
The present study showed difficulty of reading
material to be an important factor to consider in
studies of beginning reading. Further refinement of
the process whereby easy and difficult material is
devised for below and above average readers is needed.
To make fair generalizations about the effect of any
other variable, difficulty levels for superior and below
average readers need to be equated at the beginning of
testing. This is not always easy to do for the be-
ginning reader of below average ability whose sight
vocabulary may not be very large to start with. The
present procedure was only partially successful because
below average readers made more errors than superior
readers even on easy material, but represents an improvement
over past studies which did not consider it as a varia-
ble. The study did allow superior and below average
readers to perform at comparable levels in many cases
(e.g., on comprehension of easy stories. Also, certain
other effects (e.g., the effects of level and practice)
were significant only for difficult or easy material,
and this is an important extension over previous studies.
An example of this was the significant effect of level
found only for the difficult stories on nonsense errors
superior readers made more nonsense errors than below
average readers on difficult stories without aid,
another example was that found for differences in before
and after word recognition --- for superior readers, the
effect was significant only for difficult material
which may have been due to the basal effect for superior
readers on easy material.
Learning, or improvement in word recognition across
trials occurred for all groups on difficult material
and for below average readers on easy material as well.
Even though practice effects were likely across so many
trials, the effects of aid from the experimenter were
most likely also contributory, because in the case of
superior and control subjects, no improvement occurred
in the two trials before aid was given though significant
improvement did occur in both groups after aid. This
result would seem obvious since giving the subject the
word would allow learning, but some practice effect of
trial and error may also have occurred across trials.
Learning appeared to be most enhanced for below
average readers, by a simple reading of a word list with
individual aid given on the unknown or missed words, and
a post-test of the same list to reinforce the learning.
No significant word knowledge was gained by superior
readers on easy material, but this was probably due to
the fact that the superior readers usually had fewer
new words to learn in the first place, and so their
improvement might be small in comparison to that of
below average readers. Yet, one might also expect the
opposite finding --- with fewer new words to learn,
superior readers might be expected to be able to master
them more easily than if they had many new words to learn.
The possibility exists that these readers were not
challenged by the easy material, which brings up the
question of how best to improve learning in superior
readers. Since pictures did not help or hinder their
reading and stories vs. word lists did not help or hinder,
the present data suggest material of sufficient difficulty
may be helpful in increasing these readers' sight vocabulary.
Types of Story Category Errors
Repetition, reversal, insertion and omission errors.
The present study found no differences between groups on
repetition reversal or insertion errors. The result for
reversals was somewhat surprising in view of the fact
that poor readers (especially dyslexic children) are
typically seen as evidencing more of these kinds of
errors (Meier, 1972). However, none of the first graders
in the present study had been identified as specifically
learning disabled nor did any attend a resource room as
part of their school program. Many of the poor readers
had been recommended for a summer remedial reading
program, but were ineligible to participate because
their scores were above the 20th and 30th percentiles
on the Metropolitan Reading test. So, the population
of poor readers studied here may be distinct from more
severely disabled readers in terms of their error patterns.
There were also no differences between groups on
numbers of repetitions or insertions. Thus, both superior
and below average readers may repeat words for other
reasons than difficulty of the word. Likewise, below
average readers in general did not appear to insert
irrelevant or extraneous words into stories more often
than superior readers, as might have been guessed. However,
below average readers did omit significantly more words
than superior readers and this may have been their pattern
of approach to a difficult or unknown word. Below average
readers also made more self-corrections than superior
readers on easy stories, but both groups made equal
numbers on difficult material. This result may again be
due to the basal effects for superior readers on easy
Types of substitution errors. Stem, affix, substi-
tution and nonsense errors may all be seen as particular
types of substitutions of words other than the text
words. As might be expected on an error type which con-
tains part of the correct word, superior readers made
more affix errors than below average readers. This was
also the case for nonsense errors on difficult stories
without aid. Nonsense errors and graphic similarity
were highly correlated at .72, so that this finding may
not be as surprising as one might at first think.
Nonsense errors are simply incomplete or meaningless
words and may still represent an attempt to decode the
material. However, this finding was reversed on stories
with aid, in which below average readers made more
nonsense errors. It may be that, as attention was drawn
to the words with individual aid, initial attempts at
recognition, or a vacillation between word and picture,
yielded these types of errors. Also, the effect of aid
may have been to motivate the poor readers to try to
decode or sound out the words which would also produce
more nonsense errors. The general results may also
suggest though, that the superior readers were at first
reading difficult material word by word, not relying on
the contextual cues of the sentences, but with aid and
practice began to do so, or that with aid, they began
to use both contextual and graphic cues, as Biemiller
(1970) suggests. The fact that poorer readers made
more substitutions of complete and meaningful words
than better readers also implies that the poorer readers
may rely heavily on meaning and contextual cues in
reading, but that these strategies are largely unsuc-
cessful for them in learning to recognize new words.
The results of the present investigation point
out several important findings. Firstly, pictures and
contextual cues do not appear to contribute significantly
to beginning reading and especially appear to interfere
with poorer readers' acquisition of new word vocabulary.
Secondly, difficulty of the material used can be an
important variable in studies comparing below and above
average reading. Thirdly, several interpretations were
made about the relative differences between these two
groups on particular types of errors. The characterization
of the superior reader as an efficient user of both
context and graphic detail may be questionable in certain
cases of difficult material, and characterization of
poorer readers as strugglers for meaning in reading may
be justified on the basis of these results. When material
is very difficult, superior readers may tend to decode
words graphically rather than using both contextual and
graphic information. Poorer readers tend to rely on
inappropriate strategies, utilizing meaning and contextual
cues on difficult material but aid may assist them in more
attempts to decode words.
Many future studies suggested here may hold
interesting answers to the questions raised by this
study. For example, an extension of the present study
over second and third grades, with a follow-up at grade
seven would provide much needed data on the develop-
mental trends which may be present in the use of pic-
torial and contextual cues in easy and challenging
Barr, A. J., Goodnight, J.H., Sail, J.P., and Helwig, J.T.
A User's Guide to SAS, Raleigh, N.C., SAS Institute,
Biemiller, A. The development of the use of graphic and
contextual information as children learn to read.
Reading Research Quarterly, 1970, 6, 75-96.
Bourisseau, W., Davis, O.L. & Yamamoto, K. Sense-im-
pression responses to differing pictorial and verbal
stimuli. Audio-Visual Communication Review, 1965.
Bloomfield, L., & Barnhart, C.L. Let's Read (experimental
ed.), 1963, Bronxville, N.Y. Clarence L. Barnhart,
Carden, M. The Carden Method, Mae Carden, Inc., l3en
Rock, N.J.7, 1T5.
Chall, J.S. Learning to Read: The Great Debate, New York:
Mc raw- iI7-, 1677
Cohen, A.S. Oral reading errors of first grade children
taught by a code emphasis approach. Reading Research
Quarterly, 1975, lo, 616-650.
Concannon, S.J. Illustrations in books for children
review of research. The Reading Teacher, 1975,
Cronbach, L.J. Coefficient alpha and the internal structure
of tests. Psychometrika, 1951, 16, 297-334.
Duell, 0. An analysis of prompting procedures for teaching
a sight vocabulary. American Educational Research
Journal, 1968, 5, 675-66.
Drum, P.A. Types of recall response by good and poor
readers. Paper presented at the Second Annual Boston
University Conference on Language Development, Sep.
30-Oct. 1, 1977.
Elkind, D., Larson, M., & Van Doorninck, W. Perceptual
decentration learning and performance in slow and
average readers. Journal of Educational Psychology,
1965, 56, 50-56.
Engelmann, S., & Bruner, E.C. Distar Reading System,
1974, Science Research Associates, Inc., Chicago.
Fairbanks, 3. The relation between eye-movements and
voice in the oral reading of good and poor silent
readers. Psychological Monographs, 1937, 48, 78-107.
Gibson, C.M., & Richards, I.A. Language through Pictures
Series, 1963, Washington Square Press, Pocket BooKs,
Inc., New York.
Soodman, K.S. A linguistic study of cues and miscues in
reading. Elementary English, 1965, 42, 639-643.
Goodman, Y.M. Longitudinal study of children's oral
reading behavior. Final report. Contract No. 03E
-5-9-325062-0046. U.S. Offices of Education, 1971.
Goodman, Y.M., & Burke, C.L. The Reading Miscue Inventory.
New York: Macmillan, 1972.
Hartley, R.N. Effects of list types and cues on the
learning of word lists. Reading Research Quarterly,
1970, 6, 97-121,
Hood, J. Qualitative analysis of oral reading errors: the
inter-judge reliability of scores. Reading Research
Quarterly, 1976, 4, 577-598.
Marchbanks, G., & Levin, H. Cues by which children
recognize words. Journal of Educational Psychology,
1965, 56, 57-61,
Meier, J.H. Learning disabilities found in elementary
schools. In P. Satz and J.J. Ross (Eds.), The
disabled learner, 1973, Rotterdam University-Press,
Rotterdam, Netherlands, pp. 101-120.
Miller, W.A. Reading with and without pictures. Elementary
School Journal, 1938, 38, 676-682.
Piaget, J., & Morf, A. Les isomorphismes partiet/s entire
les structures logiques et les structures perceptives.
In J. Piaget (Ed.) Etudes d'Epistemologie Genetique,
Vol. 6, Paris: Presses Universitaries de trance, 1958.
Richards, I.A. Book review of J. Chall's Learning to
Read: The Sreat Debate. Harvard Educational
Review, --6B -7 -7357-64.
Sabatino, D.A., & Ysseldyke, J.E. Effect of extraneous
"background" on visual-perceptual performance of
readers and non-readers. Perceptual and Motor Skills,
1972, 35, 323-328.
Samuels, S.J. Attentional processes in reading: the
effects of pictures on the acquisition of reading
responses. Journal of Educational Psychology,
1967, 58, 337-342. -
Schonell, F.J. The Psychology and Teaching of Reading,
1961, Philosophical Library,- New York.
Singer, H., Samuels, S.J., & Spiroff, J. The effects of
pictures and contextual conditions on learning
responses to printed words. Reading Research
Quarterly, 1974, 4, 555-567.
Tarver, S.3., & Hallahan, D.P. Attention deficits in
children with learning disabilities: a review,
Journal of Learning Disabilities, 1974, 9, 560-569.
Underwood, 3.J. Stimulus selection in verbal learning.
In C.N. Cofer & B.S. Musgrave (Eds.), Verbal Behavior
and Learning: Problems and Processes, 1976 Mc3raw-
Hl, New York.
Vernon, M.D. The instruction of children by pictorial
illustrations. British Journal of Educational
Psychology, 1954,- 171--T79. -
The value of pictorial illustrations.
ritishr Journal of Educational Psychology, 1953, 23,
Wardle, K.F. Textbook illustrations do they aid reading
comprehension? Paper presented at the American
Psychological Association, San Francisco, August, 1977.
Weber, R.M. First-graders' use of grammatical context in
reading. In H. Levin & J.P. Williams (Eds.), Basic
Studies on Reading, 1970, Basic Books, Inc., New York.
SThe study of oral reading errors a survey
of the literature. Reading Research Quarterly, 1968,
Weintraub, S. The effect of pictures of the comprehension
of a second grade basal reader. Dissertation
Abstracts, Dec., 1960, 1428-1429.
Whipple, G. Appraisal of the interest appeal of illus-
trations. Elementary School Journal, 1953, 53,
Winer, 3.J. Statistical Principles in Experimental Design,
1962, Mc -aw-ri, New York.
S SS p
Table Al (cont.)
Source df SS ?
Difficulty=l, Aid=l 1 28.679 3.39 n.s.
Error 26 219.666
Difficulty=l, Aid=2 1 26.263 5.13 <0.322
Error 26 133.237
Difficulty=2, Aid=l 1 100.676 10.58 \0.0032
Error 26 247.524
Difficulty=2, Aid=2 1 39.035 3.26 n.s.
Error 26 311.074
Source df SS p
Table A2 (cont.)
Source df SS p
Table A2 (cont.)
Source df SS ? p
Table A2 (cont.)
source at ss
Table A2 (cont.)
Source df SS p
Source d SS p
Level=l, Aid=l 1 0.013 0.03 n.s.
Table A3 (cont.)
Source df SS
Difficulty=l, Aid=l 1 0.0-& 0.07 n.s.
Error 26 16.983
Difficulty=l, Aid=2 1 0.132 0.40 n.s.
Error 26 8.61l
Difficulty=2, Aid=l 1 1.702 2.78 n.s.
Error 26 15.907
Difficulty=2, Aid=2 1 0.231 0.53 n.s.
Error 26 11.296
Source df SS p
Table A4 (cont.)
Source df SS
Difficulty=l, Aid=l 1 0.070 0.98 n.s,
Error 26 1.857
Difficulty=l, Aid=2 CANNOT BE COMPUTED
Difficulty=2, Aid=l 1 30.764 26.36 0.0001
Error 26 30.345
Difficulty=2, Aid=2 1 14.669 38.03 0.o0001
Error 26 10.030
Source df SS F p
Table A5 (cont.)
SSource df SS p
Table A6 (cont.)
Source df SSI p
Source Fd-F SS F p
Table A7 (cont.)
dl 55 7
Table A8 (cont.)
Source df 5
Difficulty=l, Aid=l 1 27.844 5.90 40.0224
Error 26 122.764
Difficulty=l, Aid=2 1 13.313 13.05 <0.0013
Error 26 26.524
Difficulty=2, Aid=l 1 0.186 0.06 n.s.
Error 26 83,114
Difficulty=2, Aid=2 1 4.182 2.04 n.s.
Error 26 53.345
- Source -f SS ? p
Table A9 (cont.)
Source df SS
Difficulty=l, Aid=l 1 356.481 38.86 0.o0001
Error 26 238.519
Difficulty=l, Aid=2 1 19.654 17.96 <0.0002
Error 26 28.455
Difficulty=2, Aid=l 1 1060.036 33.97 <0.0001
Error 26 811.345
Difficulty=2, Aid=2 1 368.997 42.54 <0.0001
Error 26 225.530