|
EFFECT OF A COGNITIVE STRATEGY ON
THE WRITING ABILITY OF COLLEGE STUDENTS
WITH LEARNING DISABILITIES
By
BETH LORENE TULBERT
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1992
This work is dedicated to the memory of
my brother, Bont Franklin Tulbert. He
taught me to not give up, to never take
no for an answer, and to live life as if
this were my last day. A part of him is
with me always.
ACKNOWLEDGEMENTS
These last three years have been three of the most
challenging, difficult, and growth-filled years of my life.
Several people have helped support me and are due
considerable thanks. First, and foremost, I would like to
thank my parents. They have supported me emotionally and
financially during this trying and exciting time. They
taught me that I could succeed at anything if I would only
try. Without them, I could never have made it through this
rigorous and demanding program. This degree is as much
theirs as it is mine.
I would like to collectively thank my committee members,
Dr. Cecil Mercer, Dr. Linda Crocker, Dr. Cynthia Griffin,
Dr. William Reid, Dr. Jeanne Repetto, and Dr. Stuart
Schwartz. Their guidance and support have enabled me to
complete a demanding and rigorous program culminating in this
dissertation. I appreciate the time and effort given by each
of them on my behalf.
I would like to thank my advisor and mentor, Dr. Cecil
Mercer. He has guided me in the right direction and helped
me get from this program all the things that I would have
said that I wanted if I had known what to ask for in the
beginning. I am glad to have had him as a role model these
iii
last three years and hope that I can continue to follow in
his thoughtful, positive foot steps.
I would also like to thank Dr. Stuart Schwartz. He
helped prepare me for the demands of this program in Trends
and instilled in me the belief that I could succeed. He
included me in a variety of opportunities, challenged me to
do my best, and provided me with a standard of excellence
toward which to strive.
I would like to thank my comrades in arms, Donna Wandry
and Glenn Buck. Together we made it through Trends, the
various doctoral seminars, and all of the research courses.
We were the Happy Trio and without them both, the last three
years would just not have been the same.
I would like to thank Dr. Jeanne Repetto. She listened
and supported me throughout this program, especially during
"the week before," and has been a wonderful mentor and
friend. I am sure I will look back at all of this with "fond
memories."
I would like to thank Dr. Mary Brownell for our little
talks in the back room. She has been a soul-mate, good
friend, and role model. And, yes, it is too late.
Finally, I would like to thank Ken Osfield at the
University of Florida and Issac Jones at Santa Fe Community
College for all of their time and effort in making this study
possible. Without their cooperation and commitment to this
project, this study would never have been completed.
TABLE OF CONTENTS
Pacre
ACKNOWLEDGEMENTS .......................................... iii
LIST OF TABLES ......................................... viii
ABSTRACT .................................................. ix
CHAPTERS
1 INTRODUCTION ......................................... 1
Statement of the Problem ................. .......... 4
Rationale ................. ........................... 5
Definition of Terms ................................. 7
Delimitations of the Study ......................... 11
Limitations of the Study ............................. 12
Summary and Overview of Remaining Chapters ......... 12
2 REVIEW OF RELATED LITERATURE ....................... 14
Selection of Relevant Literature .................... 14
Interventions for College Students with
Learning Disabilities ................................. 16
History of Postsecondary Programs ............... 16
Description of Postsecondary Programs .......... 20
Learning Strategies ................................... 29
Background on Strategy Instruction .............. 29
Research on Writing Strategy Instruction ........ 38
Summary .............................................. 42
3 METHODOLOGY .......................................... 44
Hypotheses ........................................... 44
Subjects and Setting ................................. 45
Measurement Procedures ................................ 48
Screening for PENS Class ........................... 48
Pretest and Posttest .............................. 48
Course Content and Instructional Methods ........ 54
Reliability of Procedures ......................... 56
Interscorer Agreement .............................. 56
Generalizability Study ............................. 56
Procedural Reliability ............................. 57
Training of Personnel ................................. 58
Training of Student Assistant .................. 58
Training of Observers .............................. 59
Training the Scorers for Reliability Checks ..... 59
PENS Instructional Procedures ...................... 60
Experimental Design .................................. 62
Analysis of the Data ................................. 64
Summary ............................................ 64
4 RESULTS ................ .............................. 66
Descriptive Analysis of the Data ................... 66
Fluency ........................................... 66
Syntactic Maturity ................................. 68
Vocabulary ........................................ 69
Mechanics ......................................... 70
Composition Organization ........................... 71
Inferential Statistical Analyses of the Data ....... 71
PENS Writing Strategy Subjects .................. 71
Hypotheses ........................................ 72
Exploratory Analysis ............................... 74
Reliability ........................................ 80
Procedural Reliability ............................. 80
Scorer Reliability ................................. 80
Generalizability Study ............................... 83
Summary of Findings ................................... 87
5 DISCUSSION ........................................... 89
Discussion of Hypotheses .............................. 89
Feedback from Students and Staff ........... ........ 93
Educational Implications .............................. 96
Services for College Students with Disabilities 96
PENS Writing Strategy Instruction ............... 99
Future Research ...................................... 100
Summary .............................................. 102
APPENDICES
A PROGRAMS IN POSTSECONDARY SETTING FOR
STUDENTS WITH LD.............................. 104
B LETTER TO UF STUDENTS AND TENTATIVE
COURSE SYLLABUS .............................. 106
C INFORMED CONSENT FOR STUDENTS IN PENS CLASS... 110
D INFORMED CONSENT FOR STUDENTS IN GORDON RULE
AND WRITING LABS ............................. 113
E SCORER SHEET .................................. 116
F SCORE CALCULATION FORM......................... 118
G FACULTY QUESTIONNAIRE ......................... 120
H OBSERVATION FORM .............................. 125
REFERENCES ............................................. 126
BIOGRAPHICAL SKETCH ..................................... 133
vii
LIST OF TABLES
Table Page
1 The PENS Strategy .................................. 9
2 The PENS Strategy Sentence Formulas ............... 10
3 Description of Subjects .......................... 47
4 Study Design ..................................... 63
5 Means, (Standard Deviations), and [Range] of
Pretest and Posttest Scores on the Five Writing
Measures .......................................... 67
6 Individual t-tests for the UF and SFCC
Subjects in the PENS Class........................ 74
7 ANCOVA Analysis of the Five Measures .............. 77
8 Follow-up Test p-values for Comparisons
Between Groups Using the Least Squares Means ...... 79
9 Pretest and Posttest Correlations.................. 79
10 Correlations Between Pretest and Posttest
Scores for All Subjects .............. ............ 81
11 Correlations Between Pretest and Posttest
Scores for Subjects by Treatment .................. 81
12 Procedural Reliability for PENS Instructor
Behaviors.. ................................ ....... 82
13 Subjects Rated by the Five Scorers ................ 83
14 Reliability for Fluency Scores .................... 84
15 Reliability for Nonsentence, Simple/Compound
Sentence, Complex Sentence, and Organization
Scores .. ......................................... 84
16 Reliability for Vocabulary Scores ................. 85
17 Reliability for Mechanics Scores .................. 85
18 Generalizability Coefficients ..................... 88
viii
Abstract of Dissertation Presented to the
Graduate School of the University of Florida
in Partial Fulfillment of the
Requirements for the Degree of
Doctor of Philosophy
EFFECT OF A COGNITIVE STRATEGY ON
THE WRITING ABILITY OF COLLEGE STUDENTS
WITH LEARNING DISABILITIES
By
Beth Lorene Tulbert
August 1992
Chair: Cecil D. Mercer
Major Department: Special Education
The purpose of this study was to (a) determine the
effectiveness of cognitive writing strategy (PENS)
instruction and (b) compare the effect of PENS learning
strategy instruction, Developmental Writing course
instruction, and Gordon Rule course instruction on written
expression skills of college students with learning
disabilities. The subjects in this investigation were
college students with learning disabilities who have
registered with the Office of Disabled Student Services on
the campuses of the University of Florida and Santa Fe
Community College. The study included a pretest, an
implementation phase, and a posttest. Scores were obtained
from writing samples on fluency, syntactic maturity,
vocabulary, mechanics, and organization of written work. A
nonrandom pretest-posttest design with one between
(educational experience) and one within (performance over
time) group factor was used. A MANCOVA procedure failed to
reject the multivariate null hypothesis (no treatment group
effect on the five criterion measures after adjustment on the
pretest scores). Exploratory analyses were conducted.
Individual ANCOVAs were used to test for significant
treatment effect for the five criterion measures between the
three groups on posttest scores after adjusting for the
pretest scores. No significant difference was found between
the treatment groups for Fluency, Syntactic Maturity, and
Organization. Significant differences were found among the
treatment group means on Vocabulary and Mechanics. Follow-up
tests using pairwise contrasts found no significant
differences for Mechanics and Vocabulary; however, two
pairwise tests approached significance for Vocabulary
suggesting that PENS subjects made gains in Vocabulary while
subjects in the Developmental Writing course and the Gordon
Rule courses did not make improvements in Vocabulary.
Two-way Repeated Measures ANOVAs were also used to evaluate
the success of each treatment in improving writing skills on
each of the five criterion measures from the pretest to the
posttest occasion. On the average, across the three
treatments, a gain was made in Syntactic Maturity (writing
more complex sentences). These results have educational
implications for college students with learning disabilities.
CHAPTER 1
INTRODUCTION
Educational opportunities for individuals with learning
disabilities have increased rapidly over the last two decades
through the combined efforts of the Education for All
Handicapped Children Act of 1975 (EHA), Section 504 of the
Rehabilitation Act of 1973 (Section 504) (Apostoli, 1986),
and the Americans with Disabilities Act (ADA) (Linthicum,
Cole, & D'Alonzo, 1991). Often referred to as civil rights
legislation, Section 504 and ADA stipulate that colleges and
universities must ensure the rights of qualified students
with disabilities to enter colleges and universities and to
participate fully in all programs (Vogel, 1982; Linthicum,
Cole, & D'Alonzo, 1991). Specifically, Section 504 and ADA
prohibit discrimination against persons with disabilities in
recruitment, admissions, and treatment in postsecondary
education and require that colleges and universities make
reasonable accommodations or adjustments to assist students
with disabilities in meeting academic requirements.
Officials and administrators in institutions of higher
education are attempting to resolve the dilemma of
appropriately meeting the needs of students with learning
disabilities without compromising their schools' academic
standards. Extending support services for students with
disabilities to the college level can provide a solution to
this dilemma (Dalke & Schmitt, 1987). Providing support
services for students with learning disabilities in
postsecondary institutions (a) is a logical extension of
services offered at the secondary school level, (b) can be an
important component of the reasonable accommodations mandated
by law, and (c) ensures their rights to participate fully in
educational programs at the college level for several
reasons.
First, several characteristics of adolescents and adults
with learning disabilities hinder their success in college.
Many college students with learning disabilities continue to
report problems with reading, writing, notetaking (Ryan &
Heikkila, 1988), mathematics, study skills, interpersonal
skills, and oral communication (Vogel, 1985). Deficits in
written expressive language among individuals with learning
disabilities are often severe, persist into adulthood, and
may be more prevalent than earlier identified reading
difficulties (Vogel & Konrad, 1988). It has been estimated
that 90% of adults with learning disabilities experience
written language disorders (Vogel, 1987). Additionally, the
ability to express oneself clearly and precisely in writing
is considered by some faculty to be synonymous with a
bachelor's degree. Others consider the writing process to be
a catalyst for the thinking process itself (Vogel, 1987).
Therefore, written expression is an area of great concern to
college faculty and students with learning disabilities
(Vogel, 1982).
Second, students with learning disabilities often are
unprepared for the demands of college (Prater & Minner,
1986). A survey of curriculum approaches used in high
schools with students with learning disabilities revealed
five major types of programs (Deshler, Lowery, & Alley,
1979), including a basic skills remediation model, a tutorial
model, a work-study model, a functional curriculum model, and
a learning strategies model. The remedial, tutorial, work-
study, and functional curriculum models of instruction do not
prepare students with learning disabilities to be independent
academic learners. The ability to learn independently is a
critical skill in higher education where students are
required to locate, learn, and retain information with little
assistance from college professors (Prater & Minner, 1986).
Third, setting demands in college (expectations for
college students) may create seemingly insurmountable
problems for students with learning disabilities. All
college students are compelled to grapple with intense
demands on managing their time and this may be a particular
problem for students with learning disabilities (Ryan &
Heikkila, 1988). Moreover, students with learning
disabilities often face negative attitudes from professors in
higher education and they may lack necessary support
personnel and systems to help them succeed (Prater & Minner,
1986).
Consequently, support services are needed for college
students with learning disabilities. Interventions that can
(a) address specific academic weaknesses of students with
learning disabilities, (b) mediate setting demands of higher
education, and (c) prepare students to function independently
in the college setting are needed. The University of Kansas
Institute for Research in Learning Disabilities (KU-IRLD)
learning strategies model was designed to meet similar needs
of high school students with learning disabilities. The
KU-IRLD strategies have been shown to be an effective and
efficient intervention for high school students with learning
disabilities and may be appropriate for use at the
postsecondary level.
Statement of the Problem
This study was designed to investigate the effectiveness
of learning strategies as an intervention for college
students with learning disabilities. Specifically, the
KU-IRLD sentence writing strategy (PENS) was used with
college students who are experiencing problems with written
expression. The experimental questions were as follows:
1. Is the KU-IRLD sentence writing strategy (PENS) an
effective intervention for college students with learning
disabilities who are experiencing problems with written
expression?
2. Is the KU-IRLD sentence writing strategy (PENS) a
more effective intervention for college students with
learning disabilities who are experiencing problems with
5
written expression than currently available options, such as
enrollment in Gordon Rule courses and writing labs?
The study includes problems for investigation that are
important for several reasons. First, the study focuses on
evaluating the effectiveness of the KU-IRLD learning
strategies for college students with learning disabilities.
Second, the study features a comparison of alternatives that
are currently available in Florida for improving the written
expression skills of college students with learning
disabilities. Included in the comparison are the KU-IRLD
sentence writing strategy (PENS), a developmental writing
course, and Gordon Rule courses (content courses in which
writing is stressed). Third, the study adds to the limited
research data on appropriate interventions for college
students with learning disabilities.
Rationale
The current state of affairs in service delivery for
students with learning disabilities at the postsecondary
level represents a sincere effort to meet student needs, but
is often not grounded in theory or supported by efficacy or
evaluation data (McGuire, Norlander, & Shaw, 1990). Many
colleges and universities offer auxiliary aids and services,
which are available to all students or to all students with
disabilities, but this generic approach to service delivery
may not be appropriate for many students with learning
disabilities.
Some postsecondary settings provide comprehensive,
diagnostically based programs with trained specialists, while
officials at other institutions address the needs of students
with learning disabilities through remedial and developmental
approaches, peer-tutoring models, or content tutoring
(McGuire, Norlander, & Shaw, 1990).
Despite the diversity of service delivery options
available at the postsecondary level and the documented need
for appropriate services, there has been little systematic
research regarding the effectiveness of various interventions
for college students with learning disabilities. McGuire,
Norlander, and Shaw (1990) propose that additional efficacy
studies are necessary to make informed decisions as to the
effectiveness of various interventions. Empirical research
must be employed to increase the existing database related to
effective and efficient interventions for college students
with learning disabilities. Information derived from such
systematic examinations can improve the support services
available to college students with learning disabilities.
Improved interventions will increase the likelihood that
college students with learning disabilities will experience
success in postsecondary institutions.
Additionally, college students with learning
disabilities display a variety of deficits in written
expression (Fourqurean & LaCourt, 1990). Punctuation,
capitalization, and basic sentence structure continue to be a
problem for students with learning disabilities in
postsecondary settings (Hughes & Smith, 1990). Vogel and
Konrad (1988) found that college students with learning
disabilities scored significantly lower than students without
disabilities on mechanics (spelling and punctuation),
spelling, subject/predicate/number agreement, percentage of
complete sentences, and percentage of complicated sentences.
Cowen (1988) surveyed college students with learning
disabilities and found that 76% reported problems with
spelling, punctuation, and grammar. Research conducted with
students with learning disabilities underscores the need for
support services to provide interventions in written
expression.
Definition of Terms
The following section presents the terms and their
definitions as used in this study. Terms selected for
inclusion are critical to understanding implementation
procedures and observed results.
Learning disability (LD) refers to a disorder in one or
more of the basic psychological processes involved in
understanding or using language, spoken or written, which may
manifest itself in an imperfect ability to listen, think,
speak, write, spell, read, or do mathematical calculations.
The term includes such conditions as perceptual handicaps,
brain injury, minimal brain dysfunction, dyslexia, and
developmental aphasia. The term does not include children
who have learning problems which are primarily the result of
visual, hearing, or motor handicaps; mental retardation;
emotional disturbances; or environmental, cultural, or
economic disadvantages (United States Office of Education,
1987, p. 65083, as cited in Mercer, 1987).
Learning strategies are techniques, principles, or rules
that enable a person to learn, solve problems, and complete
tasks independently (Alley & Deshler, 1979).
The University of Kansas Institute for Research in
Learning Disabilities (KU-IRLD) was initially funded in 1977
with a primary mission of developing and validating an
intervention model for adolescents with learning disabilities
that meets both effectiveness and feasibility criteria
(Deshler & Schumaker, 1986).
Written expression "can be defined as a visual
representation of thoughts, feelings, and ideas using symbols
of the writer's language system for the purpose of
communication or recording" (Poteet, 1980, p.88).
The PENS sentence writing strategy is part of a writing
strategy curriculum developed by the researchers at KU-IRLD,
including Sentence Writing, Paragraph Writing, Error-
Monitoring, and Theme Writing strategies (Schumaker &
Sheldon, 1985). The mnemonic PENS (see Table 1) was
developed to increase the likelihood that subjects would
remember the strategy components in actual writing situations
(Pressley, Levin, & Delaney, 1982). PENS does not cover
every sentence variation that might arise in a developmental
language course, but students who master all of the sentence
Table 1
The PENS Strategy
E Pick a formula
E Explore words to fit the formula
N Note the words
S Subject-verb identification
Look for the action or state of being words) to
find the verb
Ask the "Who or what questions" to find the
subject
types included in the strategy will be able to write
sentences that fit more than 14 different sentence formulas.
Sentence formulas for writing simple, compound, complex,
and compound-complex sentences (see Table 2) are taught in
PENS. A simple sentence is a sentence that is made up of one
independent clause. An independent clause is a group of
words with a subject and verb that can stand alone. A
compound sentence is a sentence with two or more independent
clauses joined by appropriate punctuation and a conjunction.
A complex sentence is a sentence that contains one
independent clause and one or more dependent clauses
joined by the appropriate punctuation and conjunctions. A
dependent clause is a group of words with a subject and verb
that cannot stand alone. A compound-complex sentence is a
combination of compound and complex sentences and consists of
two or more independent clauses and at least one dependent
clause joined by the appropriate punctuation and
conjunctions.
Table 2
The PENS Strategy Sentence Formulas
Simple Sentences
I=S V
I=SS V
I=S VV
I=SS VV
Compound Sentences
I,cI
I;I
Complex Sentences
D,I
ID
Compound-Complex Sentences
D,I,cI
ID,cI
I,cID
D,I;I
ID;I
I;ID
I=independent clause
S=subject
V=verb
c=conjunction
D=dependent clause
Fluency is defined as "the degree to which the student
becomes more proficient at writing down words and sentences
into compositions of gradually increasing length" (Isaacson,
1985, p. 409). Fluency is measured by finding the average
sentence length (Tindal & Marston, 1990).
Syntactic maturity is defined as "the degree to which a
student uses expanded, more complex sentences" (Isaacson,
1985, p. 410). One way to measure syntactic maturity is to
count the number of sentences that fall into several
categories (nonsentences, simple sentences, compound
sentences, and complex sentences) and compute the percentage
of each type of sentence in the writing sample (Tindal &
Marston, 1990).
Vocabulary focuses on the uniqueness or maturity of
words that are used in a composition (Tindal & Marston,
1990). Scoring is accomplished by counting the number of
unique or mature (large) words used in the writing sample.
Mechanics incorporates spelling, punctuation, usage, and
grammar skills and can be measured using correct word
sequences (CWS), a multiple-factor measure of written
expression (Tindal & Marston, 1990). Correct word sequences
is defined as "two, adjacent, correctly spelled words that
are acceptable within the context of the phrase to a native
English speaker" (Videen, Deno, & Marston, 1982, p. 11).
When punctuation is missing, words are misspelled, or usage
is incorrect, the word sequence is scored as incorrect.
Therefore, using CWS provides a measure of the mechanical
conventions that make a composition grammatically correct and
presentable to others.
Composition organization refers to the structure and
organization within the writing sample and focuses on (a) the
use of topic sentences, supporting sentences, and concluding
statements in paragraphs and (b) the flow of ideas from
paragraph to paragraph or sentence to sentence.
Delimitations of the Study
The scope of this study was delimited in three ways.
First, this study was restricted geographically to Alachua
County within the state of Florida and to students at the
University of Florida and Santa Fe Community College.
Second, only college students with learning disabilities who
are experiencing problems with written expression were
included. Third, only students who are registered with the
colleges' Office for Disabled Student Services were
considered for the study. These students met the college
criteria for being identified as learning disabled. No
additional educational or psychological testing was completed
as part of this study.
Limitations of the Study
Since this study included only college students with
learning disabilities who are experiencing problems with
written expression, the findings should not be generalized to
other college students with or without disabilities.
Moreover, the results of this study cannot be generalized to
skills other than written expression without replication.
Caution should be exercised in extrapolating results of this
study to students outside Alachua County and at schools other
than University of Florida and Santa Fe Community College.
Summary and Overview of Remaining Chapters
Students with learning disabilities are enrolling in
2-year and 4-year colleges in increasing numbers. However,
characteristics of students with learning disabilities, poor
preparation for college in high school and setting demands in
higher education, usually combine to make college a
frustrating experience. To improve the success rate of
college students with learning disabilities in postsecondary
settings, effective and efficient interventions are needed.
The purpose of this study was to investigate the efficacy of
the sentence writing strategy (PENS) on the written
expression skills of college students with learning
disabilities. It is the intent of the researcher conducting
this study to contribute to the research findings on strategy
instruction and effective interventions for college students
with learning disabilities. The results of this study have
direct and immediate implications for college support service
providers.
A review of literature relevant to this study is
presented in Chapter 2. Methodology used for implementation
is discussed in Chapter 3. The results and their
implications is reported in Chapters 4 and 5.
CHAPTER 2
REVIEW OF RELATED LITERATURE
In Chapter 2, a summary and analysis of the professional
literature involving interventions for college students with
learning disabilities and writing strategy instruction are
presented. The chapter is divided into four major sections.
First, selection criteria for the literature that was
reviewed are presented. Second, the historical development
and analysis of research on interventions for college
students with learning disabilities are presented. Third,
the historical development and analysis of research on
teaching writing using cognitive strategy instruction were
presented. Fourth, the historical development and analysis
of research on KU-IRLD learning strategy were presented. The
chapter concludes with a summary and implications of previous
research as it related to the present study.
Selection of Relevant Literature
An initial step in the review of the literature was that
of determining the criteria for the inclusion of references.
All relevant studies completed in the last ten years (1981-
1991) were examined. In addition, any notable research cited
in the literature earlier than the 10-year period was
considered.
Professional literature concerning interventions for
students with learning disabilities and writing strategy
instruction was required to meet the following criteria to be
included in the review:
1. The subjects and the settings in which the
experimentation took place had to be thoroughly
described.
2. The treatment conditions and experimental
procedures were detailed enough to permit
replication.
3. The experimental design and data analysis
procedures were presented without significant
losses of information.
4. The interpretations of the experimenter had to be
consistent with the results displayed.
To complete an exhaustive review of the literature
relating to interventions for college students with learning
disabilities, KU-IRLD learning strategies, and writing
strategy instruction, the reference sources used included
Dissertation Abstracts International, Educational Resources
Information Clearinghouse (ERIC), Psychological Abstracts,
Current Index to Journals in Education (CIJE), and Higher
Education Abstracts. References initially selected were
located through the library at the University of Florida,
through the interlibrary loan system, or through other
professionals in the field. Descriptors used in this
literature search included learning strategies, learning
disabilities, college students, interventions, support
services, postsecondary education, writing instruction,
writing strategies, writing difficulties, written expression
problems, learning processes, metacognition, and cognitive
strategies.
The references that were selected were critically
reviewed and those that described empirical investigations
were chosen based on the investigator's judgment that the
references presented a clear description of subject
selection, methodology, and results. Professional literature
other than empirical investigations were also included if, in
the author's judgment, the information that was included
provided a valuable contribution to the knowledge base about
or an understanding of interventions for college students
with disabilities or of writing strategy instruction.
Interventions for College Students
with Learning Disabilities
History of Postsecondary Programs
Special programs and services to meet the needs of
college and university students with learning disabilities
are a relatively new phenomenon. Efforts to provide services
were found in the early 1970s when a few colleges attempted
to develop and implement the necessary conditions for
students with learning disabilities (Strichart & Mangrum,
1986). Cordoni (1982) reports that small private colleges
were the pioneers in providing services for college students
with learning disabilities. The programs began at small
private institutions were of questionable value since they
were (a) mainly tutorial in nature, (b) based on limited
assessment of the needs of students with learning
disabilities, and (c) influenced by the theoretical
disposition or practical experience of the person who
conceived or developed them. As Ostertag, Baker, Howard, and
Best (1982) point out, the continued participation of
students with learning disabilities in postsecondary
education requires a re-evaluation of the types of
educational programs and services offered to these students
at the postsecondary level.
The number of students with learning disabilities
applying to and attending postsecondary institutions is
greater than ever before (Scott, 1990), but the availability
of support programs for students with learning disabilities
does not keep up with this growth in enrollment. However, in
recent years the number of college support programs is
increasing for several reasons (Strichart & Mangrum, 1986).
First, the initial focus of learning disabilities programs
was on the needs of young children. As the population of
individuals with learning disabilities aged, it became
apparent that learning disabilities could not be cured, but
persisted into adulthood. Consequently, programs for
elementary students with learning disabilities were extended
to middle schools and high schools. The emergence of college
programs for students with learning disabilities is the next
logical step.
Second, many students with learning disabilities are
interested in attending college. White, Alley, Deshler,
Schumaker, Warner, and Clark (1982) reported that 67% of
young adults diagnosed with learning disabilities by their
schools at some point in their elementary or secondary
programming, had plans for postsecondary education.
Membership of the Vocational Committee of the Association for
Children and Adults with Learning Disabilities (Vocational
Committee, 1982) reported that 14% of the adults (responding
to their survey) had tried college and dropped out, 32% were
currently attending college, and 9% had completed bachelor
degrees. Additionally, Mitaug, Horiuchi, and Fanning (1985)
reported that 55-60% of the students with learning
disabilities surveyed planned to pursue postsecondary
education or training.
Third, increased pressure exerted by parents, adult
groups, and concerned professionals has also been a factor in
the emergence of college programs for students with learning
disabilities. Breakthroughs in services for individuals with
disabilities has often been stimulated by the activities of
professional and nonprofessional advocates. Fourth, the
economic reality of declining student enrollments in many
colleges throughout the nation has private and public
institutions searching for ways to increase student
enrollment. Since students with learning disabilities have
been portrayed as one of the largest under-educated group of
persons in our nation (Moss & Fox, 1980), more colleges are
providing programs to attract these students.
Fifth, and perhaps most important, federal legislation
has required that all students with disabilities be given an
opportunity to receive an appropriate education. The
Education for All Handicapped Children Act of 1975 (P.L.
94-142) guaranteed the educational rights of students with
disabilities ages 3-21 through secondary school level.
Section 504 of the Rehabilitation Act of 1973 and the
Americans with Disabilities Act of 1990 address the
postsecondary rights of individuals with disabilities and
prohibit discrimination in admissions, recruitment, and
treatment after admissions (reasonable accommodations).
College has become a goal for many students with
learning disabilities. The issue currently facing the fields
of special education and higher education is how to make this
goal attainable. An important factor in the success of any
college student with learning disabilities is the
availability of appropriate programs and support services.
Program components can include (a) counseling services
(academic, career or vocational), (b) instructional
accommodations (services provided by the college or
adaptations provided by the faculty), and (c) administrative
accommodations (alternative admission criteria and special
remedial courses) (Nelson & Lignugaris-Kraft, 1989). The
following section contains a review of descriptions and
surveys of programs in postsecondary settings for students
with learning disabilities (see Appendix A).
Description of Postsecondary Programs
The membership of the National Joint Committee on
Learning Disabilities (NJCLD) (1986) developed a list of
recommendations for federal, state, and local agencies, as
well as postsecondary and vocational training programs that
were consistent with regulations implementing Section 504.
The membership of the NJCLD suggested that postsecondary
institutions and agencies continue to develop and implement
effective programs that will allow adults with learning
disabilities an opportunity to attain career goals. If
adults with learning disabilities are to gain access to and
profit from postsecondary programs, innovative planning and
cooperation will be necessary to develop programs and
services. The membership of the NJCLD recommended that
persons responsible for postsecondary programs for students
with learning disabilities should establish an advisory
council that will develop, implement, and monitor necessary
procedures and services. Recommended procedures and services
include (a) admissions criteria and procedures;
(b) assessment procedures for determining an individual's
status and needs; (c) guidelines for course selection and
sequences; (d) guidelines on alternative methods for
evaluating a student's learning of course content material;
(e) support systems that will assist with the development of
study skills, reasoning abilities, and decision-making
I
skills, as well as enhancement of listening, speaking,
reading, writing, and mathematical abilities;
(f) modification of instructional methods as indicated;
(g) opportunity for individual and group psychological
assistance; (h) establishment of peer support groups; and,
(i) opportunity for career counseling. Most colleges and
universities offer programs for students with learning
disabilities (Barbaro, 1982; Bireley & Manley, 1980; Gajar,
Murphy, & Hunt, 1982) that lack important components
recommended by the membership of the NJCLD (1986).
Descriptions of single programs. The Wright State
University program for students with learning disabilities
was described by Bireley and Manley (1980). Three students
with learning disabilities participating in the program were
interviewed and followed throughout their college program.
Services offered to students at Wright State University
included (a) counseling (academic and personal) and
(b) instructional accommodations (course tutors, tape-
recorded textbooks, and proctors for special examination
administration). Of the three students participating in the
study, one graduated from Wright State University, one
transferred to another school, and one failed to fulfill the
terms of the contract established with the program
coordinator and was dropped from the program. A significant
weakness of this study was the small number of subjects. The
graduation rate of the program was poor (16%-33%) calling
into question the effectiveness and appropriateness of the
procedures and services offered.
In another study, Barbaro (1982) described the program
for college students with learning disabilities at Adelphi
University. Twenty-two students with learning disabilities
who participated in a 5-week diagnostic program at Adelphi
University were surveyed. Nineteen of the students were
enrolled at Adelphi in the fall. Program components provided
to the students included (a) counseling (diagnostic
assessment, academic, and personnel counseling),
(b) instructional accommodations (full-time tutors), and
(c) administrative accommodations (special admission
procedures and study skills classes funded by a tuition
surcharge). At the end of the first year, 84% of the
students were still enrolled. By the end of the second year,
78% remained. Most students' grade point averages fell
between 2.0 and 3.2. The strength of this study is that it
provided a 2-year evaluation of the program services for
college students with learning disabilities.
In a similar study, the Pennsylvania State University
program for students with learning disabilities was described
by Gajar, Murphy, and Hunt (1982). Twelve students
participating in the Pennsylvania State University program
were surveyed. Program components included (a) counseling
(diagnostic assessment), (b) instructional accommodations
(taped textbooks, typing lessons, and extended examination
periods), and (c) administrative accommodations (study skills
workshop and alternative classes for foreign language
requirements). During the study, two students completed
their degrees, nine students received passing grades, and one
student was not achieving passing grades in all classes in
which the student was enrolled.
Data reported in these studies allow for the conclusion
that services provided to college students with learning
disabilities in the late 1970s and early 1980, generally
included diagnostic assessments, counseling, tutoring, taped
textbooks, extended examination time, modification of college
requirements, and study skills courses. Even though
descriptions of single programs are useful in identifying the
program and service options, a broader picture of program
offerings is important.
National surveys of programs. Four state and national
studies have gathered data from a wide array of service
providers (Blalock & Dixon, 1982; Mangrum & Strichart, 1983a,
1983b; Ostertag, Baker, Howard, & Best, 1982; Parks,
Antonoff, Drake, Skiba, & Soberman, 1987). Data reported in
these studies allow for the conclusion that there is wide
variability across the nation in the types of procedures and
services available to college students with learning
disabilities.
The service providers for students with disabilities in
106 California community colleges were surveyed by Ostertag,
Baker, Howard, and Best (1982). The respondents included
coordinators of services for disabled students (41% of the
programs), instructors in programs for learning disabled
students (27% of the programs), and psychologists,
counselors, or other faculty (26% of the programs). Services
provided by most of the California community colleges
included (a) counseling services (diagnostic assessment;
academic, personal, and career counseling), (b) instructional
accommodations (course tutors, notetakers, textbook readers,
and various classroom accommodations), and (c) administrative
accommodations (extended time to complete course or
graduation requirements, registration services, course
substitutions, and reduced class load). The researchers
indicated that reading, writing, spelling, mathematical
computations, and oral communication were the most difficult
academic skills for college students with learning
disabilities in the California community college system. In
addition, it was reported that no provisions were made by the
California community college system to remediate the various
academic skill weaknesses experienced by the students with
learning disabilities.
A national telephone survey of program administrators
for college programs for students with learning disabilities
was conducted by Blalock and Dixon (1982). Program
administrators were identified in the American Council on
Education bulletin and other directories. The services
provided in the colleges and universities represented by the
respondents included (a) counseling (diagnostic assessment
and personal counseling), (b) instructional accommodations
(notetakers, typing service, course tutors, taped textbooks,
and remediation of reading, spelling, and writing skills),
and (c) administrative accommodations (classes specifically
designed for students with learning disabilities, study
skills classes, waiver of selected program requirements, and
allowed reductions in course load). The respondents
recommended development of assessment procedures for adults
suspected of having a learning disability, but emphasized
that such assessment procedures may be less important than
identifying and developing effective interventions.
Mangrum and Strichart (1983a, 1983b) also surveyed
directors of programs for college students with learning
disabilities and reported results similar to Blalock and
Dixon (1982). The program directors were identified through
a number of national college directories. Services and
program components provided by the respondents to Mangrum and
Strichart's survey included (a) counseling (diagnostic
assessment; academic and personal counseling),
(b) instructional accommodations (peer tutors from courses,
notetakers, tape-recorded textbooks, alternative test
procedures, additional time to complete courses, and
permission to tape faculty lectures), and (c) administrative
accommodations (special admissions procedures; special course
offerings in study skills, learning strategies, time
management, and stress management; late withdrawal from
courses). Data provided by the respondents in the two
national surveys allow for the conclusion that many of the
services and procedures recommended by the membership of the
NJCLD (1986) are available in some programs nationwide, but
not necessarily in any specific program (Barbaro, 1982;
Bireley & Manley, 1980; Gajar, Murphy, & Hunt, 1982).
Parks, Antonoff, Drake, Skiba, and Soberman (1987)
surveyed 223 graduate and professional schools. Surveys were
sent to 132 graduate schools, 46 law schools, 28 dental
schools, and 17 medical schools with a return rate of 32%.
Counseling services were provided by a majority of the
respondents and included diagnostic assessment and
counseling. Instructional support services offered by at
least half of the schools surveyed included course tutors,
instruction in test-taking skills, special instructions on
library use, extra time on tests, and oral instead of written
examinations. Less that half of the schools offered audio-
video resources, word processing, use of microcomputers, help
in preparation for tests, use of calculators, taped responses
to assignments and tests, and instruction in reading,
writing, spelling, and listening skills. Very few schools
offered administrative accommodations, such as special
admissions criteria (3%-18% of the respondents), except that
over half of the schools allowed for extra time to complete
the program of study. The data from this study may not be
representative of undergraduate or community college services
offered to students with learning disabilities because of the
difference in program goals between the two types of
institutions. Most graduate and professional schools,
however, are part of a postsecondary institution with an
undergraduate component and may share the same program
procedures and services. An additional concern in
interpreting the results of the study is the low response
rate (32%).
It is notable that most of the postsecondary
institutions surveyed or described provided (a) counseling
and diagnostic assessment, (b) basic instructional
accommodations, such as course tutors, notetakers, and taped
textbooks, and (c) administrative accommodations, such as
extended time to complete degrees, special admissions
procedures, and reduced class loads. However, few colleges
and universities reported providing courses specifically
designed to remediate academic weaknesses characteristic of
students with learning disabilities (Blalock & Dixon, 1982;
Ostertag et al., 1982; Parks et al., 1987). The emphasis on
instructional and academic accommodations in lieu of remedial
courses or strategy instruction for students with learning
disabilities requires that faculty and administrators be
willing to make the recommended modifications. Without
faculty support in the classroom, the effectiveness of
programs for college students with learning disabilities will
be lessened (Schumaker & Deshler, 1984).
Faculty attitudes. Two studies in which faculty
attitudes and willingness to provide students different
accommodations were assessed (Matthews, Anderson, & Skolnick,
1987; Nelson, Dodd, & Smith, 1990) and allowed for the
determination that most faculty members were willing to make
some basic modifications. The type of accommodations
allowed, however, varied from department to department and
professor to professor. Matthews, Anderson, and Skolnick
(1987) surveyed all faculty members (100) at a small
northeastern public university (65% return rate) and found
that most faculty would allow accommodations such as
extension of deadlines, alternative assignments, extra time
to complete tests, and oral instead of written tests or
assignment presentation. The faculty members generally did
not allow substitution of alternative courses for required
courses and were not willing to provide copies of lecture
notes. The researchers recommended (a) inservice training
for faculty and admissions staff concerning students with
learning disabilities and (b) the development of advisory
committees.
Nelson, Dodd, and Smith (1990) also investigated faculty
attitudes toward providing accommodations to students with
learning disabilities in college and university settings and
reported results similar to Matthews et al. (1987). The
researchers surveyed 107 faculty in colleges of education,
business, and arts and sciences at a northwestern college
(76% response rate). The respondents reported that they
would allow instructional accommodations including tape
recorders, copies of lecture notes, proofreaders for written
assignments, assignment modifications, and testing
modifications. The faculty in the college of education
reported more willingness to provide support services than
the other faculty which may reflect a greater knowledge about
student variance, special education students, and/or
effective teaching practices. The researchers recommended
future research concerning effective and efficient program
components.
The program descriptions and survey findings discussed
above provide for a review of faculty attitudes, procedures,
and services available in schools across the nation.
However, research comparing different postsecondary programs
or evaluating program components is lacking. Accurate
descriptions of the current state of postsecondary program
options are important. The identification of effective and
efficient program components is a priority due to funding
shortages facing many institutions and the variety of
services needed by students with learning disabilities
currently enrolled in colleges and universities.
Learning Strategies
Background on Strategy Instruction
From the late 1940s to the mid 1970s, behaviorism had a
major impact on the advancement of psychological and
educational practices. Beginning with Thorndike, the idea
that behavior was controlled by its consequences had a
powerful impact on American learning theories (Swenson,
1980). The premise was that most human behavior was acquired
or modified through learning procedures or environmental
influences. Theories developed by researchers such as
Pavlov, Hull, Spence, and Skinner were based on data derived
from extensive research efforts with animal subjects.
Complex human and animal behaviors were seen as determined by
environmental contingencies and lawful relationships between
stimuli and responses. All of these theorists attempted to
mold psychology on the model of natural and physical sciences
and strongly opposed explaining behavior in terms of mental
variables or processes. Their theories were most successful
in predicting simple behaviors and were less successful when
applied to complex behaviors (Swenson, 1980).
Behaviorism made a number of assumptions about the way
in which learning occurs (Ashman & Conway, 1989):
1. Behaviors can be defined in instructional terms.
2. Behaviors can be learned and modified.
3. Behaviors are environmentally determined and
reinforced.
The behavioral framework had a profound effect on the
organization of classroom teaching strategies. Highly task-
analyzed curricula were developed to ensure the success of
instruction (Koorland, 1986). Such curricula included
discrete and systematic steps for instruction that, although
designed to impart specific knowledge, often lead to teaching
many discrete splinter skills without the learner
understanding the overall learning process (Ashman & Conway,
1989). The focus of these curricula was on the product and
not the process of learning.
31
There is no doubt that behaviorism has made a
significant contribution to education, especially in the area
of behavior management, but it has become increasingly clear
that organisms do not simply learn by associations from blind
trial and error (Schwartz, 1989). Recent years have seen
increased attention on the role of the learner as an active
participant in the teaching-learning process (Weinstein &
Mayer, 1986). In particular, this focus suggests that the
success of teaching is partly dependent upon what the learner
knows, such as the learner's prior knowledge, and what the
learner thinks about during the learning experience, such as
the learner's active cognitive processing. This change in
thinking about learning had several important implications
for education. Teachers entering the classroom now have two
distinctly different kinds of goals for their students:
(a) goals concerning the product of learning (what students
should know or be able to do as a result of learning) and
(b) goals concerning the process of learning (techniques and
strategies students can use to accomplish learning)
(Weinstein & Mayer, 1986).
This interest in learning strategies is a natural
outgrowth of the change in orientation from behaviorist
theories to cognitive theories of learning. While
behaviorism was built on the stimulus-response theory,
cognitive psychology has filled in the "-" separating
"stimulus-response" with the activities of learning (mental
structures and processes). The cognitive approach to
learning attempts to understand how information is processed
and stored in memory. This cognitive approach has changed
our conception of the teaching-learning process in several
ways (Weinstein & Mayer, 1986). First, learning is viewed as
an active process that occurs within the learner and which
can be influenced by the learner. Second, the outcome of
learning depends jointly on what information is presented and
on how the learner processes that information. Therefore,
there are two types of activities that influence the learning
process, (a) teaching strategies and (b) learning strategies.
Weinstein and Mayer (1986) present a framework for
describing the teaching-learning process:
1. Teacher characteristics--knowledge of how to teach
and content area material required for the teaching
strategy selected.
2. Teaching strategies--teacher's performance during
teaching, including what is taught, how it is
taught, and when it is taught.
3. Learner characteristics--learner's knowledge of
facts, procedures, and strategies required for the
learning strategy selected.
4. Learning strategies--behaviors that the learner
engages in during learning intended to influence
affective and cognitive processing during encoding.
5. Encoding process--internal cognitive processes
during learning, such as how the learner selects,
organizes, and integrates new information.
6. Learning outcome--newly acquired knowledge that
depends on both teaching and learning strategies.
7. Performance--behavior on tests of retention and
transfer. (p. 316)
As seen above, instruction in learning strategies can effect
learners' performance by making specific strategies and
methods available to the learner which can improve the
encoding process and learning outcome. During the 1980s,
strategy instruction became a major focus of educational
research with students with learning disabilities (Graham,
Harris, MacArthur, & Schwartz, 1991). The main premise
underlying strategy instruction was that students could
profit from being taught more effective strategies than they
would discover on their own. The emphasis was placed on
developing effective and efficient learning strategies and
methods of strategy instruction.
Palinscar, David, Wisen, and Stevens (1991) describe six
prominent cognitive strategy instruction models which include
Cognitive Behavior Management, Direct Instruction, Direct
Explanation, Informed Strategies for Learning, Reciprocal
Teaching, and Strategy Intervention Model, and review
research concerning each approach. Cognitive Behavior
Management (CBM) was developed by Dansereau (1978).
Cognitive Behavior Management focuses on teaching students to
regulate their own performance by using an internalized set
of monitoring statements before, during, and after a learning
task. These statements typically focus on problem
definition, attention focusing, self-reinforcing, and self-
coping. The CBM monitoring statements are taught in a manner
similar to Direct Instruction and have been effective with
above average students. However, research has not addressed
flexibility of strategy use, generalized use of strategies,
and changes in attitudes or awareness following instruction
(Palinscar, David, Wisen, & Stevens, 1991).
Direct Instruction (DI), developed by Engelmann and his
associates at the University of Oregon, teaches content
knowledge and strategies in a sequential fashion with
students practicing material as the teacher uses explicit
instruction, development of mastery, elaborated feedback,
gradual fading of prompts, adequate practice, and cumulative
review (Gersten, Carnine, & Woodward, 1987). White (1988)
completed a meta-analysis of research studies on DI in
special education assessing target skills including reading,
math, language, spelling, social, writing, and health. While
none of the outcome measures in the 25 studies reviewed by
White significantly favored the comparison group, 53% of the
outcome measures significantly favored DI. Even though some
educators contend that DI teaches lower-order cognitive
skills (such as word-attack) at the expense of higher-order
skills (such as comprehension) (Palinscar, David, Wisen, &
Stevens, 1991), a follow-up analysis of the study-weighted
mean effect sizes on subdivided reading measures
(comprehension, word-attack, and total reading measures) did
not support such arguments (White, 1988). Research findings
allows for the conclusion that DI is an effective means of
teaching strategic and content knowledge.
The Direct Explanation (DE) strategy model, developed by
Duffy et al. (1987), is unlike the previous models in several
ways. Using DE, the instructor provides declarative (what
the strategy is), procedural (how the strategy is used), and
conditional (when the strategy is used) knowledge. In an
effort to teach academic skills as strategies, the instructor
uses verbal modeling to teach effective strategy use. The
steps used in DE correspond to modeling, guided practice, and
independent practice used in the University of Kansas
approach discussed below. Researchers have shown DE to
improve procedural and conditional knowledge in reading
strategies, but results concerning achievement measures are
mixed (Palinscar, David, Wisen, & Stevens, 1991).
Paris (1986) developed the Informed Strategies for
Learning (ISL) model, a curricular approach to strategy
instruction in reading. Informed Strategies Learning
consists of 20 modules addressing four comprehension
processes: (a) planning for reading, (b) identifying
meaning, (c) reasoning while reading, and (d) monitoring
comprehension. This model uses group dialog to emphasize
personal aspects of strategy use, thereby increasing student
awareness of strategies. Palinscar, David, Wisen, and
Stevens (1991) report that results of research with ISL on
reading comprehension have been less than encouraging.
The Reciprocal Teaching (RT) model was developed by
Brown and Palinscar (1989) and features instruction in four
strategies taught as a set of complimentary activities to be
used flexibly as the text demands and learner needs change.
Unlike other approaches, RT emphasizes collaboration between
the teacher and students rather than direct instruction to
teach strategy use. This model incorporates a questioning
technique that is faded from teacher control to student
control and is very successful in content area classes.
Reciprocal Teaching has significantly increased students'
ability to use targeted strategies and attain higher scores
on comprehension measures. These gains also have been shown
to maintain over time and generalize beyond the instructional
setting.
Deshler and his associates at the University of Kansas
have developed the Strategy Intervention Model (SIM) for use
with learning disabled and low achieving students (Deshler &
Lenz, 1989). The strategies, developed as a part of SIM,
have been proven effective in improving strategy use and
content learning. The SIM approach to learning strategies
instruction has been described as a comprehensive and
realistic model (Pressley, Symons, Snyder, & Cariglia-Bull,
1989). Like DI and CBM, the SIM strategies are taught as a
series of steps. Unlike the other strategies, the SIM
incorporates two steps, the acquisition phase (what the
strategy is, and how and when to use it) and the
generalization phase (application of the strategy in the
regular education setting). Additionally, strategy
instruction is preceded by assessment regarding the student's
current strategy use in comparable situations. The SIM
strategies are taught using modeling and verbal rehearsal,
guided and independent practice with elaborated feedback,
materials controlled for difficulty, and criteria to measure
mastery. The main strengths of the SIM approach and
curriculum are the (a) successful generalization of the
strategies taught, (b) instructor's ability to motivate
students (goal setting activities and charted progress),
(c) resulting intensive and extensive strategy instruction,
and (d) incorporation of effective teaching practices into
the strategy instructional procedure. Research indicates
that sustained instruction is successful in improving
strategy use and content area learning.
Independent student functioning is one of the goals of
strategy instruction and a setting demand of postsecondary
institutions. In the generalization steps of the KU-IRLD
strategies, students are taught to be active agents in the
intervention process by using self-monitoring techniques with
the goal being to develop self-regulated learning and
behavior (Reid & Harris, 1989). Ghatala (1986), in studies
with elementary students, found that students who fail to
monitor the effectiveness of strategies used strategies less
often and less effectively. The general monitoring
difficulties being detected and reported in the literature
and the importance of self-regulation allow for the
conclusion that more effort should be expended in developing
and utilizing interventions that teach and encourage self-
regulated learning (Pressley, Goodchild, Fleet, Zajchowski, &
Evans, 1989). The KU-IRLD strategy model has been
extensively researched with high school students with
learning disabilities and is considered to be one of the most
highly developed strategy models available (Pressley &
Harris, 1991); however, there is little documentation in the
literature as to its effectiveness or appropriateness with
college students.
Research on Writing Strategy Instruction
In writing, strategy instruction has been used as an aid
to assist students in learning to internalize and regulate
the cognitive activities considered central to the effective
planning, production, and revision of written text (Graham,
Harris, MacArthur, & Schwartz, 1991). Writing strategies
developed and used with students with learning disabilities
have concentrated on such diverse areas as self-monitoring of
productivity, planning the text, content generation, text
production skills (such as sentence writing), and editing and
revising.
Graham and Harris (1989) conducted a study on a strategy
for planning and writing essays with three 6th-grade students
with learning disabilities receiving resource room services.
The subjects were taught a text planning strategy (TREE)
using modeling, mastery learning, and generalization
training. After training, all subjects demonstrated
substantial gains in writing performance over baseline
levels. The study was designed and data analyzed using
single subject methodology and no comparison was made with
other writing strategy approaches.
Reynolds, Hill, Swassing, and Ward (1988) evaluated the
effectiveness of (a) Evaluative and Directive Phrases (EDP),
developed by Bereiter and Scardamalia and (b) the Error-
Monitoring Strategy (COPS), developed by KU-IRLD. Reynolds
et al. compared two strategy approaches for revising written
text. Fifty-four students with learning disabilities in the
sixth, seventh, and eighth grades participated in this study.
The study was conducted in four resource rooms for an
unspecified length of time. Classes were assigned to one of
three groups, (a) control (tell to revise with no
instruction), (b) EDP, or (c) COPS. Pretest and posttest
scores were analyzed with an ANOVA (the Tukey HSD procedure
was used for post hoc comparisons). No evaluation of the
students' writing was made prior to intervention. The
researchers reported that both the EDP and COPS groups
improved in mechanics, but that none of the groups improved
in text content.
Welch and Jensen (1990) compared strategy intervention
to a more traditional grammar-based approach and addressed
some of the weaknesses of the studies discussed above.
However, the subjects were not students with learning
disabilities. The researchers identified 114 at-risk
students (failed at least two classes in the past year) from
middle to low socioeconomic levels participating in a summer
school session in a metropolitan school district. Two school
sites (n = 71 and n = 43) were chosen and each site received
a different treatment (strategy instruction and traditional
grammar-based language instruction). The strategy
instruction (P.L.E.A.S.E. strategy) was video-assisted and
focused on paragraph writing skills over a 6-week summer
school session. Data were analyzed by ANOVA and ANCOVA
procedures. Results support the conclusion that students who
received the strategy instruction became significantly more
proficient in paragraph structure and pre-writing planning
than did the students in the traditional grammar-based
language instruction.
In a more recent study, Welch (1992) investigated the
use of the P.L.E.A.S.E. paragraph writing strategy with seven
6th grade students identified as having mild learning
disabilities. This study incorporated a control group
(eleven students receiving traditional writing instruction)
at a different school and matched students in the
experimental and control groups on socioeconomic status and
student characteristics. Data were collected using a student
survey (knowledge of paragraphs and metacognitive processes
related to writing paragraphs), writing samples (pre- and
post- intervention), and an attitude measure (feelings about
ability to write paragraphs). Data analysis was completed
using ANCOVAs and ANOVAs on the surveys, writing samples, and
the attitude measure. Welch reported that the control and
experimental groups were the same in knowledge, paragraph
writing skills, and attitude towards writing on the pretest
measures. On the posttest, the P.L.E.A.S.E. strategy group
significantly improved on the posttest in all three areas,
but the control group did not. In conducting this study,
Welch has addressed most of the concerns of the previous
studies and was able to add to the research base regarding
the positive effect of strategy instruction on improving
writing skills for students with learning disabilities.
Published research on the effectiveness of the PENS
sentence writing strategy is limited. One study completed by
Schmidt (Schmidt, Deshler, Schumaker, & Alley, 1988) was
designed using single subject methodology. No comparison was
made with other writing strategy approaches. Schmidt
conducted the study with the PENS strategy using seven high
school students with learning disabilities. The strategy was
taught as per the procedures and instructions in the
Instructor's Manual. Before training in PENS, the students
averaged 70% complete sentences and 18% complicated
(compound, complex, and compound-complex sentences) sentences
in resource room written products. After instruction and
training in generalization, the students averaged 92%
complete sentences and 42% complicated sentences in resource
room written products and 92% complete and 48% complicated
sentences in regular class written products. The results
lead to the conclusion that PENS is effective in improving
sentence writing skills. However, the question still remains
unanswered: Is PENS better than other available programs?
Researchers have shown that when students with learning
disabilities are taught appropriate task and self-regulation
strategies, student performance improves (Graham & Harris,
1990). Educators and support service providers can promote
good writing by teaching students to use self-instructional
techniques that combine training in the use of composition
and self-monitoring strategies (Levy & Rosenberg, 1990).
Summary
Postsecondary programs for students with learning
disabilities have primarily been developed in the last 20
years. The recent concern about appropriate intervention
programs is due to the increasing number of students with
learning disabilities pursuing postsecondary education in
community colleges, traditional 4-year higher education
institutions, and graduate and professional schools (Nelson &
Lignugaris-Kraft, 1989). Increasing enrollment of students
with learning disabilities has been brought about by changes
in attitudes (student, parent, and teacher) and recent
legislation (Section 504 and ADA). Researchers investigating
procedures and services available to students with learning
disabilities in postsecondary settings have reported a
diversity of programming options and are recommending
research to identify effective and efficient program
components.
Strategy instruction has been identified as an effective
intervention for students with learning disabilities by
various researchers, but there is little research on the
effectiveness of using learning strategies with college
students. Problems identified with research on writing
strategy instruction include (a) small numbers of subjects,
43
(b) lack of comparison with other approaches, (c) no
replications or limited replications of research findings,
and (d) little research with college students with learning
disabilities.
CHAPTER 3
METHODOLOGY
Chapter 3 presents the methods and procedures of the
study. For the purposes of presentation, the chapter has
been divided into five sections. The sections include a
description of (a) the hypotheses, (b) the subjects and
setting, (c) the measurement procedures, (d) the reliability
of procedures, (e) the training of personnel, (f) the PENS
instructional procedures, (g) the experimental design, and
(h) the analysis of the data.
Hypotheses
This study was designed to (a) examine differences
between three different approaches for remediating written
expression deficits of college students with learning
disabilities and (b) investigate the effectiveness of PENS
for remediating written expression deficits of college
students with learning disabilities. Written expression
skills evaluated in the study include fluency, syntactic
maturity (non-sentences, simple/compound sentences and
complex sentences), vocabulary, mechanics, and composition
organization. The following null hypotheses were posited for
testing in the study:
HI: There will be no difference between the three
educational experiences (Gordon Rule courses, Developmental
Writing course, or PENS class) on posttest scores after
adjusting for pretest scores in fluency.
H2: There will be no difference between the three
educational experiences (Gordon Rule courses, Developmental
Writing course, or PENS class) on posttest scores after
adjusting for pretest scores in syntactic maturity.
H3: There will be no difference between the three
educational experiences (Gordon Rule courses, Developmental
Writing course, or PENS class) on posttest scores after
adjusting for pretest scores in vocabulary.
H4: There will be no difference between the three
educational experiences (Gordon Rule courses, Developmental
Writing course, or PENS class) on posttest scores after
adjusting for pretest scores in mechanics.
H5: There will be no difference between the three
educational experiences (Gordon Rule courses, Developmental
Writing course, or PENS class) on posttest scores after
adjusting for pretest scores in composition organization.
Subjects and Setting
The subjects of this study were college students with
learning disabilities at the University of Florida (UF) and
Santa Fe Community College (SFCC). Each subject was eligible
for placement according to the criteria required for
registration as learning disabled with the Office of Disabled
Student Services (ODSS) on the campuses of either UF or SFCC.
Further descriptions of the data are provided in Table 3
(Smith, Deshler, Hallahan, Lovitt, Robinson, Voress, &
Ysseldyke, 1984). Achievement and IQ data was based on test
scores found in psychological reports in the students'
confidential files.
Students were referred for participation in the PENS
class by personnel in ODSS at Santa Fe Community College and
the University of Florida. The criteria for referral to the
PENS class included a history of written expression problems
identified by test data, faculty referral, or self-report by
the student. Letters were sent to all students with learning
disabilities registered with ODSS who were considered
eligible for referral at the University of Florida by ODSS
during the Summer 1991 semester. The letters described the
proposed PENS class and included a proposed course syllabus
(see Appendix B). Placement into the PENS class followed the
screening procedures described below. Each student in the
PENS class signed an informed consent form (see Appendix C).
Additionally, personnel in ODSS identified students with
learning disabilities enrolled in Gordon Rule courses and the
Developmental Writing course and contacted those students by
letter (informed consent form) for possible participation in
the study. The signed consent form (see Appendix D)
described the proposed study and asked the student to
participate by providing writing samples as well as access to
descriptive data in the student's confidential folder.
Table 3
Description of Subjects
PENS Strategy
ClAss
Developmental
Writing Course
Gordon Rule
Courses
male = 10
female = 2
total = 12
mean = 24.9
range = 18-50
mean = 107.8
range = 85-104
male = 7
female = 6
total = 13
mean = 25.6
range = 20-35
mean = 86.6
range = 81-96
male = 4
female = 9
total = 13
mean = 26.8
range = 20-51
mean = 105.4
range = 86-122
Tests used = WAIS-R, WISC-R, TONI, SIT
Math:
Standard Score
mean = 92.0
range = 74-114
Percentile
mean = 36.0
range = 4-70
Grade Equivalent
mean = 9.8
range = 5.9-14.5
mean = 83.5
range = 76-91
mean = 33.0
range = 27-39
mean = 10.6
range = 9.0-12.9
mean = 102.5
range = 85-129
mean = 51.6
range = 16-97
mean = 10.9
range = 8.8-13.0
Tests used = WRAT-R, WRAT, PIAT, TABE, WJPEB, SDRT
Reading:
Standard Score
mean = 94.6
range = 83-122
Percentile
mean = 40.7
range = 13-83
Grade Equivalent
mean = 11.1
range = 7.9-15.6
mean = 69.0
range = 63-75
mean = 3.0
range = 1-5
mean = 6.3
range = 3-10
mean = 86.0
range = 80-92
mean = 10.3
range = 3-19
mean = 8.1
range = 7.3-8.9
Tests used = WRAT-R, WRAT, PIAT, TABE, WJPEB, SDRT
Spelling:
Standard Score
mean = 78.2
range = 65-99
Percentile
mean = 15.6
range = 1-32
Grade Equivalent
mean = 6.4
range = 3.9-8.3
Tests used = WRAT-R, WRAT, PIAT,
mean = 68.0
range = 63-73
mean = 2.5
range = 1-4
mean = 5.4
range = 3.0-8.2
TABE, WJPEB, SDRT
mean = 82.6
range = 74-93
mean = 18.0
range = 4-32
mean = 6.9
range = 5.0-9.9
Subjects:
Age:
Writina Course -
Measurement Procedures
Several measurement procedures were used during the
experiment. The procedures included (a) screening for
placement in the PENS class, (b) scoring the pretest and
posttest writing samples, and (c) obtaining information on
course content and instructional methods for the Gordon Rule
and writing courses.
Screening for PENS Class
Screening for participation in the PENS class was
completed using the pretest component of the PENS sentence
writing strategy provided in the PENS Instructor's Manual.
The pretest requires that students write a paragraph of at
least six sentences using a topic from a list provided. The
writing sample was evaluated on the percentage of complete
sentences, complicated sentences, and complicated sentences
punctuated correctly using the PENS scoring sheet and scoring
directions. To be included in the PENS class, the students
scored less than 100% on complete sentences, 50% on
complicated sentences, and 66% on complicated sentences
punctuated correctly in the PENS pretest writing sample.
Pretest and Posttest
The pretest and posttest evaluations were conducted as
follows. One or two writing samples were collected from each
student participating in the study at the beginning of the
semester (during the first three weeks) and counted as the
pretest. One or two additional writing samples were collected
from each student at the end of the semester (during the last
three weeks) and counted as the posttest. In an effort to
provide a more stable estimate of each student's writing
ability, the researcher attempted to collect two writing
samples collected for both the pretest and posttest measures
instead of one writing sample. These writing samples were
assignments from the respective classes. Assignments in the
PENS class were three to five pages in length, assignments
from the Gordon Rule courses were three to six pages in
length, and assignments from the Developmental Writing course
were one paragraph to one page in length.
The pretest and posttest writing samples were scored by
at least two scorers using the following procedures. Each
student was randomly assigned a number when they agreed to
participate in the study. When the writing samples were
collected, the papers were given to a typist who transcribed
each writing sample as written with no corrections. Each
writing sample was coded with the number assigned to each
subject and was scored in the areas of (a) fluency,
(b) syntactic maturity--nonsentences, simple/compound
sentences, and complex sentences (complex and compound-
complex sentences), (c) vocabulary, (d) mechanics, and
(e) composition organization. Scores for each paper were
recorded on the Scorer Sheet (see Appendix E). Scores for
the pretest and posttest writing samples were averaged to
obtain a single pretest or posttest score for each measure
using the Score Calculation Form (see Appendix F).
Writing samples were evaluated in the areas of fluency,
syntactic maturity, vocabulary, mechanics, and composition
organization. Fluency, syntactic maturity, vocabulary, and
correct word sequences are included in most objective scoring
systems of written work; composition organization is an often
used subjective assessment of written work (Tindal & Marston,
1990). Evaluation of writing samples in the areas mentioned
above will allow for a comprehensive assessment of written
expression skills (not just sentence writing skills) and will
enable the researcher to better evaluate the effectiveness of
each type of instruction in improving the written expression
abilities of college students with learning disabilities.
Fluency. Fluency has been found to be highly correlated
with performance on both norm-referenced tests of written
expression and teacher judgments of quality in written
expression in the elementary grades and is thought to
increase with educational level (Deno, Marston, & Mirkin,
1982). The reliability of this measure of written expression
skills is considered to be quite high (Marston & Deno, 1981).
The total number of words in each writing sample was
determined by completing a word count for each writing sample
using the Microsoft Word word processing program on the
Macintosh computer. Fluency is measured by determining the
average sentence length (Mercer & Mercer, 1989):
Number of total words
Number of sentences
Syntactic maturity. Syntactic maturity is the extent to
which individuals use expanded and more complex sentences
(Issacson, 1985) and has been shown to increase throughout
the school years (Hunt, 1977). Syntactic maturity can be
measured by counting the number of sentences that fall into a
given category (Marston & Tindal, 1990) including
(a) nonsentence--a grammatically incorrect sentence
(fragment, run-on, nonsensical), (b) simple sentence--one
independent clause, (c) compound sentence--has two
independent clauses, and (d) complex sentence--has an
independent clause and at least one dependent clause.
Syntactic maturity was measured by finding the percentage of
nonsentences, simple/compound sentences, and complex
sentences:
Number of nonsentences X 100
Total number of sentences
Number of simple and compound sentences X 100
Total number of sentences
Number of complex sentences X 100
Total number of sentences
Vocabulary. Vocabulary scoring focuses on the
uniqueness or maturity of words used in a composition and can
be measured by counting the number of large words contained
in the writing sample. In an examination of written
compositions, Deno, Marston, and Mirkin (1980) found moderate
correlations ranging from .47 to .60 between the number of
large words and other criterion measures (Test Of Written
Language, Developmental Sentence Scoring, and The Stanford
Language Subtest). The long word method of calculating
vocabulary is also used in the Test of Written Language-2 as
a measure of vocabulary maturity (Hammill & Larsen, 1988).
The vocabulary score is calculated by finding the number of
different words in the writing sample that have seven or more
letters (excluding -s, -ing, and -ed endings when adding them
would make the root word have seven letters if it did not
before) (Deno, Marston, & Mirkin, 1980). Scoring of
vocabulary is the percentage of large words used in the
writing sample:
Number of large words X 100
Total number of words
Correct word sequences. Spelling, punctuation, and word
usage skills can be assessed by computing the number of
correct word sequences (CWS) in a writing sample. Therefore,
using CWS provides a measure of the mechanical conventions
that makes a composition grammatically correct and
presentable to others. Correct word sequences have been
found to have a high relationship with fluency, words spelled
correctly, holistic composition ratings, and word scores
weighted according to developmental level (Tindal & Marston,
1990). The number of CWS is found by evaluating each pair of
two adjacent words (a word sequence). To be scored as a
correct word sequence, each pair of words must be spelled
correctly and must be acceptable within the context of the
phrase to a native English speaker (have appropriate
punctuation between them, demonstrate appropriate subject-
verb or noun-pronoun agreement, and be capitalized if
necessary) (Tindal & Marston, 1990). Single words beginning
and ending a sentence are counted as correct if they are
spelled correctly, capitalized correctly, and if the sentence
has the correct end punctuation. Scoring of CWS is the
percent of the CWS in the writing sample:
Number of CWS X 100
Number of Word Sequences
Organization. Composition organization focuses on the
use of topic sentences, supporting sentences, and concluding
Statements to organize or structure the writing sample. The
evaluation of composition organization will be evaluated by
assigning points to each writing sample using the following
guidelines (Tindal & Marston, 1990):
5--The writer uses paragraphs that are not only intact
in terms of sentences within them, but also in terms of
the flow from one paragraph to the next. Structure is
well developed; ideas are expressed in a systematic
fashion. The paper starts at a good point, has a sense
of movement, gets somewhere, and then stops.
4--Paragraphs are present, topical structure and
supporting detail is well developed, and concluding
sentences are used.
3--Paragraphs are present and have some structure, but
are not well connected. The organization of the paper
is standard and conventional. There is usually a one-
paragraph introduction, three main points treated in
each paragraph, and a conclusion that often seems tacked
on or forced.
2--Paragraphs are present, but lack a topic sentence;
sentences within a paragraph are marginally related to
each other.
1--The writer has no paragraph structure. The main
points are clearly separated from one another and they
come in a random order as though the writer had not
given any thought to what he or she intended to say
before starting to write.
Course Content and Instructional Methods
All undergraduate students in the State of Florida are
required to complete nine semester hours in Gordon Rule
courses. Gordon Rule courses are any approved course within
the university system requiring students to write 2500 words
during the semester. Courses are typically in the English
department, but other departments also offer Gordon Rule
courses, such as History and Psychology. The Gordon Rule
requirement was instituted in an effort to improve the
writing skills of all college graduates. Faculty teaching
students involved in this study were sent a faculty
questionnaire (see Appendix G). Twelve English, one History,
and one Psychology instructor were sent questionnaires. Four
English (33% return rate), one History (100%), and one
Psychology (100%) instructor returned their questionnaire.
Responses to the questionnaire indicated that all four of the
responding English instructors addressed planning, editing,
and organizational (theme or paragraph) skills through direct
instruction and feedback on student papers. Three of the
English instructors also addressed mechanics (spelling,
capitalization, and punctuation), correct word usage
(subject-verb or noun-pronoun agreement), and sentence
writing skills through direct instruction and feedback on
student papers. Two of the English instructors also reported
covering vocabulary through direct instruction and feedback
on student papers. The history instructor reported using
direct instruction for organizational skills, but feedback on
papers for mechanics (spelling, capitalization, and
punctuation), correct word usage (subject-verb or noun-
pronoun agreement), planning, editing, vocabulary, and
sentence writing skills. The psychology instructor reported
using feedback on papers to address mechanics (spelling,
capitalization, and punctuation), correct word usage
(subject-verb or noun-pronoun agreement), planning,
vocabulary, and organizational skills (theme or paragraph).
Students at SFCC are referred to the Developmental
Writing course because of poor scores on admissions tests or
students may elect to enroll in the Developmental Writing
course because of problems in meeting the demands of their
required courses. Students enrolled in the Developmental
Writing course attend both a large lecture class, where
material is introduced, and a small class, where lecture
material is reviewed and writing samples are evaluated. Five
Developmental Writing course instructors, who were teaching
students participating in this study, were sent a faculty
questionnaire (see Appendix G). The questionnaire was
returned by two instructors (40% return rate). Discussion
with two instructors not returning their questionnaire
revealed that they agreed with the responses on the returned
questionnaires. Responses on the questionnaires indicated
that the Developmental Writing course specifically addresses
(a) mechanics (spelling, capitalization, and punctuation),
(b) correct word usage (subject-verb or noun-pronoun
agreement), (c) planning, (d) editing, (e) organization
skills (theme or paragraph), and (f) sentence writing skills,
through both direct instruction and feedback on papers.
Reliability of Procedures
Several procedures were used during the study to
establish the reliability of measurement and instructional
procedures. Interscorer agreement and procedural reliability
checks were implemented to obtain reliability information.
Interscorer Agreement
To establish interscorer agreement, all of the pretest
and posttest writing samples were scored by at least two
trained scorers using the same scoring procedures (described
above). Pearson correlation coefficients were calculated
between the scorers on all writing samples.
Generalizability Study
Additionally, a generalizability study (G-study) was
conducted to obtain a type of reliability coefficient
appropriate when multiple raters are used in the decision
study. Information gathered from the G-study was used to
evaluate whether the number of raters were sufficient to
obtain generalizable findings. Generalizability coefficients
were calculated for each measure on groups of two, three,
and/or four scorers depending on the number of scorers rating
each measure. Results from One-way Repeated Measures ANOVAs
using a raters by subjects design were used to calculate the
generalizability coefficients using the following formula
(Crocker & Algina, 1986, p. 167):
MSp MSr
MSp + (n n')MSr/n'
Procedural Reliability
All PENS strategy instruction was conducted by the
researcher or a trained strategy instructor. To ensure
procedural reliability, two independent observers randomly
observed five instructional sessions to ascertain whether
instructional procedures used with the strategy instruction
were consistent between different groups of subjects and
different instructors. The observers had already been
trained in the KU-IRLD strategies, so they only received
training in administration of the probes.
The observers recorded teacher behaviors in the areas of
(a) following the content of the script, (b) following the
sequence of instructional procedures, and (c) using the
instructional materials appropriately. An interval recording
system was used to measure the teachers' behaviors. While
the instruction was being delivered by the teacher, the
observers listened to a cassette tape designed to beep at
30-second intervals. Each time the beep was heard, the
observer recorded either whether the instructor was engaged
in appropriate or inappropriate behaviors on the form to be
provided (see Appendix H). The tape allowed for 40
observation intervals of 30 seconds each.
Behaviors were rated as (a) appropriate,
(b) inappropriate, or (c) not applicable and were reported as
percentages using the following formulas:
Number of appropriate behaviors X 100
40
Number of inappropriate behaviors X 100
40
Number of not applicable behaviors X 100
40
Training of Personnel
Prior to beginning the study, a student assistant was
trained in the PENS instructional procedures. The two
independent observers were trained during the first month of
the study to provide procedural reliability checks.
Training of Student Assistant
The student assistant acted as an instructor for the
PENS class. Before the PENS class began, the student
assistant was trained in the PENS strategy instructional
procedures. The student assistant was introduced to the PENS
strategy, the sequence of instruction, and the procedures for
scoring the PENS writing samples. The researcher is a
certified trainer of the KU-IRLD strategies and conducted the
2-hour training session using demonstration and modeling of
the PENS procedures as described in the trainer's manuals.
Training of Observers
The two independent observers were provided with
training in using the cassette tape recorder with the audible
beeps and in recording the teacher's behavior on the interval
form. Since the two observers had already been trained in
the KU-IRLD strategies, they did not require training in the
PENS strategy procedures.
Training the Scorers for Reliability Checks
Four independent scorers were selected to score the
pretest and posttest writing samples. The researcher served
as a fifth scorer for reliability purposes only. One of the
independent scorers evaluated all of the writing samples in
each area and those scores were used in the data analysis.
All scorers were provided with a training session (up to 1
hour) on how to score the writing samples in fluency,
syntactical maturity, vocabulary, mechanics, and
organization. Scoring methods for each area as discussed in
this chapter were demonstrated, criteria for decisions were
explained, and scoring methods were practiced. The
researcher provided the training and was available for any
questions regarding the scoring procedures during the scoring
process.
PENS Instructional Procedures
The sentence writing strategy was taught in four parts,
including instruction in (a) simple, (b) compound,
(c) complex, and (d) compound-complex sentences. Each
sentence type was taught using an 8-step instructional
procedure.
Step 1--Pretest and obtain commitment to learn. Before
any instruction began, a pretest was given to obtain a
measure of each student's sentence writing skills (the
pretest component of the PENS writing strategy). After
completing the assignment, students were informed of their
strengths and weaknesses relative to their sentence writing
ability and a verbal commitment to improve their skills was
obtained. Instruction in each of the four sentence types
followed the seven-step instructional sequence described
below.
Step 2--Describe. The PENS strategy was described and
students were provided rationales showing them how they can
benefit from mastering each part of this strategy. Once the
students became motivated to learn the PENS strategy, they
set goals with regard to the pace of instruction.
Step 3--Model. In the model step, the instructor
demonstrated the application of the PENS strategy in writing
different kinds of sentences while explaining the thought
processes involved in completing the strategy, so that the
students can witness the necessary cognitive processes as
well as the overt behaviors involved in performing the
strategy.
Step 4--Verbal Rehearsal. Before students were asked to
use the strategy, they were asked to learn certain concepts
and strategy steps at an automatic level. During the verbal
rehearsal step, students were asked to verbally rehearse
definitions of important terms and strategy steps. Practice
was carried out in small groups using practice techniques
where students were asked to recite the strategy steps
rapidly. Each student was quizzed on the strategy procedures
to determine mastery, which was required before moving on to
Step 5.
Step 5--Controlled Practice and Feedback. The students
were asked to use the strategy in writing sentences using
specific lessons provided in the PENS Instructor's Manual.
Students were provided individual feedback each time they
practiced the strategy. Student progress was charted on
graphs provided in the PENS Instructor's Manual. Before
moving on to Step 6, each student continued to practice the
strategy until mastery was achieved.
Step 6--Grade-Appropriate Practice/Posttest and
Feedback. In Step 6, students practiced in the use of PENS.
Assignments were similar to those assigned in a regular
educational setting. The last practice at the grade-
appropriate level served as the posttest for the type of
sentence being mastered. The instructor faded prompts and
cues given in the earlier stages. Feedback was given after
each practice and involved asking the students to analyze the
appropriateness of their performance. Before moving on to
Step 7, each student continued to practice the strategy until
mastery is achieved.
Step 7--Obtain Commitment to Generalize. Step 7
involved a reviewing of student progress as well as obtaining
student commitment to use the PENS strategy in a variety of
settings.
Step 8--Generalization. Students (a) were made aware of
new situations and circumstances where the PENS strategy can
be used, (b) practiced using the strategy in a wide variety
of settings by completing assignments requiring the
application of the strategy in a broad array of settings, and
(c) reported strategy use periodically by completing self-
report forms.
Experimental Design
The design of the study was a nonrandom pretest-posttest
group statistical design (see Table 4) (Cook & Campbell,
1979). The study was nonrandom because the students were
enrolled in classes and could not be randomly assigned to
treatment conditions. Pretests and posttests were used to.
determine the effectiveness of the three instructional
classes available to college students with learning
disabilities. A control group was not used in the study
because most students with learning disabilities experiencing
difficulties in college are receiving some type
Table 4
Study Design
PENS class
Gordon Rule courses
Developmental
Writing course
Pre Post
X1 0 X1
X2 0 X2
X3 0 X3
X4 0 X4
X5 0 X5
X6 0 X6
X7 0 X7
X1 0 X1
X2 0 X2
X3 0 X3
X4 0 X4
X5 0 X5
X6 0 X6
X7 0 X7
X1 0 X1
X2 0 X2
X3 0 X3
X4 0 X4
X5 0 X5
X6 0 X6
X7 0 X7
(fluency)
(nonsentences)
(simple and compound
sentences)
(complex sentences)
(vocabulary)
(mechanics)
(organization)
(fluency)
(nonsentences)
(simple and compound
sentences)
(complex sentences)
(vocabulary)
(mechanics)
(organization)
(fluency)
(nonsentences)
(simple and compound
sentences)
(complex sentences)
(vocabulary)
(mechanics)
(organization)
of service and it would be inappropriate to deny services to
students with disabilities needing assistance.
A group statistical design was chosen for two reasons.
First, the exploratory research on the KU-IRLD learning
strategies using single subject methodology has lead to the
conclusion that this strategy approach is an effective
intervention for secondary students with learning
disabilities. The next logical step in researching the
KU-IRLD strategies is to use larger numbers of students,
requiring a group statistical design. Second, the
effectiveness of the KU-IRLD strategies needs to be compared
to the effectiveness of other instructional options. Such a
comparison is best accomplished using a group statistical
procedure with a large number of subjects. For example, a
study reported by Welch and Jensen (1990) used a group design
with a large number of subjects to evaluate the effect of the
three interventions.
Analysis of the Data
A MANCOVA was computed to determine whether differences
existed among the levels of the instructional experiences
(Gordon Rule courses, Developmental Writing course, or PENS
class) with respect to combinations of the measures of
written expression (fluency, syntactic maturity, vocabulary,
mechanics, and organization) (Kennedy & Bush, 1985). The .05
level of significance was used as the basis for rejection of
the null multivariate hypothesis. The independent between
group variable was the type of instructional experience and
the within group variable was the occasion. The dependent
variables were the scores on fluency, syntactic maturity
(nonsentences, simple/compound sentences, and complex
sentences), vocabulary, mechanics, and organization.
Summary
The subjects of this study were college students with
learning disabilities registered with the Office of Disabled
Student Services (ODSS) on the campuses of the University of
Florida and Santa Fe Community College, in Gainesville,
Florida. Subjects on both campuses were referred for
strategy instruction by personnel in the ODSS. Students
enrolled on both campuses in Gordon Rule courses and
Developmental Writing courses were identified for
participation in the study. The design of the study was a
nonrandom pretest-posttest design with comparison groups
consisting of a PENS class, Developmental Writing course, and
Gordon Rule courses. A MANCOVA was computed to determine
whether differences exist among the levels of the
instructional experiences.
CHAPTER 4
RESULTS
This chapter presents the data gathered to answer the
experimental questions. First, descriptive analyses of the
data are reported. Second, the inferential statistical
analyses of the data are provided. Third, the procedural and
interscorer reliabilities are reported. Fourth, results of
the generalizability study are stated. Fifth, a summary is
given.
Descriptive Analysis of the Data
During the beginning of the semester and again at the
end of the semester, subjects in the three treatments were
asked to complete writing samples within their educational
setting (PENS class, Developmental Writing course, or Gordon
Rule courses). These samples were scored on the measures of
fluency, syntactic maturity, vocabulary, mechanics, and
composition organization. Table 5 presents the means,
standard deviations, and the ranges of the scores observed
for each measure by treatment group. Results of inferential
statistical tests for differences in observed means are
presented in subsequent sections.
Fluency
On the pretest writing samples, PENS students were
writing sentences averaging 17.26 words in length,
Table 5
Means. (Standard Deviations), and fRangel of Pretest and
t-h FiTvt Writ-inn MnP1nSre5
PENS Developmental Gordon Rule
Measure Occasion Class Writing Course Courses
Fluency
pre
post
Nonsentences
pre
post
Simple/
Compound
Sentences
pre
post
Complex
Sentences
pre
post
Vocabulary
pre
post
Mechanics
pre
post
Organization pre
post
17.26
(3.96)
[10.8 24.2]
18.07
(4.51)
[13.1 27.9]
14.57
(10.68)
[0.0 33.8]
11.88
(12.05)
[0.0 36.7]
51.16
(10.94)
[36.6 72.2]
43.85
(11.58)
[26.7 64.3]
34.85
(6.97)
[26.9 47.7]
44.16
(10.17)
[25.0 58.2]
10.38
(2.71)
[4.0 13.3]
13.94
(4.83)
[9.5 27.3]
85.80
(6.32)
[75.9 94.6]
85.73
(7.41)
[71.2 94.0]
3.04
(0.75)
[1.5 4.0]
3.12
(0.60)
2.0 4.0
16.00
(7.13)
[8.7 34.4]
18.18
(5.33)
[9.5 29.9]
25.62
(28.75)
[0.0 100.0]
19.23
(16.74)
[0.0 47.7]
50.31
(22.45)
[0.0 92.9]
50.98
(18.74)
[13.4 92.9]
24.17
(16.63)
[0.0 42.9]
29.76
(16.47)
[0.0 57.1]
9.42
(3.75)
[3.7 14.9]
8.9
(3.99)
[5.0 19.8]
82.09
(8.86)
[65.3 96.5]
85.06
(8.05)
[69.4 97.4]
2.76
(0.56)
[2.0 4.0]
2.65
(0.51)
2 n 5
20.23
(3.71)
[15.0 27.8]
21.52
(4.67)
[15.0 27.8]
15.17
(12.67)
[0.0 41.7]
7.56
(8.65)
[0.0 30.0]
49.00
(13.18)
[11.1 63.6]
50.07
(19.89)
[0.0 70.3]
35.48
(13.13)
[18.2 55.7]
42.41
(15.55)
[27.9 85.7]
11.39
(3.30)
[5.0 15.8]
10.24
(2.42)
[6.7 15.7]
92.66
(4.54)
[82.9 87.8]
93.55
(4.11)
[84.5 98.6]
2.92
(0.64)
[2.0 4.0]
2.88
(0.46)
2.0 4.0
1) 1- f- f- qr -n-rsmq nn ----' ---"--
. .
P~E+fP9+ ~PA~P~ ~n
Developmental Writing course students were writing sentences
averaging 16.00 words in length, and Gordon Rule students
were writing sentences averaging 20.23 words in length. On
the posttest writing samples, PENS students were writing
sentences averaging 18.07 words in length, Developmental
Writing course students were writing sentences averaging
18.18 words in length, and Gordon Rule students were writing
sentences averaging 21.52 words in length. The subjects in
the Gordon Rule courses scored higher in the use of
vocabulary than the subjects in either of the other treatment
groups at the pretest and maintained their advantage on the
posttest; however, there appears to be little difference in
the amount of improvement between the treatment groups on the
Vocabulary measure.
Syntactic Maturity
The three measures in Syntactic Maturity (nonsentences,
simple/compound sentences, and complex sentences) are closely
related to each other, and a change in one measure requires a
change in another. These measures are percentages of the
total number of sentences in the sample and scores on the
three measures total 100% for each subject. Improvement in
Syntactic Maturity would be noted by a reduction in
nonsentences and an increase in either simple/compound
sentences, complex sentences, or both.
On the pretest writing samples, students in the PENS
class were writing an average of 14.57% nonsentences, 51.16%
simple and compound sentences, and 34.85% complex sentences.
Students in the Developmental Writing course were writing an
average of 25.62% nonsentences, 50.31% simple and compound
sentences, and 24.17% complex sentences. Students in the
Gordon Rule courses were writing an average of 15.17%
nonsentences, 49.00% simple and compound sentences, and
35.48% complex sentences. On the posttest writing samples,
students in the PENS class were writing an average of 11.88%
nonsentences, 43.85% simple and compound sentences, and
41.16% complex sentences. Students in the Developmental
Writing course were writing an average of 19.23%
nonsentences, 50.98% simple and compound sentences, and
29.76% complex sentences. Students in the Gordon Rule
courses were writing an average of 7.56% nonsentences,
50.07% simple and compound sentences, and 42.41% complex
sentences. Students in all three treatments appeared to
reduce the number of nonsentences and increase the number of
complex sentences. The subjects in the PENS class made the
largest mean gain in increasing Complex Sentences and
subjects in the Gordon Rule courses made the largest mean
gain in reducing nonsentences. Notable also is the reduction
in range of percentage of nonsentences written (from 0.00-
100.0 to 0.00-47.7) for the subjects in the Developmental
Writing course from the pretest to the posttest. The
Developmental Writing course appears to be more successful
with students writing a large number of nonsentences.
Vocabulary
On the pretest writing samples, PENS students were using
an average of 10.38 large (7 or more letters) words,
Developmental Writing course students were using an average
of 9.42 large words, and Gordon Rule students were using an
average of 11.39 large words. On the posttest writing
samples, PENS students were using an average of 13.94 large
words, Developmental Writing course students were using an
average of 8.93 large words, and Gordon Rule students were
using an average of 10.24 large words. Students in the PENS
class appeared to make gains in Vocabulary, while students in
the Developmental Writing course and Gordon Rule courses did
not.
Mechanics
On the pretest writing samples, students in the PENS
class were writing an average of 85.80% correct word
sequences, students in the Developmental Writing course were
writing an average of 82.09% correct word sequences, and
students in the Gordon Rule courses were writing an average
of 92.66% correct word sequences. On the posttest writing
samples, students in the PENS class were writing an average
of 85.70% correct word sequences, students in the
Developmental Writing course were writing an average of
85.06% correct word sequences, and students in the Gordon
Rule courses were writing an average of 93.55% correct word
sequences. The subjects in the Gordon Rule courses scored
higher in the use of correct mechanics than subjects in
either of the other treatment groups at the pretest and
maintained their advantage on the posttest; however, there
appears to be little difference in the amount of improvement
between the treatment groups on the Mechanics measure.
Composition Organization
On the pretest writing samples, PENS students were
writing papers that were generally average in organization
(3.04), Developmental Writing course students were writing
papers that were generally average (2.76) in organization,
and Gordon Rule students were writing papers that were
generally average (2.92) in organization. On the posttest
writing samples, PENS students were writing papers that were
generally average in organization (3.12), Developmental
Writing course students were writing papers that were
generally average (2.65) in organization, and Gordon Rule
students were writing papers that were generally average
(2.88) in organization. A visual inspection of the means,
standard deviations, and ranges suggests there was little
difference between the three treatment groups in
Organization.
Inferential Statistical Analysis of the Data
PENS Writing Strategy Subjects
All subjects participating in the Gordon Rule courses
and Developmental Writing course attended Santa Fe Community
College (SFCC), but subjects in the PENS class were from both
the University of Florida (UF) and Santa Fe Community
College. Because these two groups of subjects were used as
one treatment group (the PENS class), t-tests were computed
for the pretest scores to rule out the possibility that the
UF and SFCC subjects in the PENS class were different before
instruction began. Separate t-tests were computed for each
measure (fluency, nonsentences, simple/compound sentences,
complex sentences, vocabulary, mechanics, and organization)
with a .01 level of significance. The null hypotheses stated
that the mean scores for the UF and SFCC subjects were not
significantly different on each of the measures. Two-tailed
t-tests were used since the direction of the possible
differences in the means was not known. None of the null
hypotheses could be rejected. Furthermore, the p-values
associated with the t-tests indicated that significant
differences between UF and SFCC subjects would not have been
detected even if a more liberal alpha level had been
specified. Table 6 lists the individual t-tests.
Hypotheses
In this pretest-posttest design the independent between
group variable was the type of instructional experience
(Gordon Rule courses, Developmental Writing course, or PENS
class). The dependent variables were the posttest scores on
fluency, syntactic maturity (nonsentences, simple/compound
sentences, and complex sentences), vocabulary, mechanics, and
organization. The pretest scores on these variables served
as covariates. To evaluate significance of differences among
means on the dependent variables the researcher tested five
statistical null hypotheses which are restated here.
HI: There will be no difference among the groups
receiving the three educational experiences (Gordon Rule
courses, Developmental Writing course, and PENS class) on
posttest scores after adjusting for pretest scores in
fluency.
H2: There will be no difference among the groups
receiving the three educational experiences (Gordon Rule
courses, Developmental Writing course, and PENS class) on
posttest scores after adjusting for pretest scores in
syntactic maturity.
H3: There will be no difference among the groups
receiving the three educational experiences (Gordon Rule
courses, Developmental Writing course, and PENS class) on
posttest scores after adjusting for pretest scores in
vocabulary.
H4: There will be no difference among the groups
receiving the three educational experiences (Gordon Rule
courses, Developmental Writing course, and PENS class) on
posttest scores after adjusting for pretest scores in
mechanics.
H5: There will be no difference among the groups
receiving the three educational experiences (Gordon Rule
courses, Developmental Writing course, and PENS class) on
posttest scores after adjusting for pretest scores in
composition organization.
Table 6
Individual t-tests for the UF and SFCC Subjects in the PENS
Class
Degrees
Measure of Freedom
Fluency 10
Nonsentences 10
Simple/Comp.Sentences 10
Complex Sentences 10
Vocabulary 10
Mechanics 10
Organization 10
t Statistic
-1.084
.3185
-.6234
.8864
-.0072
-.4663
-.9447
The multivariate hypothesis was analyzed with a MANCOVA
procedure to determine whether there was a treatment group
effect on the five criterion measures after adjustment on the
five covariates pretestt scores). This hypothesis was not
rejected based on a Pillai's Trace F(14,46) = 1.27 (r =
0.1771). When all five dependent measures were evaluated as
a vector, none of the educational treatments were more
beneficial in improving writing skills for college students
with learning disabilities after adjustment for the pretest
scores.
Exploratory Analysis
Exploratory analyses were conducted even though the
MANCOVA failed to produce significant findings because of the
possibility of collinearity, the loss of degrees of freedom
.3038
.7567
.5470
.3962
.9944
.6510
.3671
due to the effect of the large number of covariates, and the
possibility of aptitude by treatment interaction that could
mask a treatment effect. Additional variables that could
have contributed to the lack of significant treatment
differences include the small number of subjects, the
moderate correlation of the covariates pretestt scores), and
the possibility of a salutary treatment effect.
Aptitude by treatment interaction. Univariate ANCOVA
analyses were run on each dependent measure to evaluate
whether there was an interaction between the treatment groups
and the occasion. An .01 level of significance was used to
evaluate these analyses due to the number of comparisons
being made (n = 5). The test for significant effect of the
covariate by treatment interaction was not significant on
Fluency [F(2,32) = 0.96; p = .3945], Nonsentences [F(2,32) =
0.26; p = .7700], Simple/Compound Sentences [F(2,32) = 0.45;
p = .6407], Complex Sentences [F(2,32) = 2.38; 2 = .1092],
Vocabulary [F(2,32) = 0.28; p = .7595], Mechanics [F(2,32) =
0.06; 1 = .9416], and Organization [F(2,32) = 0.17; p =
.8430]. When each dependent measure was analyzed separately,
none of the educational treatments interacted with the
initial levels of writing on that variable to affect the
students' performance on outcome measures. These tests for
significant aptitude by treatment interactions also were
tests for violations of the assumption of homogeneity of
slopes for the regression of outcomes on pretest measures and
must be completed prior to evaluation of the ANCOVAs without
the aptitude by treatment interaction.
Analysis of each dependent measure. Univariate ANCOVA
analyses were completed on each dependent measure without
using the aptitude by treatment interaction to evaluate
whether there was a difference among the treatment groups on
outcome measures adjusted for pretest scores (see Table 7).
An .01 level of significance was used to evaluate these
analyses due to the number of comparisons being made. The
test for significant treatment effect between groups on
posttest scores adjusted for pretest scores was not
significant on (a) Fluency [F(2,32) = 2.10; p = .1383];
(b) Syntactic Maturity -- Nonsentences [F(2,32) = 3.24;
p = .0517], Simple Sentences [F(2,32) = .60; p = .5623], and
Complex Sentences [F(2,32) = 3.86; = .0309]; and
(c) Organization [F(2,32) = 3.75; p = .0338]. Significance
was found on the pretest measures for Nonsentences [F(2,32) =
7.93; = .0080], Mechanics [F(2,32) = 16.58; p = .0003], and
Organization [F(2,32) = 19.18; 1 = .0001], indicating that
the pretest measures were useful covariates. Significant
differences were found among the treatment group
means on Vocabulary [F(2,32) = 5.64; p = .0077] and Mechanics
[F(2,32) = 9.14; p = .0007].
Follow-up tests were conducted for the Vocabulary and
Mechanics scores using the least squares means and a
Bonferroni alpha correction of an .003 significance level
(.01/3) to identify between which groups the mean differences
Table 7
ANCOVA Analysis of the Five Measures
Dependent
Variable
Fluency
Group
Pre
Error
98.42
29.97
925.96
Nonsentences
Group
Pre
Error
Simple Sentences
Group
Pre
Error
Complex Sentences
Group
Pre
Error
Vocabulary
Group
Pre
Error
Mechanics
Organization
Group
Pre
Error
Group
Pre
Error
* significant at the .01
49.21
29.97
23.45
2.10
1.28
.1383
.2662
905.16
1108.46
6767.98
370.89
7.67
10434.97
1574.80
360.84
6937.37
166.43
17.58
501.90
573.67
520.34
1066.96
452.58
1108.46
139.83
185.44
7.67
306.91
787.4
360.84
204.04
83.21
17.58
14.76
486.83
520.34
31.38
0.69
3.54
0.18
3.24
7.93
.60
.02
3.86
1.77
5.64
1.19
9.14
16.58
3.75
19.18
.0517
.0080*
.5623
.8753
.0309
.1924
.0077*
.2827
.0007*
.0003*
.0338
.0001*
1.38
3.54
6.28
level
occurred (see Table 8). No significant group mean
differences were found on the follow-up tests for Mechanics
and Vocabulary, indicating that no single pairwise contrast
contributed to the significant finding; however, two
follow-up tests approached significance for Vocabulary (PENS
subjects compared to Developmental Writing subjects and PENS
subjects compared to Gordon Rule subjects) and would have
been significant if a less conservative alpha level had been
chosen. PENS subjects appeared to make gains in Vocabulary
(from a mean of 10.38 large words to a mean of 13.94 large
words) while subjects in the Developmental Writing course
(from a mean of 9.42 large words to a mean of 8.9 large
words) and the Gordon Rule courses (from a mean of 11.39
large words to a mean of 10.24 large words) did not appear to
make improvements in Vocabulary.
Pretest and posttest correlations. In order to evaluate
whether multivariate or univariate statistics were needed, the
correlations of the pretest measures to each other and the
posttest measures to each other were obtained (see Table 9).
Since many of the correlations were moderately high, the
multivariate analysis was appropriate.
Pretest to posttest correlations. The stability of the
measures obtained was evaluated using correlations between
the pretest and posttest scores on the measures for all of
the subjects together and for the subjects by treatment
group. The correlations of pretest to posttest scores for
the subjects as a whole group ranged from -.33 to .68
Table 8
Follow-up Test p-values for Comparisons Between Groups Using
the Least Squares Means
Bonferroni Correction alpha = .003
Measure Group PENS DW GR
Vocabulary
PENS .0039 .0165
DW .5733
GR
Mechanics
PENS .5406 .1146
DW .3416
GR
Table 9
Pretest and Posttest Correlations
Occasion Measure F NS SS CS V M O
Pretest F .449 -.676 .154 -.063 .021 -.051
NS -.729 -.565 -.477 -.487 -.469
SS -.147 .239 .172 .200
CS .377 .472 .436
V .436 .299
M .179
0
Posttest F .018 -.396 .412 .141 .329 .200
NS -.507 -.315 -.090 -.556 -.280
SS -.655 -.075 .162 .052
CS .165 .309 .171
V .056 .341
M .147
0
(see Table 10). The correlations of pretest to posttest
scores for the subjects by treatment groups ranged from -.23
to .69 (see Table 11). These low correlations indicate that
writing performance of the subjects from the pretest to the
posttest (i.e., from the beginning to the end of the
semester) was not stable.
Reliability
Procedural Reliability
A time sampling recording form was used to obtain a
measure of the PENS strategy instructors' ability to
implement the PENS instruction according to the outlined
procedure. All instructor behaviors were rated as either
appropriate or not applicable. Individual ratings for each
instructor are presented in Table 12.
Scorer Reliability
Five scorers rated the subjects' writing samples,
including an individual trained in the PENS writing strategy
(R4), two public school teachers who teach writing skills (R1
and R3), a preservice elementary teacher (R2), and the
researcher (R5). All of the scorers, with the exception of
the researcher, were unaware of the conditions of this study.
While R4 scored all writing samples on all measures, the
other scorers rated all or part of the writing samples on
only part of the measures (see Table 13). The reliability
coefficients were Pearson Correlation Coefficients obtained
using the SAS computer program. The reliability coefficients
range from .73 to .99. Reliability for the scores were all
Table 10
Correlations Between Pretest and Posttest Scores for All
Subjects
Measure
Fluency
Nonsentences
Simple/Compound Sentences
Complex Sentences
Vocabulary
Mechanics
Organization
Correlation
.274
.478
-.330
.346
.185
.684
.614
Table 11
Correlations Between Pretest and Posttest Scores for Subjects
by Treatment
PENS
;1 a q
Measure
Fluency
Nonsentences
Simple/Compound
Sentences
Complex Sentences
Vocabulary
Mechanics
Organization
Developmental
Writina Course
.512
.458
.375
.429
.178
.514
.683
.036
.441
-.139
.520
.306
.634
.419
Gordon Rule
Courses
.244
.465
-.028
-.238
.011
.497
.699
""'~ ~'-"~-~~~ ~-----
Table 12
Procedural Reliability for PENS Instructor Behaviors
Instructor Script Sequence Materials
Instructor 1 (researcher)
Observation #1
appropriate 100% 100%
inappropriate 0% 0%
not applicable 0% 0%
Observation #2
appropriate 0% 100%
inappropriate 0% 0%
not applicable 100% 0%
Instructor 2 (student assistant fall semester)
Observation #1
appropriate 0% 100%
inappropriate 0% 0%
not applicable 100% 0%
Observation #2
appropriate 100% 100%
inappropriate 0% 0%
not applicable 0% 0%
Instructor 3 (student assistant spring semester)
Observation #1
appropriate 100% 100%
inappropriate 0% 0%
not applicable 0% 0%
100%
0%
0%
100%
0%
0%
100%
0%
0%
100%
0%
0%
100%
0%
0%
Table 13
Subjects Rated by the Five Scorers
Scorer
R3
Measure Rl
Fluency 100%
Nonsentences
Simple/Compound
Sentences
Complex Sentences
Vocabulary 100%
Mechanics 55%a
Organization
a = The 55% rated by R1
by R3.
100%
100%
100%
55%
55%
55%
is a different
R4
100%
100%
100%
100%
100%
100%
100%
55% than
R5
100%
100%
100%
100%
100%
100%
that rated
within an acceptable range (above .70) and allow the
conclusion to be made that the measures were reliably scored.
Tables 14 to 17 present the correlations estimating the
reliability of the scores for all raters on all measures.
Generalizability Study
A Generalizability study (G-study) was conducted to
provide another method to evaluate reliability of the measures
and errors in measurement that may have occurred when multiple
raters are used. Data gathered from this G-study helped
determine whether a sufficient number of raters were used to
obtain generalizable scores on each measure.
Generalizability coefficients were calculated for each
measure on groups of two, three, and/or four scorers
depending on the number of scorers rating each measure.
Table 14
Reliability for Fluency Scores
Scorer
R1
R2
R3
R2
.94
.95
Occasion
pre
post
pre
post
pre
post
R3
.98
.98
.96
.92
Scorer
R4
.92
.97
.89
.94
.81
.95
pre
post
R5
.98
.98
.94
.96
.97
.98
.93
.98
Table 15
Reliability for Nonsentence, Simple/Compound Sentence,
Complex Sentence, and Organization Scores
Scorer
Measure Scorer Occasion R5
Nonsentences R4 pre .97
post .93
Simple/Compound R4 pre .90
Sentence post .89
Complex R4 pre .88
Sentence post .90
Organization R4 pre .77
post .84
RI~1 IU1-Vi1 -~ fcVr VccI-h,,1 YYrw
Scorer
Scorer Occasion R3 R4 R5
R1 pre .83 .88 .85
post .75 .80 .80
R3 pre .88 .88
post .75 .73
R4 pre .99
post .97
Table 17
Reliability for Mechanics Scores
Scorer
Scorer Occasion R2 R3 R4
R1 pre .85 .87 .84
post .91 .85 .88
R2 pre .89 .86
post .87 .89
R3 pre .89
post .85
Table 16
i i- fr Vnfhn1n r .qr-nrTPq
Results from One-way Repeated Measures ANOVAs using a raters
by subjects design were used to calculate the
generalizability coefficients using the following formula
(Crocker & Algina, 1986, p. 167):
MSp MSr
MSp + (n n')MSr/n'
For example, the first generalizability coefficient for
Fluency was calculated by using the posttest scores from R1
and R2 in a One-way Repeated Measures ANOVA to obtain the
mean squares used in the formula (generalizability
coefficient = .987). The second generalizability coefficient
for Fluency was calculated by using the posttest scores from
R1, R2, and R4 in a One-way Repeated Measures ANOVA to obtain
the mean squares used in the formula (generalizability
coefficient = .999). And the third generalizability
coefficient for Fluency was calculated by using the posttest
scores from R1, R2, R4, and R5 in a One-way Repeated Measures
ANOVA to obtain the mean squares used in the formula
(generalizability coefficient = .999). The subsequent
generalizability coefficients were calculated in the same
manner.
The G-study coefficients were high for the Fluency
(.98), Nonsentences (.97), Vocabulary (.93), Mechanics (.88),
and Organization (.99) measures. The G-study coefficients
were moderate for the Simple/Compound Sentences (.73)
measure. The G-study coefficients were low for the Complex
Sentences (.14) measure. Results from the G-study enable the
conclusion to be made that two raters were sufficient for
scoring Fluency, Nonsentences, and Organization. Three
scorers were sufficient for scoring Mechanics and Vocabulary.
More than two scorers were needed to score Simple/Compound
Sentences and Complex Sentences. Table 18 contains the
G-study coefficients.
Summary of Findings
The hypotheses regarding the lack of interactions
between the treatment groups and the pretest and posttest
scores on the dependent measures were retained. Exploratory
analyses and visual inspection of the group means confirmed
the initial findings; however, a nearly statistically
significant gain was made by subjects in the PENS class in
increasing the percentage of large words (vocabulary) in
their writing samples. A discussion of these results is
reported in Chapter 5.
Table 18
Generalizability Coefficients
Scorers Generalizability
Measure R1 R2 R3 R4 R5 Coefficient
Fluency X X .987
X X X .999
X X X X .999
Nonsentences X X .972
Simple/Compound X X .733
Sentences
Complex Sentences X X .144
Vocabulary X X .757
X X X .888
X X X X .932
Mechanics X X .688
x X x .889
Organization X X .998
X denotes the raters used in the ANOVA analysis
CHAPTER 5
DISCUSSION
The purpose of this chapter is to summarize the results
of this research. Statistical and descriptive discussions of
the hypotheses are presented. Feedback from students and
staff is presented. Educational implications are stated.
Future research needs are reported.
Discussion of Hypotheses
The multivariate hypothesis stated that there would be
no treatment group effect on the five criterion measures
after adjustment on the five covariates pretest measures of
Fluency, Syntactic Maturity, Vocabulary, Mechanics, and
Composition Organization. There were no statistically
significant mean differences among the treatment groups after
adjusting for the pretest differences on any of the measures
of writing ability, indicating that none of the educational
experiences were more beneficial than the others.
Exploratory analyses of the data generally confirmed the
lack of significant findings of differences in group
performance; however, significant differences were noted on
Vocabulary and Mechanics among the treatment groups on the
posttest measures when the posttest was adjusted for pretest
differences. While no significant group mean differences
were found on the follow-up tests for Mechanics and
Vocabulary, two follow-up tests approached significance for
Vocabulary (PENS subjects compared to Developmental Writing
subjects and PENS subjects compared to Gordon Rule subjects).
These follow-up tests were for pairwise contrasts only. PENS
subjects appeared to make gains in Vocabulary while subjects
in the Developmental Writing course and the Gordon Rule
courses did not appear to make improvements in Vocabulary.
Visual inspection of the means, standard deviations, and
ranges of the pretest and posttest scores confirms the lack
of significant differences between the subjects in the three
educational experiences; however, several points should be
noted. First, very little change could be detected through
visual inspection from the pretest to the posttest for all
three treatment groups on measures of Fluency, Mechanics, and
Organization. Second, visual inspection supports the finding
of the statistical analysis that only the subjects in the
PENS class increased the number of large words (Vocabulary)
used in their writing. Third, visual inspection also
detected changes in Syntactic Maturity. Students in each of
the three treatment groups appeared to reduce the number of
nonsentences and increased the percentage of complex
sentences written. Subjects in the PENS class appeared to
make the largest gain in writing complex sentences. Subjects
in the Gordon Rule courses appeared to make the largest gain
in reducing the percentage of nonsentences. Notable also is
the reduction in range of percentage of nonsentences written
(from 0.00-100.0 to 0.00-47.7) for the subjects in the
|
