Effect of a cognitive strategy on the writing ability of college students with learning disabilities

EFFECT OF A COGNITIVE STRATEGY ON
THE WRITING ABILITY OF COLLEGE STUDENTS
WITH LEARNING DISABILITIES
















By

BETH LORENE TULBERT


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


1992

































This work is dedicated to the memory of
my brother, Bont Franklin Tulbert. He
taught me to not give up, to never take
no for an answer, and to live life as if
this were my last day. A part of him is
with me always.















ACKNOWLEDGEMENTS


These last three years have been three of the most

challenging, difficult, and growth-filled years of my life.

Several people have helped support me and are due

considerable thanks. First, and foremost, I would like to

thank my parents. They have supported me emotionally and

financially during this trying and exciting time. They

taught me that I could succeed at anything if I would only

try. Without them, I could never have made it through this

rigorous and demanding program. This degree is as much

theirs as it is mine.

I would like to collectively thank my committee members,

Dr. Cecil Mercer, Dr. Linda Crocker, Dr. Cynthia Griffin,

Dr. William Reid, Dr. Jeanne Repetto, and Dr. Stuart

Schwartz. Their guidance and support have enabled me to

complete a demanding and rigorous program culminating in this

dissertation. I appreciate the time and effort given by each

of them on my behalf.

I would like to thank my advisor and mentor, Dr. Cecil

Mercer. He has guided me in the right direction and helped

me get from this program all the things that I would have

said that I wanted if I had known what to ask for in the

beginning. I am glad to have had him as a role model these


iii










last three years and hope that I can continue to follow in

his thoughtful, positive foot steps.

I would also like to thank Dr. Stuart Schwartz. He

helped prepare me for the demands of this program in Trends

and instilled in me the belief that I could succeed. He

included me in a variety of opportunities, challenged me to

do my best, and provided me with a standard of excellence

toward which to strive.

I would like to thank my comrades in arms, Donna Wandry

and Glenn Buck. Together we made it through Trends, the

various doctoral seminars, and all of the research courses.

We were the Happy Trio and without them both, the last three

years would just not have been the same.

I would like to thank Dr. Jeanne Repetto. She listened

and supported me throughout this program, especially during

"the week before," and has been a wonderful mentor and

friend. I am sure I will look back at all of this with "fond

memories."

I would like to thank Dr. Mary Brownell for our little

talks in the back room. She has been a soul-mate, good

friend, and role model. And, yes, it is too late.

Finally, I would like to thank Ken Osfield at the

University of Florida and Issac Jones at Santa Fe Community

College for all of their time and effort in making this study

possible. Without their cooperation and commitment to this

project, this study would never have been completed.
















TABLE OF CONTENTS
Pacre

ACKNOWLEDGEMENTS .......................................... iii

LIST OF TABLES ......................................... viii

ABSTRACT .................................................. ix

CHAPTERS

1 INTRODUCTION ......................................... 1

Statement of the Problem ................. .......... 4
Rationale ................. ........................... 5
Definition of Terms ................................. 7
Delimitations of the Study ......................... 11
Limitations of the Study ............................. 12
Summary and Overview of Remaining Chapters ......... 12

2 REVIEW OF RELATED LITERATURE ....................... 14

Selection of Relevant Literature .................... 14
Interventions for College Students with
Learning Disabilities ................................. 16
History of Postsecondary Programs ............... 16
Description of Postsecondary Programs .......... 20
Learning Strategies ................................... 29
Background on Strategy Instruction .............. 29
Research on Writing Strategy Instruction ........ 38
Summary .............................................. 42

3 METHODOLOGY .......................................... 44

Hypotheses ........................................... 44
Subjects and Setting ................................. 45
Measurement Procedures ................................ 48
Screening for PENS Class ........................... 48
Pretest and Posttest .............................. 48
Course Content and Instructional Methods ........ 54
Reliability of Procedures ......................... 56
Interscorer Agreement .............................. 56
Generalizability Study ............................. 56
Procedural Reliability ............................. 57
Training of Personnel ................................. 58
Training of Student Assistant .................. 58
Training of Observers .............................. 59









Training the Scorers for Reliability Checks ..... 59
PENS Instructional Procedures ...................... 60
Experimental Design .................................. 62
Analysis of the Data ................................. 64
Summary ............................................ 64

4 RESULTS ................ .............................. 66

Descriptive Analysis of the Data ................... 66
Fluency ........................................... 66
Syntactic Maturity ................................. 68
Vocabulary ........................................ 69
Mechanics ......................................... 70
Composition Organization ........................... 71
Inferential Statistical Analyses of the Data ....... 71
PENS Writing Strategy Subjects .................. 71
Hypotheses ........................................ 72
Exploratory Analysis ............................... 74
Reliability ........................................ 80
Procedural Reliability ............................. 80
Scorer Reliability ................................. 80
Generalizability Study ............................... 83
Summary of Findings ................................... 87

5 DISCUSSION ........................................... 89

Discussion of Hypotheses .............................. 89
Feedback from Students and Staff ........... ........ 93
Educational Implications .............................. 96
Services for College Students with Disabilities 96
PENS Writing Strategy Instruction ............... 99
Future Research ...................................... 100
Summary .............................................. 102

APPENDICES

A PROGRAMS IN POSTSECONDARY SETTING FOR
STUDENTS WITH LD.............................. 104

B LETTER TO UF STUDENTS AND TENTATIVE
COURSE SYLLABUS .............................. 106

C INFORMED CONSENT FOR STUDENTS IN PENS CLASS... 110

D INFORMED CONSENT FOR STUDENTS IN GORDON RULE
AND WRITING LABS ............................. 113

E SCORER SHEET .................................. 116

F SCORE CALCULATION FORM......................... 118

G FACULTY QUESTIONNAIRE ......................... 120

H OBSERVATION FORM .............................. 125











REFERENCES ............................................. 126

BIOGRAPHICAL SKETCH ..................................... 133


vii















LIST OF TABLES


Table Page

1 The PENS Strategy .................................. 9

2 The PENS Strategy Sentence Formulas ............... 10

3 Description of Subjects .......................... 47

4 Study Design ..................................... 63

5 Means, (Standard Deviations), and [Range] of
Pretest and Posttest Scores on the Five Writing
Measures .......................................... 67

6 Individual t-tests for the UF and SFCC
Subjects in the PENS Class........................ 74

7 ANCOVA Analysis of the Five Measures .............. 77

8 Follow-up Test p-values for Comparisons
Between Groups Using the Least Squares Means ...... 79

9 Pretest and Posttest Correlations.................. 79

10 Correlations Between Pretest and Posttest
Scores for All Subjects .............. ............ 81

11 Correlations Between Pretest and Posttest
Scores for Subjects by Treatment .................. 81

12 Procedural Reliability for PENS Instructor
Behaviors.. ................................ ....... 82

13 Subjects Rated by the Five Scorers ................ 83

14 Reliability for Fluency Scores .................... 84

15 Reliability for Nonsentence, Simple/Compound
Sentence, Complex Sentence, and Organization
Scores .. ......................................... 84

16 Reliability for Vocabulary Scores ................. 85

17 Reliability for Mechanics Scores .................. 85

18 Generalizability Coefficients ..................... 88


viii















Abstract of Dissertation Presented to the
Graduate School of the University of Florida
in Partial Fulfillment of the
Requirements for the Degree of
Doctor of Philosophy

EFFECT OF A COGNITIVE STRATEGY ON
THE WRITING ABILITY OF COLLEGE STUDENTS
WITH LEARNING DISABILITIES

By

Beth Lorene Tulbert

August 1992


Chair: Cecil D. Mercer
Major Department: Special Education

The purpose of this study was to (a) determine the

effectiveness of cognitive writing strategy (PENS)

instruction and (b) compare the effect of PENS learning

strategy instruction, Developmental Writing course

instruction, and Gordon Rule course instruction on written

expression skills of college students with learning

disabilities. The subjects in this investigation were

college students with learning disabilities who have

registered with the Office of Disabled Student Services on

the campuses of the University of Florida and Santa Fe

Community College. The study included a pretest, an

implementation phase, and a posttest. Scores were obtained

from writing samples on fluency, syntactic maturity,

vocabulary, mechanics, and organization of written work. A










nonrandom pretest-posttest design with one between

(educational experience) and one within (performance over

time) group factor was used. A MANCOVA procedure failed to

reject the multivariate null hypothesis (no treatment group

effect on the five criterion measures after adjustment on the

pretest scores). Exploratory analyses were conducted.

Individual ANCOVAs were used to test for significant

treatment effect for the five criterion measures between the

three groups on posttest scores after adjusting for the

pretest scores. No significant difference was found between

the treatment groups for Fluency, Syntactic Maturity, and

Organization. Significant differences were found among the

treatment group means on Vocabulary and Mechanics. Follow-up

tests using pairwise contrasts found no significant

differences for Mechanics and Vocabulary; however, two

pairwise tests approached significance for Vocabulary

suggesting that PENS subjects made gains in Vocabulary while

subjects in the Developmental Writing course and the Gordon

Rule courses did not make improvements in Vocabulary.

Two-way Repeated Measures ANOVAs were also used to evaluate

the success of each treatment in improving writing skills on

each of the five criterion measures from the pretest to the

posttest occasion. On the average, across the three

treatments, a gain was made in Syntactic Maturity (writing

more complex sentences). These results have educational

implications for college students with learning disabilities.















CHAPTER 1
INTRODUCTION


Educational opportunities for individuals with learning

disabilities have increased rapidly over the last two decades

through the combined efforts of the Education for All

Handicapped Children Act of 1975 (EHA), Section 504 of the

Rehabilitation Act of 1973 (Section 504) (Apostoli, 1986),

and the Americans with Disabilities Act (ADA) (Linthicum,

Cole, & D'Alonzo, 1991). Often referred to as civil rights

legislation, Section 504 and ADA stipulate that colleges and

universities must ensure the rights of qualified students

with disabilities to enter colleges and universities and to

participate fully in all programs (Vogel, 1982; Linthicum,

Cole, & D'Alonzo, 1991). Specifically, Section 504 and ADA

prohibit discrimination against persons with disabilities in

recruitment, admissions, and treatment in postsecondary

education and require that colleges and universities make

reasonable accommodations or adjustments to assist students

with disabilities in meeting academic requirements.

Officials and administrators in institutions of higher

education are attempting to resolve the dilemma of

appropriately meeting the needs of students with learning

disabilities without compromising their schools' academic

standards. Extending support services for students with










disabilities to the college level can provide a solution to

this dilemma (Dalke & Schmitt, 1987). Providing support

services for students with learning disabilities in

postsecondary institutions (a) is a logical extension of

services offered at the secondary school level, (b) can be an

important component of the reasonable accommodations mandated

by law, and (c) ensures their rights to participate fully in

educational programs at the college level for several

reasons.

First, several characteristics of adolescents and adults

with learning disabilities hinder their success in college.

Many college students with learning disabilities continue to

report problems with reading, writing, notetaking (Ryan &

Heikkila, 1988), mathematics, study skills, interpersonal

skills, and oral communication (Vogel, 1985). Deficits in

written expressive language among individuals with learning

disabilities are often severe, persist into adulthood, and

may be more prevalent than earlier identified reading

difficulties (Vogel & Konrad, 1988). It has been estimated

that 90% of adults with learning disabilities experience

written language disorders (Vogel, 1987). Additionally, the

ability to express oneself clearly and precisely in writing

is considered by some faculty to be synonymous with a

bachelor's degree. Others consider the writing process to be

a catalyst for the thinking process itself (Vogel, 1987).

Therefore, written expression is an area of great concern to










college faculty and students with learning disabilities

(Vogel, 1982).

Second, students with learning disabilities often are

unprepared for the demands of college (Prater & Minner,

1986). A survey of curriculum approaches used in high

schools with students with learning disabilities revealed

five major types of programs (Deshler, Lowery, & Alley,

1979), including a basic skills remediation model, a tutorial

model, a work-study model, a functional curriculum model, and

a learning strategies model. The remedial, tutorial, work-

study, and functional curriculum models of instruction do not

prepare students with learning disabilities to be independent

academic learners. The ability to learn independently is a

critical skill in higher education where students are

required to locate, learn, and retain information with little

assistance from college professors (Prater & Minner, 1986).

Third, setting demands in college (expectations for

college students) may create seemingly insurmountable

problems for students with learning disabilities. All

college students are compelled to grapple with intense

demands on managing their time and this may be a particular

problem for students with learning disabilities (Ryan &

Heikkila, 1988). Moreover, students with learning

disabilities often face negative attitudes from professors in

higher education and they may lack necessary support

personnel and systems to help them succeed (Prater & Minner,

1986).










Consequently, support services are needed for college

students with learning disabilities. Interventions that can

(a) address specific academic weaknesses of students with

learning disabilities, (b) mediate setting demands of higher

education, and (c) prepare students to function independently

in the college setting are needed. The University of Kansas

Institute for Research in Learning Disabilities (KU-IRLD)

learning strategies model was designed to meet similar needs

of high school students with learning disabilities. The

KU-IRLD strategies have been shown to be an effective and

efficient intervention for high school students with learning

disabilities and may be appropriate for use at the

postsecondary level.

Statement of the Problem

This study was designed to investigate the effectiveness

of learning strategies as an intervention for college

students with learning disabilities. Specifically, the

KU-IRLD sentence writing strategy (PENS) was used with

college students who are experiencing problems with written

expression. The experimental questions were as follows:

1. Is the KU-IRLD sentence writing strategy (PENS) an

effective intervention for college students with learning

disabilities who are experiencing problems with written

expression?

2. Is the KU-IRLD sentence writing strategy (PENS) a

more effective intervention for college students with

learning disabilities who are experiencing problems with







5


written expression than currently available options, such as

enrollment in Gordon Rule courses and writing labs?

The study includes problems for investigation that are

important for several reasons. First, the study focuses on

evaluating the effectiveness of the KU-IRLD learning

strategies for college students with learning disabilities.

Second, the study features a comparison of alternatives that

are currently available in Florida for improving the written

expression skills of college students with learning

disabilities. Included in the comparison are the KU-IRLD

sentence writing strategy (PENS), a developmental writing

course, and Gordon Rule courses (content courses in which

writing is stressed). Third, the study adds to the limited

research data on appropriate interventions for college

students with learning disabilities.

Rationale

The current state of affairs in service delivery for

students with learning disabilities at the postsecondary

level represents a sincere effort to meet student needs, but

is often not grounded in theory or supported by efficacy or

evaluation data (McGuire, Norlander, & Shaw, 1990). Many

colleges and universities offer auxiliary aids and services,

which are available to all students or to all students with

disabilities, but this generic approach to service delivery

may not be appropriate for many students with learning

disabilities.










Some postsecondary settings provide comprehensive,

diagnostically based programs with trained specialists, while

officials at other institutions address the needs of students

with learning disabilities through remedial and developmental

approaches, peer-tutoring models, or content tutoring

(McGuire, Norlander, & Shaw, 1990).

Despite the diversity of service delivery options

available at the postsecondary level and the documented need

for appropriate services, there has been little systematic

research regarding the effectiveness of various interventions

for college students with learning disabilities. McGuire,

Norlander, and Shaw (1990) propose that additional efficacy

studies are necessary to make informed decisions as to the

effectiveness of various interventions. Empirical research

must be employed to increase the existing database related to

effective and efficient interventions for college students

with learning disabilities. Information derived from such

systematic examinations can improve the support services

available to college students with learning disabilities.

Improved interventions will increase the likelihood that

college students with learning disabilities will experience

success in postsecondary institutions.

Additionally, college students with learning

disabilities display a variety of deficits in written

expression (Fourqurean & LaCourt, 1990). Punctuation,

capitalization, and basic sentence structure continue to be a

problem for students with learning disabilities in










postsecondary settings (Hughes & Smith, 1990). Vogel and

Konrad (1988) found that college students with learning

disabilities scored significantly lower than students without

disabilities on mechanics (spelling and punctuation),

spelling, subject/predicate/number agreement, percentage of

complete sentences, and percentage of complicated sentences.

Cowen (1988) surveyed college students with learning

disabilities and found that 76% reported problems with

spelling, punctuation, and grammar. Research conducted with

students with learning disabilities underscores the need for

support services to provide interventions in written

expression.

Definition of Terms

The following section presents the terms and their

definitions as used in this study. Terms selected for

inclusion are critical to understanding implementation

procedures and observed results.

Learning disability (LD) refers to a disorder in one or

more of the basic psychological processes involved in

understanding or using language, spoken or written, which may

manifest itself in an imperfect ability to listen, think,

speak, write, spell, read, or do mathematical calculations.

The term includes such conditions as perceptual handicaps,

brain injury, minimal brain dysfunction, dyslexia, and

developmental aphasia. The term does not include children

who have learning problems which are primarily the result of

visual, hearing, or motor handicaps; mental retardation;










emotional disturbances; or environmental, cultural, or

economic disadvantages (United States Office of Education,

1987, p. 65083, as cited in Mercer, 1987).

Learning strategies are techniques, principles, or rules

that enable a person to learn, solve problems, and complete

tasks independently (Alley & Deshler, 1979).

The University of Kansas Institute for Research in

Learning Disabilities (KU-IRLD) was initially funded in 1977

with a primary mission of developing and validating an

intervention model for adolescents with learning disabilities

that meets both effectiveness and feasibility criteria

(Deshler & Schumaker, 1986).

Written expression "can be defined as a visual

representation of thoughts, feelings, and ideas using symbols

of the writer's language system for the purpose of

communication or recording" (Poteet, 1980, p.88).

The PENS sentence writing strategy is part of a writing

strategy curriculum developed by the researchers at KU-IRLD,

including Sentence Writing, Paragraph Writing, Error-

Monitoring, and Theme Writing strategies (Schumaker &

Sheldon, 1985). The mnemonic PENS (see Table 1) was

developed to increase the likelihood that subjects would

remember the strategy components in actual writing situations

(Pressley, Levin, & Delaney, 1982). PENS does not cover

every sentence variation that might arise in a developmental

language course, but students who master all of the sentence










Table 1
The PENS Strategy


E Pick a formula
E Explore words to fit the formula
N Note the words
S Subject-verb identification
Look for the action or state of being words) to
find the verb
Ask the "Who or what questions" to find the
subject




types included in the strategy will be able to write

sentences that fit more than 14 different sentence formulas.

Sentence formulas for writing simple, compound, complex,

and compound-complex sentences (see Table 2) are taught in

PENS. A simple sentence is a sentence that is made up of one

independent clause. An independent clause is a group of

words with a subject and verb that can stand alone. A

compound sentence is a sentence with two or more independent

clauses joined by appropriate punctuation and a conjunction.

A complex sentence is a sentence that contains one

independent clause and one or more dependent clauses

joined by the appropriate punctuation and conjunctions. A

dependent clause is a group of words with a subject and verb

that cannot stand alone. A compound-complex sentence is a

combination of compound and complex sentences and consists of

two or more independent clauses and at least one dependent

clause joined by the appropriate punctuation and

conjunctions.










Table 2
The PENS Strategy Sentence Formulas


Simple Sentences
I=S V
I=SS V
I=S VV
I=SS VV

Compound Sentences
I,cI
I;I

Complex Sentences
D,I
ID

Compound-Complex Sentences
D,I,cI
ID,cI
I,cID
D,I;I
ID;I
I;ID


I=independent clause
S=subject
V=verb



c=conjunction



D=dependent clause


Fluency is defined as "the degree to which the student

becomes more proficient at writing down words and sentences

into compositions of gradually increasing length" (Isaacson,

1985, p. 409). Fluency is measured by finding the average

sentence length (Tindal & Marston, 1990).

Syntactic maturity is defined as "the degree to which a

student uses expanded, more complex sentences" (Isaacson,

1985, p. 410). One way to measure syntactic maturity is to

count the number of sentences that fall into several

categories (nonsentences, simple sentences, compound

sentences, and complex sentences) and compute the percentage










of each type of sentence in the writing sample (Tindal &

Marston, 1990).

Vocabulary focuses on the uniqueness or maturity of

words that are used in a composition (Tindal & Marston,

1990). Scoring is accomplished by counting the number of

unique or mature (large) words used in the writing sample.

Mechanics incorporates spelling, punctuation, usage, and

grammar skills and can be measured using correct word

sequences (CWS), a multiple-factor measure of written

expression (Tindal & Marston, 1990). Correct word sequences

is defined as "two, adjacent, correctly spelled words that

are acceptable within the context of the phrase to a native

English speaker" (Videen, Deno, & Marston, 1982, p. 11).

When punctuation is missing, words are misspelled, or usage

is incorrect, the word sequence is scored as incorrect.

Therefore, using CWS provides a measure of the mechanical

conventions that make a composition grammatically correct and

presentable to others.

Composition organization refers to the structure and

organization within the writing sample and focuses on (a) the

use of topic sentences, supporting sentences, and concluding

statements in paragraphs and (b) the flow of ideas from

paragraph to paragraph or sentence to sentence.

Delimitations of the Study

The scope of this study was delimited in three ways.

First, this study was restricted geographically to Alachua

County within the state of Florida and to students at the










University of Florida and Santa Fe Community College.

Second, only college students with learning disabilities who

are experiencing problems with written expression were

included. Third, only students who are registered with the

colleges' Office for Disabled Student Services were

considered for the study. These students met the college

criteria for being identified as learning disabled. No

additional educational or psychological testing was completed

as part of this study.

Limitations of the Study

Since this study included only college students with

learning disabilities who are experiencing problems with

written expression, the findings should not be generalized to

other college students with or without disabilities.

Moreover, the results of this study cannot be generalized to

skills other than written expression without replication.

Caution should be exercised in extrapolating results of this

study to students outside Alachua County and at schools other

than University of Florida and Santa Fe Community College.

Summary and Overview of Remaining Chapters

Students with learning disabilities are enrolling in

2-year and 4-year colleges in increasing numbers. However,

characteristics of students with learning disabilities, poor

preparation for college in high school and setting demands in

higher education, usually combine to make college a

frustrating experience. To improve the success rate of

college students with learning disabilities in postsecondary










settings, effective and efficient interventions are needed.

The purpose of this study was to investigate the efficacy of

the sentence writing strategy (PENS) on the written

expression skills of college students with learning

disabilities. It is the intent of the researcher conducting

this study to contribute to the research findings on strategy

instruction and effective interventions for college students

with learning disabilities. The results of this study have

direct and immediate implications for college support service

providers.

A review of literature relevant to this study is

presented in Chapter 2. Methodology used for implementation

is discussed in Chapter 3. The results and their

implications is reported in Chapters 4 and 5.















CHAPTER 2
REVIEW OF RELATED LITERATURE


In Chapter 2, a summary and analysis of the professional

literature involving interventions for college students with

learning disabilities and writing strategy instruction are

presented. The chapter is divided into four major sections.

First, selection criteria for the literature that was

reviewed are presented. Second, the historical development

and analysis of research on interventions for college

students with learning disabilities are presented. Third,

the historical development and analysis of research on

teaching writing using cognitive strategy instruction were

presented. Fourth, the historical development and analysis

of research on KU-IRLD learning strategy were presented. The

chapter concludes with a summary and implications of previous

research as it related to the present study.

Selection of Relevant Literature

An initial step in the review of the literature was that

of determining the criteria for the inclusion of references.

All relevant studies completed in the last ten years (1981-

1991) were examined. In addition, any notable research cited

in the literature earlier than the 10-year period was

considered.










Professional literature concerning interventions for

students with learning disabilities and writing strategy

instruction was required to meet the following criteria to be

included in the review:

1. The subjects and the settings in which the

experimentation took place had to be thoroughly

described.

2. The treatment conditions and experimental

procedures were detailed enough to permit

replication.

3. The experimental design and data analysis

procedures were presented without significant

losses of information.

4. The interpretations of the experimenter had to be

consistent with the results displayed.

To complete an exhaustive review of the literature

relating to interventions for college students with learning

disabilities, KU-IRLD learning strategies, and writing

strategy instruction, the reference sources used included

Dissertation Abstracts International, Educational Resources

Information Clearinghouse (ERIC), Psychological Abstracts,

Current Index to Journals in Education (CIJE), and Higher

Education Abstracts. References initially selected were

located through the library at the University of Florida,

through the interlibrary loan system, or through other

professionals in the field. Descriptors used in this

literature search included learning strategies, learning










disabilities, college students, interventions, support

services, postsecondary education, writing instruction,

writing strategies, writing difficulties, written expression

problems, learning processes, metacognition, and cognitive

strategies.

The references that were selected were critically

reviewed and those that described empirical investigations

were chosen based on the investigator's judgment that the

references presented a clear description of subject

selection, methodology, and results. Professional literature

other than empirical investigations were also included if, in

the author's judgment, the information that was included

provided a valuable contribution to the knowledge base about

or an understanding of interventions for college students

with disabilities or of writing strategy instruction.

Interventions for College Students
with Learning Disabilities

History of Postsecondary Programs

Special programs and services to meet the needs of

college and university students with learning disabilities

are a relatively new phenomenon. Efforts to provide services

were found in the early 1970s when a few colleges attempted

to develop and implement the necessary conditions for

students with learning disabilities (Strichart & Mangrum,

1986). Cordoni (1982) reports that small private colleges

were the pioneers in providing services for college students

with learning disabilities. The programs began at small










private institutions were of questionable value since they

were (a) mainly tutorial in nature, (b) based on limited

assessment of the needs of students with learning

disabilities, and (c) influenced by the theoretical

disposition or practical experience of the person who

conceived or developed them. As Ostertag, Baker, Howard, and

Best (1982) point out, the continued participation of

students with learning disabilities in postsecondary

education requires a re-evaluation of the types of

educational programs and services offered to these students

at the postsecondary level.

The number of students with learning disabilities

applying to and attending postsecondary institutions is

greater than ever before (Scott, 1990), but the availability

of support programs for students with learning disabilities

does not keep up with this growth in enrollment. However, in

recent years the number of college support programs is

increasing for several reasons (Strichart & Mangrum, 1986).

First, the initial focus of learning disabilities programs

was on the needs of young children. As the population of

individuals with learning disabilities aged, it became

apparent that learning disabilities could not be cured, but

persisted into adulthood. Consequently, programs for

elementary students with learning disabilities were extended

to middle schools and high schools. The emergence of college

programs for students with learning disabilities is the next

logical step.










Second, many students with learning disabilities are

interested in attending college. White, Alley, Deshler,

Schumaker, Warner, and Clark (1982) reported that 67% of

young adults diagnosed with learning disabilities by their

schools at some point in their elementary or secondary

programming, had plans for postsecondary education.

Membership of the Vocational Committee of the Association for

Children and Adults with Learning Disabilities (Vocational

Committee, 1982) reported that 14% of the adults (responding

to their survey) had tried college and dropped out, 32% were

currently attending college, and 9% had completed bachelor

degrees. Additionally, Mitaug, Horiuchi, and Fanning (1985)

reported that 55-60% of the students with learning

disabilities surveyed planned to pursue postsecondary

education or training.

Third, increased pressure exerted by parents, adult

groups, and concerned professionals has also been a factor in

the emergence of college programs for students with learning

disabilities. Breakthroughs in services for individuals with

disabilities has often been stimulated by the activities of

professional and nonprofessional advocates. Fourth, the

economic reality of declining student enrollments in many

colleges throughout the nation has private and public

institutions searching for ways to increase student

enrollment. Since students with learning disabilities have

been portrayed as one of the largest under-educated group of










persons in our nation (Moss & Fox, 1980), more colleges are

providing programs to attract these students.

Fifth, and perhaps most important, federal legislation

has required that all students with disabilities be given an

opportunity to receive an appropriate education. The

Education for All Handicapped Children Act of 1975 (P.L.

94-142) guaranteed the educational rights of students with

disabilities ages 3-21 through secondary school level.

Section 504 of the Rehabilitation Act of 1973 and the

Americans with Disabilities Act of 1990 address the

postsecondary rights of individuals with disabilities and

prohibit discrimination in admissions, recruitment, and

treatment after admissions (reasonable accommodations).

College has become a goal for many students with

learning disabilities. The issue currently facing the fields

of special education and higher education is how to make this

goal attainable. An important factor in the success of any

college student with learning disabilities is the

availability of appropriate programs and support services.

Program components can include (a) counseling services

(academic, career or vocational), (b) instructional

accommodations (services provided by the college or

adaptations provided by the faculty), and (c) administrative

accommodations (alternative admission criteria and special

remedial courses) (Nelson & Lignugaris-Kraft, 1989). The

following section contains a review of descriptions and










surveys of programs in postsecondary settings for students

with learning disabilities (see Appendix A).

Description of Postsecondary Programs

The membership of the National Joint Committee on

Learning Disabilities (NJCLD) (1986) developed a list of

recommendations for federal, state, and local agencies, as

well as postsecondary and vocational training programs that

were consistent with regulations implementing Section 504.

The membership of the NJCLD suggested that postsecondary

institutions and agencies continue to develop and implement

effective programs that will allow adults with learning

disabilities an opportunity to attain career goals. If

adults with learning disabilities are to gain access to and

profit from postsecondary programs, innovative planning and

cooperation will be necessary to develop programs and

services. The membership of the NJCLD recommended that

persons responsible for postsecondary programs for students

with learning disabilities should establish an advisory

council that will develop, implement, and monitor necessary

procedures and services. Recommended procedures and services

include (a) admissions criteria and procedures;

(b) assessment procedures for determining an individual's

status and needs; (c) guidelines for course selection and

sequences; (d) guidelines on alternative methods for

evaluating a student's learning of course content material;

(e) support systems that will assist with the development of

study skills, reasoning abilities, and decision-making


I










skills, as well as enhancement of listening, speaking,

reading, writing, and mathematical abilities;

(f) modification of instructional methods as indicated;

(g) opportunity for individual and group psychological

assistance; (h) establishment of peer support groups; and,

(i) opportunity for career counseling. Most colleges and

universities offer programs for students with learning

disabilities (Barbaro, 1982; Bireley & Manley, 1980; Gajar,

Murphy, & Hunt, 1982) that lack important components

recommended by the membership of the NJCLD (1986).

Descriptions of single programs. The Wright State

University program for students with learning disabilities

was described by Bireley and Manley (1980). Three students

with learning disabilities participating in the program were

interviewed and followed throughout their college program.

Services offered to students at Wright State University

included (a) counseling (academic and personal) and

(b) instructional accommodations (course tutors, tape-

recorded textbooks, and proctors for special examination

administration). Of the three students participating in the

study, one graduated from Wright State University, one

transferred to another school, and one failed to fulfill the

terms of the contract established with the program

coordinator and was dropped from the program. A significant

weakness of this study was the small number of subjects. The

graduation rate of the program was poor (16%-33%) calling










into question the effectiveness and appropriateness of the

procedures and services offered.

In another study, Barbaro (1982) described the program

for college students with learning disabilities at Adelphi

University. Twenty-two students with learning disabilities

who participated in a 5-week diagnostic program at Adelphi

University were surveyed. Nineteen of the students were

enrolled at Adelphi in the fall. Program components provided

to the students included (a) counseling (diagnostic

assessment, academic, and personnel counseling),

(b) instructional accommodations (full-time tutors), and

(c) administrative accommodations (special admission

procedures and study skills classes funded by a tuition

surcharge). At the end of the first year, 84% of the

students were still enrolled. By the end of the second year,

78% remained. Most students' grade point averages fell

between 2.0 and 3.2. The strength of this study is that it

provided a 2-year evaluation of the program services for

college students with learning disabilities.

In a similar study, the Pennsylvania State University

program for students with learning disabilities was described

by Gajar, Murphy, and Hunt (1982). Twelve students

participating in the Pennsylvania State University program

were surveyed. Program components included (a) counseling

(diagnostic assessment), (b) instructional accommodations

(taped textbooks, typing lessons, and extended examination

periods), and (c) administrative accommodations (study skills










workshop and alternative classes for foreign language

requirements). During the study, two students completed

their degrees, nine students received passing grades, and one

student was not achieving passing grades in all classes in

which the student was enrolled.

Data reported in these studies allow for the conclusion

that services provided to college students with learning

disabilities in the late 1970s and early 1980, generally

included diagnostic assessments, counseling, tutoring, taped

textbooks, extended examination time, modification of college

requirements, and study skills courses. Even though

descriptions of single programs are useful in identifying the

program and service options, a broader picture of program

offerings is important.

National surveys of programs. Four state and national

studies have gathered data from a wide array of service

providers (Blalock & Dixon, 1982; Mangrum & Strichart, 1983a,

1983b; Ostertag, Baker, Howard, & Best, 1982; Parks,

Antonoff, Drake, Skiba, & Soberman, 1987). Data reported in

these studies allow for the conclusion that there is wide

variability across the nation in the types of procedures and

services available to college students with learning

disabilities.

The service providers for students with disabilities in

106 California community colleges were surveyed by Ostertag,

Baker, Howard, and Best (1982). The respondents included

coordinators of services for disabled students (41% of the










programs), instructors in programs for learning disabled

students (27% of the programs), and psychologists,

counselors, or other faculty (26% of the programs). Services

provided by most of the California community colleges

included (a) counseling services (diagnostic assessment;

academic, personal, and career counseling), (b) instructional

accommodations (course tutors, notetakers, textbook readers,

and various classroom accommodations), and (c) administrative

accommodations (extended time to complete course or

graduation requirements, registration services, course

substitutions, and reduced class load). The researchers

indicated that reading, writing, spelling, mathematical

computations, and oral communication were the most difficult

academic skills for college students with learning

disabilities in the California community college system. In

addition, it was reported that no provisions were made by the

California community college system to remediate the various

academic skill weaknesses experienced by the students with

learning disabilities.

A national telephone survey of program administrators

for college programs for students with learning disabilities

was conducted by Blalock and Dixon (1982). Program

administrators were identified in the American Council on

Education bulletin and other directories. The services

provided in the colleges and universities represented by the

respondents included (a) counseling (diagnostic assessment

and personal counseling), (b) instructional accommodations










(notetakers, typing service, course tutors, taped textbooks,

and remediation of reading, spelling, and writing skills),

and (c) administrative accommodations (classes specifically

designed for students with learning disabilities, study

skills classes, waiver of selected program requirements, and

allowed reductions in course load). The respondents

recommended development of assessment procedures for adults

suspected of having a learning disability, but emphasized

that such assessment procedures may be less important than

identifying and developing effective interventions.

Mangrum and Strichart (1983a, 1983b) also surveyed

directors of programs for college students with learning

disabilities and reported results similar to Blalock and

Dixon (1982). The program directors were identified through

a number of national college directories. Services and

program components provided by the respondents to Mangrum and

Strichart's survey included (a) counseling (diagnostic

assessment; academic and personal counseling),

(b) instructional accommodations (peer tutors from courses,

notetakers, tape-recorded textbooks, alternative test

procedures, additional time to complete courses, and

permission to tape faculty lectures), and (c) administrative

accommodations (special admissions procedures; special course

offerings in study skills, learning strategies, time

management, and stress management; late withdrawal from

courses). Data provided by the respondents in the two

national surveys allow for the conclusion that many of the










services and procedures recommended by the membership of the

NJCLD (1986) are available in some programs nationwide, but

not necessarily in any specific program (Barbaro, 1982;

Bireley & Manley, 1980; Gajar, Murphy, & Hunt, 1982).

Parks, Antonoff, Drake, Skiba, and Soberman (1987)

surveyed 223 graduate and professional schools. Surveys were

sent to 132 graduate schools, 46 law schools, 28 dental

schools, and 17 medical schools with a return rate of 32%.

Counseling services were provided by a majority of the

respondents and included diagnostic assessment and

counseling. Instructional support services offered by at

least half of the schools surveyed included course tutors,

instruction in test-taking skills, special instructions on

library use, extra time on tests, and oral instead of written

examinations. Less that half of the schools offered audio-

video resources, word processing, use of microcomputers, help

in preparation for tests, use of calculators, taped responses

to assignments and tests, and instruction in reading,

writing, spelling, and listening skills. Very few schools

offered administrative accommodations, such as special

admissions criteria (3%-18% of the respondents), except that

over half of the schools allowed for extra time to complete

the program of study. The data from this study may not be

representative of undergraduate or community college services

offered to students with learning disabilities because of the

difference in program goals between the two types of

institutions. Most graduate and professional schools,










however, are part of a postsecondary institution with an

undergraduate component and may share the same program

procedures and services. An additional concern in

interpreting the results of the study is the low response

rate (32%).

It is notable that most of the postsecondary

institutions surveyed or described provided (a) counseling

and diagnostic assessment, (b) basic instructional

accommodations, such as course tutors, notetakers, and taped

textbooks, and (c) administrative accommodations, such as

extended time to complete degrees, special admissions

procedures, and reduced class loads. However, few colleges

and universities reported providing courses specifically

designed to remediate academic weaknesses characteristic of

students with learning disabilities (Blalock & Dixon, 1982;

Ostertag et al., 1982; Parks et al., 1987). The emphasis on

instructional and academic accommodations in lieu of remedial

courses or strategy instruction for students with learning

disabilities requires that faculty and administrators be

willing to make the recommended modifications. Without

faculty support in the classroom, the effectiveness of

programs for college students with learning disabilities will

be lessened (Schumaker & Deshler, 1984).

Faculty attitudes. Two studies in which faculty

attitudes and willingness to provide students different

accommodations were assessed (Matthews, Anderson, & Skolnick,

1987; Nelson, Dodd, & Smith, 1990) and allowed for the










determination that most faculty members were willing to make

some basic modifications. The type of accommodations

allowed, however, varied from department to department and

professor to professor. Matthews, Anderson, and Skolnick

(1987) surveyed all faculty members (100) at a small

northeastern public university (65% return rate) and found

that most faculty would allow accommodations such as

extension of deadlines, alternative assignments, extra time

to complete tests, and oral instead of written tests or

assignment presentation. The faculty members generally did

not allow substitution of alternative courses for required

courses and were not willing to provide copies of lecture

notes. The researchers recommended (a) inservice training

for faculty and admissions staff concerning students with

learning disabilities and (b) the development of advisory

committees.

Nelson, Dodd, and Smith (1990) also investigated faculty

attitudes toward providing accommodations to students with

learning disabilities in college and university settings and

reported results similar to Matthews et al. (1987). The

researchers surveyed 107 faculty in colleges of education,

business, and arts and sciences at a northwestern college

(76% response rate). The respondents reported that they

would allow instructional accommodations including tape

recorders, copies of lecture notes, proofreaders for written

assignments, assignment modifications, and testing

modifications. The faculty in the college of education










reported more willingness to provide support services than

the other faculty which may reflect a greater knowledge about

student variance, special education students, and/or

effective teaching practices. The researchers recommended

future research concerning effective and efficient program

components.

The program descriptions and survey findings discussed

above provide for a review of faculty attitudes, procedures,

and services available in schools across the nation.

However, research comparing different postsecondary programs

or evaluating program components is lacking. Accurate

descriptions of the current state of postsecondary program

options are important. The identification of effective and

efficient program components is a priority due to funding

shortages facing many institutions and the variety of

services needed by students with learning disabilities

currently enrolled in colleges and universities.

Learning Strategies

Background on Strategy Instruction

From the late 1940s to the mid 1970s, behaviorism had a

major impact on the advancement of psychological and

educational practices. Beginning with Thorndike, the idea

that behavior was controlled by its consequences had a

powerful impact on American learning theories (Swenson,

1980). The premise was that most human behavior was acquired

or modified through learning procedures or environmental

influences. Theories developed by researchers such as










Pavlov, Hull, Spence, and Skinner were based on data derived

from extensive research efforts with animal subjects.

Complex human and animal behaviors were seen as determined by

environmental contingencies and lawful relationships between

stimuli and responses. All of these theorists attempted to

mold psychology on the model of natural and physical sciences

and strongly opposed explaining behavior in terms of mental

variables or processes. Their theories were most successful

in predicting simple behaviors and were less successful when

applied to complex behaviors (Swenson, 1980).

Behaviorism made a number of assumptions about the way

in which learning occurs (Ashman & Conway, 1989):

1. Behaviors can be defined in instructional terms.

2. Behaviors can be learned and modified.

3. Behaviors are environmentally determined and

reinforced.

The behavioral framework had a profound effect on the

organization of classroom teaching strategies. Highly task-

analyzed curricula were developed to ensure the success of

instruction (Koorland, 1986). Such curricula included

discrete and systematic steps for instruction that, although

designed to impart specific knowledge, often lead to teaching

many discrete splinter skills without the learner

understanding the overall learning process (Ashman & Conway,

1989). The focus of these curricula was on the product and

not the process of learning.







31


There is no doubt that behaviorism has made a

significant contribution to education, especially in the area

of behavior management, but it has become increasingly clear

that organisms do not simply learn by associations from blind

trial and error (Schwartz, 1989). Recent years have seen

increased attention on the role of the learner as an active

participant in the teaching-learning process (Weinstein &

Mayer, 1986). In particular, this focus suggests that the

success of teaching is partly dependent upon what the learner

knows, such as the learner's prior knowledge, and what the

learner thinks about during the learning experience, such as

the learner's active cognitive processing. This change in

thinking about learning had several important implications

for education. Teachers entering the classroom now have two

distinctly different kinds of goals for their students:

(a) goals concerning the product of learning (what students

should know or be able to do as a result of learning) and

(b) goals concerning the process of learning (techniques and

strategies students can use to accomplish learning)

(Weinstein & Mayer, 1986).

This interest in learning strategies is a natural

outgrowth of the change in orientation from behaviorist

theories to cognitive theories of learning. While

behaviorism was built on the stimulus-response theory,

cognitive psychology has filled in the "-" separating

"stimulus-response" with the activities of learning (mental

structures and processes). The cognitive approach to










learning attempts to understand how information is processed

and stored in memory. This cognitive approach has changed

our conception of the teaching-learning process in several

ways (Weinstein & Mayer, 1986). First, learning is viewed as

an active process that occurs within the learner and which

can be influenced by the learner. Second, the outcome of

learning depends jointly on what information is presented and

on how the learner processes that information. Therefore,

there are two types of activities that influence the learning

process, (a) teaching strategies and (b) learning strategies.

Weinstein and Mayer (1986) present a framework for

describing the teaching-learning process:

1. Teacher characteristics--knowledge of how to teach
and content area material required for the teaching
strategy selected.

2. Teaching strategies--teacher's performance during
teaching, including what is taught, how it is
taught, and when it is taught.

3. Learner characteristics--learner's knowledge of
facts, procedures, and strategies required for the
learning strategy selected.

4. Learning strategies--behaviors that the learner
engages in during learning intended to influence
affective and cognitive processing during encoding.

5. Encoding process--internal cognitive processes
during learning, such as how the learner selects,
organizes, and integrates new information.

6. Learning outcome--newly acquired knowledge that
depends on both teaching and learning strategies.

7. Performance--behavior on tests of retention and
transfer. (p. 316)










As seen above, instruction in learning strategies can effect

learners' performance by making specific strategies and

methods available to the learner which can improve the

encoding process and learning outcome. During the 1980s,

strategy instruction became a major focus of educational

research with students with learning disabilities (Graham,

Harris, MacArthur, & Schwartz, 1991). The main premise

underlying strategy instruction was that students could

profit from being taught more effective strategies than they

would discover on their own. The emphasis was placed on

developing effective and efficient learning strategies and

methods of strategy instruction.

Palinscar, David, Wisen, and Stevens (1991) describe six

prominent cognitive strategy instruction models which include

Cognitive Behavior Management, Direct Instruction, Direct

Explanation, Informed Strategies for Learning, Reciprocal

Teaching, and Strategy Intervention Model, and review

research concerning each approach. Cognitive Behavior

Management (CBM) was developed by Dansereau (1978).

Cognitive Behavior Management focuses on teaching students to

regulate their own performance by using an internalized set

of monitoring statements before, during, and after a learning

task. These statements typically focus on problem

definition, attention focusing, self-reinforcing, and self-

coping. The CBM monitoring statements are taught in a manner

similar to Direct Instruction and have been effective with

above average students. However, research has not addressed










flexibility of strategy use, generalized use of strategies,

and changes in attitudes or awareness following instruction

(Palinscar, David, Wisen, & Stevens, 1991).

Direct Instruction (DI), developed by Engelmann and his

associates at the University of Oregon, teaches content

knowledge and strategies in a sequential fashion with

students practicing material as the teacher uses explicit

instruction, development of mastery, elaborated feedback,

gradual fading of prompts, adequate practice, and cumulative

review (Gersten, Carnine, & Woodward, 1987). White (1988)

completed a meta-analysis of research studies on DI in

special education assessing target skills including reading,

math, language, spelling, social, writing, and health. While

none of the outcome measures in the 25 studies reviewed by

White significantly favored the comparison group, 53% of the

outcome measures significantly favored DI. Even though some

educators contend that DI teaches lower-order cognitive

skills (such as word-attack) at the expense of higher-order

skills (such as comprehension) (Palinscar, David, Wisen, &

Stevens, 1991), a follow-up analysis of the study-weighted

mean effect sizes on subdivided reading measures

(comprehension, word-attack, and total reading measures) did

not support such arguments (White, 1988). Research findings

allows for the conclusion that DI is an effective means of

teaching strategic and content knowledge.

The Direct Explanation (DE) strategy model, developed by

Duffy et al. (1987), is unlike the previous models in several










ways. Using DE, the instructor provides declarative (what

the strategy is), procedural (how the strategy is used), and

conditional (when the strategy is used) knowledge. In an

effort to teach academic skills as strategies, the instructor

uses verbal modeling to teach effective strategy use. The

steps used in DE correspond to modeling, guided practice, and

independent practice used in the University of Kansas

approach discussed below. Researchers have shown DE to

improve procedural and conditional knowledge in reading

strategies, but results concerning achievement measures are

mixed (Palinscar, David, Wisen, & Stevens, 1991).

Paris (1986) developed the Informed Strategies for

Learning (ISL) model, a curricular approach to strategy

instruction in reading. Informed Strategies Learning

consists of 20 modules addressing four comprehension

processes: (a) planning for reading, (b) identifying

meaning, (c) reasoning while reading, and (d) monitoring

comprehension. This model uses group dialog to emphasize

personal aspects of strategy use, thereby increasing student

awareness of strategies. Palinscar, David, Wisen, and

Stevens (1991) report that results of research with ISL on

reading comprehension have been less than encouraging.

The Reciprocal Teaching (RT) model was developed by

Brown and Palinscar (1989) and features instruction in four

strategies taught as a set of complimentary activities to be

used flexibly as the text demands and learner needs change.

Unlike other approaches, RT emphasizes collaboration between










the teacher and students rather than direct instruction to

teach strategy use. This model incorporates a questioning

technique that is faded from teacher control to student

control and is very successful in content area classes.

Reciprocal Teaching has significantly increased students'

ability to use targeted strategies and attain higher scores

on comprehension measures. These gains also have been shown

to maintain over time and generalize beyond the instructional

setting.

Deshler and his associates at the University of Kansas

have developed the Strategy Intervention Model (SIM) for use

with learning disabled and low achieving students (Deshler &

Lenz, 1989). The strategies, developed as a part of SIM,

have been proven effective in improving strategy use and

content learning. The SIM approach to learning strategies

instruction has been described as a comprehensive and

realistic model (Pressley, Symons, Snyder, & Cariglia-Bull,

1989). Like DI and CBM, the SIM strategies are taught as a

series of steps. Unlike the other strategies, the SIM

incorporates two steps, the acquisition phase (what the

strategy is, and how and when to use it) and the

generalization phase (application of the strategy in the

regular education setting). Additionally, strategy

instruction is preceded by assessment regarding the student's

current strategy use in comparable situations. The SIM

strategies are taught using modeling and verbal rehearsal,

guided and independent practice with elaborated feedback,










materials controlled for difficulty, and criteria to measure

mastery. The main strengths of the SIM approach and

curriculum are the (a) successful generalization of the

strategies taught, (b) instructor's ability to motivate

students (goal setting activities and charted progress),

(c) resulting intensive and extensive strategy instruction,

and (d) incorporation of effective teaching practices into

the strategy instructional procedure. Research indicates

that sustained instruction is successful in improving

strategy use and content area learning.

Independent student functioning is one of the goals of

strategy instruction and a setting demand of postsecondary

institutions. In the generalization steps of the KU-IRLD

strategies, students are taught to be active agents in the

intervention process by using self-monitoring techniques with

the goal being to develop self-regulated learning and

behavior (Reid & Harris, 1989). Ghatala (1986), in studies

with elementary students, found that students who fail to

monitor the effectiveness of strategies used strategies less

often and less effectively. The general monitoring

difficulties being detected and reported in the literature

and the importance of self-regulation allow for the

conclusion that more effort should be expended in developing

and utilizing interventions that teach and encourage self-

regulated learning (Pressley, Goodchild, Fleet, Zajchowski, &

Evans, 1989). The KU-IRLD strategy model has been

extensively researched with high school students with










learning disabilities and is considered to be one of the most

highly developed strategy models available (Pressley &

Harris, 1991); however, there is little documentation in the

literature as to its effectiveness or appropriateness with

college students.

Research on Writing Strategy Instruction

In writing, strategy instruction has been used as an aid

to assist students in learning to internalize and regulate

the cognitive activities considered central to the effective

planning, production, and revision of written text (Graham,

Harris, MacArthur, & Schwartz, 1991). Writing strategies

developed and used with students with learning disabilities

have concentrated on such diverse areas as self-monitoring of

productivity, planning the text, content generation, text

production skills (such as sentence writing), and editing and

revising.

Graham and Harris (1989) conducted a study on a strategy

for planning and writing essays with three 6th-grade students

with learning disabilities receiving resource room services.

The subjects were taught a text planning strategy (TREE)

using modeling, mastery learning, and generalization

training. After training, all subjects demonstrated

substantial gains in writing performance over baseline

levels. The study was designed and data analyzed using

single subject methodology and no comparison was made with

other writing strategy approaches.










Reynolds, Hill, Swassing, and Ward (1988) evaluated the

effectiveness of (a) Evaluative and Directive Phrases (EDP),

developed by Bereiter and Scardamalia and (b) the Error-

Monitoring Strategy (COPS), developed by KU-IRLD. Reynolds

et al. compared two strategy approaches for revising written

text. Fifty-four students with learning disabilities in the

sixth, seventh, and eighth grades participated in this study.

The study was conducted in four resource rooms for an

unspecified length of time. Classes were assigned to one of

three groups, (a) control (tell to revise with no

instruction), (b) EDP, or (c) COPS. Pretest and posttest

scores were analyzed with an ANOVA (the Tukey HSD procedure

was used for post hoc comparisons). No evaluation of the

students' writing was made prior to intervention. The

researchers reported that both the EDP and COPS groups

improved in mechanics, but that none of the groups improved

in text content.

Welch and Jensen (1990) compared strategy intervention

to a more traditional grammar-based approach and addressed

some of the weaknesses of the studies discussed above.

However, the subjects were not students with learning

disabilities. The researchers identified 114 at-risk

students (failed at least two classes in the past year) from

middle to low socioeconomic levels participating in a summer

school session in a metropolitan school district. Two school

sites (n = 71 and n = 43) were chosen and each site received

a different treatment (strategy instruction and traditional










grammar-based language instruction). The strategy

instruction (P.L.E.A.S.E. strategy) was video-assisted and

focused on paragraph writing skills over a 6-week summer

school session. Data were analyzed by ANOVA and ANCOVA

procedures. Results support the conclusion that students who

received the strategy instruction became significantly more

proficient in paragraph structure and pre-writing planning

than did the students in the traditional grammar-based

language instruction.

In a more recent study, Welch (1992) investigated the

use of the P.L.E.A.S.E. paragraph writing strategy with seven

6th grade students identified as having mild learning

disabilities. This study incorporated a control group

(eleven students receiving traditional writing instruction)

at a different school and matched students in the

experimental and control groups on socioeconomic status and

student characteristics. Data were collected using a student

survey (knowledge of paragraphs and metacognitive processes

related to writing paragraphs), writing samples (pre- and

post- intervention), and an attitude measure (feelings about

ability to write paragraphs). Data analysis was completed

using ANCOVAs and ANOVAs on the surveys, writing samples, and

the attitude measure. Welch reported that the control and

experimental groups were the same in knowledge, paragraph

writing skills, and attitude towards writing on the pretest

measures. On the posttest, the P.L.E.A.S.E. strategy group

significantly improved on the posttest in all three areas,










but the control group did not. In conducting this study,

Welch has addressed most of the concerns of the previous

studies and was able to add to the research base regarding

the positive effect of strategy instruction on improving

writing skills for students with learning disabilities.

Published research on the effectiveness of the PENS

sentence writing strategy is limited. One study completed by

Schmidt (Schmidt, Deshler, Schumaker, & Alley, 1988) was

designed using single subject methodology. No comparison was

made with other writing strategy approaches. Schmidt

conducted the study with the PENS strategy using seven high

school students with learning disabilities. The strategy was

taught as per the procedures and instructions in the

Instructor's Manual. Before training in PENS, the students

averaged 70% complete sentences and 18% complicated

(compound, complex, and compound-complex sentences) sentences

in resource room written products. After instruction and

training in generalization, the students averaged 92%

complete sentences and 42% complicated sentences in resource

room written products and 92% complete and 48% complicated

sentences in regular class written products. The results

lead to the conclusion that PENS is effective in improving

sentence writing skills. However, the question still remains

unanswered: Is PENS better than other available programs?

Researchers have shown that when students with learning

disabilities are taught appropriate task and self-regulation

strategies, student performance improves (Graham & Harris,










1990). Educators and support service providers can promote

good writing by teaching students to use self-instructional

techniques that combine training in the use of composition

and self-monitoring strategies (Levy & Rosenberg, 1990).

Summary

Postsecondary programs for students with learning

disabilities have primarily been developed in the last 20

years. The recent concern about appropriate intervention

programs is due to the increasing number of students with

learning disabilities pursuing postsecondary education in

community colleges, traditional 4-year higher education

institutions, and graduate and professional schools (Nelson &

Lignugaris-Kraft, 1989). Increasing enrollment of students

with learning disabilities has been brought about by changes

in attitudes (student, parent, and teacher) and recent

legislation (Section 504 and ADA). Researchers investigating

procedures and services available to students with learning

disabilities in postsecondary settings have reported a

diversity of programming options and are recommending

research to identify effective and efficient program

components.

Strategy instruction has been identified as an effective

intervention for students with learning disabilities by

various researchers, but there is little research on the

effectiveness of using learning strategies with college

students. Problems identified with research on writing

strategy instruction include (a) small numbers of subjects,






43


(b) lack of comparison with other approaches, (c) no

replications or limited replications of research findings,

and (d) little research with college students with learning

disabilities.















CHAPTER 3
METHODOLOGY


Chapter 3 presents the methods and procedures of the

study. For the purposes of presentation, the chapter has

been divided into five sections. The sections include a

description of (a) the hypotheses, (b) the subjects and

setting, (c) the measurement procedures, (d) the reliability

of procedures, (e) the training of personnel, (f) the PENS

instructional procedures, (g) the experimental design, and

(h) the analysis of the data.

Hypotheses

This study was designed to (a) examine differences

between three different approaches for remediating written

expression deficits of college students with learning

disabilities and (b) investigate the effectiveness of PENS

for remediating written expression deficits of college

students with learning disabilities. Written expression

skills evaluated in the study include fluency, syntactic

maturity (non-sentences, simple/compound sentences and

complex sentences), vocabulary, mechanics, and composition

organization. The following null hypotheses were posited for

testing in the study:

HI: There will be no difference between the three

educational experiences (Gordon Rule courses, Developmental










Writing course, or PENS class) on posttest scores after

adjusting for pretest scores in fluency.

H2: There will be no difference between the three

educational experiences (Gordon Rule courses, Developmental

Writing course, or PENS class) on posttest scores after

adjusting for pretest scores in syntactic maturity.

H3: There will be no difference between the three

educational experiences (Gordon Rule courses, Developmental

Writing course, or PENS class) on posttest scores after

adjusting for pretest scores in vocabulary.

H4: There will be no difference between the three

educational experiences (Gordon Rule courses, Developmental

Writing course, or PENS class) on posttest scores after

adjusting for pretest scores in mechanics.

H5: There will be no difference between the three

educational experiences (Gordon Rule courses, Developmental

Writing course, or PENS class) on posttest scores after

adjusting for pretest scores in composition organization.

Subjects and Setting

The subjects of this study were college students with

learning disabilities at the University of Florida (UF) and

Santa Fe Community College (SFCC). Each subject was eligible

for placement according to the criteria required for

registration as learning disabled with the Office of Disabled

Student Services (ODSS) on the campuses of either UF or SFCC.

Further descriptions of the data are provided in Table 3

(Smith, Deshler, Hallahan, Lovitt, Robinson, Voress, &










Ysseldyke, 1984). Achievement and IQ data was based on test

scores found in psychological reports in the students'

confidential files.

Students were referred for participation in the PENS

class by personnel in ODSS at Santa Fe Community College and

the University of Florida. The criteria for referral to the

PENS class included a history of written expression problems

identified by test data, faculty referral, or self-report by

the student. Letters were sent to all students with learning

disabilities registered with ODSS who were considered

eligible for referral at the University of Florida by ODSS

during the Summer 1991 semester. The letters described the

proposed PENS class and included a proposed course syllabus

(see Appendix B). Placement into the PENS class followed the

screening procedures described below. Each student in the

PENS class signed an informed consent form (see Appendix C).

Additionally, personnel in ODSS identified students with

learning disabilities enrolled in Gordon Rule courses and the

Developmental Writing course and contacted those students by

letter (informed consent form) for possible participation in

the study. The signed consent form (see Appendix D)

described the proposed study and asked the student to

participate by providing writing samples as well as access to

descriptive data in the student's confidential folder.











Table 3
Description of Subjects


PENS Strategy
ClAss


Developmental
Writing Course


Gordon Rule
Courses


male = 10
female = 2
total = 12


mean = 24.9
range = 18-50

mean = 107.8
range = 85-104


male = 7
female = 6
total = 13


mean = 25.6
range = 20-35

mean = 86.6
range = 81-96


male = 4
female = 9
total = 13


mean = 26.8
range = 20-51

mean = 105.4
range = 86-122


Tests used = WAIS-R, WISC-R, TONI, SIT


Math:
Standard Score
mean = 92.0
range = 74-114

Percentile
mean = 36.0
range = 4-70

Grade Equivalent
mean = 9.8
range = 5.9-14.5


mean = 83.5
range = 76-91


mean = 33.0
range = 27-39


mean = 10.6
range = 9.0-12.9


mean = 102.5
range = 85-129


mean = 51.6
range = 16-97


mean = 10.9
range = 8.8-13.0


Tests used = WRAT-R, WRAT, PIAT, TABE, WJPEB, SDRT


Reading:
Standard Score
mean = 94.6
range = 83-122

Percentile
mean = 40.7
range = 13-83

Grade Equivalent
mean = 11.1
range = 7.9-15.6


mean = 69.0
range = 63-75


mean = 3.0
range = 1-5


mean = 6.3
range = 3-10


mean = 86.0
range = 80-92


mean = 10.3
range = 3-19


mean = 8.1
range = 7.3-8.9


Tests used = WRAT-R, WRAT, PIAT, TABE, WJPEB, SDRT

Spelling:
Standard Score


mean = 78.2
range = 65-99

Percentile
mean = 15.6
range = 1-32

Grade Equivalent
mean = 6.4
range = 3.9-8.3

Tests used = WRAT-R, WRAT, PIAT,


mean = 68.0
range = 63-73


mean = 2.5
range = 1-4


mean = 5.4
range = 3.0-8.2

TABE, WJPEB, SDRT


mean = 82.6
range = 74-93


mean = 18.0
range = 4-32


mean = 6.9
range = 5.0-9.9


Subjects:


Age:


Writina Course -










Measurement Procedures

Several measurement procedures were used during the

experiment. The procedures included (a) screening for

placement in the PENS class, (b) scoring the pretest and

posttest writing samples, and (c) obtaining information on

course content and instructional methods for the Gordon Rule

and writing courses.

Screening for PENS Class

Screening for participation in the PENS class was

completed using the pretest component of the PENS sentence

writing strategy provided in the PENS Instructor's Manual.

The pretest requires that students write a paragraph of at

least six sentences using a topic from a list provided. The

writing sample was evaluated on the percentage of complete

sentences, complicated sentences, and complicated sentences

punctuated correctly using the PENS scoring sheet and scoring

directions. To be included in the PENS class, the students

scored less than 100% on complete sentences, 50% on

complicated sentences, and 66% on complicated sentences

punctuated correctly in the PENS pretest writing sample.

Pretest and Posttest

The pretest and posttest evaluations were conducted as

follows. One or two writing samples were collected from each

student participating in the study at the beginning of the

semester (during the first three weeks) and counted as the

pretest. One or two additional writing samples were collected

from each student at the end of the semester (during the last










three weeks) and counted as the posttest. In an effort to

provide a more stable estimate of each student's writing

ability, the researcher attempted to collect two writing

samples collected for both the pretest and posttest measures

instead of one writing sample. These writing samples were

assignments from the respective classes. Assignments in the

PENS class were three to five pages in length, assignments

from the Gordon Rule courses were three to six pages in

length, and assignments from the Developmental Writing course

were one paragraph to one page in length.

The pretest and posttest writing samples were scored by

at least two scorers using the following procedures. Each

student was randomly assigned a number when they agreed to

participate in the study. When the writing samples were

collected, the papers were given to a typist who transcribed

each writing sample as written with no corrections. Each

writing sample was coded with the number assigned to each

subject and was scored in the areas of (a) fluency,

(b) syntactic maturity--nonsentences, simple/compound

sentences, and complex sentences (complex and compound-

complex sentences), (c) vocabulary, (d) mechanics, and

(e) composition organization. Scores for each paper were

recorded on the Scorer Sheet (see Appendix E). Scores for

the pretest and posttest writing samples were averaged to

obtain a single pretest or posttest score for each measure

using the Score Calculation Form (see Appendix F).










Writing samples were evaluated in the areas of fluency,

syntactic maturity, vocabulary, mechanics, and composition

organization. Fluency, syntactic maturity, vocabulary, and

correct word sequences are included in most objective scoring

systems of written work; composition organization is an often

used subjective assessment of written work (Tindal & Marston,

1990). Evaluation of writing samples in the areas mentioned

above will allow for a comprehensive assessment of written

expression skills (not just sentence writing skills) and will

enable the researcher to better evaluate the effectiveness of

each type of instruction in improving the written expression

abilities of college students with learning disabilities.

Fluency. Fluency has been found to be highly correlated

with performance on both norm-referenced tests of written

expression and teacher judgments of quality in written

expression in the elementary grades and is thought to

increase with educational level (Deno, Marston, & Mirkin,

1982). The reliability of this measure of written expression

skills is considered to be quite high (Marston & Deno, 1981).

The total number of words in each writing sample was

determined by completing a word count for each writing sample

using the Microsoft Word word processing program on the

Macintosh computer. Fluency is measured by determining the

average sentence length (Mercer & Mercer, 1989):

Number of total words
Number of sentences










Syntactic maturity. Syntactic maturity is the extent to

which individuals use expanded and more complex sentences

(Issacson, 1985) and has been shown to increase throughout

the school years (Hunt, 1977). Syntactic maturity can be

measured by counting the number of sentences that fall into a

given category (Marston & Tindal, 1990) including

(a) nonsentence--a grammatically incorrect sentence

(fragment, run-on, nonsensical), (b) simple sentence--one

independent clause, (c) compound sentence--has two

independent clauses, and (d) complex sentence--has an

independent clause and at least one dependent clause.

Syntactic maturity was measured by finding the percentage of

nonsentences, simple/compound sentences, and complex

sentences:


Number of nonsentences X 100
Total number of sentences

Number of simple and compound sentences X 100
Total number of sentences

Number of complex sentences X 100
Total number of sentences


Vocabulary. Vocabulary scoring focuses on the

uniqueness or maturity of words used in a composition and can

be measured by counting the number of large words contained

in the writing sample. In an examination of written

compositions, Deno, Marston, and Mirkin (1980) found moderate

correlations ranging from .47 to .60 between the number of

large words and other criterion measures (Test Of Written










Language, Developmental Sentence Scoring, and The Stanford

Language Subtest). The long word method of calculating

vocabulary is also used in the Test of Written Language-2 as

a measure of vocabulary maturity (Hammill & Larsen, 1988).

The vocabulary score is calculated by finding the number of

different words in the writing sample that have seven or more

letters (excluding -s, -ing, and -ed endings when adding them

would make the root word have seven letters if it did not

before) (Deno, Marston, & Mirkin, 1980). Scoring of

vocabulary is the percentage of large words used in the

writing sample:

Number of large words X 100
Total number of words

Correct word sequences. Spelling, punctuation, and word

usage skills can be assessed by computing the number of

correct word sequences (CWS) in a writing sample. Therefore,

using CWS provides a measure of the mechanical conventions

that makes a composition grammatically correct and

presentable to others. Correct word sequences have been

found to have a high relationship with fluency, words spelled

correctly, holistic composition ratings, and word scores

weighted according to developmental level (Tindal & Marston,

1990). The number of CWS is found by evaluating each pair of

two adjacent words (a word sequence). To be scored as a

correct word sequence, each pair of words must be spelled

correctly and must be acceptable within the context of the

phrase to a native English speaker (have appropriate










punctuation between them, demonstrate appropriate subject-

verb or noun-pronoun agreement, and be capitalized if

necessary) (Tindal & Marston, 1990). Single words beginning

and ending a sentence are counted as correct if they are

spelled correctly, capitalized correctly, and if the sentence

has the correct end punctuation. Scoring of CWS is the

percent of the CWS in the writing sample:

Number of CWS X 100
Number of Word Sequences

Organization. Composition organization focuses on the

use of topic sentences, supporting sentences, and concluding

Statements to organize or structure the writing sample. The

evaluation of composition organization will be evaluated by

assigning points to each writing sample using the following

guidelines (Tindal & Marston, 1990):

5--The writer uses paragraphs that are not only intact

in terms of sentences within them, but also in terms of

the flow from one paragraph to the next. Structure is

well developed; ideas are expressed in a systematic

fashion. The paper starts at a good point, has a sense

of movement, gets somewhere, and then stops.

4--Paragraphs are present, topical structure and

supporting detail is well developed, and concluding

sentences are used.

3--Paragraphs are present and have some structure, but

are not well connected. The organization of the paper

is standard and conventional. There is usually a one-










paragraph introduction, three main points treated in

each paragraph, and a conclusion that often seems tacked

on or forced.

2--Paragraphs are present, but lack a topic sentence;

sentences within a paragraph are marginally related to

each other.

1--The writer has no paragraph structure. The main

points are clearly separated from one another and they

come in a random order as though the writer had not

given any thought to what he or she intended to say

before starting to write.

Course Content and Instructional Methods

All undergraduate students in the State of Florida are

required to complete nine semester hours in Gordon Rule

courses. Gordon Rule courses are any approved course within

the university system requiring students to write 2500 words

during the semester. Courses are typically in the English

department, but other departments also offer Gordon Rule

courses, such as History and Psychology. The Gordon Rule

requirement was instituted in an effort to improve the

writing skills of all college graduates. Faculty teaching

students involved in this study were sent a faculty

questionnaire (see Appendix G). Twelve English, one History,

and one Psychology instructor were sent questionnaires. Four

English (33% return rate), one History (100%), and one

Psychology (100%) instructor returned their questionnaire.

Responses to the questionnaire indicated that all four of the










responding English instructors addressed planning, editing,

and organizational (theme or paragraph) skills through direct

instruction and feedback on student papers. Three of the

English instructors also addressed mechanics (spelling,

capitalization, and punctuation), correct word usage

(subject-verb or noun-pronoun agreement), and sentence

writing skills through direct instruction and feedback on

student papers. Two of the English instructors also reported

covering vocabulary through direct instruction and feedback

on student papers. The history instructor reported using

direct instruction for organizational skills, but feedback on

papers for mechanics (spelling, capitalization, and

punctuation), correct word usage (subject-verb or noun-

pronoun agreement), planning, editing, vocabulary, and

sentence writing skills. The psychology instructor reported

using feedback on papers to address mechanics (spelling,

capitalization, and punctuation), correct word usage

(subject-verb or noun-pronoun agreement), planning,

vocabulary, and organizational skills (theme or paragraph).

Students at SFCC are referred to the Developmental

Writing course because of poor scores on admissions tests or

students may elect to enroll in the Developmental Writing

course because of problems in meeting the demands of their

required courses. Students enrolled in the Developmental

Writing course attend both a large lecture class, where

material is introduced, and a small class, where lecture

material is reviewed and writing samples are evaluated. Five










Developmental Writing course instructors, who were teaching

students participating in this study, were sent a faculty

questionnaire (see Appendix G). The questionnaire was

returned by two instructors (40% return rate). Discussion

with two instructors not returning their questionnaire

revealed that they agreed with the responses on the returned

questionnaires. Responses on the questionnaires indicated

that the Developmental Writing course specifically addresses

(a) mechanics (spelling, capitalization, and punctuation),

(b) correct word usage (subject-verb or noun-pronoun

agreement), (c) planning, (d) editing, (e) organization

skills (theme or paragraph), and (f) sentence writing skills,

through both direct instruction and feedback on papers.

Reliability of Procedures

Several procedures were used during the study to

establish the reliability of measurement and instructional

procedures. Interscorer agreement and procedural reliability

checks were implemented to obtain reliability information.

Interscorer Agreement

To establish interscorer agreement, all of the pretest

and posttest writing samples were scored by at least two

trained scorers using the same scoring procedures (described

above). Pearson correlation coefficients were calculated

between the scorers on all writing samples.

Generalizability Study

Additionally, a generalizability study (G-study) was

conducted to obtain a type of reliability coefficient










appropriate when multiple raters are used in the decision

study. Information gathered from the G-study was used to

evaluate whether the number of raters were sufficient to

obtain generalizable findings. Generalizability coefficients

were calculated for each measure on groups of two, three,

and/or four scorers depending on the number of scorers rating

each measure. Results from One-way Repeated Measures ANOVAs

using a raters by subjects design were used to calculate the

generalizability coefficients using the following formula

(Crocker & Algina, 1986, p. 167):


MSp MSr
MSp + (n n')MSr/n'

Procedural Reliability

All PENS strategy instruction was conducted by the

researcher or a trained strategy instructor. To ensure

procedural reliability, two independent observers randomly

observed five instructional sessions to ascertain whether

instructional procedures used with the strategy instruction

were consistent between different groups of subjects and

different instructors. The observers had already been

trained in the KU-IRLD strategies, so they only received

training in administration of the probes.

The observers recorded teacher behaviors in the areas of

(a) following the content of the script, (b) following the

sequence of instructional procedures, and (c) using the

instructional materials appropriately. An interval recording










system was used to measure the teachers' behaviors. While

the instruction was being delivered by the teacher, the

observers listened to a cassette tape designed to beep at

30-second intervals. Each time the beep was heard, the

observer recorded either whether the instructor was engaged

in appropriate or inappropriate behaviors on the form to be

provided (see Appendix H). The tape allowed for 40

observation intervals of 30 seconds each.

Behaviors were rated as (a) appropriate,

(b) inappropriate, or (c) not applicable and were reported as

percentages using the following formulas:

Number of appropriate behaviors X 100
40

Number of inappropriate behaviors X 100
40

Number of not applicable behaviors X 100
40


Training of Personnel

Prior to beginning the study, a student assistant was

trained in the PENS instructional procedures. The two

independent observers were trained during the first month of

the study to provide procedural reliability checks.

Training of Student Assistant

The student assistant acted as an instructor for the

PENS class. Before the PENS class began, the student

assistant was trained in the PENS strategy instructional

procedures. The student assistant was introduced to the PENS










strategy, the sequence of instruction, and the procedures for

scoring the PENS writing samples. The researcher is a

certified trainer of the KU-IRLD strategies and conducted the

2-hour training session using demonstration and modeling of

the PENS procedures as described in the trainer's manuals.

Training of Observers

The two independent observers were provided with

training in using the cassette tape recorder with the audible

beeps and in recording the teacher's behavior on the interval

form. Since the two observers had already been trained in

the KU-IRLD strategies, they did not require training in the

PENS strategy procedures.

Training the Scorers for Reliability Checks

Four independent scorers were selected to score the

pretest and posttest writing samples. The researcher served

as a fifth scorer for reliability purposes only. One of the

independent scorers evaluated all of the writing samples in

each area and those scores were used in the data analysis.

All scorers were provided with a training session (up to 1

hour) on how to score the writing samples in fluency,

syntactical maturity, vocabulary, mechanics, and

organization. Scoring methods for each area as discussed in

this chapter were demonstrated, criteria for decisions were

explained, and scoring methods were practiced. The

researcher provided the training and was available for any

questions regarding the scoring procedures during the scoring

process.










PENS Instructional Procedures

The sentence writing strategy was taught in four parts,

including instruction in (a) simple, (b) compound,

(c) complex, and (d) compound-complex sentences. Each

sentence type was taught using an 8-step instructional

procedure.

Step 1--Pretest and obtain commitment to learn. Before

any instruction began, a pretest was given to obtain a

measure of each student's sentence writing skills (the

pretest component of the PENS writing strategy). After

completing the assignment, students were informed of their

strengths and weaknesses relative to their sentence writing

ability and a verbal commitment to improve their skills was

obtained. Instruction in each of the four sentence types

followed the seven-step instructional sequence described

below.

Step 2--Describe. The PENS strategy was described and

students were provided rationales showing them how they can

benefit from mastering each part of this strategy. Once the

students became motivated to learn the PENS strategy, they

set goals with regard to the pace of instruction.

Step 3--Model. In the model step, the instructor

demonstrated the application of the PENS strategy in writing

different kinds of sentences while explaining the thought

processes involved in completing the strategy, so that the

students can witness the necessary cognitive processes as










well as the overt behaviors involved in performing the

strategy.

Step 4--Verbal Rehearsal. Before students were asked to

use the strategy, they were asked to learn certain concepts

and strategy steps at an automatic level. During the verbal

rehearsal step, students were asked to verbally rehearse

definitions of important terms and strategy steps. Practice

was carried out in small groups using practice techniques

where students were asked to recite the strategy steps

rapidly. Each student was quizzed on the strategy procedures

to determine mastery, which was required before moving on to

Step 5.

Step 5--Controlled Practice and Feedback. The students

were asked to use the strategy in writing sentences using

specific lessons provided in the PENS Instructor's Manual.

Students were provided individual feedback each time they

practiced the strategy. Student progress was charted on

graphs provided in the PENS Instructor's Manual. Before

moving on to Step 6, each student continued to practice the

strategy until mastery was achieved.

Step 6--Grade-Appropriate Practice/Posttest and

Feedback. In Step 6, students practiced in the use of PENS.

Assignments were similar to those assigned in a regular

educational setting. The last practice at the grade-

appropriate level served as the posttest for the type of

sentence being mastered. The instructor faded prompts and

cues given in the earlier stages. Feedback was given after










each practice and involved asking the students to analyze the

appropriateness of their performance. Before moving on to

Step 7, each student continued to practice the strategy until

mastery is achieved.

Step 7--Obtain Commitment to Generalize. Step 7

involved a reviewing of student progress as well as obtaining

student commitment to use the PENS strategy in a variety of

settings.

Step 8--Generalization. Students (a) were made aware of

new situations and circumstances where the PENS strategy can

be used, (b) practiced using the strategy in a wide variety

of settings by completing assignments requiring the

application of the strategy in a broad array of settings, and

(c) reported strategy use periodically by completing self-

report forms.

Experimental Design

The design of the study was a nonrandom pretest-posttest

group statistical design (see Table 4) (Cook & Campbell,

1979). The study was nonrandom because the students were

enrolled in classes and could not be randomly assigned to

treatment conditions. Pretests and posttests were used to.

determine the effectiveness of the three instructional

classes available to college students with learning

disabilities. A control group was not used in the study

because most students with learning disabilities experiencing

difficulties in college are receiving some type










Table 4
Study Design


PENS class










Gordon Rule courses










Developmental
Writing course


Pre Post

X1 0 X1
X2 0 X2
X3 0 X3

X4 0 X4
X5 0 X5
X6 0 X6
X7 0 X7


X1 0 X1
X2 0 X2
X3 0 X3

X4 0 X4
X5 0 X5
X6 0 X6
X7 0 X7



X1 0 X1
X2 0 X2
X3 0 X3

X4 0 X4
X5 0 X5
X6 0 X6
X7 0 X7


(fluency)
(nonsentences)
(simple and compound
sentences)
(complex sentences)
(vocabulary)
(mechanics)
(organization)


(fluency)
(nonsentences)
(simple and compound
sentences)
(complex sentences)
(vocabulary)
(mechanics)
(organization)



(fluency)
(nonsentences)
(simple and compound
sentences)
(complex sentences)
(vocabulary)
(mechanics)
(organization)


of service and it would be inappropriate to deny services to

students with disabilities needing assistance.

A group statistical design was chosen for two reasons.

First, the exploratory research on the KU-IRLD learning

strategies using single subject methodology has lead to the

conclusion that this strategy approach is an effective

intervention for secondary students with learning










disabilities. The next logical step in researching the

KU-IRLD strategies is to use larger numbers of students,

requiring a group statistical design. Second, the

effectiveness of the KU-IRLD strategies needs to be compared

to the effectiveness of other instructional options. Such a

comparison is best accomplished using a group statistical

procedure with a large number of subjects. For example, a

study reported by Welch and Jensen (1990) used a group design

with a large number of subjects to evaluate the effect of the

three interventions.

Analysis of the Data

A MANCOVA was computed to determine whether differences

existed among the levels of the instructional experiences

(Gordon Rule courses, Developmental Writing course, or PENS

class) with respect to combinations of the measures of

written expression (fluency, syntactic maturity, vocabulary,

mechanics, and organization) (Kennedy & Bush, 1985). The .05

level of significance was used as the basis for rejection of

the null multivariate hypothesis. The independent between

group variable was the type of instructional experience and

the within group variable was the occasion. The dependent

variables were the scores on fluency, syntactic maturity

(nonsentences, simple/compound sentences, and complex

sentences), vocabulary, mechanics, and organization.

Summary

The subjects of this study were college students with

learning disabilities registered with the Office of Disabled










Student Services (ODSS) on the campuses of the University of

Florida and Santa Fe Community College, in Gainesville,

Florida. Subjects on both campuses were referred for

strategy instruction by personnel in the ODSS. Students

enrolled on both campuses in Gordon Rule courses and

Developmental Writing courses were identified for

participation in the study. The design of the study was a

nonrandom pretest-posttest design with comparison groups

consisting of a PENS class, Developmental Writing course, and

Gordon Rule courses. A MANCOVA was computed to determine

whether differences exist among the levels of the

instructional experiences.















CHAPTER 4
RESULTS


This chapter presents the data gathered to answer the

experimental questions. First, descriptive analyses of the

data are reported. Second, the inferential statistical

analyses of the data are provided. Third, the procedural and

interscorer reliabilities are reported. Fourth, results of

the generalizability study are stated. Fifth, a summary is

given.

Descriptive Analysis of the Data

During the beginning of the semester and again at the

end of the semester, subjects in the three treatments were

asked to complete writing samples within their educational

setting (PENS class, Developmental Writing course, or Gordon

Rule courses). These samples were scored on the measures of

fluency, syntactic maturity, vocabulary, mechanics, and

composition organization. Table 5 presents the means,

standard deviations, and the ranges of the scores observed

for each measure by treatment group. Results of inferential

statistical tests for differences in observed means are

presented in subsequent sections.

Fluency

On the pretest writing samples, PENS students were

writing sentences averaging 17.26 words in length,











Table 5
Means. (Standard Deviations), and fRangel of Pretest and


t-h FiTvt Writ-inn MnP1nSre5


PENS Developmental Gordon Rule
Measure Occasion Class Writing Course Courses


Fluency


pre


post


Nonsentences


pre


post


Simple/
Compound
Sentences


pre


post


Complex
Sentences


pre


post


Vocabulary


pre


post


Mechanics


pre


post



Organization pre



post


17.26
(3.96)
[10.8 24.2]

18.07
(4.51)
[13.1 27.9]

14.57
(10.68)
[0.0 33.8]

11.88
(12.05)
[0.0 36.7]

51.16
(10.94)
[36.6 72.2]

43.85
(11.58)
[26.7 64.3]

34.85
(6.97)
[26.9 47.7]

44.16
(10.17)
[25.0 58.2]

10.38
(2.71)
[4.0 13.3]

13.94
(4.83)
[9.5 27.3]

85.80
(6.32)
[75.9 94.6]

85.73
(7.41)
[71.2 94.0]

3.04
(0.75)
[1.5 4.0]

3.12
(0.60)
2.0 4.0


16.00
(7.13)
[8.7 34.4]

18.18
(5.33)
[9.5 29.9]

25.62
(28.75)
[0.0 100.0]

19.23
(16.74)
[0.0 47.7]

50.31
(22.45)
[0.0 92.9]

50.98
(18.74)
[13.4 92.9]

24.17
(16.63)
[0.0 42.9]

29.76
(16.47)
[0.0 57.1]

9.42
(3.75)
[3.7 14.9]

8.9
(3.99)
[5.0 19.8]

82.09
(8.86)
[65.3 96.5]

85.06
(8.05)
[69.4 97.4]

2.76
(0.56)
[2.0 4.0]

2.65
(0.51)
2 n 5


20.23
(3.71)
[15.0 27.8]

21.52
(4.67)
[15.0 27.8]

15.17
(12.67)
[0.0 41.7]

7.56
(8.65)
[0.0 30.0]

49.00
(13.18)
[11.1 63.6]

50.07
(19.89)
[0.0 70.3]

35.48
(13.13)
[18.2 55.7]

42.41
(15.55)
[27.9 85.7]

11.39
(3.30)
[5.0 15.8]

10.24
(2.42)
[6.7 15.7]

92.66
(4.54)
[82.9 87.8]

93.55
(4.11)
[84.5 98.6]

2.92
(0.64)
[2.0 4.0]

2.88
(0.46)
2.0 4.0


1) 1- f- f- qr -n-rsmq nn ----' ---"--


. .


P~E+fP9+ ~PA~P~ ~n










Developmental Writing course students were writing sentences

averaging 16.00 words in length, and Gordon Rule students

were writing sentences averaging 20.23 words in length. On

the posttest writing samples, PENS students were writing

sentences averaging 18.07 words in length, Developmental

Writing course students were writing sentences averaging

18.18 words in length, and Gordon Rule students were writing

sentences averaging 21.52 words in length. The subjects in

the Gordon Rule courses scored higher in the use of

vocabulary than the subjects in either of the other treatment

groups at the pretest and maintained their advantage on the

posttest; however, there appears to be little difference in

the amount of improvement between the treatment groups on the

Vocabulary measure.

Syntactic Maturity

The three measures in Syntactic Maturity (nonsentences,

simple/compound sentences, and complex sentences) are closely

related to each other, and a change in one measure requires a

change in another. These measures are percentages of the

total number of sentences in the sample and scores on the

three measures total 100% for each subject. Improvement in

Syntactic Maturity would be noted by a reduction in

nonsentences and an increase in either simple/compound

sentences, complex sentences, or both.

On the pretest writing samples, students in the PENS

class were writing an average of 14.57% nonsentences, 51.16%

simple and compound sentences, and 34.85% complex sentences.










Students in the Developmental Writing course were writing an

average of 25.62% nonsentences, 50.31% simple and compound

sentences, and 24.17% complex sentences. Students in the

Gordon Rule courses were writing an average of 15.17%

nonsentences, 49.00% simple and compound sentences, and

35.48% complex sentences. On the posttest writing samples,

students in the PENS class were writing an average of 11.88%

nonsentences, 43.85% simple and compound sentences, and

41.16% complex sentences. Students in the Developmental

Writing course were writing an average of 19.23%

nonsentences, 50.98% simple and compound sentences, and

29.76% complex sentences. Students in the Gordon Rule

courses were writing an average of 7.56% nonsentences,

50.07% simple and compound sentences, and 42.41% complex

sentences. Students in all three treatments appeared to

reduce the number of nonsentences and increase the number of

complex sentences. The subjects in the PENS class made the

largest mean gain in increasing Complex Sentences and

subjects in the Gordon Rule courses made the largest mean

gain in reducing nonsentences. Notable also is the reduction

in range of percentage of nonsentences written (from 0.00-

100.0 to 0.00-47.7) for the subjects in the Developmental

Writing course from the pretest to the posttest. The

Developmental Writing course appears to be more successful

with students writing a large number of nonsentences.










Vocabulary

On the pretest writing samples, PENS students were using

an average of 10.38 large (7 or more letters) words,

Developmental Writing course students were using an average

of 9.42 large words, and Gordon Rule students were using an

average of 11.39 large words. On the posttest writing

samples, PENS students were using an average of 13.94 large

words, Developmental Writing course students were using an

average of 8.93 large words, and Gordon Rule students were

using an average of 10.24 large words. Students in the PENS

class appeared to make gains in Vocabulary, while students in

the Developmental Writing course and Gordon Rule courses did

not.

Mechanics

On the pretest writing samples, students in the PENS

class were writing an average of 85.80% correct word

sequences, students in the Developmental Writing course were

writing an average of 82.09% correct word sequences, and

students in the Gordon Rule courses were writing an average

of 92.66% correct word sequences. On the posttest writing

samples, students in the PENS class were writing an average

of 85.70% correct word sequences, students in the

Developmental Writing course were writing an average of

85.06% correct word sequences, and students in the Gordon

Rule courses were writing an average of 93.55% correct word

sequences. The subjects in the Gordon Rule courses scored

higher in the use of correct mechanics than subjects in










either of the other treatment groups at the pretest and

maintained their advantage on the posttest; however, there

appears to be little difference in the amount of improvement

between the treatment groups on the Mechanics measure.

Composition Organization

On the pretest writing samples, PENS students were

writing papers that were generally average in organization

(3.04), Developmental Writing course students were writing

papers that were generally average (2.76) in organization,

and Gordon Rule students were writing papers that were

generally average (2.92) in organization. On the posttest

writing samples, PENS students were writing papers that were

generally average in organization (3.12), Developmental

Writing course students were writing papers that were

generally average (2.65) in organization, and Gordon Rule

students were writing papers that were generally average

(2.88) in organization. A visual inspection of the means,

standard deviations, and ranges suggests there was little

difference between the three treatment groups in

Organization.

Inferential Statistical Analysis of the Data

PENS Writing Strategy Subjects

All subjects participating in the Gordon Rule courses

and Developmental Writing course attended Santa Fe Community

College (SFCC), but subjects in the PENS class were from both

the University of Florida (UF) and Santa Fe Community

College. Because these two groups of subjects were used as










one treatment group (the PENS class), t-tests were computed

for the pretest scores to rule out the possibility that the

UF and SFCC subjects in the PENS class were different before

instruction began. Separate t-tests were computed for each

measure (fluency, nonsentences, simple/compound sentences,

complex sentences, vocabulary, mechanics, and organization)

with a .01 level of significance. The null hypotheses stated

that the mean scores for the UF and SFCC subjects were not

significantly different on each of the measures. Two-tailed

t-tests were used since the direction of the possible

differences in the means was not known. None of the null

hypotheses could be rejected. Furthermore, the p-values

associated with the t-tests indicated that significant

differences between UF and SFCC subjects would not have been

detected even if a more liberal alpha level had been

specified. Table 6 lists the individual t-tests.

Hypotheses

In this pretest-posttest design the independent between

group variable was the type of instructional experience

(Gordon Rule courses, Developmental Writing course, or PENS

class). The dependent variables were the posttest scores on

fluency, syntactic maturity (nonsentences, simple/compound

sentences, and complex sentences), vocabulary, mechanics, and

organization. The pretest scores on these variables served

as covariates. To evaluate significance of differences among

means on the dependent variables the researcher tested five

statistical null hypotheses which are restated here.










HI: There will be no difference among the groups

receiving the three educational experiences (Gordon Rule

courses, Developmental Writing course, and PENS class) on

posttest scores after adjusting for pretest scores in

fluency.

H2: There will be no difference among the groups

receiving the three educational experiences (Gordon Rule

courses, Developmental Writing course, and PENS class) on

posttest scores after adjusting for pretest scores in

syntactic maturity.

H3: There will be no difference among the groups

receiving the three educational experiences (Gordon Rule

courses, Developmental Writing course, and PENS class) on

posttest scores after adjusting for pretest scores in

vocabulary.

H4: There will be no difference among the groups

receiving the three educational experiences (Gordon Rule

courses, Developmental Writing course, and PENS class) on

posttest scores after adjusting for pretest scores in

mechanics.

H5: There will be no difference among the groups

receiving the three educational experiences (Gordon Rule

courses, Developmental Writing course, and PENS class) on

posttest scores after adjusting for pretest scores in

composition organization.










Table 6
Individual t-tests for the UF and SFCC Subjects in the PENS
Class


Degrees
Measure of Freedom

Fluency 10

Nonsentences 10

Simple/Comp.Sentences 10

Complex Sentences 10

Vocabulary 10

Mechanics 10

Organization 10


t Statistic

-1.084

.3185

-.6234

.8864

-.0072

-.4663

-.9447


The multivariate hypothesis was analyzed with a MANCOVA

procedure to determine whether there was a treatment group

effect on the five criterion measures after adjustment on the

five covariates pretestt scores). This hypothesis was not

rejected based on a Pillai's Trace F(14,46) = 1.27 (r =

0.1771). When all five dependent measures were evaluated as

a vector, none of the educational treatments were more

beneficial in improving writing skills for college students

with learning disabilities after adjustment for the pretest

scores.

Exploratory Analysis

Exploratory analyses were conducted even though the

MANCOVA failed to produce significant findings because of the

possibility of collinearity, the loss of degrees of freedom


.3038

.7567

.5470

.3962

.9944

.6510

.3671










due to the effect of the large number of covariates, and the

possibility of aptitude by treatment interaction that could

mask a treatment effect. Additional variables that could

have contributed to the lack of significant treatment

differences include the small number of subjects, the

moderate correlation of the covariates pretestt scores), and

the possibility of a salutary treatment effect.

Aptitude by treatment interaction. Univariate ANCOVA

analyses were run on each dependent measure to evaluate

whether there was an interaction between the treatment groups

and the occasion. An .01 level of significance was used to

evaluate these analyses due to the number of comparisons

being made (n = 5). The test for significant effect of the

covariate by treatment interaction was not significant on

Fluency [F(2,32) = 0.96; p = .3945], Nonsentences [F(2,32) =

0.26; p = .7700], Simple/Compound Sentences [F(2,32) = 0.45;

p = .6407], Complex Sentences [F(2,32) = 2.38; 2 = .1092],

Vocabulary [F(2,32) = 0.28; p = .7595], Mechanics [F(2,32) =

0.06; 1 = .9416], and Organization [F(2,32) = 0.17; p =

.8430]. When each dependent measure was analyzed separately,

none of the educational treatments interacted with the

initial levels of writing on that variable to affect the

students' performance on outcome measures. These tests for

significant aptitude by treatment interactions also were

tests for violations of the assumption of homogeneity of

slopes for the regression of outcomes on pretest measures and










must be completed prior to evaluation of the ANCOVAs without

the aptitude by treatment interaction.

Analysis of each dependent measure. Univariate ANCOVA

analyses were completed on each dependent measure without

using the aptitude by treatment interaction to evaluate

whether there was a difference among the treatment groups on

outcome measures adjusted for pretest scores (see Table 7).

An .01 level of significance was used to evaluate these

analyses due to the number of comparisons being made. The

test for significant treatment effect between groups on

posttest scores adjusted for pretest scores was not

significant on (a) Fluency [F(2,32) = 2.10; p = .1383];

(b) Syntactic Maturity -- Nonsentences [F(2,32) = 3.24;

p = .0517], Simple Sentences [F(2,32) = .60; p = .5623], and

Complex Sentences [F(2,32) = 3.86; = .0309]; and

(c) Organization [F(2,32) = 3.75; p = .0338]. Significance

was found on the pretest measures for Nonsentences [F(2,32) =

7.93; = .0080], Mechanics [F(2,32) = 16.58; p = .0003], and

Organization [F(2,32) = 19.18; 1 = .0001], indicating that

the pretest measures were useful covariates. Significant

differences were found among the treatment group

means on Vocabulary [F(2,32) = 5.64; p = .0077] and Mechanics

[F(2,32) = 9.14; p = .0007].

Follow-up tests were conducted for the Vocabulary and

Mechanics scores using the least squares means and a

Bonferroni alpha correction of an .003 significance level

(.01/3) to identify between which groups the mean differences











Table 7
ANCOVA Analysis of the Five Measures


Dependent
Variable

Fluency


Group

Pre

Error


98.42

29.97

925.96


Nonsentences


Group

Pre

Error

Simple Sentences

Group

Pre

Error

Complex Sentences

Group

Pre

Error

Vocabulary

Group

Pre

Error


Mechanics


Organization


Group

Pre

Error


Group

Pre

Error


* significant at the .01


49.21

29.97

23.45


2.10

1.28


.1383

.2662


905.16

1108.46

6767.98



370.89

7.67

10434.97


1574.80

360.84

6937.37



166.43

17.58

501.90



573.67

520.34

1066.96


452.58

1108.46

139.83



185.44

7.67

306.91


787.4

360.84

204.04



83.21

17.58

14.76



486.83

520.34

31.38


0.69

3.54

0.18


3.24

7.93




.60

.02


3.86

1.77


5.64

1.19


9.14

16.58


3.75

19.18


.0517

.0080*




.5623

.8753


.0309

.1924


.0077*

.2827


.0007*

.0003*


.0338

.0001*


1.38

3.54

6.28


level










occurred (see Table 8). No significant group mean

differences were found on the follow-up tests for Mechanics

and Vocabulary, indicating that no single pairwise contrast

contributed to the significant finding; however, two

follow-up tests approached significance for Vocabulary (PENS

subjects compared to Developmental Writing subjects and PENS

subjects compared to Gordon Rule subjects) and would have

been significant if a less conservative alpha level had been

chosen. PENS subjects appeared to make gains in Vocabulary

(from a mean of 10.38 large words to a mean of 13.94 large

words) while subjects in the Developmental Writing course

(from a mean of 9.42 large words to a mean of 8.9 large

words) and the Gordon Rule courses (from a mean of 11.39

large words to a mean of 10.24 large words) did not appear to

make improvements in Vocabulary.

Pretest and posttest correlations. In order to evaluate

whether multivariate or univariate statistics were needed, the

correlations of the pretest measures to each other and the

posttest measures to each other were obtained (see Table 9).

Since many of the correlations were moderately high, the

multivariate analysis was appropriate.

Pretest to posttest correlations. The stability of the

measures obtained was evaluated using correlations between

the pretest and posttest scores on the measures for all of

the subjects together and for the subjects by treatment

group. The correlations of pretest to posttest scores for

the subjects as a whole group ranged from -.33 to .68










Table 8
Follow-up Test p-values for Comparisons Between Groups Using
the Least Squares Means

Bonferroni Correction alpha = .003

Measure Group PENS DW GR

Vocabulary

PENS .0039 .0165
DW .5733
GR

Mechanics

PENS .5406 .1146
DW .3416
GR



Table 9
Pretest and Posttest Correlations


Occasion Measure F NS SS CS V M O

Pretest F .449 -.676 .154 -.063 .021 -.051
NS -.729 -.565 -.477 -.487 -.469
SS -.147 .239 .172 .200
CS .377 .472 .436
V .436 .299
M .179
0


Posttest F .018 -.396 .412 .141 .329 .200
NS -.507 -.315 -.090 -.556 -.280
SS -.655 -.075 .162 .052
CS .165 .309 .171
V .056 .341
M .147
0










(see Table 10). The correlations of pretest to posttest

scores for the subjects by treatment groups ranged from -.23

to .69 (see Table 11). These low correlations indicate that

writing performance of the subjects from the pretest to the

posttest (i.e., from the beginning to the end of the

semester) was not stable.

Reliability

Procedural Reliability

A time sampling recording form was used to obtain a

measure of the PENS strategy instructors' ability to

implement the PENS instruction according to the outlined

procedure. All instructor behaviors were rated as either

appropriate or not applicable. Individual ratings for each

instructor are presented in Table 12.

Scorer Reliability

Five scorers rated the subjects' writing samples,

including an individual trained in the PENS writing strategy

(R4), two public school teachers who teach writing skills (R1

and R3), a preservice elementary teacher (R2), and the

researcher (R5). All of the scorers, with the exception of

the researcher, were unaware of the conditions of this study.

While R4 scored all writing samples on all measures, the

other scorers rated all or part of the writing samples on

only part of the measures (see Table 13). The reliability

coefficients were Pearson Correlation Coefficients obtained

using the SAS computer program. The reliability coefficients

range from .73 to .99. Reliability for the scores were all










Table 10
Correlations Between Pretest and Posttest Scores for All
Subjects


Measure

Fluency

Nonsentences

Simple/Compound Sentences

Complex Sentences

Vocabulary

Mechanics

Organization


Correlation

.274

.478

-.330

.346

.185

.684

.614


Table 11
Correlations Between Pretest and Posttest Scores for Subjects
by Treatment


PENS
;1 a q


Measure

Fluency

Nonsentences

Simple/Compound
Sentences

Complex Sentences

Vocabulary

Mechanics

Organization


Developmental
Writina Course


.512

.458


.375

.429

.178

.514

.683


.036


.441


-.139

.520

.306

.634

.419


Gordon Rule
Courses

.244

.465


-.028

-.238

.011

.497

.699


""'~ ~'-"~-~~~ ~-----










Table 12
Procedural Reliability for PENS Instructor Behaviors

Instructor Script Sequence Materials


Instructor 1 (researcher)
Observation #1

appropriate 100% 100%

inappropriate 0% 0%

not applicable 0% 0%

Observation #2

appropriate 0% 100%

inappropriate 0% 0%

not applicable 100% 0%


Instructor 2 (student assistant fall semester)
Observation #1

appropriate 0% 100%

inappropriate 0% 0%

not applicable 100% 0%

Observation #2

appropriate 100% 100%

inappropriate 0% 0%

not applicable 0% 0%


Instructor 3 (student assistant spring semester)
Observation #1

appropriate 100% 100%

inappropriate 0% 0%

not applicable 0% 0%


100%

0%

0%



100%

0%

0%





100%

0%

0%



100%

0%

0%


100%

0%

0%










Table 13
Subjects Rated by the Five Scorers


Scorer
R3


Measure Rl

Fluency 100%

Nonsentences

Simple/Compound
Sentences

Complex Sentences

Vocabulary 100%

Mechanics 55%a

Organization

a = The 55% rated by R1
by R3.


100%






100%





100%


55%









55%

55%


is a different


R4

100%

100%


100%

100%

100%

100%

100%

55% than


R5

100%

100%


100%

100%

100%



100%

that rated


within an acceptable range (above .70) and allow the

conclusion to be made that the measures were reliably scored.

Tables 14 to 17 present the correlations estimating the

reliability of the scores for all raters on all measures.

Generalizability Study

A Generalizability study (G-study) was conducted to

provide another method to evaluate reliability of the measures

and errors in measurement that may have occurred when multiple

raters are used. Data gathered from this G-study helped

determine whether a sufficient number of raters were used to

obtain generalizable scores on each measure.

Generalizability coefficients were calculated for each

measure on groups of two, three, and/or four scorers

depending on the number of scorers rating each measure.










Table 14
Reliability for Fluency Scores


Scorer


R1



R2



R3


R2


.94
.95


Occasion


pre
post


pre
post


pre
post


R3


.98
.98


.96
.92


Scorer
R4


.92
.97


.89
.94


.81
.95


pre
post


R5


.98
.98


.94
.96


.97
.98


.93
.98


Table 15
Reliability for Nonsentence, Simple/Compound Sentence,
Complex Sentence, and Organization Scores


Scorer
Measure Scorer Occasion R5



Nonsentences R4 pre .97
post .93


Simple/Compound R4 pre .90
Sentence post .89


Complex R4 pre .88
Sentence post .90


Organization R4 pre .77
post .84












RI~1 IU1-Vi1 -~ fcVr VccI-h,,1 YYrw


Scorer
Scorer Occasion R3 R4 R5



R1 pre .83 .88 .85
post .75 .80 .80


R3 pre .88 .88
post .75 .73


R4 pre .99
post .97




Table 17
Reliability for Mechanics Scores


Scorer
Scorer Occasion R2 R3 R4



R1 pre .85 .87 .84
post .91 .85 .88


R2 pre .89 .86
post .87 .89


R3 pre .89
post .85


Table 16


i i- fr Vnfhn1n r .qr-nrTPq










Results from One-way Repeated Measures ANOVAs using a raters

by subjects design were used to calculate the

generalizability coefficients using the following formula

(Crocker & Algina, 1986, p. 167):

MSp MSr
MSp + (n n')MSr/n'


For example, the first generalizability coefficient for

Fluency was calculated by using the posttest scores from R1

and R2 in a One-way Repeated Measures ANOVA to obtain the

mean squares used in the formula (generalizability

coefficient = .987). The second generalizability coefficient

for Fluency was calculated by using the posttest scores from

R1, R2, and R4 in a One-way Repeated Measures ANOVA to obtain

the mean squares used in the formula (generalizability

coefficient = .999). And the third generalizability

coefficient for Fluency was calculated by using the posttest

scores from R1, R2, R4, and R5 in a One-way Repeated Measures

ANOVA to obtain the mean squares used in the formula

(generalizability coefficient = .999). The subsequent

generalizability coefficients were calculated in the same

manner.

The G-study coefficients were high for the Fluency

(.98), Nonsentences (.97), Vocabulary (.93), Mechanics (.88),

and Organization (.99) measures. The G-study coefficients

were moderate for the Simple/Compound Sentences (.73)

measure. The G-study coefficients were low for the Complex










Sentences (.14) measure. Results from the G-study enable the

conclusion to be made that two raters were sufficient for

scoring Fluency, Nonsentences, and Organization. Three

scorers were sufficient for scoring Mechanics and Vocabulary.

More than two scorers were needed to score Simple/Compound

Sentences and Complex Sentences. Table 18 contains the

G-study coefficients.

Summary of Findings

The hypotheses regarding the lack of interactions

between the treatment groups and the pretest and posttest

scores on the dependent measures were retained. Exploratory

analyses and visual inspection of the group means confirmed

the initial findings; however, a nearly statistically

significant gain was made by subjects in the PENS class in

increasing the percentage of large words (vocabulary) in

their writing samples. A discussion of these results is

reported in Chapter 5.










Table 18
Generalizability Coefficients

Scorers Generalizability
Measure R1 R2 R3 R4 R5 Coefficient



Fluency X X .987
X X X .999
X X X X .999


Nonsentences X X .972


Simple/Compound X X .733
Sentences
Complex Sentences X X .144


Vocabulary X X .757
X X X .888
X X X X .932


Mechanics X X .688
x X x .889


Organization X X .998


X denotes the raters used in the ANOVA analysis















CHAPTER 5
DISCUSSION


The purpose of this chapter is to summarize the results

of this research. Statistical and descriptive discussions of

the hypotheses are presented. Feedback from students and

staff is presented. Educational implications are stated.

Future research needs are reported.

Discussion of Hypotheses

The multivariate hypothesis stated that there would be

no treatment group effect on the five criterion measures

after adjustment on the five covariates pretest measures of

Fluency, Syntactic Maturity, Vocabulary, Mechanics, and

Composition Organization. There were no statistically

significant mean differences among the treatment groups after

adjusting for the pretest differences on any of the measures

of writing ability, indicating that none of the educational

experiences were more beneficial than the others.

Exploratory analyses of the data generally confirmed the

lack of significant findings of differences in group

performance; however, significant differences were noted on

Vocabulary and Mechanics among the treatment groups on the

posttest measures when the posttest was adjusted for pretest

differences. While no significant group mean differences

were found on the follow-up tests for Mechanics and










Vocabulary, two follow-up tests approached significance for

Vocabulary (PENS subjects compared to Developmental Writing

subjects and PENS subjects compared to Gordon Rule subjects).

These follow-up tests were for pairwise contrasts only. PENS

subjects appeared to make gains in Vocabulary while subjects

in the Developmental Writing course and the Gordon Rule

courses did not appear to make improvements in Vocabulary.

Visual inspection of the means, standard deviations, and

ranges of the pretest and posttest scores confirms the lack

of significant differences between the subjects in the three

educational experiences; however, several points should be

noted. First, very little change could be detected through

visual inspection from the pretest to the posttest for all

three treatment groups on measures of Fluency, Mechanics, and

Organization. Second, visual inspection supports the finding

of the statistical analysis that only the subjects in the

PENS class increased the number of large words (Vocabulary)

used in their writing. Third, visual inspection also

detected changes in Syntactic Maturity. Students in each of

the three treatment groups appeared to reduce the number of

nonsentences and increased the percentage of complex

sentences written. Subjects in the PENS class appeared to

make the largest gain in writing complex sentences. Subjects

in the Gordon Rule courses appeared to make the largest gain

in reducing the percentage of nonsentences. Notable also is

the reduction in range of percentage of nonsentences written

(from 0.00-100.0 to 0.00-47.7) for the subjects in the