Citation
The effects of prevocational guidance on the relationship between vocational ability and interests of educable mentally retarded adolescents

Material Information

Title:
The effects of prevocational guidance on the relationship between vocational ability and interests of educable mentally retarded adolescents
Creator:
Stodden, Robert Andrew, 1943-
Publication Date:
Language:
English
Physical Description:
viii, 127 leaves : ; 28 cm.

Subjects

Subjects / Keywords:
Adolescents ( jstor )
Developmental disabilities ( jstor )
Educational evaluation ( jstor )
Labor ( jstor )
Maturity groups ( jstor )
Mental retardation ( jstor )
Personnel evaluation ( jstor )
Psychological assessment ( jstor )
Vocational education ( jstor )
Work sample tests ( jstor )
Dissertations, Academic -- Special Education -- UF
People with mental disabilities -- Education -- Florida ( lcsh )
People with mental disabilities -- Rehabilitation -- Florida ( lcsh )
Special Education thesis Ph. D
Vocational guidance -- Florida ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Thesis:
Thesis--University of Florida.
Bibliography:
Includes bibliographical references (leaves 94-105).
Additional Physical Form:
Also available online.
General Note:
Typescript.
General Note:
Vita.
Statement of Responsibility:
by Robert Andrew Stodden.

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
Copyright [name of dissertation author]. Permission granted to the University of Florida to digitize, archive and distribute this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
Resource Identifier:
025554952 ( ALEPH )
03028902 ( OCLC )

Downloads

This item has the following downloads:


Full Text










THE EFFECTS OF PREVOCATIOYAL GUIDANCE
07 THE RELATIONSHIP BETWEEN VOCATIONAL ABILITY
AND INTERESTS OF EDUCABLE MENTALLY RETARDED ADOLESCENTS










By

ROBERT ANDREW STODDEN


A DISSERTATION PRESENTED TO THE GRADUATE COUNCIL OF
THE UNIVERSITY OF FLORIDA
IN PARTIAL FULFILLMENT OF THE REQL'IRE'.ENTS FOR THE
DEGREE OF DOCTOR OF PHILOSOPHY











UNIVERSITY OF FLORIDA


1976














ACKNOWLEDGMENTS


The author wishes to acknowledge the following people

who helped with completion of this study:

Mr. Joe Casale, Ms. Debbie Hartzell, Ms. Vickie

Gonzales, Mr. Steve Boissoneault, and Mr. Jim McBride,

graduate assistants in the Department of Special Educa-

tion, University of Florida, for their assistance and

participation in the treatment stage of this study.

The secondary level special education teachers in

the City of Gainesville, Florida, for allowing their

students to participate in the experimental stage of this

study.

Dr. Stuart E. Schwartz, Dr. William R. Reid, Dr.

Cecil D. Mercer, Dr. Cary L. Reichard, and Dr. James W.

IIensel, members of the writer's supervisory committee,

whose attention and assistance have been essential.















TABLE OF CONTENTS



ACKNOWLEDGMENTS . .

LIST OF TABLES . ..


ABSTRACT ..


CHAPTER


I. INTRODUCTION .


Statement of the Problem .
Purpose of the Study .
Statement of the Hypothesis.
Delimitations. ..
Assumptions ..
Definition of Terms ....

II. REVIEW OF RELATED LITERATURE


Historical Overview . ..
Vocational Evaluation Approaches .
Standardized Testing Approach ....
Work Sample Approach . .
Rating Scale and Situational Assess-
ment Approach ...........
Vocational Interests and Needs ..
Prevocational Guidance and the Mentally
Retarded . ...
Summary of the Literature Review .

III. PROCEDURES . .

Statement of Null Hypotheses .
Design . . .
Sample . . .
Instrumentation . .
Picture Interest Exploration Survey
(PIES) ...... ................
Worker Trait Components: Dictionary
of Occupational Titles .
Method ......... ....................
Analysis of Data . .


iii


. vi


. . 1


S. 5
S. 5
S 6
S 7
So 7
S 8











IV. RESULTS. . . 71

Introduction . 71
Findings . . 72
Hypotheses . 81

V. SUMMIARY, DISCUSSION, AND RECOMMENDATIONS 85

Summary . .. 85
Discussion . 88
Recommendations . ... 91

REFERENCES . . 94

APPENDIX

A. P.I.E.S. SLIDE DESCRIPTION .. .107

B. WORKER TRAIT LEVELS . ... .111

C. PREVOCATIONAL EXPLORATORY TREATMENT 118

D. GROUP CELL MEANS . .. 122

E. TREATMENT TAPES . ... 125

BIOGRAPHICAL SKETCH . .. 126














LIST OF TABLES



Table Page

1 Analysis of Variance Source Table 73

2 Simple Main Effects Source Table 77

3 Simple Effects Source Table .. 80














Abstract of Dissertation Presented to the
Graduate Council of the University of Florida
in Partial Fulfillment of the Requirements
for the Degree of Doctor of Philosophy



THE EFFECTS OF PREVOCATIONAL GUIDANCE
ON THE RELATIONSHIP BETWEEN VOCATIONAL ABILITY
AND INTERESTS OF EDUCABLE MENTALLY RETARDED ADOLESCENTS



By

Robert Andrew Stodden

August, 1976

Chairman: Dr. William R. Reid
Major Department: Special Education

This study assessed the effects of a prevocational

exploratory treatment upon the relationship between

assessed vocational interests and vocational ability

levels of educable mentally retarded adolescents. The

study investigated three major hypotheses concerned with

pre- and postobservation differences or interactions in

the relationship between vocational interests and voca-

tional ability levels, accorded to the effects of a pre-

vocational exploration treatment, different modes of

treatment presentation and subject maturity level.

A sample of 78 educable mentally retarded adolescents

in grades 9 through 12 were randomly assigned within










their school to one of three groups, stratified according

to grade level. The subjects were pretested with the

Picture Interest Exploration Survey and divided into the

following randomly formed groups: Group I received a

directive presentation of treatment; Group II received

a nondirective presentation; and Group III received similar

attention with different content.

A two hour exploratory treatment involved the ex-

panding of career alternatives as to subject needs and

interests, desired levels of education or training, and

local community needs. Data were obtained for the effects

of three independent variables (treatment, presentation,

maturity levels) upon the dependent variable (formulated

by correlating inventoried vocational interests with

vocational ability levels established through the worker

trait components of the Dictionary of Occupational Titles).

The statistical procedures consisted of a multivariate

analysis of variance followed by appropriate univariate

analysis. A significance level of .05 was selected as

the level indicative of the need for further investigation.

Results for the multivariate analysis indicated a

lack of significant differences among treatment groups

or nestings for preobservations or postobservations when

investigated separately. Significant pre-post observation

differences were found for the directive presentation of


vii










treatment and for the high maturity level (llth and 12th

grades) nested within the directive treatment group.

The author concluded that a prevocational exploratory

experience for educable mentally retarded adolescents

was statistically effective in contributing to a positive

change in the relationship between vocational interests

and ability, when presented in a directive manner with

higher grade level students.


viii














CHAPTER 1

INTRODUCTION



During the past 15 years there has been a progres-

sively increasing interest in the social and prevoca-

tional competencies that are critical determinants of

postschool vocational adjustment for educable mentally

retarded adolescents (Brolin, 1976; Kolstoe & Frey, 1965).

Although sources estimate that the majority of retarded

persons have potential for satisfactory social and voca-

tional adjustment (Goldstein, 1964; Halpern, 1973; Ko-

kaska, 1968; Strickland, 1967), many studies indicate

this level is usually not attained or is achieved at

varying levels of success (Dinger, 1961; HcFall, 1966;

Peterson & Smith, 1960; Saenger, 1967; Tobias, 1970).

Numerous other writers are optimistic about the possi-

bilities of occupational success for most retarded per-

sons, particularly if provided with occupationally ori-

ented guidance and training (Chaffin, Haring, & Smith,

1971; DiMichael, 1967; Katz, 1968; Olshansky, 1969;

Wolfensberger, 1967). These studies suggest that (a)

the retarded have many untapped potentialities in the

personal, social, and occupational domains, (b) little

is known about the mentally retarded individual's social










and vocational abilities, and (c) given further research

and better training programs, the retarded individual

can reach a higher level of performance than was earlier

thought attainable.

Martin (1972) estimated that only 21% of the handi-

capped children leaving school in the next four years

will be fully employed, 40% underemployed, 26% unemployed,

10% sheltered, and 3% almost totally dependent. These

figures seem generally valid in light of current reports

of vocational adjustment of the mentally retarded. In

determining the extent of vocational adjustment success

with the mentally retarded, Brolin, Durand, Kromer, and

Muller (1975) followed up 71 former mildly retarded

students and found that only 21% had an average or above

adjustment, 35% had a fair adjustment, and 44% had a poor

adjustment. These follow-up estimates were further sub-

stantiated through findings by Olshansky and Beach (1974)

that only slightly more than half of the 237 former

workers at a rehabilitation workshop were working a year

or more after placement.

Kokaska's (1968) review of 11 follow-up studies

found that many former special class students were trained,

placed, and remained employed in varying service positions,

but perhaps training and placement personnel may be char-

acterizing the mentally retarded as being too limited

concerning vocational abilities. In a later article,










Kokaska (1971) warned that the limited research and

knowledge of vocational training and adjustment of spe-

cial class students may be affecting the socio-occupa-

tional success of mentally retarded individuals, forcing

them to become marginal workers within our labor force.

One of the greatest problems that mentally retarded

individuals encounter is the continual misunderstanding

and underestimation of their vocational potentials. The

successful evaluation and habilitation of the mentally

retarded individual is precluded by several assumptions

underlying successful vocational adjustment (Olshansky,

1969): (a) that we understand the level of intelligence

required for different types of jobs, (b) that a slow

learner is a poor learner, (c) that intelligence is a

constant and global quality, (d) that the retarded person

can tolerate boring tasks, and (e) that all retarded

persons have the same personality characteristics.

Since many of these misconceptions continue to pre-

vail among professionals and employers of the retarded,

the task of meaningful prevocational evaluation and

habilitation is quite difficult. Brolin (1972) found

that 40% of the mentally retarded clients, who had been

extensively evaluated at a diagnostic center, were not

achieving their vocational potential despite continued

rehabilitation services.










Several researchers indicate that due to the many

factors related to vocational development and adjustment,

no single measure is likely to be of help in programming

or predicting the extent of vocational adjustment success

or failure for the mentally retarded (Butler & Browning,

1970; Goldstein & Heber, 1961; Heber, 1963; Windle, 1962).

The current absence of a unitary vocational prediction

formula suggests that successful personal, social, and

vocational outcomes are the product of many interacting

variables. Scattered studies investigate individual

variables (Becker, 1973, inventoried interest; Farber,

1968, basic societal expectancies; Salomone, Lehmann, &

Green, 1973, vocational sophistication), while few in-

vestigate the relationship between variables involved in

the prevocational evaluation process (Gold, 1973, produc-

tivity and task complexity; Lofquist, Dawis, & Weiss,

1968, work personality and vocational choice).

Despite the vocational adjustment research and expert

opinions that indicate most mentally retarded individuals

can reach a successful vocational level, further research

indicates a great number of retarded individuals are

having difficulty adjusting to the world of work. Un-

employment and underemployment are constant problems for

the mentally retarded due to underestimations and miscon-

ceptions as to their vocational potentials, lack of

knowledge and understanding as to which variables or










interaction of variables contribute to vocational ad-

justment, and the lack of experimental research upon which

to base prevocational exploration and evaluation programs.


Statement of the Problem

Research concerned with vocational adjustment of

the mentally retarded has failed to identify variables

or the interaction of variables necessary for successful

vocational adjustment in the world of work and the develop-

ment of effective prevocational exploration and evaluation

programs. In an effort to meet the total life career

development needs of the mentally retarded adolescent, a

recent emphasis upon career education (Clark, 1974) and

prevocational evaluation (Pruitt & Longfellow, 1970) has

evolved, without direction from consistent, sound research

studies.

The problem for the present research study is a

question commonly encountered in career education and

prevocational evaluation and exploration programs for the

educable mentally retarded. What is the relationship

between vocational interest and vocational ability, and

what are the effects of prevocational exploratory treat-

ments upon this relationship?


Purpose of the Study

The purpose of this study was to examine the effects

of a prevocational guidance experience upon the relationship










between assessed vocational interests and vocational

ability levels for educable mentally retarded adolescents.

Further concerns were the examination of differences and

interactions in the relationship between vocational in-

terests and vocational ability levels accorded to direc-

tive versus nondirective presentation of treatment and

different maturity levels for educable mentally retarded

adolescents.

Selection of the particular variables was based upon

an expressed need in the professional literature to study

experimentally some of the areas of misconception and

assumption with regards to prevocational exploration

and evaluation of the mentally retarded. The expressed

lack of research based knowledge, the conflicting findings

and arguments, and the intense methodological criticism

of existing studies provide purpose for the present re-

search study.


Statement of the Hypothesis

The variables of a prevocational guidance experience,

in either a directive or nondirective mode of presentation

and the differing maturity levels of secondary level edu-

cable mentally retarded individuals will not individually

or in interactive combination statistically affect the

relationship between assessed vocational interests and

vocational ability.










Delimitations

The topic for this research concerned the relation-

ship between vocational interests and vocational ability

of educable mentally retarded adolescents. The target

population for this study was composed of secondary

school EMR students who had been placed in special self-

contained classes prior to entering secondary school.

The inaccessability of resource room or mainstream models

limited generalizations to those EMR students in self-

contained classrooms.

The generalizability of the study was also limited

by the geographical location, nature, and size of the

sample; a single urban area (university city) in North

Central Florida. Therefore caution should be exercised

in specific attempts to generalize findings from this

study to populations outside of the sample selected.


Assumptions

For the purposes of this study it was assumed

I. that educable mentally retarded students assigned to

special classes actually have needs that cannot be met

in present administrative and instructional arrangements

serving children of normal intelligence.

2. that the sampled students responded according to their

actual interests and not according to the expectancy of

others.










3. that the random sample of educable mentally retarded

adolescents was representative of the general population

of educable mentally retarded.

4. that the dependent variable was meaningful to the

independent variable.


Definition of Terms

Vocational Evaluation: the appraisal of the individual's

capacity including patterns of work behavior, ability to

acquire occupational skills, and the selection of appro-

priate vocational goals

Educable Mentally Retarded Adolescents: a secondary

level school student between the chronological ages of

14 and 21, who is assigned to a special self-contained

class for educable mentally retarded students in the

public schools.

Directive: refers to a presentation of treatment from

the frame of reference of the group leader (leader made

suggestions and comments from his background and past

experience concerning career choices of the subject).

Nondirective: refers to a presentation of treatment from

the frame of reference of the mentally retarded student

(leader did not make suggestions and comments from his

background or past experience concerning career choices;

suggestions came from information offered by the student).

Maturity Levels: refers to grade achievement levels 9







9


through 10 and 11 through 12 (grade achievement levels

9 through 10 were the low level maturity group; grade

achievement levels 11 through 12 were the high level

maturity group).

Prevocational: refers to any counseling, guidance, or

training of an occupational nature before the actual

employment stage.














CHAPTER II

REVIEW OF RELATED LITERATURE



This review of the literature is divided into three

sections pertaining to the topic of study. The first

section consists of a brief historical overview of pre-

vocational evaluation and training of the mentally re-

tarded. The second section consists of a research review

of vocational evaluation approaches and assessment measures

appropriate with the mentally retarded (standardized

ability assessments, work samples, interests and needs

assessments). The third section consists of a review

of studies in the area of prevocational counseling and

guidance efforts with the mentally retarded.


Historical Overview

The development of prevocational training programs

for the mentally retarded has been a relatively recent

phenomenon in the history of American education. By the

1940s several secondary level training programs existed

in larger urban centers (Hungerford, 1941; Hungerford &

Rosenweig, 1944; Mones, 1941, 1948). Hungerford (1941)

formulated the concept of occupational education with

the intention of shifting the educator's allegiance from










abstract academics to a vigorous emphasis upon the re-

tarded individual's ability to assume vocational respon-

sibility.

The labor deficit created by the Second World War

assisted the mentally retarded in demonstrating their

employability and many capabilities. The end of the war

brought about new realizations concerning the goals, pur-

poses, and functions of public education, community agencies,

and the federal government.

Interest among rehabilitation researchers in predict-

ing vocational success among the mentally retarded began

when Cowdery (1922) verified that persons of limited mental

specifications could successfully take part in the con-

structive and useful work of society. Until recently,

the mentally retarded had been visualized as performing

only at unskilled levels (DiMichael, 1967; Kokaska, 1968,

1971; Oswald, 1968).

A new appreciation of the work potential of the

mentally retarded evolved from an increased understanding

of the vocational assets of this group, which are often

not reflected in standardized mental tests (Kennedy, 1948;

O'Connor & Tizard, 1951). Taylor (1964) concluded spe-

cifically that general intelligence tests, with their

emphasis on scholastic skills, may leave untapped other

variables of crucial importance for the mentally retarded.

Appell, Williams, and Fishell (1964) found that variables










such as general ability, vocational approach, attitudinal

tone, learning speed, and psychomotor aptitude related

factors were able to significantly differentiate repeatedly

clients from those who became gainfully employed. In

the process of distinguishing successful from nonsuccessful

retarded clients, with respect for job placement, the

authors found that neither intelligence nor educational

level were discriminating variables.

In further investigation for a predictor of voca-

tional success, Gragent (1962) examined a group of retarded

adults and adolescents in a Goodwill Industries vocational

evaluation and training program, noting that the intelli-

gence quotient was a less effective predictor of employ-

ment than were interviews and projective techniques. Sali

and Amir (1971) studied mental age as a dominant factor

in the determination of rehabilitation prospects. De-

pendent variables were success at work and specific job

level. Independent variables were personality factors

affecting the work of the sample of 305 retarded individuals

(over age 14; IQ, 30 to 65). Results indicated (a) suc-

cess at work was most clearly related to motor coordina-

tion, (b) the relationship between IQ and output was weak,

(c) personality characteristics and performance were highly

correlated (+.66 to +.80), and (d) performance was related

to social adjustment (+.80).










Albin (1973) examined the relationship of IQ to

vocational success (average production in three different,

nonconsecutive weeks) with 35 employees in a sheltered

workshop. The data were analyzed with a Spearman correla-

tion and the analysis indicated no significant relation-

ship existed. Implications were that IQ does not seem

to be a prerequisite or valid predictor for vocational

production. These studies tend to contribute to the general

hypothesis that mental age is not the dominant factor in

the determination of vocational prospects for the mentally

retarded, and that a comprehensive assessment in the form

of a complete prevocational evaluation is a contributing

force to socio-occupational success of mentally retarded

individuals.

It is the assumption of many (Bozarth, 1968; Rosenberg,

1967) that the evaluation and prediction of vocational

potential and/or success is a highly developed process.

The main thrust of research in vocational evaluation has

been to seek scientific support for the use of predictive

devices (Overs, 1970).

In a comprehensive review of studies concerning the

predictive assessment of the social and vocational adjust-

ment of the retarded, Cobb (1969) concluded that judgments

and assessments more often than not underestimate the

ability of retarded persons to satisfactorily meet community

and occupational demands. He summarized his review as follows:









The most consistent and outstanding finding
of all follow-up studies is in the high pro-
portion of adult retarded who achieve satis-
factory adjustment, by whatever criteria are
employed. It is more appropriate to make an
assumption of positive adaptation on some
meaningful criteria of employment until nega-
tive evidence appears, rather than assume a
poor diagnosis until positive evidence appears.
(p. 49)

As emphasized by Cobb (1969), traditionally work evalua-

tion or prevocational exploration has been utilized to

provide an effective method of judging client performance.

In recent years the concept of work evaluation has been

modified or broadened toward an interaction process (Gell-

man, 1968; Moed, 1960), an examination of the person, the

work situation, and the interaction between them which

facilitates the transition to employment of vocationally

handicapped persons.

Current objectives for vocational evaluation services,

as expressed by the Vocational Evaluation and Work Adjust-

ment Association (VEWAA) Vocational Evaluation Project,

Final Report (1975), reflect the trend away from prediction

of success or failure as a role of evaluation programs.

Prevocational evaluation as a comprehensive assessment

as well as a remedial treatment process seems to be evidenced

by current reports (Vocational Evaluation Project, Final

Report, 1975).

Neff (1966) suggests the following differing approaches

to vocational evaluation: (a) standardized psychological

testing approach, (b) work sample approach, (c) job analysis










approach, and (d) situational assessment approach (interest

and personality factors). The next portion of this review

concerns itself with the various approaches to vocational

evaluation and research pertaining to their use with

mentally retarded individuals.


Vocational Evaluation Approaches

Standardized Testing Approach

Standardized testing has had wide use through attempts

to predict vocational success (Overs, 1970), sharing the

attributes of easy and inexpensive administration and

objectivity, and acceptable reliability. According to

Neff (1966), their main disadvantages are with low pre-

dictive validity, artificiality, and general inappropriate-

ness for use with even the borderline mentally retarded.

A brief review of arguments pertaining to the use of

standardized vocational ability tests with the mentally

retarded yielded the following statement:

by definition, retardation assumes a slow
rate of learning. Standardized tests that
measure one trial learning will reinforce the
initial diagnosis of retardation without sup-
plying the information necessary for the de-
termination of vocational feasibility. (Tobias,
1960, p. 122)

In relation to the mentally retarded, Gellman (1968)

suggests that the very strength of standardized tests may

be their greatest weakness. They are designed and ad-

ministered to maximize performance, thus resulting in a










poor reflection of the client's general orientation toward

work or what would be expected from him in an actual job

performance situation. Timmerman and Doctor (1974) dis-

cuss three cautions concerning the use of standard tests

with the mentally retarded: (a) the specific results of

standard tests are too often generalized to indicate some

all-encompassing work ability, (b) frequently, inadequate

work attitudes and behaviors cannot be assessed during

a short standard test in an isolated administration setting,

and (c) standard tests are frequently kept short to elim-

inate fatigue or declining motivation, thus inappropriately

reflecting the employment setting.

While working with the mentally retarded, Tobias and

Gorelick (1960) made an effort to determine the predic-

tive ability of a single standardized test, the Purdue

Pegboard. Eighty-one clients, with IQ ranges of retarda-

tion from 30 to 80, made up the sample. Performance on

the Purdue Pegboard was correlated with quality of pro-

duction on two separate bench operations. One was a

ball-point pen assembly and the other a wire clamp assembly.

Significant differences were found in performance on the

Purdue, not only between retarded and normal adults, but

also between different levels within the retarded range.

The resulting scores on the various subtests of the Purdue

Pegboard correlated highly (.79, .75, .69) with production

output on two separate operations. The authors suggest a










substantial relationship between intellectual functioning

(as measured by the WAIS) and manipulative dexterity.

Despite these high correlations, Timmerman and Doctor

(1974) concluded that the Purdue Pegboard had questionable

value and must be used cautiously with the mentally re-

tarded for the following reasons: (a) the high relation-

ship between the Purdue Pegboard and workshop assembly

tasks cannot guarantee significant correlations with

other types of tasks such as those in service occupations

and manual tasks requiring different movements (both

tasks were assembly operations, expected to correlate

with the Purdue Pegboard), and (b) the study dealt with

only one aspect of productivity, quantity, to the exclu-

sion of quality (does not mention whether or not incor-

rectly assembled units were counted as finished pieces).

Even though a significant implication from the study,

that there are apparent levels of dexterity corresponding

to levels of retardation, it remains unclear as to whether

or not loss of dexterity is due to the level of retarda-

tion. Other factors, such as lack of vocational exposure,

should be considered before making the sweeping conclu-

sions about the relationship between dexterity and intel-

ligence within the retarded range.

Wagner and Hawver (1965) obtained somewhat similar

results in an attempt to identify one or more tests which

would predict workshop success for severely retarded adults.










A sample included 27 subjects assessed by the Stanford-

Binet to have an IQ of less than 55. The test instruments

were the O'Connor Finger and Tweezer Dexterity Tests,

subtest of the Minnesota Rate of Manipulation Test, the

Goodenough Harris Draw a Man Test, the Active Score of

the Hand Test, and the Bender Gestalt Test. Clients were

administered the battery and the instructor ranked all

subjects according to established criteria, such as respect

for authority and directions, completes assignments with

good quality, gets along with co-workers, and learns new

task skills without too much difficulty.

All of the test scores registered a significant re-

lationship with the instructor's rankings of clients.

The Bender-Gestalt displayed the highest correlation

(+.89) suggesting the possibility of predicting rankings

of work performance with some degree of accuracy. Wagner

and Hawver (1965) suggest two implications from their

study: first, that there seems to be a single intactness

factor found in low level mental ranges, or that there is

a single factor in retardation which causes repression

in other areas of learning and performance, due to the

high degree of correlation between several different tests

and the criteria measured, and second, that the high cor-

relations support the validity of psychological testing

with a retarded population.










Timmerman and Doctor (1974) considered the following

limitations to the Wagner and Hawver (1965) study: that

of the difficulty of generalizing the results due to

small sample size and geographic areas, that the results

could only purport to predict performance in a sheltered

workshop (noncompetitive situation), and that predictive-

ness appears to apply only to bench work and woodworking

tasks. Despite the limitations and criticisms of aptitude

assessment as a prediction tool, instruments such as the

Purdue Pegboard appear to be valid and useful instruments,

especially for the lower level retarded individuals.

When assessing the abilities of the mildly retarded, dif-

ferent generalizations would appear to apply and different

assumptions need to be examined.

An instrument having some applicability for the

mentally retarded is the General Aptitude Test Battery

(GATB), a test used commonly in employee selection. De-

veloped by the United States Employment Service, this

instrument battery was designed to assess vocationally

significant aptitudes for vocational counseling, job

selection, and job placement (Brolin, 1976). The battery

makes predictions for about 500 occupations, primarily

of the unskilled or semi-skilled type, and measures apti-

tudes in nine areas: general learning, verbal aptitude,

numerical ability, spatial aptitude, form perception,

clerical perception, motor coordination, finger dexterity,

and manual dexterity.










According to the United States Department of Labor

(1970), the GATB is the most widely used aptitude test

and one of the best validated and most carefully researched

psychometric tests in existence. A particular strength

is its close integration with the Dictionary of Occupa-

tional Titles (DOT) which constitutes the most comprehen-

sive taxonomy of the American job market (Bacher, 1972).

Unfortunately, the GATB requires a reading ability for

direction and calculations at about the seventh grade

level. For this reason, the Manpower Administration

initiated research on a battery of tests that would mea-

sure the same nine aptitude areas as the GATB, but would

also be suitable for use with many nonreading and disad-

vantaged clients. The resulting instrument, the Nonreading

Aptitude Test Battery (NATB), has similar subtests as

the GATB and supposingly is understood by individuals

with limited verbal abilities.

Carbuhn and Wells (1973) attempted to establish con-

current validity of NATB factors which would significantly

differentiate the vocational trainees rated as good and

excellent workers from those rated as fair and poor com-

petitive workers by on-the-job training supervisors.

Their sample included 102 subjects with IQs ranging from

42 to 83 and a mean reading ability of 3.0 grade level.

The criterion for admission to the replacement program

was assumed potential for returning to the community as










semi-independent and economically productive individuals.

Predictive measures included the WAIS, WRAT, and the

NATB, while criterion measures were three ratings on a

nine-factor, five-point rating scale of each trainee's

job performance. Results indicated that the NATB Factor I

(space or nonverbal intelligence) is related to such job

characteristics as learning time and resourcefulness for

females. Factor II (perceptual speed) was related to

learning time for males. Factors III and IV (internaliza-

tion of rules and general learning ability) were highly

related to females' learning time, ability to handle

equipment, and resourcefulness. Factor V (dexterity and

speed) was the most important characteristic of overall

job performance for females. The authors concluded that

the NATB could be used with levels of mental retardation

above a minimum of 40 IQ (mild and moderate levels). Both

Carbuhn and Wells (1973) and Brolin (1976) in reviews of

aptitude tests recommended the NATB as a valuable source

of measurement of vocational aptitude for higher level

mentally retarded individuals.

Despite limitations of many of the studies conducted

with standardized vocational or psychological instruments

and the mentally retarded, Cobb (1969) suggests from his

review of predictive studies "that the most stable pre-

dictive indicators have been found to be standard measures

of manual dexterity to criteria of work competence" (p. 143).










In reviewing manual dexterity tests, Overs (1970) suggests

that standardized instruments of dexterity predict as

well as or better than job samples, and may be administered

in a much shorter time at a greater savings to clients

and staff.

As indicated by Super and Crites (1962) manual dex-

terity is not a singular trait. The authors view isolated

movements as lacking significance when an entire patterning

of movements is required by workers on the job. Implica-

tions are drawn that standardized tests of manual dexterity

should be recommended for use, but with caution, when

working with the mentally retarded.

Jacobs and Weingold (1958) indicate that psychological

testing of the mentally retarded has value in that the

results (a) allow for an objective comparison of the

client with his peers, (b) give general indicators as to

areas to be stressed or eliminated, (c) allow for an

assessment of changing capabilities of the client, and

(c) may suggest that beginning or continuing training is

not feasible for the client. Burdett (1963) viewed the

value of psychological examinations of the mentally re-

tarded to be in the ability of the interpretation of

test results to discover areas of training as well as

giving an adequate estimate of motivational abilities,

personality, and level of functioning. Bitter (1967)

viewed standardized instruments as valuable in exploring










personality and adjustment difficulties which could

potentially affect job performance.

Several authors have heavily critized standardized

testing as the dominant predictor of vocational success

with the mentally retarded. Wolfensberger (1967), in

his extensive review of literature of vocational re-

habilitation of the mentally retarded, critized the con-

ceptualization of the evaluation process as the "sacred

cow" of vocational rehabilitation programs, despite the

lack of evidence as to its value. Wolfensberger dis-

cussed several criticisms common with studies using psy-

chological predictive instruments. He mentions the

common practice of validating a predictive technique

with workshop performance rather than competitive employ-

ment, the poor control and design of studies, often plagued

with selection problems, and the consistent failure to

emphasize variables other than retardation as significant

causal factors.

Wolfensberger (1967) and Neff (1970) argue the need

for improved experimental conditions and new, more pro-

ductive evaluation approaches to the vocational evalua-

tion of the mentally retarded. Neff (1970) further

critized the traditional psychological assessment approach

as being roundabout and indirect in achieving a measure-

ment of crucial variables necessary in the real world of

work. The following criticisms are posed: (a) the time










span is usually very short, (b) an encouragement to work

toward full potential is given in the testing situation

but is lacking in the employment situation, and (c) lack

of reality in assessment situations.

The heavy criticism of studies involving standardized

psychological or mental assessment instruments as an

approach to vocational evaluation (Hoffman, 1970; Neff,

1970; Overs, 1968; Wolfensberger, 1967) has led to evalua-

tion approaches providing a valid and more realistic

picture of actual work capabilities.


Work Sample Approach

A work sample, according to Neff (1966), is a "mock

up, a close simulation of an actual industrial operation,

not different in its essentials from the kind of work

a potential employee would be required to perform on an

ordinary job" (p. 27). A number of vocational evaluators

view work samples as superior to standardized tests,

especially with a mentally retarded population (Gold,

1973; Hoffman, 1970; Neff, 1970). Yet, others indicate

that standardized tests reveal as much if not more than

work samples (Cobb, 1969; Sankovsky, Arthur, & Mann,

1968; Super & Crities, 1962).

Several arguments were noted in the literature in

favor of the use of work samples as an approach to voca-

tional assessment: (a) that they assess the same skills,

aptitudes, and abilities required by competitive employment










situations (JEVS, 1968; Overs, 1968; Usdane, 1963),

(b) since they are much like real work, they motivate

clients more than standardized tests (Hoffman, 1970;

Neff, 1966; Overs, 1968), and (c) work samples are much

less affected than standardized tests by factors such

as educational level, speech and hearing impairments,

and excessive anxiety (JEVS, 1968; Lustig, 1966; Overs,

1968).

Although favoring the increased use of the work

sample approach to evaluation, Timmerman and Doctor

(1974) present general cautions against a heavy re-

liance upon the use of work samples with the mentally

retarded. These cautions include the fact that the re-

semblance of work samples to actual jobs does not assure

their predictive validity since many job differences

cannot be duplicated in work samples, the problem of sub-

jective observation and interpretation by raters, and the

existence of a widespread lack of standardization among

work samples, with very few being validated with the

mentally retarded.

In the field of work evaluation for the handicapped,

some efforts have been made to devise batteries of work

samples which would have universal significance. One

well accepted work sample system is the Wide Range Em-

ployment Sample Test (WREST), also titled the Jastak-

King Work Samples, a recent development of Jastak and










King (1972). It consists of a short battery of 10 work

samples developed at a workshop for the mentally and

physically handicapped. The primary purpose of the WREST,

according to its authors, is to evaluate dexterity and

perceptual abilities in small groups within a two hour

time period. Brolin (1976) discusses the strengths of

the battery, for use with the mentally retarded client,

as providing the opportunity for the client to practice

exercises prior to timing and evaluation of the work

sample, by listing industrial norms (both competitive and

workshop), along with short administration time and pre-

cise instructions.

In contrast, Timmerman and Doctor (1974) reported

the normative data for the WREST as sketchy, found the

instructions as confusing and obscure, thus making it

necessary to modify instructions and procedures in several

areas to closer approximate the needs of the mentally

retarded. In criticism, Brolin (1976) also noted a need

for further and more systematic procedures concerning

the observation of client behavior when using the WREST

battery. A further problem concerned the application of

results of the evaluation to any specific jobs or occupa-

tional groups, since the battery does not relate its work

samples to the DOT or any other job classification system.

Botterbusch (1973) noted that there was no way of knowing

what competitive jobs were related to specific samples or










to the battery as a whole. Despite the obvious short-

comings the test battery appears to have potential for

both mild and moderately retarded groups (Brolin, 1976;

Timmerman & Doctor, 1974).

One of the better known work evaluation systems is

the TOWER (Testing, Orientation, and Work Evaluation in

Rehabilitation), developed in 1936 at the Institute for

the Crippled and Disabled (ICD). The TOWER system includes

14 broad areas of work evaluation and over 110 work samples

in all. Evaluation takes approximately three weeks with

each occupational task presented in a gradual sequence

from simple to complex.

Some of the advantages of the system, according to

Rosenberg (1967), are that service procedures are flexible

rather than rigid making the tasks adaptable to the men-

tally retarded, and the tasks relate directly to skill

potential needed in vocational training. Rosenberg views

the TOWER as an approach to vocational evaluation rather

than an exact "test" of the client's vocational potential.

In turn, the same flexibility of instruction and adminis-

tration (modifications made to compensate for high reading

and comprehension levels, pretimed practice trials, and

adaptability to local placement opportunities) would

appear to effect the already weak validity reports of

the test samples, as well as the norms concerning quality

and quantity of performance (Rabucha, 1975; Timmerman &










Doctor, 1974). Brolin (1976) suggests that the TOWER

has applicability with the mentally retarded, at mild

levels, at least in some work sample areas, with appro-

priate modifications. One of the more advantageous reasons

for its applicability with the mentally retarded is the

wide variety of occupational exposure available through

the work task samples.

Another evaluation instrument, the Jewish Employment

Vocational Service (JEVS), was developed under a contract

with the Manpower Administration, Department of Labor.

The system consists of 28 work samples covering 20 dif-

ferent work areas within 10 worker trait groups (Brolin,

1976). Several of the work tasks have been adapted (VIEWS,

1976) to assess the work potential of persons with learning

disabilities and various levels of mental retardation.

The evaluation period, involving approximately two weeks,

is initiated with the simplest work samples and proceeds

towards increasing complexity. Each work sample is ad-

ministered under actual work conditions for that specific

task, with the client observed, timed, and rated on a

three point scale; the higher scores indicative of per-

formance and behavior required in competitive employment.

One of the major strengths of the system (Brolin,

1976) is that each work sample is directly related to the

"work trait group" arrangement in the Dictionary of Oc-

cupational Titles (DOT), allowing for a direct comparison










between actual occupational categories and evaluated skills.

Weaknesses of the JEVS system (Brolin, 1976; Timmerman &

Doctor, 1974) are that the instructions for the tasks

are not suited to reading levels of the mentally retarded,

that the pressure of time limits upon the client causes

undue anxiety and mistakes, the inability of the client

to conceive the work samples as related to real work

situations, and the need to disassemble the work samples

before they can be used again. With an increasing use

of JEVS by state employment agencies, and the introduction

of the new work samples (VIEWS, 1976) for the mentally

retarded, consideration is warranted by those involved

in the vocational evaluation of the mentally retarded.

Another evaluation unit, the Singer Vocational Evalua-

tion System, is a work-oriented screening device designed

to help the individual make a vocational choice through

a "hands on" exploration experience of several job tasks.

The system utilizes an audiovisual approach to present

programmed instruction on the performance of specific

tasks selected in 17 occupational clusters, self-contained

within work stations, each outfitted with the necessary

instructional tools. Purportedly the system avoids the

pitfalls of reading deficiencies, through the combined

film strip and tape presentation, and also permits the

trainee to work at his own pace. The "hands on" nature

of the system is considered a very positive aspect for










prevocational exploration and awareness, particularly

for the mildly retarded (Brolin, 1976), yet caution must

be exercised when using the system for purposes of evalua-

tion. Only one or two of the work stations are considered

appropriate for the more seriously retarded individual,

with indications that some of the work tasks were too

involved for the retarded population. According to Rabucha

(1974), problems have existed in the oral presentation

of instructions, the small screen size, and poor audio

when used with the retarded or handicapped.

The Goodwill Industries of Chicago and Cook County

Inc. produced the Multidimensional Objective Vocational

Evaluation (MOVE) system, a computerized method of as-

sessing an individual's vocational abilities. The system

consists of 32 factor-pure tests (completely objective)

which take approximately three hours to administer (Hester,

1975). After testing the raw aptitude scores and addi-

tional personality characteristics of the individual,

these are all fed into the MOVE computer program. The

data are analyzed by the computer to determine how the

individual's abilities relate to jobs contained in the

Dictionary of Occupational Titles. According to Hester

(1975), the MOVE system does not duplicate information

offered by the other major systems, thus it can be used

in conjunction with JEVS and TOWER as a further guidance

information aid. Hester suggests that because of the










factor-pure nature of the MOVE tests, the system can be

used with all levels of persons regardless of background

or handicap (with the exception of the blind). Due to

the relative newness of the system and the lack of use

outside of Chicago, only the original test reports could

be found on the system.

The MacDonald Vocational Capacity Scale (VCS) de-

veloped at the MacDonald Training Center (Pinkard, Gil-

more, Richer, & Williams, 1963) consists of eight separate

tests or ratings administered in an evaluation setting

where observations can be made over a two week period,

for the purpose of predicting the vocational potentialities

of young retarded adults. The VCS purports to differen-

tiate among persons needing close and constant supervision,

those of workshop potential and those capable of some

level of competitive employment. As initially designed,

the VCS was comprised of eight measures: (a) work habits,

(b) physical capacity, (c) social maturity, (d) general

health, (e) manual skills, (f) arithmetic, (g) direction

following, and (h) motivation. Further research by Ho

(1972) revealed that physical capacity, general health,

and motivation were not significantly predictive to warrant

their inclusion in the VCS. Based on an individual's

scores on each of the eight measures, which have varying

weights, the scores are recorded and tallied on a profile

sheet and a prediction of the individual's vocational










capacity as day care, sheltered, borderline, or competi-

tive is ascertained.

The VCS was developed especially for use with the

mentally retarded, yet some serious questions have been

raised concerning some of the procedures involved in the

original study with the instrument. Windle (1962) sug-

gests that the MacDonald (VCS) study contained such serious

methodological shortcomings that it was worthless as a

predictive study. He noted in particular that the reasons

for placement were not independent of the descriptive

characteristics of the client, and thus retrospectively

related to placement (self-fulfilling prophecy). This

would entail placing a client at a certain level due to

his performance on a test battery and then using the place-

ment to establish the validity of the test battery, without

consideration as to whether the client performed any

better. Windle (1962) also critized the use of chi squares

with unequal group cells and pointed out serious computa-

tional errors in the original VCS study.

Dayan (1968) in another study, attempted to validate

the VCS using a population of mentally retarded young

adults. Three hundred and sixty-six institutional resi-

dents were each administered the VCS twice, with care

taken to avoid contamination of ratings by staff members.

The VCS scores were not used in making programming de-

terminations, thus avoiding the main objection to the










MacDonald study. Dayan indicated significant findings

concerning vocational capacity with the mentally retarded.

These included little or no discrimination accorded to

age, some relationship existing between IQ and the level

of employment, and disproportionate percentages of minor-

ities in the competitively employed group.

Comparison between the mean scores of each VCS mea-

sure and various criterion groups indicated that differences

were all in the same direction; competitively employed,

sheltered workshop, nonemployed, and day care. According

to Brolin (1976) and Timmerman and Doctor (1974), the

VCS does appear to have some predictive value for dis-

crimination levels of placement for the mentally retarded.

Wright and Trotter (1968) and Wolfensberger (1967) recom-

mended further research with the VCS to determine the

validity of the scale, by increasing sample size, further

cross-validation, and studies of the instrument's predic-

tive value. Further revision of the scale by Ho (1972)

is a significant starting point in this direction.


Rating Scale and Situational Assessment Approach

According to Dunn (1973) vocational evaluation pre-

dictors can be expected to reach their greatest validity

when they closely approximate a real work setting. This

philosophy has given birth to the situational assessment,

defined by Dunn (1973) as "a systematic procedure for ob-

serving, recording, and interpreting work behavior .











applicable to a variety of real or simulated work situa-

tions" (p. 5).

Brolin (1976) provides several advantages and dis-

advantages of the situational assessment approach to voca-

tional evaluation (assessed through a rating scale) for

the mentally retarded. Some advantages are (a) activity

approximates the real work situation, (b) eliminates the

typical testing situation (anxiety), (c) is possible to

assess many typical work behaviors (interpersonal rela-

tionships, cooperation, pressures, authority), (d) gives

the individual more time to adjust to the situation before

assessment, and (e) individual can be evaluated under

various conditions, with several supervisors, and in

several work situations. Some disadvantages are (a)

is dependent upon accurate interpretation of observations,

(b) problem of variance among raters on observed behavior,

and (c) the group setting may effect the rater's evalua-

tions.

Several rating instruments on the vocational behavior

of the mentally retarded have been developed. Daniels

(1972) designed the Vocational Adjustment Rating Scale

(VARS) with 90 items, 42 of which reflected positive

vocational adjustment and 48 which are negative. Re-

liability was ascertained by comparing scores of three

raters who independently assessed each group of trainees

in each of the four job areas rated by the instruments.










The test population was composed of 40 young adult males

(age range, 17 to 28) (IQ, 50 to 80) enrolled in a re-

habilitation center. A basic weakness of the study,

according to Timmerman and Doctor (1974), was that the

rating scale consisted of only a small part of a project

on self-concept and vocational adjustment, thus efforts

to establish predictive validity for the scale never

occurred.

Another behavior rating scale is the Work Adjustment

Rating Form (WARF) developed by Bitter and Bolanovich

(1970) designed to predict job readiness of mentally

retarded individuals. It was designed to provide a sys-

tematic observation, relevance, reliability observations,

and verifications of behavior patterns. A study by Bitter

and Bolanovich (1970) to validate the form used a sample

of 40 clients (mean age, 19.41; mean IQ, 59.25) in light

of four criteria mentioned. The ratings were completed

by three counselors and one foreman after the third and

16th weeks of training. The WARF contains eight sub-

scales, consisting of areas of work behavior, each broken

down into five levels of performance.

Correlations were conducted between counselor ratings,

after both three and 16 weeks, and job success. Correla-

tional studies were also conducted involving one counselor

and the foreman pertaining to each subscale of the test

and eventual job success. In general, the WARF ratings










corresponded to the counselor's pooled judgment of em-

ployability at the end of the 16 week training period

with actual job success determined by six months employ-

ment. Significant differences in the number of clients

assessed at each time by each rater make the correlations

relatively useless for determining the effect of the

additional 13 weeks. Bitter and Bolanovich (1970) indicate

a need for further research in the areas of identification

of specific behaviors, attainment of observer consistency,

and the development of normative information.

An assessment tool developed by Levine and Freeman

(1968), the San Francisco Vocational Competency Scale,

is described as assessing vocational competence of mentally

retarded adults. The scale contains 30 items relating

to motor skills, cognition, responsibility, and social-

emotional behavior. Two protests, each involving 330

individuals, were conducted to produce the 30 items.

These were then normed using a population of 562 mentally

retarded individuals in 45 workshops representing all

geographic areas of the United States. According to

Downie (1972) the test appears to have face validity,

but the potential responses are in ambiguous terms, such

as "hardly ever." Recommendations are for further study

with the instrument to establish predictive validity

through additional norms.










The Vocational Adjustment Rating Scale for the Re-

tarded (Song & Song, 1971) is used to measure the per-

formance and specific behaviors of the client in all types

of work settings. The test population was composed of

113 clients (age range, 15-44; IQ range, 36 to 76) from

a rehabilitation center. Five areas (work ability, work

habits, withdrawn behavior, aggressive behavior, and

bizarre behavior) were rated on a five point continuum

ranging from "very poor" to "excellent." The same super-

visory personnel who had completed the rating scale also

completed classification of the workers, thus possibly

contaminating the results and explaining the high correla-

tions that were found. On a two week follow-up, employers'

ratings only displayed predictive validity (r = .42) on

scale number two (work habits). The authors blame the

lack of predictive validity and differences in criteria

between that of the workshop and that required on the job.

The use of work samples and rating scales in voca-

tional evaluation of the retarded has received sharp criti-

cism by authorities in the field of special education.

Wolfensberger (1967) contends that the work sample has

rather doubtful value in evaluating the mentally retarded

because of a number of behavioral as well as practical

considerations. He suggests that caution be taken in

that work sampling can deaden motivation, since no reward

is generally given for working on samples. Also, the










work is often contrived or make believe and clients often

do not exert their optimum effort in performing these

tasks. Wolfensberger also indicated that some work samples

appear to measure skills that should be taught for long

periods of time before being evaluated, such as the co-

ordinated use of tools. He warned that in working through

a series of work tasks, clients are often shifted from

task to task and do not remain long enough in one activity

to reach their highest level of performance (limited

task performance may not accurately predict later per-

formance in the retarded).

Wolfensberger (1967) suggests that we reduce evalua-

tion time and increase training experiences with regard

to mentally retarded clients. Jastak and King (1972)

and Brolin (1976) encourage multiple administration of

evaluation batteries, with the role of training or ex-

ploring as an objective, to increase the client's func-

tioning ability. Patterson (1964) points out that what

often is overlooked in discussions of the work sampling

approach is that research indicates "that it is not only

or mainly, inability or lack of manual skill which makes

for failure in employment" (p. 145). Thus the research

tends to support the need to further investigate the use

of present work samples as a predictive or evaluative

instrument with the mentally retarded. Rather it may be

necessary to investigate other variables important for










successful employment and other uses of the work sample

approach, such as exploratory or training tools.

Studies support the hypothesis of poor adaptability,

rather than poor ability, as an explanation for low

scores on initial trials on motor tasks with the mentally

retarded. Clarke (1958) suggests that with practice the

mentally retarded closely approach the performance level

of normal individuals who have also had the advantage

of practice. Gordan (1969) and Reynolds and Adams (1954)

found that the difference in performance level between

subjects of high and low ability diminished with increasing

practice. Lower rates of improvement for normal individuals

were explained by a high initial performance, "an activating

of a pool of previously acquired relevant responses carried

over from psycho-motor tasks encountered in everyday situa-

tions" (Reynolds & Adams, 1954, p. 276). -

Current studies relating to performance on work tasks

by the mentally retarded are beginning to show that even

individuals with lower levels of retardation can achieve

success on high level tasks with proper training practice.

Gold (1973), in attempting to obtain base-rate data of

nonreinforced performance on acquisition and production

with moderately and severely retarded individuals produc-

ing a complex assembly, found an exceptionally high rate

and quality of performance in the absence of pay. The

findings offered an indication that more should be done










to increase the level and value of work which the re-

tarded do, instead of attempting to develop larger evalua-

tion and reinforcement systems. With the general increase

in productivity and the drop in mean errors, the author

concluded that the task itself had strong reinforcing

properties.

Gold (1973) also noted a nonsignificant relationship

between acquisition and production, rejecting the assump-

tion that a strong relationship exists between learning

ability and production (assumption made by TOWER (1967),

JEVS (1968), and other evaluation systems and standardized

instruments). The data (Gold, 1973) indicate that current

evaluation procedures result in a misleading underestimate

of the performance capabilities of the retarded (Gold,

1972) and raise serious questions about the limiting ef-

fects of current vocational diagnostic and evaluative

procedures for the mentally retarded. Gold (1973) suggests

that future research focus on (a) program development

rather than organization, (b) procedures of evaluation

which produce demonstrably useful information rather than

more prediction, and (c) development of universal rein-

forcement systems and clarification of training objectives.

As reflected by the VEWAA Vocational Evaluation

Project Final Report (1975), the current objectives for

vocational evaluation services seem to be progressing

toward a more treatment oriented service, rather than










the strict assessment and predictive procedure. The

current objectives are to identify an optimal outcome

for the individual, to intensify the functional compe-

tencies and functional disabilities of the individual

(medical, psychological, social, vocational, educational,

cultural, economic), to identify those services needed

to overcome the functional disabilities, and to reduce

or eliminate functional disabilities of the individual

in a new situation.

There appears to be evidence to suggest that current

assessment and evaluation systems affect the client in a

positive manner. Gwillian (1970) identified several direct

client benefits from vocational evaluation, including

greater awareness of goals, a better understanding of

personal abilities, capacity, and potential, along with

more realistic aspirations. The remaining section of

this literature review indicates the importance of these

variables (awareness, interests, and aspirations) in the

career development of the mentally retarded individual.


Vocational Interests and Needs

A number of studies on vocational placement of the

retarded include follow-up and analysis of reasons for

job failure. In the majority of cases the clients were

able to meet the skill and strength demends of the job,

but failures appeared to reflect difficulties pertaining

to work interest, habits, motivation, and understanding










of job requirements (Cohen, 1960; Dinger, 1961; O'Connor,

1957; Peckham, 1951; Peterson & Smith, 1960; Windle,

1961).

For the above reasons there has been a steadily

increasing concern shown in the area of vocational in-

terest evaluation of the mentally retarded. Timmerman

and Doctor (1974) suggest three possible reasons why the

area of interest assessment has lagged behind other areas

of vocational evaluation for the mentally retarded.

These are that the mentally retarded have not been con-

sidered as serious candidates for competitive employment

in the past, that due to the mentally retarded individual's

limited social maturity and exposure to job differences,

their ability to make occupational choices has been ques-

tioned, and lastly, that very few interest assessment

instruments are appropriate for the mentally retarded

population.

Well-known and commonly used interest inventories

such as the Strong Vocational Interest Blank (SVIB) and

the Kuder have proven to be unsatisfactory for use with

the mentally retarded because of their dependence upon

verbal abilities and a wide range of experiences (Brolin,

1976).

With the desire to assess the validity of occupational

choices among various levels of the mentally retarded

population, Cohen and Rusalem (1964) conducted a study










using three subgroups: normal, noninstitutionalized

mentally retarded, and institutionalized mentally re-

tarded. The students were presented nine occupational

values: advancement, benefits, independence, interesting

work, prestige, interpersonal relations, salary, security,

and working conditions. Each student was asked to identify

the most important occupational value (oral and written

presentation). The mean ranks for the nine items were

computed both for the three major groups and for subgroups

of boys and girls. Results indicated a greater resemblance

between subgroups of girls than between groups of boys

which was viewed as a reflection of greater similarities

of perceived roles across intellectual levels. Both

sexes and subgroups placed high value upon job advance-

ment, suggesting that despite intellectual capacity, stu-

dents have high expectations of achieving high-level

vocational objectives. The mentally retarded boys placed

greater emphasis on job benefits and less upon relations

with others and interesting work, thus indicating that

boys seemed to prefer immediate or early gratification.

Whether or not the choices reflect legitimate, consistent

interests was not really established. The author indicated

the need for similar studies with older age groups to

evaluate if interests expressed at the younger age level

remain constant.










Burg and Burnett (1965) noted that with interest

assessment of the mentally retarded (a) tests requiring

even a minimum of reading ability were unrealistic, and

(b) that even in picture tests, response was frequently

to specific people or items within the picture rather

than to the occupation illustrated. To counteract this

affect, the authors devised a set of verbal descriptions

to accompany a standardized interest instrument, the

Geist Picture Interest Inventory (GPII). In some cases

original titles were used, while for the majority of

titles, simplifications of a more descriptive nature were

adopted. During the test administration, the examiner

reads the questions and describes each picture, then

asks the client to indicate his preference by raising

the appropriate number of fingers or pointing to her

preference. This bi-sensory form of administration for

the retarded (no established validity) yielded scores

which were compared with scores from the GEIST manual.

Comparisons indicated that the retarded sample obtained

mean scores exceeding the general population in the areas

of mechanical, clerical, outdoor, social service, and

dramatic. The authors do not draw specific conclusions

from the findings, leaving open the question of valid

interest patterns for the mentally retarded.

Another interest inventory, the Vocational Interest

and Sophistication Assessment (VISA) developed as part










of the Johnstone Project by Parnicky and Kahn (1963),

was designed specifically for a mentally retarded pop-

ulation. The interest portion of the instrument consists

of a number of line drawings showing people working at

various jobs usually considered as suitable for the re-

tarded (males in the areas of construction-maintenance,

farm-grounds, food service, garage, industrial and laundry;

females in the areas of business-clerical, food science,

housekeeping and laundry). The client is shown each

drawing and asked if he would like to perform the work

"a lot," "a little," or "not at all" (problem of response

set with the client).

The sophistication portion of the VISA consists of

a series of questions asked about pictures taken from

the interest portion of the test, to assess the subject's

knowledge of the jobs which are illustrated. According

to the authors, the premise is that for an individual's

interest to have high validity, the interest should re-

flect knowledge of the job area.

Parnicky, Kahn, and Burdett (1968) reported temporal

reliability data for terms of one month and 12 to 18

month test-retest correlation coefficients. The mean,

short-term correlation coefficient for males and females

was +.74. The mean, long-term correlation coefficient

for males was +.85; for females, the r = +.97. Validity

reports consisted of VISA scores favorably compared with










ratings of work supervisors. A later report by Parnicky,

Kahn, and Burdett (1971) indicated that interests, as

measured by the VISA, appear to be independent of the

level of intelligence, CA, and knowledge of jobs. The

strength of associations between IQ and sophistication

scores were low to moderate. Also, a high degree of

construct validity was found through a factor analysis

for both male and female forms of the interest scales.

In a study to determine the effects of various ex-

ploratory treatments upon the sophistication level of

mildly retarded students, Salomone, Lehmann, and Green

(1973) administered the VISA with some perceived short-

comings: (a) the personality of the VISA administrator

may affect the scores, (b) a need for extensive normative

data on the VISA with normal individuals, and (c) the

need for a greater number of realistic, potential job

employment areas for the retarded. Timmerman and Doctor

(1974), in their review of assessment instruments for

the mentally retarded, offer some positive aspects of the

VISA: it gives appropriate jobs in simple, line drawings,

it attempts to deal with the problem of sophistication,

and it has the advantage of the client not having to make

forced choices between pictures. Several problems with

use of the VISA are also pointed out by Timmerman and

Doctor (1974): the simple response format often does not

discriminate between job areas, some of the drawings are










ambiguous as to what the job is or which of the persons

pictured is related to the job, the sophistication element

of the test requires constant updating and change, and

it offers a limited view of a small number of fields which

are illustrated.

The Reading Free Vocational Interest Inventory was

developed by Becker (1967, 1969, 1973) to objectively

assess a retarded individual's performance for particular

work areas. The Guide to Jobs for the Mentally Retarded

(Peterson & Jones, 1964) served as the primary source of

work areas which the mentally retarded have demonstrated

proficiency and productivity within. Eleven male job

cluster scales and eight female scales were developed.

The client was presented with three pictures of job areas

and was allowed to make only one choice, although all

three areas may have been of interest to him.

The normative sample for the study (Becker, 1969)

consisted of 6400 EMR subjects from all geographical

sections of the United States. Norms are available for

male and female subjects in the public schools and from

institutions. Validity of the inventory was obtained

with concurrent testing using the Geist Picture Interest

Inventory (GPII), since it contains male and female forms

similar to the Reading Free Vocational Interest Inventory.

Coefficients of correlation were computed with the interest

scales, resulting in a positive relationship. The test-retest










method of establishing reliability coefficients (two-

week interval) for males and females yielded the follow-

ing values: .75 (public school, male); .92 (institution,

male); .72 (public school, female); .88 (institution,

female). There remains a need to establish predictive

validity of the scales for the Reading Free Vocational

Interest Inventory and to assess its vulnerability to

the same criticisms brought against the Geist and VISA,

due to the many similarities among the three instruments.

The Wide Range Interest Opinion Test (WRIOT) is a

recent test published by Jastak and Jastak (1972), which

is composed of 150 sets of three pictures each. The client

is to respond to the most and least liked pictures in

each set. No reading is required on the part of the client.

The instrument was normed on the general population (adults

and high school students) including a high percentage of

rehabilitation clients. Varying validity coefficients

were obtained through concurrent testing using the GPII,

but the authors discount the value of validity estimates

by implying that the instrument should be validated each

time it is used. Reliability coefficients were obtained

through a split-half method for males and females within

each of the cluster areas. The length of the test is

rather long, especially for those with limited attention

spans, making use of the instrument with the mentally re-

tarded questionable.










Another instrument which focuses specifically upon

competencies needed for postschool adjustment is the

Social and Prevocational Information Battery (SPIB) which

was developed by Halpern, Raffeld, Irvin, and Link (1975)

to evaluate the impact of prevocational (work-study)

programs upon the mildly retarded. The SPIB assesses

knowledge in nine skill areas related to community ad-

justment. Norms were gathered on the mildly retarded

for sample groups from different regions within the United

States. Reliability estimates were established with vary-

ing sample sizes through the test-retest method. Validity

figures were obtained by comparing the relationship between

the nine tests of the battery and the five criterion sub-

scales over a one year period. Only tentative coefficients

were available due to the newness of the instrument.

While reporting work with the Minnesota studies,

Dawis (1967) introduced a theory of work adjustment to

assess the appropriateness of job placement according to

the degree of correspondence between an individual's work

personality (abilities and needs) and the work environment

(abilities required and needs satisfiable by the reinforcer

system of the job). To match an assessed individual with

an assessed work environment satisfactorily assumes that

work environments and personalities are stable and con-

sistent. Dawis (1967) indicates that mental ability and

interests are found to be stable, but admits that the










assumption is a weak point in the theory. The work ad-

justment theory has been used with disadvantaged individuals

primarily, but seems to have implications for prevocational

programs for the mentally retarded.

The need to investigate variables or combinations

of variables other than just vocational aptitude or ability

seems to be well supported in the literature. The failure

of many aptitude instruments to predict or contribute

to the vocational success of the mentally retarded has

encouraged the development of several interest and needs

inventories. This review of literature indicates varying

levels of success, with conflicting conclusions concerning

assessment in this important area of study. Further con-

cerns are with the use of the results of interest assess-

ments to best help the mentally retarded individual to

attain the highest level of career development possible.


Prevocational Guidance and the Mentally Retarded

With the development of several prevocational interest

assessment instruments there is concern for the career

development levels attainable by the mentally retarded

and the necessary prevocational guidance programs necessary

to facilitate this development.

Related to the current interest in vocational adjust-

ment of the mentally retarded, several studies have in-

vestigated variables of concern in prevocational guidance

programs. Ginzberg (1951) described three periods in










the development of occupational choice for individuals

of average ability. During the fantasy period, extending

to 11 years of age, the individual is motivated by hero

figures; there is little reality testing of the appro-

priateness of his choice, which often lacks stability.

During the second stage, 11 to 17 years, the individual

becomes aware of the qualifications (ability and training)

and personal values supporting his choices, with more

realistic conceptions of the preparations needed to attain

the job.

A review of selected studies indicates that the

mildly retarded adolescent may have problems which hinder

him in the development of realistic vocational goals.

Kuhn (1966) interviewed mildly retarded junior high school

students and normal CA and MA controls, reporting that

the mildly retarded students had a significantly lower

level of understanding of their selected future occupa-

tions than the other groups. Katzen (1966), working with

the mildly retarded in a special class, found that the

degree of realism in vocational preference was related

to IQ and degree of self-acceptance. The author also

found that for all subjects there was an inverse relation-

ship between self-acceptance and realism of vocational

choice, but for male groups the degree of realism was

also a function of IQ.










Jones (1967) investigated the relationship between

vocational outlook and type of program with 373 special

class students. Using a self-constructed instrument,

the findings indicated that the appropriateness of voca-

tional preference increased with IQ for both sexes; in

general, females were less realistic in their choices than

males, the degree of realism of vocational choice also

increased with chronological age, and white students were

more realistic than nonwhite.

Folman and Budoff (1971) also studied the relationship

of IQ to the development of realistic vocational choice

(interests and aspirations). The retarded population

was further divided into groupings of high scorers or

gainers and nongainers (determined by scores on the Kohs'

Block Design Test). Findings consisted of the following:

(a) there were few or no differences between retarded

and nonretarded adolescents in vocational development,

(b) the high learning potential retarded students aspired

for and expected to obtain lower level jobs (minimized

the risk of failure), but displayed realistic awareness

and involvement in the future job choice. (c) the nongaining

retarded responded very positively initially concerning

vocational aspirations (wishful thinking), but could not

offer any solid evidence about the reality of his responses

(appeared to be in the fantasy period), and (d) the high

level retarded tend to express the same outcomes as their











socially deprived class peers, thus reflecting the effect

of socioeconomic, noneducational factors upon vocational

development.

Sears (1940) and Ringness (1961) reported that many

retarded children exhibit unrealistically high or low

self-appraisal which tends to reflect in high or low ex-

pectations of occupational choice. Knight (1972) con-

ducted a study to determine whether mildly retarded males

were realistic with regard to occupational aspirations.

A sample of 83 males in special classes (IQ mean, 63.83;

age mean, 12.5) were administered a questionnaire asking,

"What kind of job would you like when you grow up?" (p. 55).

Responses were categorized according to the Dictionary

of Occupational Titles. Results indicated the most fre-

quently mentioned jobs to be of a service nature, yet

these were still viewed as appropriate by the author.

In assessing realisticness the subjects were asked

to respond to the question, "What kind of job do you

think you will have when you grow up?" (Knight, 1972,

p. 55). Chi-square comparisons of the responses to the

two questions were made. A significant change toward

the more realistic aspirations was found for the white

males, but no significant differences between responses

could be found for the black special class males. The

author suggested that the black group had already given

predominantly realistic responses to the first question.










Salomone, Lehmann, and Green (1973) assessed the

vocational sophistication of retarded secondary level

students, and the effectiveness of two different explora-

tory procedures directed toward increasing vocational

sophistication. A sample consisted of 71 students (16

to 21 years old; IQ range 50 to 80). Two treatments

consisted of (a) field trips to several industries and

businesses and (b) lectures about occupational oppor-

tunities for high school graduates. A repeated measures

paradigm involving pre- and posttreatment assessment (VISA)

of the two exploration procedures was used. Findings

indicated a significant difference between the field trip

sophistication gain scores and the control group sophis-

tication gain scores for females. The lecture group gain

scores were significantly greater than the gain scores

for the control group for both males and females. The

authors suggest that occupational experiences which are

not seen as school related and which occur in a small

group can stimulate vocational development in retarded

youth. Several questions were raised from the study con-

cerning the appropriateness of the VISA, the long-term

impact of short treatment, and whether the results re-

flected sound and realistic vocational decisions on the

part of the subjects.

Another explanation for socio-occupational failure

with retarded individuals has been attributed to guidance







55


counselors' limited understanding of the client's per-

sonality and potential within the counseling situation

and the work situation (Brolin, 1976; Cortazzo & Runnels,

1970). Sarason (1953) indicated that counseling may have

only questionable value to the mentally retarded child

because of the difficulty involved in conceptualizing the

behavior of oneself and of others, and retarded individuals

being too limited in their expressive abilities and per-

ceptual skills to gain much benefit.

Fine (1969), using a client centered approach, at-

tempted to conduct group counseling with five retarded

students (IQ 50-80) at the middle school level. After

eight sessions the group was terminated due to the un-

ruliness of the students. Fine suggested the use of a

highly structured setting implementing play therapy or

activity therapy principles. However, studies in the

area of counseling and guidance with the mentally retarded

have been poorly designed, weak in statistical measures

and control, and leave many questions unanswered (Brolin,

1976).

Despite considerable methological problems positive

results have been reported for individual and group coun-

seling efforts with the mentally retarded (Mann, Beaber,

& Jacobson, 1969; Nitzberg, 1972; Snyder & Sechrest, 1959).

Rotman and Golburgh (1967) conducted a group counseling

program with three groups of institutionalized clients










(age 18 to 23; IQ 60 to 75) for the purpose of improving

self-concept and lessening the number of runaways. Even

though the fact that evaluation and assessment techniques

were not very objective, results appeared to be positive.

By the seventh week of the group experience, behavior

had improved, clients appeared to know themselves better,

and a lack of continual runaways was reported for those

in the counseling groups.

Humes, Adamczyk, and Myco (1969) attempted group

counseling with mildly retarded students in the public

school setting, applying standardized instruments and

rating scales as means of assessment. Twelve group

sessions were conducted using a combination structured-

nonstructured approach. Following the sessions, a

battery of self-concept and teacher rating scales were

administered. Results indicated that counseled group

members exhibited significantly more adjustment, as seen

by teacher ratings, than noncounseled group members.

Also, higher scores were found on standardized personality

instruments (California Test of Personality) for the

counseled group. Other measures of self-concept and socio-

metric position did not indicate significance. Several

limitations to the study were pointed out in the areas

of variable control and sample size.

Woody and Herr (1965) and Sternlicht (1966) suggest

that group counseling which is more directive will be more










successful with the retarded than that which is nondirec-

tive. Mann (1969) found self-concept could be improved

with group counseling within a school setting. While

counseling teen-age, mildly retarded black and white

males, anxiety was reduced, and general school behavior

improved. The importance of the establishment of good

rapport for this kind of endeavor was strongly emphasized

from the study findings.

In a survey by Woody and Billy (1966) 94 psychologists

rated the value of counseling with the retarded as follows:

IQs 75-90, of great value; IQs 50-75, some value; IQs

25-50, undecided; and IQs under 25, little value. Some

evidence is offered that professionals are gradually

changing their opinions about the efficacy of counseling

with the retarded, especially at the mild and moderate

levels.

As indicated by Bialer (1967) and Brolin (1976),

the real question is not whether counseling and guidance

is feasible with the mentally retarded, but rather what

methods and procedures will work most effectively in im-

proving the socio-occupational behavior of the population.


Summary of the Literature Review

This review of the literature has examined the major

conceptual areas related to the purposed research: the

areas of prevocational evaluation, including assessment

measures and differing approaches concerning vocational










ability and vocational interest. The last section of

the review dealt with career development of the mentally

retarded and the contributions of prevocational counseling

and guidance toward this development.

From a historical vantage point, the literature in-

dicates tremendous growth in the areas of prevocational

evaluation and assessment, yet a lack of consistent direc-

tion is indicated by the continual development of differing

assessment instruments and evaluation systems. The con-

tinual uncovering of conflicting research findings in the

area contributes to a general conclusion that a sound

knowledge base is lacking in the development of approaches

to prevocational evaluation. Most research findings,

related to prevocational evaluation and treatment of

mentally retarded individuals, are plagued with conflicting

findings and severe methodological criticisms. If any

direction for research can be obtained from the literature,

it would seem to be that it is time to assess and evaluate

the mentally retarded individual from the viewpoint of

what can be done to treat and satisfy the individual's

existing needs and skills, rather than try to predict

success or failure of selected groups. Despite the

paucity of studies related specifically to the effects

of treatment upon the relationship of vocational ability

and vocational interests, the review seems to support

the need and feasibility of such an experiment.














CHAPTER III

PROCEDURES



This study was concerned with variables of impor-

tance in the prevocational evaluation of the mentally

retarded individual. The study investigated the ef-

fects of prevocational guidance treatments upon the re-

lationship between vocational interest and vocational

ability of educable mentally retarded adolescents. Also

investigated were the differences accorded to modes of

treatment presentation and maturity level of sample

groupings.


Statement of Hull Hypotheses

The null hypotheses for this research were

1. There will be no statistically significant differ-

ences ( ( = .05) between pre- and postobservations for

the treatment groups or the control group concerning

the relationship of vocational interest and vocational

ability for educable mentally retarded adolescents.

2. There will be no statistically significant differences

or interactions (o( = .05) between directive presenta-

tion of treatment and nondirective presentation of treat-

ment as to pre- and postobservations concerning the










relationship of vocational interests and vocational

ability for educable mentally retarded adolescents.

3. There will be no statistically significant interac-

tions or differences (< = .05) between presentations

of treatment and maturity levels of the subjects on pre-

and postobservations regarding the relationship of voca-

tional interests and vocational ability for educable

mentally retarded adolescents.


Design

A two-way factorial, repeated measures design was

selected to assess differences accorded to different

presentations of prevocational guidance treatment based

upon the relationship between vocational interests and

vocational abilities (the percentage and directionality

of variation). Individual subject differences were

dealt with through randomization, thus controlling for

subject heterogeneity (Kirk, 1968).

A total of 98 subjects representing three randomly

selected groups were obtained from a common population

of educable mentally retarded adolescents in the City

of Gainesville, Florida. The three randomly selected

groups of subjects were assigned to the P = 3 levels

of (A) (two differing presentations of prevocational

guidance treatment and a Hawthorne group) and observed

under all q = 2 levels of (C) pretestt and posttest

measures).










Structural Model

(ABCS)


Pretest

C1


Direc-
tive A1



Nondirec-
tive A2


Hawthorne
A3


Posttest

C2


(Campbell &
Stanley, 1968)

Experimental
Design 4







R O X1 0

R 0 X2 0


RO 0


B1 = 9th and 10th grades (low level)

B2 = llth and 12th grades (high level)


R = random assign-
ment of subjects

X = treatment

0 = observations


The relationship between vocational interests and

vocational ability (dependent variable) may be attribut-

able to the influence of several different independent

variables. The primary objective of the design for this

research was to assess statistically the influence of

two differing presentations of a prevocational exploratory

treatment upon the dependent variable. The differentia-

tion in treatment presentation consisted of directive

versus nondirective modes of presentation. Literature


(Kirk, 1968)


---------- --------------



-------------------------





----------- t ----------- I










in the area of treatment presentation with the mentally

retarded suggests conflicting evidence concerning use of

the two modes of presentation (Fine, 1969; Sarason, 1953;

Sternlicht, 1966; Woods & Herr, 1965), indicating a need

to determine what methods and techniques will work in the

most effective manner with the mentally retarded, individ-

ually or in small group situation (Bialer, 1967; Brolin,

1976).

A secondary objective of the design for this research

was to determine the interactive influence of maturity

level upon the experimental and control groups and upon

the mode of presentation, as to the relationship between

vocational interests and abilities. Many studies in the

literature are concerned with the vocational development

of the mentally retarded individual and question the re-

lationship between the maturity level and the degree of

realism for vocational interests or goals (Folman &

Budoff, 1971; Jones, 1967; Katzen, 1966; Kuhn, 1966).

The literature review supported the structure estab-

lished for the design and variables tested within the struc-

ture. The selected factorial design allowed for tests

of significant difference between suggested independent

variables, as to their effect upon the dependent variable.


Sample

The population of educable mentally retarded adoles-

cents was drawn from the City of Gainesville in Alachua











County, State of Florida. An available population of

students was selected from each of the three secondary

level schools with classes for the educable mentally

retarded, producing a total of 98 subjects. The selected

subjects in each school were then randomly assigned to

one of three groups, stratified according to grade level.

The random assignment was made to one of two treatment

presentations and one Hawthorne situation.

Each subject

1. was between the ages of 15 and 20 years of age and

in grades 9 through 12.

2. had been administered the Stanford-Binet or Wechsler

Intelligence Scale within the last three years.

3. was placed and was participating in a special educa-

tion program for educable mentally retarded students.

The subjects sampled were free of auditory and visual

deficits unless corrected to within normal limits. The

subjects reflected the population ratio of male-female

and black-white for educable mentally retarded classes

in North Central Florida.


Instrumentation

The pretest and posttest assessment over the two

treatment groups and single control consisted of a mea-

surement of vocational interests and an analysis of vo-

cational ability. Care was taken in the selection of

measurement instrumentation to insure appropriateness










for the population sampled (low or nonreading level)

and the desired variables under consideration (a wide

range of occupational areas in which the educable men-

tally retarded adolescent might have an initial interest).


Picture Interest Exploration Survey (PIES)

The Picture Interest Exploratory Survey is a visual,

nonreading format, career interest inventory developed

by Educational Achievement Corporation (1975). PIES

is unique in that occupations are represented on color

slides showing only workers' hands performing tasks repre-

sentative of occupations. This format is to help the

student to focus on the task rather than on the sex of

the worker, the worker's physical attractiveness, and

the worker's racial or ethnic characteristics. The in-

strument was developed to satisfy the need for an interest

survey that would not require students to read, that

would by easy to administer and score, that would hold

students' attention, and would allow students not familiar

with the terminology of the world of work to respond

accurately.

The PIES contains 13 occupational cluster areas,

with 12 specific careers represented within each cluster,

based upon the Occupational Outlook Handbook (OOH). The

156 slides that are in the PIES were selected from over

1100 slides based on the following criteria: representa-

tiveness of a specific occupation and of a general cluster










(Appendix A), quality of photography, and the realistic-

ness of the occupation pictured. An audio tape, placed

for an eight second interval between slides, accompanies

the slide set and provides pacing for the inventory.

The administration and scoring of the instrument are

simple and quick processes involving only the skills of

circling numbers and counting up the number circles. The

response form contains 13 rows each representing a cluster

area, thus making interpretation of high interest areas

for specific occupations and general clusters relatively

simple.


Worker Trait Components: Dictionary of Occupational

Titles

An analysis of vocational abilities, as related

to the job areas of expressed interests made use of the

Worker Trait component of the Dictionary of Occupational

Titles (DOT). The abilities, personal traits, and in-

dividual characteristics required of a worker in order

to achieve average successful job performance are referred

to as worker traits (U.S. Department of Labor, 1965).

Occupational information, within 22 areas of work,

was based on the analysis of required worker traits broken

down in six distinct components: (a) the amount of general

educational development and specific vocational prepara-

tion a worker must have, (b) the specific capabilities

and abilities required in order to learn or perform tasks










of the job, (c) preferences for certain types of work

activities, (d) types of work situations to which an

individual must adjust, (e) physical activities required

in a work situation, and (f) physical surroundings prevalent

in jobs.

A qualification profile for each worker trait group

indicated the range of required traits and/or level of

traits for each of the six components. For the purposes

of assessing worker trait abilities for this study only

the first two components were utilized: (a) educational

development and specific vocational preparation levels

and (b) specific capacities and abilities within appro-

priate areas. Levels from high to low attainment or

capacities were established (U.S. Department of Labor,

1965) for the areas of general educational development,

specific vocational training, and selected aptitude areas

(See Appendix B).

A qualification profile for each worker trait group

is based on the established levels indicating adequate

or satisfactory levels necessary for success within that

occupational area. Appropriate levels for the educable

mentally retarded were readily established across the

profile areas.


Method

The three secondary level public high schools in

the City of Gainesville, Alachua County, in the State of










Florida served as sites for instrument assessment and

conduction of the experiment. Trained graduate assis-

tants from the Department of Special Education, University

of Florida, served as assistants during the assessment

and treatment stages of the experiment. All participating

assistants were trained to perform standardized tasks

within the treatment group in which they were assigned

(Appendix C).

Initially, all 98 subjects were administered the

Picture Interest Exploration Survey (PIES) in small group

settings of 15 subjects or less. A procedural check

for visual and auditory problems among the subjects was

conducted as part of the assessment instructions. The

interest assessment involved approximately 30 minutes,

providing a survey of interests over 156 jobs with 13

cluster areas (Appendix A). Each subject identified

those job areas that he had knowledge of and had a general

interest by means of circling "yes" or "no" on the answer

form.

Subjects were selected from each of the three secon-

dary level schools and randomly assigned in a stratified

manner, according to maturity levels, to the three groups.

At the completion of the assessment process treatment

began for Groups I and II, and similar attention with

unrelated activities began with Group III (Hawthorne

group). Treatment Groups I and II experienced a standardized










prevocational exploratory treatment based upon Phase II,

Expanding Career Alternatives: An Exploratory Exercise

(Friel & Carkhuff, 1974), with minor adaptations for use

with the mentally retarded. The approximately two hour

treatment involved an expanding of career alternatives

in terms of the subject's individual needs and interests,

desired levels of education and training, and local com-

munity needs. An in depth standardized structure of the

treatment, with examples, is provided in Appendix C.

The two treatment groups were differentiated by the

mode of presentation of career related information to the

subjects within the group. Group I received treatment

in a directive manner (all direction, suggestive informa-

tion of a career or occupational nature was from the

group leader's frame of reference). Group II received

treatment in a nondirective manner (all directing, sug-

gestive information of a career or occupational nature

was from the subject's frame of reference). For the

purposes of maintaining consistency and standardization

of presentation the same group leaders and assistants

were used for all subjects within their specific group.

Group III, a Hawthorne group, participated in ac-

tivities with similar attention as given in the treatment

groups, but with materials unrelated to prevocational

exploration. Immediately following the intensive two hour

treatment stage all subjects were administered the post-

assessment of the PIES.










Analysis of Data

Since the experimental design which was applied for

this study is a repeated measures paradigm with a pre-

and posttreatment assessment of three groups, the necessary

method of statistical analysis was to determine pre-

post difference scores (percentage and directionality)

and make differential group comparisons (within and among

groups) using the difference scores (Campbell & Stanley,

1968).

To establish values for the dependent variable (re-

lationship between assessed interest and analyzed abilities)

a procedure of placing the number of job areas selected

within acceptable ability levels over the number of job

areas selected with interest was employed. If a subject

selected 90 jobs of which he indicated a general interest

and 30 of those jobs were within his ability level, as

determined by the DOT worker trait qualification profiles,

then the value would be .33.

An analysis of variance was conducted to determine

the influence of the independent variables (prevocational

treatment, modes of presentation, and maturity levels)

upon the dependent variable (relationship between assessed

vocational interests and analyzed vocational ability).

Appropriate tests of simple main effects and/or a poste-

riori comparison among means were conducted when appro-

priate to determine significance among and within groups.







70


The BMD 08V computer library program (Dixon, 1973) was

employed to compute the two-way analysis of variance and

provide mean values.














CHAPTER IV

RESULTS



Introduction

The results of this study concerned the effects of

differing presentations of prevocational guidance treat-

ments upon the relationship between the vocational interest

and vocational ability of educable mentally retarded

adolescents. Data were obtained from a total population

of 98 secondary school level EMR adolescents from three

urban high schools in the City of Gainesville, Alachua

County, Florida. Due to incomplete data, the deletion

of some subjects resulted in a final sample of 78 subjects.

In those groups containing more than the desired

26 subjects, individuals were selected for the group through

use of a table of random numbers (Walker & Lev, 1953).

A total of 78 subjects representing three randomly formed

groups of 26 subjects were stratified across grade levels

nine through 12.

Data were obtained for the effects of three inde-

pendent variables upon the dependent variable. The in-

dependent variables consisted of a prevocational explora-

tory treatment, two modes of treatment presentation (direc-

tive and nondirective), and the interactive effects of










subject maturity levels. The dependent variable was

the relationship between vocational interests and voca-

tional ability, formulated by correlating inventoried

vocational interests with vocational ability levels es-

tablished through the worker trait components of the

Dictionary of Occupational Titles.

The statistical procedures used to test for overall

effects within the repeated measures design and to screen

for interaction effects were a two-way analysis of variance

test for equal cell sizes performed on the pre-post ob-

servations for all subjects. All possible two-way inter-

actions were simultaneously investigated. A significant

F-statistic for the total multivariate analysis indicated

need for further investigation by means of tests for

simple main effects and simple effects. A significance

level of .05 was selected as the level indicative of the

need for further investigation.


Findings

An F-statistic of .21 was obtained for the main

effect A (combined pre-post observations for the three

groups) which was not statistically significant at the

.05 level (critical F value was 3.13). F values for

the multivariate analysis of variance are presented in

Table 1 and group means are presented in Appendix D.


















r-i c
C11 CYO


cl r-4 r-I cl


4-3

o <;
0 41
-ri rt 0
43 0


Q) Oc C
;-, F-
0Q Q
0 4-)


0 E,
4 4 i-
*Q o <)
0 ~-I
+-


C"] y-i cq


aI-





0
a)
u 4-1-


0


0
Q) 0
wo


E-


I
I -



0) CdC
a)





a)
m 0





)4-- Cd

4- 0 )
Cd) 914 4
CO0
-1X>0--
-l- 0+I

-il0


>1
0- > U


0

: 02 4-
0-.- 0
4 -) -)


00
a) -H
4-' 4

a) -'-+-


ErH










For the main effect B (maturity levels; low level,

grades 9 and 10 and high level, grades 11 and 12) an F-

statistic of 1.32 was obtained between the two established

maturity levels for combined pre-post observations; the

value was nonsignificant at the .05 level with a critical

F value of 3.96. These findings indicated a lack of

statistically significant differences between nestings

formed according to maturity levels when combining pre-

post observations and experimental groupings.

The test for significant differences between pre-

and postobservations over all groups and nestings,

concerning the relationship between vocational interests

and vocational ability, yielded a significant F-sta-

tistic of 21.47 at the .05 level. Thus a statistically

significant difference was found between pre- and post-

observations when combining groups and nestings.

Included with the tests of main effects was an

analysis for interaction among variables. Tests for

significant interactions between treatment groups and

maturity level nestings (AB) yielded a nonsignificant

F-statistic of .59 indicating a lack of any significant

interaction between treatment groups and nested maturity

levels.

The repeated measures design was selected because

of its power for tests of pre-post observation and treat-

ment variable interaction (Kirk, 1968). In this study










a statistically significant interaction (F-statistic of

6.67 at the .05 level) was found between treatment groups

for pre-post observations (AC). Results indicated the

need for further analysis of simple main effects to de-

termine specifically where significant interaction existed

between specific treatment groups on pre-post observations.

Significant interaction (F-statistic of 5.96) at

the .05 level was also found between maturity level nest-

ings for pre-post observations (BC). The need for further

analysis through tests of simple main effects was indicated

as necessary to determine specific areas of significance.

Discussion of the results of these findings is presented

on page 76.

A final multivariate test was for significant three-

way interaction (ABC) or significant differences or in-

teractions between specific treatment groups and maturity

levels on pre-post observations. The general analysis

of variance source table (Table 1) provides an F-statistic

of 1.22 for the three-way interaction, which was statis-

tically nonsignificant at the .05 level. In review, the

source table (Table 1) for the multivariate analysis

indicates significance for the single main effect C (over-

all pre-post observations) and significant interactions

for AC (specific treatment groups for pre-post observations)

and for BC (specific maturity levels for pre-post observa-

tions).










Further univariate analysis through tests of simple

effects was appropriate and necessary to determine spe-

cifically the location of significant interaction within

the AC (specific treatment groups for pre-post observa-

tions) and BC (maturity level nestings for pre-post ob-

servations) groupings. Results for the simple effects

analysis are presented in Table 2.

When analyzing comparisons of all treatment groups

on preobservations an F-statistic of 1.14 at the .05

level was observed as not being statistically significant

(critical F value was 3.04). Also, a nonsignificant F-

statistic of 1.07 at the .05 level was observed among

treatment groups on postobservations. The two tests in-

dicated a lack of statistically significant differences

between randomly selected treatment groups on either the

preobservations or postobservations, when analyzed sep-

arately.

Further interactive tests were conducted to determine

significant differences between treatment groups concerning

pre-post gain observations. The purpose of the tests

was to determine specifically which, if any, of the treat-

ment groups produced statistically significant gains

concerning pre-post observations. The F-statistic of

pre-post observation differences for Treatment Group I

(directive presentation of prevocational exploration

experience) was 4.64, statistically significant at the
















0 *
*ll tlO l H H t- O ti
4-3 oo 0 cq 0 0 CO
Cd
Sr- H CM Cl N14







0)
p
Cd
Q0 LO co cN HV C)t m o O o
HO H H D r-i Cq CY) CN r- D

Cd





00
CO
H0

0 C

4 u C.
o0
CO CO 0)
C<] H- C) CK Ol C) H- C) to tO 0
CO Ct') C~l CD H- Cl CO) Cl H- CD
iz1 H 00C



SC>
2 *4-
SCuCO

S^ C~l C~l H- H- H- H- H- H- H- t-
Q 0

1-I o-c

co CE En CO Cl CdC) CdU cid-' I + C>od-. cdQ_ to
H- U '-/ CO 0) CO C) '. 0
r. < 4-) E3 +-) i ls-, pq^- 04 -^ Z 0i
4-- 4-) 0 O r OU -1 0 (0'- O) ((
Cd Cd *rq ) *H0 ) *H' 4- U' 4-)U r-) H L) q > Ir
n 4-) E 4-3 E 4-) cd' S --. > +a Q) 4-)
(D 0( car Cd -3 Cd 4-J Cdr r Cd (1) Cdr it.
0) 0 a 0 > cd > Cd > 3 () CO >i-l > 4-'
0 ::0 -H 4 (U p a) 0 r-l r-0l p ?0 >1 Cd
0 0-r-1 0 4-)- Q ;-, a) F- C) ? Q cc a) co aQ ) () 4 -P
+-) ; Cd COE-i COHE-4 ( > z > zc o-) (w-rq H
0 OCd c > o .3 .3 a 0 a 0 ,0 -H .0! ? z
CO > 0 (0) a) 0 a) 4 .,1 1-1 H 0o ?- 0 3 Cd
4-) ;- 4-) (1) > > r. 4-) 4-) z 4- Q
0 (3 r. (n +-P T^ -P -H- 4-)- >1 C3 >1 Cd +- -4-3 41 + cd *-iq
(1)C Q).0 +-) ( nO4-) COO 4-) > 4-) > c 0C3 wS 4H
E,0 20 0 0 0 0 0X3 *Hr A -q4 0? OS 0
4-) 0 4-) +-P P- (L) 12L (1) C| +-P a)i a) a, "U & Cd 0)B co ~ 1 1 P- 1i I : j 3 ) 03 (n3 I &C Icu
(1) F4 ) 0 0) q-H Q -H CL) 4-) .0 4-) 0) W 0 0)
S $4 CL -, m $- m O Cd0 q:k $L, m O
E-H &A pq pq Pi S8 P,











.05 level (critical F value was 3.91). Thus, statistically

significant pre-post gains were attained through a direc-

tive presentation of treatment, concerning the relation-

ship between vocational interests and vocational ability

for EMR adolescents.

Treatment Group II nondirectivee presentation of

prevocational guidance experience) yielded a nonsignifi-

cant F-statistic (.85 at the .05 level) concerning pre-

post observations. Also, Treatment Group III (Hawthorne)

yielded a F-statistic of 1.64, which was nonsignificant

at the .05 level. Since Treatment Groups II and III did

not yield significant differences between pre-post ob-

servations, further investigation was not performed.

As indicated on the multivariate source table (Table 1)

a significant interaction was found at BC (maturity levels

nested within treatments for pre-post observations).

Univariate analysis was pursued to determine significant

pre-post differences for B1 (low maturity level, grades

9 and 10) and for B2 (high maturity level, grades 11 and

12). The F-statistic for all nestings of low maturity

levels (B1) was 1.07, not significant at the .05 level.

Nestings of high maturity levels (B2) yielded a F-statis-

tic of 4.14, for pre-post observations, which was found

to be statistically significant at the .05 level. Thus,

significant pre-post observation differences were achieved

for the nested high maturity levels. Further analysis










was indicated to determine specifically which treatment

group contained the high maturity level with significant

pre-post observation differences.

Further univariate analysis (Table 3) was conducted

to determine which treatment presentation group contained

the significant high maturity level nesting. Tests for

significant pre-post observation differences were conducted

for each maturity level within each of the three treatment

presentations. For the directive presentation of treat-

ment, the high maturity level nesting yielded a statis-

tically significant F-statistic of 4.14 at the .05 level.

The high maturity level group for nondirective presentation

of treatment yielded a F-statistic of .57 and the high

maturity level nesting with the Hawthorne group yielded

a F-statistic of .07, both being statistically nonsignifi-

cant at the .05 level. Further univariate analysis for

pre-post observation differences for the low maturity

level nesting were not performed because simple effect

differences were found to be nonsignificant.

A summary of the multivariate analysis indicates

that there were significant differences between overall

pre-post observations, significant interactions between

treatment presentations and pre-post observations, and

maturity levels and pre-post observations. Further uni-

variate analysis produced statistically significant pre-

post observation differences for the directive mode of




































































Cd 0


0 H
0a)
*H >



. > <.
>t^1 0,

a) 4) ;-1
(02 *iH 0
o2 ^
0
4-) >
+j cn *-,

0 U
I a)
() *,-i .,-H


4-a)o$-
d 0





OD-
0 a)
o s+ a
-H >
4- Q) 0

> C
i>>

0 4J-
a -J 0

0 -rq

I UO Z
(1) *-l4 0


4--) P
d 0

Ca

.1-I >


^. >> 0
a4-) Q)
O a
.,.-t

>0


$04 0


o, ,0
0

0 (-1
0 -H!C
& .C -,I =










presentation for prevocational exploration treatment and

for the high maturity level nested within the directive

treatment presentation group.


Hypotheses

Hypothesis 1. There will be no statistically significant
differences between pre- and postobservations for
the treatment groups or the Hawthorne group concern-
ing the relationship of vocational interests and
vocational ability for educable mentally retarded
adolescents.

Results of the multivariate analysis for differences

in pre-post treatment observations yielded a significant

F-statistic of 21.47 at the .05 level. Mean scores (Ap-

pendix D) for the postobservations increased significantly

over preobservations, specifically for Treatment Group A

(directive presentation). Treatment Group II nondirectivee

presentation) and the Hawthorne group did not yield sig-

nificant mean pre-post observation differences. The

critical F value necessary for one or all of the groups

to indicate significant pre-post observation differences

and for rejection of the null hypothesis was 3.98 at the

.05 level of significance. The findings support the

rejection of the first null hypothesis at the .05 level

of confidence.


Hypothesis 2. There will be no statistically significant
differences or interactions between pre- and post-
observations for directive presentation of treatment
and nondirective presentation of treatment concerning
the relationship of vocational interests and voca-
tional ability for educable mentally retarded adoles-
cents.










The multivariate analysis yielded a significant F-

statistic of 6.67 for the interaction of treatment presenta-

tion mode and pre-post observation differences (critical

F value was 3.31 at the .05 level of significance).

Further investigation of this significant interaction in

a univariate analysis yielded a F-statistic of 4.64 for

pre-post observation differences for the group receiving

directive presentation of treatment; an F-statistic of

.85 was found for pre-post observation differences with

the group receiving nondirective presentation of treat-

ment. The F-statistic for pre-post observation differences

with directive presentation of treatment was statistically

significant at the .05 level of confidence. Comparisons

of mean scores (Appendix D) on pre-post observations for

the two modes of presentation reveal larger significant

gains for the directive mode of presentation as compared

with the nondirective mode of presentation. The findings

show statistically significant pre-post differences for

the directive mode of presentation as opposed to nonsig-

nificant differences for the nondirective mode of presenta-

tion, thus supporting the rejection of the second null

hypothesis at the .05 level of confidence.


Hypothesis 3. There will be no statistically significant
interactions or differences between presentation of
treatment and maturity levels of the subjects on pre-
and postobservations concerning the relationship
of vocational interests and vocational ability for
educable mentally retarded adolescents.










The multivariate analysis revealed a F-statistic

of 5.96 for the interaction of nested maturity level and

pre-post observations, which was found to be statistically

significant at the .05 level (critical F value was 3.91).

Further investigation through univariate analysis yielded

a F-statistic of 1.07 for interaction of low maturity

level nestings and pre-post observation differences, and

a F-statistic of 4.64 for interaction of high maturity

level nestings and pre-post observation differences.

The F-statistic for interaction between pre-post observa-

tional differences and high maturity level nestings was

statistically significant at the .05 level. Comparisons

of group mean scores (Appendix D) for low maturity level

nestings (B1) and high maturity level nestings (B2) reveal

differences in pre-post observations.

Further univariate analysis determined which of the

specific treatment group-nesting interactions contributed

toward the significance. The following F-statistics

were found: 4.14 for pre-post observation differences

and high maturity level subjects nested within the direc-

tive treatment group; .57 for pre-post observation dif-

ferences and high maturity level subjects nested within

the nondirective treatment group; and .07 for pre-post

observation differences and high maturity level subjects

nested within the Hawthorne group (Table 3). The F-statistic

of 4.14 was statistically significant indicating that







84


high maturity level subjects produced higher pre-post

observation differences, specifically for the directive

mode of treatment presentation, thus supporting the re-

jection of the third null hypothesis at the .05 level

of confidence.














CHAPTER V

SUMMARY, DISCUSSION, AND RECO.I!E-1DATIONS



Summary

The purpose of this study was to investigate the

effects of a prevocational exploratory experience upon

the relationship, between assessed vocational interests

and vocational ability levels for educable mentally re-

tarded adolescents. Further investigation concerned

differences or interactions in the relationship between

vocational interests and vocational ability levels ac-

corded to directive and nondirective presentation of

treatment and high or low maturity levels. Specifically

the study investigated three main hypotheses related to

the effects of a prevocational exploration treatment,

different modes of treatment presentation and subject

maturity level upon the relationship between vocational

interests and vocational ability levels.

The available population of 98 educable mentally

retarded adolescents in grades 9 through 12 was selected

from the special class rosters of the three public sec-

ondary schools in the City of Gainesville, Alachua County,

Florida. These students were randomly assigned within

their school to one of three groups, stratified according










to grade level. The deletion of some subjects from the

total population, due to incomplete data, resulted in a

sample of 78 subjects for the study. All subjects were

pretested and separated into randomly selected groups;

one group received a directive presentation of treatment,

a second group received a nondirective presentation of

treatment, and a third group received similar attention

with different content.

The approximately two hour treatment involved an ex-

perience in expanding career alternatives, regarding the

subjects' needs and interests, desired levels of educa-

tion and training, and local community needs. Prepared

graduate assistant served as group leaders and aides,

each performing standardized tasks specific to the treat-

ment group to which they were assigned (Appendix C).

Random portions of the treatment were taped as a veri-

fication of treatment consistency and a check for differ-

ences in methods of treatment presentation (Appendix E).

The pretest and posttest assessment over the three

treatment groups consisted of a measurement of vocational

interests and an analysis of vocational ability levels.

A visual, nonreading interest inventory was administered

to all subjects, surveying interests over 156 jobs within

13 cluster areas. The Worker Trait component of the Dic-

tionary of Occupational Titles (DOT) was used to conduct

an analysis of vocational abilities. The dependent variable










was formulated through a procedure of placing the number

of job areas selected within acceptable ability levels

over the number of job areas selected with interest.

The data collection process involved six prepared graduate

assistants for a period of two weeks.

A multivariate analysis was performed on pre-post

observations and among groups for significant differences

accorded to treatment effects, differing modes of treat-

ment presentation, and maturity level of subjects. When

the analysis of variance produced a F-statistic significant

at the .05 level further investigation was done through

univariate analysis for each significant effect of inter-

action.

Findings for the multivariate analysis indicated a

lack of significant differences between treatment groups

or nestings for preobservations or postobservations when

investigated separately. A significant F-statistic was

found for pre-post observation differences for all groups

and nestings, with significant interaction between treat-

ment presentation and pre-post observations, and maturity

levels and pre-post observation differences. Further

univariate analysis indicated significant pre-post observa-

tion differences for the directive presentation of pre-

vocational exploration treatment and for the high maturity

level nested within the directive treatment group. An

examination of group means (Appendix D) indicates that










definite postobservation gains were made for both treat-

ment groups over preobservation scores, but significant

differences among treatment and control groups were not

attained. Further examination of group means for maturity

level nestings indicates significant pre-post observation

gains for the high maturity level nesting, but lacks sig-

nificant differences for postobservations between low

maturity level subjects and high maturity level subjects.


Discussion

The experience of a prevocational guidance treatment

appears to have had some effect upon the relationship

between vocational interests and vocational abilities

for mentally retarded adolescents, however findings from

this study fail to show comparative significant differences

between treatment groups and the Hawthorne group on post-

observations. The practical significant differences found

in favor of the treatment groups, specifically the direc-

tive treatment presentation, were reflected in pre-post

observation difference gains. The failure of the analysis

of posttest observations to maintain the same relation-

ships found in analysis of pre-post observation differences

indicates the need for caution concerning interpretation

of the results, even though the 78 subjects for which

complete data were available were randomly selected from

the total population.










Similar studies by Knight (1972) and Salomone,

Lehmann, and Green (1973) assessed the effectiveness of

two different exploratory procedures (field trips and

lectures) finding significant gain scores for the treat-

ment groups, as opposed to the control group. Knight

(1972) suggested that prevocational experiences which

are not seen as school related and which occur in small

groups can stimulate vocational growth in the retarded

youth. While pre-post observation gains for this study

were obtained for all groups, with the exception of the

low maturity level Hawthorne group, the only statistically

significant gains were attained with the directive presenta-

tion treatment group. This finding is in agreement with

other attempts to work in a group setting with the mentally

retarded (Fine, 1969; Woody & Herr, 1965). In concurring

agreement with studies reviewed, it appears that nondirec-

tive group leadership with the mentally retarded may

effect subject interaction and self-disclosure, but does

not appear to have a decisive effect upon decisions re-

garding career interests.

The maturity level of the mentally retarded adoles-

cent, regarding career development, has long been a ques-

tion of concern by researchers (Folman & Budoff, 1971;

Jones, 1966; Kuhn, 1966). Findings from this study in-

dicate a lack of significant differences between high

and low maturity levels on separate pre- and postobservation










comparisons, but significant pre-post observation gains

were made for the high level maturity group. Treatment,

especially through directive presentation, was statistically

more effective for llth and 12th grade level EMRs, in-

dicating that readiness for consideration of realistic

careers is more evident with the older, higher grade

level mentally retarded adolescent.

The findings for this study, even though lacking

statistical significance among group differences, offer

direction of an experimental data based nature for educa-

tors of the mentally retarded adolescent. The important

questions of what needs to be done, regarding prevocational

exploratory treatment, how it needs to be done, regarding

directive or nondirective presentation, and when it should

be done, regarding career development maturity level,

have been investigated as related to the relationship of

vocational interest and vocational ability.

The following factors may have limited the findings

of this study:

1. The prevocational guidance experience was of two

hour duration. This period may have been too short in

some cases to affect major cluster area changes often

necessary to improve the relationship between interest

and ability.

2. The instrument used in this study may not have been

sensitive to small changes, accorded to the treatment,










specifically for semiskilled jobs within traditional

professional cluster areas.

3. The research design and statistical analysis did not

control for any effect due to differing school-teacher

attitudes toward the treatment and the class in general.

4. The small loss in population size from original assign-

ment to completed data may have biased the final results.

This would be particularly suspect with those subjects

who were eliminated because of erroneous or missing data

due to lack of self-control or absenteeism.

5. Random selection of subjects for this study produced

a sample of subjects which did not necessarily seem to

perceive a need or desire for such an experience. The

lack of felt need may have prevented experimental subjects

from reaping maximum benefit from the experience.


Recommendations

Specific recommendations arising from this study

might include the following:

1. Other studies using the prevocational exploratory

experience as an experimental variable should be conducted

to clarify discrepancies in relationships found in the

analysis of group differences on postobservations and

the analysis of pre-post difference observations in this

study.

2. Studies using the prevocational exploratory experience

as an experimental variable should be conducted using an










instrument that is more sensitive to small changes in

subjects. Changes should include measures that reflect

paraprofessional and semiskilled jobs within traditional

professional work clusters.

3. Studies using the prevocational exploratory experience

as a treatment variable should assess the effects of dif-

fering lengths and concentrations of treatment.

4. Studies using the prevocational exploratory experience

as a treatment should be expanded to include multiple

treatments to test for possible differential effects of

sex and maturity with different treatments.

5. Studies using the prevocational exploratory experience

as a treatment should assess the effects of differential

group composition other than that found in the present

study.

6. Studies using the prevocational exploratory experience

as a treatment should be conducted in an actual occupa-

tional setting, as compared to an academic setting within

the public school environment.

7. Studies using the prevocational exploratory experience

as a treatment should be conducted with adolescents who

request career development services or who have a desire

for such information or assistance.

8. A follow-up study of the subjects used for this study

should be conducted to determine whether the relationship

between interest and ability remains stable, concerning

treatment effects.




Full Text

PAGE 1

7+( ())(&76 2) 35(92&$7,21$/ *8,'$1&( 21 7+( 5(/$7,216+,3 %(7:((1 92&$7,21$/ $%,/,7< $1' ,17(5(676 2) ('8&$%/( 0(17$//< 5(7$5'(' $'2/(6&(176 %\ 52%(57 $1'5(: 672''(1 $ ',66(57$7,21 35(6(17(' 72 7+( *5$'8$7( &281&,/ 2) 7+( 81,9(56,7< 2) )/25,'$ ,1 3$57,$/ )8/),//0(17 2) 7+( 5(48,5(0(176 )25 7+( '(*5(( 2) '2&725 2) 3+,/2623+< 81,9(56,7< 2) )/25,'$

PAGE 2

$&.12:/('*0(176 7KH DXWKRU ZLVKHV WR DFNQRZOHGJH WKH IROORZLQJ SHRSOH ZKR KHOSHG ZLWK FRPSOHWLRQ RI WKLV VWXG\ 0U -RH &DVDOH 0V 'HEELH +DUW]HOO 0V 9LFNLH *RQ]DOHV 0U 6WHYH %RLVVRQHDXOW DQG 0U -LP 0F%ULGH JUDGXDWH DVVLVWDQWV LQ WKH 'HSDUWPHQW RI 6SHFLDO (GXFDn WLRQ 8QLYHUVLW\ RI )ORULGD IRU WKHLU DVVLVWDQFH DQG SDUWLFLSDWLRQ LQ WKH WUHDWPHQW VWDJH RI WKLV VWXG\ 7KH VHFRQGDU\ OHYHO VSHFLDO HGXFDWLRQ WHDFKHUV LQ WKH &LW\ RI *DLQHVYLOOH )ORULGD IRU DOORZLQJ WKHLU VWXGHQWV WR SDUWLFLSDWH LQ WKH H[SHULPHQWDO VWDJH RI WKLV VWXG\ 'U 6WXDUW ( 6FKZDUW] 'U :LOOLDP 5 5HLG 'U &HFLO 0HUFHU 'U &DU\ / 5HLFKDUG DQG 'U -DPHV : ,OHQVHO PHPEHUV RI WKH ZULWHUnV VXSHUYLVRU\ FRPPLWWHH ZKRVH DWWHQWLRQ DQG DVVLVWDQFH KDYH EHHQ HVVHQWLDO

PAGE 3

7$%/( 2) &217(176 $&.12:/('*0(176 LL /,67 2) 7$%/(6 Y $%675$&7 YL &+$37(5 ,,1752'8&7,21 6WDWHPHQW RI WKH 3UREOHP 3XUSRVH RI WKH 6WXG\ 6WDWHPHQW RI WKH +\SRWKHVLV 'HOLPLWDWLRQV $VVXPSWLRQV 'HILQLWLRQ RI 7HUPV ,,5(9,(: 2) 5(/$7(' /,7(5$785( +LVWRULFDO 2YHUYLHZ 9RFDWLRQDO (YDOXDWLRQ $SSURDFKHV 6WDQGDUGL]HG 7HVWLQJ $SSURDFK :RUN 6DPSOH $SSURDFK 5DWLQJ 6FDOH DQG 6LWXDWLRQDO $VVHVVn PHQW $SSURDFK 9RFDWLRQDO ,QWHUHVWV DQG 1HHGV 3UHYRFDWLRQDO *XLGDQFH DQG WKH 0HQWDOO\ 5HWDUGHG 6XPPDU\ RI WKH /LWHUDWXUH 5HYLHZ ,,,352&('85(6 6WDWHPHQW RI 1XOO +\SRWKHVHV 'HVLJQ 6DPSOH ,QVWUXPHQWDWLRQ 3LFWXUH ,QWHUHVW ([SORUDWLRQ 6XUYH\ 3,(6f :RUNHU 7UDLW &RPSRQHQWV 'LFWLRQDU\ RI 2FFXSDWLRQDO 7LWOHV 0HWKRG $QDO\VLV RI 'DWD LLL

PAGE 4

,9 5(68/76 ,QWURGXFWLRQ )LQGLQJV +\SRWKHVHV 9 6800$5< ',6&866,21 $1' 5(&200(1'$7,216 6XPPDU\ 'LVFXVVLRQ 5HFRPPHQGDWLRQV 5()(5(1&(6 $33(1',; $ 3,(6 6/,'( '(6&5,37,21 % :25.(5 75$,7 /(9(/6 ,OO & 35(92&$7,21$/ (;3/25$725< 75($70(17 *5283 &(// 0($16 ( 75($70(17 7$3(6 %,2*5$3+,&$/ 6.(7&+ ,9

PAGE 5

/,67 2) 7$%/(6 7DEOH 3DJH $QDO\VLV RI 9DULDQFH 6RXUFH 7DEOH 6LPSOH 0DLQ (IIHFWV 6RXUFH 7DEOH 6LPSOH (IIHFWV 6RXUFH 7DEOH Y

PAGE 6

$EVWUDFW RI 'LVVHUWDWLRQ 3UHVHQWHG WR WKH *UDGXDWH &RXQFLO RI WKH 8QLYHUVLW\ RI )ORULGD LQ 3DUWLDO )XOILOOPHQW RI WKH 5HTXLUHPHQWV IRU WKH 'HJUHH RI 'RFWRU RI 3KLORVRSK\ 7+( ())(&76 2) 35(92&$7,21$/ *8,'$1&( 21 7+( 5(/$7,216+,3 %(7:((1 92&$7,21$/ $%,/,7< $1' ,17(5(676 2) ('8&$%/( 0(17$//< 5(7$5'(' $'2/(6&(176 %\ 5REHUW $QGUHZ 6WRGGHQ $XJXVW &KDLUPDQ 'U :LOOLDP 5 5HLG 0DMRU 'HSDUWPHQW 6SHFLDO (GXFDWLRQ 7KLV VWXG\ DVVHVVHG WKH HIIHFWV RI D SUHYRFDWLRQDO H[SORUDWRU\ WUHDWPHQW XSRQ WKH UHODWLRQVKLS EHWZHHQ DVVHVVHG YRFDWLRQDO LQWHUHVWV DQG YRFDWLRQDO DELOLW\ OHYHOV RI HGXFDEOH PHQWDOO\ UHWDUGHG DGROHVFHQWV 7KH VWXG\ LQYHVWLJDWHG WKUHH PDMRU K\SRWKHVHV FRQFHUQHG ZLWK SUH DQG SRVWREVHUYDWLRQ GLIIHUHQFHV RU LQWHUDFWLRQV LQ WKH UHODWLRQVKLS EHWZHHQ YRFDWLRQDO LQWHUHVWV DQG YRFDn WLRQDO DELOLW\ OHYHOV DFFRUGHG WR WKH HIIHFWV RI D SUHn YRFDWLRQDO H[SORUDWLRQ WUHDWPHQW GLIIHUHQW PRGHV RI WUHDWPHQW SUHVHQWDWLRQ DQG VXEMHFW PDWXULW\ OHYHO $ VDPSOH RI HGXFDEOH PHQWDOO\ UHWDUGHG DGROHVFHQWV LQ JUDGHV WKURXJK ZHUH UDQGRPO\ DVVLJQHG ZLWKLQ YL

PAGE 7

WKHLU VFKRRO WR RQH RI WKUHH JURXSV VWUDWLILHG DFFRUGLQJ WR JUDGH OHYHO 7KH VXEMHFWV ZHUH SUHWHVWHG ZLWK WKH 3LFWXUH ,QWHUHVW ([SORUDWLRQ 6XUYH\ DQG GLYLGHG LQWR WKH IROORZLQJ UDQGRPO\ IRUPHG JURXSV *URXS UHFHLYHG D GLUHFWLYH SUHVHQWDWLRQ RI WUHDWPHQW *URXS ,, UHFHLYHG D QRQGLUHFWLYH SUHVHQWDWLRQ DQG *URXS ,,, UHFHLYHG VLPLODU DWWHQWLRQ ZLWK GLIIHUHQW FRQWHQW $ WZR KRXU H[SORUDWRU\ WUHDWPHQW LQYROYHG WKH H[n SDQGLQJ RI FDUHHU DOWHUQDWLYHV DV WR VXEMHFW QHHGV DQG LQWHUHVWV GHVLUHG OHYHOV RI HGXFDWLRQ RU WUDLQLQJ DQG ORFDO FRPPXQLW\ QHHGV 'DWD ZHUH REWDLQHG IRU WKH HIIHFWV RI WKUHH LQGHSHQGHQW YDULDEOHV WUHDWPHQW SUHVHQWDWLRQ PDWXULW\ OHYHOVf XSRQ WKH GHSHQGHQW YDULDEOH IRUPXODWHG E\ FRUUHODWLQJ LQYHQWRULHG YRFDWLRQDO LQWHUHVWV ZLWK YRFDWLRQDO DELOLW\ OHYHOV HVWDEOLVKHG WKURXJK WKH ZRUNHU WUDLW FRPSRQHQWV RI WKH 'LFWLRQDU\ RI 2FFXSDWLRQDO 7LWOHVf 7KH VWDWLVWLFDO SURFHGXUHV FRQVLVWHG RI D PXOWLYDULDWH DQDO\VLV RI YDULDQFH IROORZHG E\ DSSURSULDWH XQLYDULDWH DQDO\VLV $ VLJQLILFDQFH OHYHO RI ZDV VHOHFWHG DV WKH OHYHO LQGLFDWLYH RI WKH QHHG IRU IXUWKHU LQYHVWLJDWLRQ 5HVXOWV IRU WKH PXOWLYDULDWH DQDO\VLV LQGLFDWHG D ODFN RI VLJQLILFDQW GLIIHUHQFHV DPRQJ WUHDWPHQW JURXSV RU QHVWLQJV IRU SUHREVHUYDWLRQV RU SRVWREVHUYDWLRQV ZKHQ LQYHVWLJDWHG VHSDUDWHO\ 6LJQLILFDQW SUHSRVW REVHUYDWLRQ GLIIHUHQFHV ZHUH IRXQG IRU WKH GLUHFWLYH SUHVHQWDWLRQ RI YLL

PAGE 8

WUHDWPHQW DQG IRU WKH KLJK PDWXULW\ OHYHO WK DQG WK JUDGHVf QHVWHG ZLWKLQ WKH GLUHFWLYH WUHDWPHQW JURXS 7KH DXWKRU FRQFOXGHG WKDW D SUHYRFDWLRQDO H[SORUDWRU\ H[SHULHQFH IRU HGXFDEOH PHQWDOO\ UHWDUGHG DGROHVFHQWV ZDV VWDWLVWLFDOO\ HIIHFWLYH LQ FRQWULEXWLQJ WR D SRVLWLYH FKDQJH LQ WKH UHODWLRQVKLS EHWZHHQ YRFDWLRQDO LQWHUHVWV DQG DELOLW\ ZKHQ SUHVHQWHG LQ D GLUHFWLYH PDQQHU ZLWK KLJKHU JUDGH OHYHO VWXGHQWV YLLL

PAGE 9

&+$37(5 ,1752'8&7,21 'XULQJ WKH SDVW \HDUV WKHUH KDV EHHQ D SURJUHVn VLYHO\ LQFUHDVLQJ LQWHUHVW LQ WKH VRFLDO DQG SUHYRFD WLRQDO FRPSHWHQFLHV WKDW DUH FULWLFDO GHWHUPLQDQWV RI SRVWVFKRRO YRFDWLRQDO DGMXVWPHQW IRU HGXFDEOH PHQWDOO\ UHWDUGHG DGROHVFHQWV %UROLQ .ROVWRH t )UH\ f $OWKRXJK VRXUFHV HVWLPDWH WKDW WKH PDMRULW\ RI UHWDUGHG SHUVRQV KDYH SRWHQWLDO IRU VDWLVIDFWRU\ VRFLDO DQG YRFDn WLRQDO DGMXVWPHQW *ROGVWHLQ +DOSHUQ .R NDVND 6WULFNODQG f PDQ\ VWXGLHV LQGLFDWH WKLV OHYHO LV XVXDOO\ QRW DWWDLQHG RU LV DFKLHYHG DW YDU\LQJ OHYHOV RI VXFFHVV 'LQJHU 0F)DOO 3HWHUVRQ t 6PLWK 6DHQJHU 7RELDV f 1XPHURXV RWKHU ZULWHUV DUH RSWLPLVWLF DERXW WKH SRVVLn ELOLWLHV RI RFFXSDWLRQDO VXFFHVV IRU PRVW UHWDUGHG SHUn VRQV SDUWLFXODUO\ LI SURYLGHG ZLWK RFFXSDWLRQDOO\ RULn HQWHG JXLGDQFH DQG WUDLQLQJ &KDIILQ +DULQJ t 6PLWK 'L0LFKDHO .DW] 2OVKDQVN\ :ROIHQVEHUJHU f 7KHVH VWXGLHV VXJJHVW WKDW Df WKH UHWDUGHG KDYH PDQ\ XQWDSSHG SRWHQWLDOLWLHV LQ WKH SHUVRQDO VRFLDO DQG RFFXSDWLRQDO GRPDLQV Ef OLWWOH LV NQRZQ DERXW WKH PHQWDOO\ UHWDUGHG LQGLYLGXDOnV VRFLDO

PAGE 10

DQG YRFDWLRQDO DELOLWLHV DQG Ff JLYHQ IXUWKHU UHVHDUFK DQG EHWWHU WUDLQLQJ SURJUDPV WKH UHWDUGHG LQGLYLGXDO FDQ UHDFK D KLJKHU OHYHO RI SHUIRUPDQFH WKDQ ZDV HDUOLHU WKRXJKW DWWDLQDEOH 0DUWLQ f HVWLPDWHG WKDW RQO\ b RI WKH KDQGLn FDSSHG FKLOGUHQ OHDYLQJ VFKRRO LQ WKH QH[W IRXU \HDUV ZLOO EH IXOO\ HPSOR\HG b XQGHUHPSOR\HG b XQHPSOR\HG b VKHOWHUHG DQG b DOPRVW WRWDOO\ GHSHQGHQW 7KHVH ILJXUHV VHHP JHQHUDOO\ YDOLG LQ OLJKW RI FXUUHQW UHSRUWV RI YRFDWLRQDO DGMXVWPHQW RI WKH PHQWDOO\ UHWDUGHG ,Q GHWHUPLQLQJ WKH H[WHQW RI YRFDWLRQDO DGMXVWPHQW VXFFHVV ZLWK WKH PHQWDOO\ UHWDUGHG %UROLQ 'XUDQG .URPHU DQG 0XOOHU f IROORZHG XS IRUPHU PLOGO\ UHWDUGHG VWXGHQWV DQG IRXQG WKDW RQO\ b KDG DQ DYHUDJH RU DERYH DGMXVWPHQW b KDG D IDLU DGMXVWPHQW DQG b KDG D SRRU DGMXVWPHQW 7KHVH IROORZXS HVWLPDWHV ZHUH IXUWKHU VXEn VWDQWLDWHG WKURXJK ILQGLQJV E\ 2OVKDQVN\ DQG %HDFK f WKDW RQO\ VOLJKWO\ PRUH WKDQ KDOI RI WKH IRUPHU ZRUNHUV DW D UHKDELOLWDWLRQ ZRUNVKRS ZHUH ZRUNLQJ D \HDU RU PRUH DIWHU SODFHPHQW .RNDVNDnV f UHYLHZ RI IROORZXS VWXGLHV IRXQG WKDW PDQ\ IRUPHU VSHFLDO FODVV VWXGHQWV ZHUH WUDLQHG SODFHG DQG UHPDLQHG HPSOR\HG LQ YDU\LQJ VHUYLFH SRVLWLRQV EXW SHUKDSV WUDLQLQJ DQG SODFHPHQW SHUVRQQHO PD\ EH FKDUn DFWHUL]LQJ WKH PHQWDOO\ UHWDUGHG DV EHLQJ WRR OLPLWHG FRQFHUQLQJ YRFDWLRQDO DELOLWLHV ,Q D ODWHU DUWLFOH

PAGE 11

.RNDVND f ZDUQHG WKDW WKH OLPLWHG UHVHDUFK DQG NQRZOHGJH RI YRFDWLRQDO WUDLQLQJ DQG DGMXVWPHQW RI VSHn FLDO FODVV VWXGHQWV PD\ EH DIIHFWLQJ WKH VRFLRRFFXSD WLRQDO VXFFHVV RI PHQWDOO\ UHWDUGHG LQGLYLGXDOV IRUFLQJ WKHP WR EHFRPH PDUJLQDO ZRUNHUV ZLWKLQ RXU ODERU IRUFH 2QH RI WKH JUHDWHVW SUREOHPV WKDW PHQWDOO\ UHWDUGHG LQGLYLGXDOV HQFRXQWHU LV WKH FRQWLQXDO PLVXQGHUVWDQGLQJ DQG XQGHUHVWLPDWLRQ RI WKHLU YRFDWLRQDO SRWHQWLDOV 7KH VXFFHVVIXO HYDOXDWLRQ DQG KDELOLWDWLRQ RI WKH PHQWDOO\ UHWDUGHG LQGLYLGXDO LV SUHFOXGHG E\ VHYHUDO DVVXPSWLRQV XQGHUO\LQJ VXFFHVVIXO YRFDWLRQDO DGMXVWPHQW 2OVKDQVN\ f Df WKDW ZH XQGHUVWDQG WKH OHYHO RI LQWHOOLJHQFH UHTXLUHG IRU GLIIHUHQW W\SHV RI MREV Ef WKDW D VORZ OHDUQHU LV D SRRU OHDUQHU Ff WKDW LQWHOOLJHQFH LV D FRQVWDQW DQG JOREDO TXDOLW\ Gf WKDW WKH UHWDUGHG SHUVRQ FDQ WROHUDWH ERULQJ WDVNV DQG Hf WKDW DOO UHWDUGHG SHUVRQV KDYH WKH VDPH SHUVRQDOLW\ FKDUDFWHULVWLFV 6LQFH PDQ\ RI WKHVH PLVFRQFHSWLRQV FRQWLQXH WR SUHn YDLO DPRQJ SURIHVVLRQDOV DQG HPSOR\HUV RI WKH UHWDUGHG WKH WDVN RI PHDQLQJIXO SUHYRFDWLRQDO HYDOXDWLRQ DQG KDELOLWDWLRQ LV TXLWH GLIILFXOW %UROLQ f IRXQG WKDW b RI WKH PHQWDOO\ UHWDUGHG FOLHQWV ZKR KDG EHHQ H[WHQVLYHO\ HYDOXDWHG DW D GLDJQRVWLF FHQWHU ZHUH QRW DFKLHYLQJ WKHLU YRFDWLRQDO SRWHQWLDO GHVSLWH FRQWLQXHG UHKDELOLWDWLRQ VHUYLFHV

PAGE 12

6HYHUDO UHVHDUFKHUV LQGLFDWH WKDW GXH WR WKH PDQ\ IDFWRUV UHODWHG WR YRFDWLRQDO GHYHORSPHQW DQG DGMXVWPHQW QR VLQJOH PHDVXUH LV OLNHO\ WR EH RI KHOS LQ SURJUDPPLQJ RU SUHGLFWLQJ WKH H[WHQW RI YRFDWLRQDO DGMXVWPHQW VXFFHVV RU IDLOXUH IRU WKH PHQWDOO\ UHWDUGHG %XWOHU t %URZQLQJ *ROGVWHLQ t +HEHU +HEHU :LQGOH f 7KH FXUUHQW DEVHQFH RI D XQLWDU\ YRFDWLRQDO SUHGLFWLRQ IRUPXOD VXJJHVWV WKDW VXFFHVVIXO SHUVRQDO VRFLDO DQG YRFDWLRQDO RXWFRPHV DUH WKH SURGXFW RI PDQ\ LQWHUDFWLQJ YDULDEOHV 6FDWWHUHG VWXGLHV LQYHVWLJDWH LQGLYLGXDO YDULDEOHV %HFNHU LQYHQWRULHG LQWHUHVW )DUEHU EDVLF VRFLHWDO H[SHFWDQFLHV 6DORPRQH /HKPDQQ t *UHHQ YRFDWLRQDO VRSKLVWLFDWLRQf ZKLOH IHZ LQn YHVWLJDWH WKH UHODWLRQVKLS EHWZHHQ YDULDEOHV LQYROYHG LQ WKH SUHYRFDWLRQDO HYDOXDWLRQ SURFHVV *ROG SURGXFn WLYLW\ DQG WDVN FRPSOH[LW\ /RITXLVW 'DZLV t :HLVV ZRUN SHUVRQDOLW\ DQG YRFDWLRQDO FKRLFHf 'HVSLWH WKH YRFDWLRQDO DGMXVWPHQW UHVHDUFK DQG H[SHUW RSLQLRQV WKDW LQGLFDWH PRVW PHQWDOO\ UHWDUGHG LQGLYLGXDOV FDQ UHDFK D VXFFHVVIXO YRFDWLRQDO OHYHO IXUWKHU UHVHDUFK LQGLFDWHV D JUHDW QXPEHU RI UHWDUGHG LQGLYLGXDOV DUH KDYLQJ GLIILFXOW\ DGMXVWLQJ WR WKH ZRUOG RI ZRUN 8Qn HPSOR\PHQW DQG XQGHUHPSOR\PHQW DUH FRQVWDQW SUREOHPV IRU WKH PHQWDOO\ UHWDUGHG GXH WR XQGHUHVWLPDWLRQV DQG PLVFRQn FHSWLRQV DV WR WKHLU YRFDWLRQDO SRWHQWLDOV ODFN RI NQRZOHGJH DQG XQGHUVWDQGLQJ DV WR ZKLFK YDULDEOHV RU

PAGE 13

LQWHUDFWLRQ RI YDULDEOHV FRQWULEXWH WR YRFDWLRQDO DGn MXVWPHQW DQG WKH ODFN RI H[SHULPHQWDO UHVHDUFK XSRQ ZKLFK WR EDVH SUHYRFDWLRQDO H[SORUDWLRQ DQG HYDOXDWLRQ SURJUDPV 6WDWHPHQW RI WKH 3UREOHP 5HVHDUFK FRQFHUQHG ZLWK YRFDWLRQDO DGMXVWPHQW RI WKH PHQWDOO\ UHWDUGHG KDV IDLOHG WR LGHQWLI\ YDULDEOHV RU WKH LQWHUDFWLRQ RI YDULDEOHV QHFHVVDU\ IRU VXFFHVVIXO YRFDWLRQDO DGMXVWPHQW LQ WKH ZRUOG RI ZRUN DQG WKH GHYHORSn PHQW RI HIIHFWLYH SUHYRFDWLRQDO H[SORUDWLRQ DQG HYDOXDWLRQ SURJUDPV ,Q DQ HIIRUW WR PHHW WKH WRWDO OLIH FDUHHU GHYHORSPHQW QHHGV RI WKH PHQWDOO\ UHWDUGHG DGROHVFHQW D UHFHQW HPSKDVLV XSRQ FDUHHU HGXFDWLRQ &ODUN f DQG SUHYRFDWLRQDO HYDOXDWLRQ 3UXLWW t /RQJIHOORZ f KDV HYROYHG ZLWKRXW GLUHFWLRQ IURP FRQVLVWHQW VRXQG UHVHDUFK VWXGLHV 7KH SUREOHP IRU WKH SUHVHQW UHVHDUFK VWXG\ LV D TXHVWLRQ FRPPRQO\ HQFRXQWHUHG LQ FDUHHU HGXFDWLRQ DQG SUHYRFDWLRQDO HYDOXDWLRQ DQG H[SORUDWLRQ SURJUDPV IRU WKH HGXFDEOH PHQWDOO\ UHWDUGHG :KDW LV WKH UHODWLRQVKLS EHWZHHQ YRFDWLRQDO LQWHUHVW DQG YRFDWLRQDO DELOLW\ DQG ZKDW DUH WKH HIIHFWV RI SUHYRFDWLRQDO H[SORUDWRU\ WUHDWn PHQWV XSRQ WKLV UHODWLRQVKLS" 3XUSRVH RI WKH 6WXG\ 7KH SXUSRVH RI WKLV VWXG\ ZDV WR H[DPLQH WKH HIIHFWV RI D SUHYRFDWLRQDO JXLGDQFH H[SHULHQFH XSRQ WKH UHODWLRQVKLS

PAGE 14

EHWZHHQ DVVHVVHG YRFDWLRQDO LQWHUHVWV DQG YRFDWLRQDO DELOLW\ OHYHOV IRU HGXFDEOH PHQWDOO\ UHWDUGHG DGROHVFHQWV )XUWKHU FRQFHUQV ZHUH WKH H[DPLQDWLRQ RI GLIIHUHQFHV DQG LQWHUDFWLRQV LQ WKH UHODWLRQVKLS EHWZHHQ YRFDWLRQDO LQn WHUHVWV DQG YRFDWLRQDO DELOLW\ OHYHOV DFFRUGHG WR GLUHFn WLYH YHUVXV QRQGLUHFWLYH SUHVHQWDWLRQ RI WUHDWPHQW DQG GLIIHUHQW PDWXULW\ OHYHOV IRU HGXFDEOH PHQWDOO\ UHWDUGHG DGROHVFHQWV 6HOHFWLRQ RI WKH SDUWLFXODU YDULDEOHV ZDV EDVHG XSRQ DQ H[SUHVVHG QHHG LQ WKH SURIHVVLRQDO OLWHUDWXUH WR VWXG\ H[SHULPHQWDOO\ VRPH RI WKH DUHDV RI PLVFRQFHSWLRQ DQG DVVXPSWLRQ ZLWK UHJDUGV WR SUHYRFDWLRQDO H[SORUDWLRQ DQG HYDOXDWLRQ RI WKH PHQWDOO\ UHWDUGHG 7KH H[SUHVVHG ODFN RI UHVHDUFK EDVHG NQRZOHGJH WKH FRQIOLFWLQJ ILQGLQJV DQG DUJXPHQWV DQG WKH LQWHQVH PHWKRGRORJLFDO FULWLFLVP RI H[LVWLQJ VWXGLHV SURYLGH SXUSRVH IRU WKH SUHVHQW UHn VHDUFK VWXG\ 6WDWHPHQW RI WKH +\SRWKHVLV 7KH YDULDEOHV RI D SUHYRFDWLRQDO JXLGDQFH H[SHULHQFH LQ HLWKHU D GLUHFWLYH RU QRQGLUHFWLYH PRGH RI SUHVHQWDWLRQ DQG WKH GLIIHULQJ PDWXULW\ OHYHOV RI VHFRQGDU\ OHYHO HGXn FDEOH PHQWDOO\ UHWDUGHG LQGLYLGXDOV ZLOO QRW LQGLYLGXDOO\ RU LQ LQWHUDFWLYH FRPELQDWLRQ VWDWLVWLFDOO\ DIIHFW WKH UHODWLRQVKLS EHWZHHQ DVVHVVHG YRFDWLRQDO LQWHUHVWV DQG YRFDWLRQDO DELOLW\

PAGE 15

'HOLPLWDWLRQV 7KH WRSLF IRU WKLV UHVHDUFK FRQFHUQHG WKH UHODWLRQn VKLS EHWZHHQ YRFDWLRQDO LQWHUHVWV DQG YRFDWLRQDO DELOLW\ RI HGXFDEOH PHQWDOO\ UHWDUGHG DGROHVFHQWV 7KH WDUJHW SRSXODWLRQ IRU WKLV VWXG\ ZDV FRPSRVHG RI VHFRQGDU\ VFKRRO (05 VWXGHQWV ZKR KDG EHHQ SODFHG LQ VSHFLDO VHOI FRQWDLQHG FODVVHV SULRU WR HQWHULQJ VHFRQGDU\ VFKRRO 7KH LQDFFHVVDELOLW\ RI UHVRXUFH URRP RU PDLQVWUHDP PRGHOV OLPLWHG JHQHUDOL]DWLRQV WR WKRVH (05 VWXGHQWV LQ VHOI FRQWDLQHG FODVVURRPV 7KH JHQHUDOL]DELOLW\ RI WKH VWXG\ ZDV DOVR OLPLWHG E\ WKH JHRJUDSKLFDO ORFDWLRQ QDWXUH DQG VL]H RI WKH VDPSOH D VLQJOH XUEDQ DUHD XQLYHUVLW\ FLW\f LQ 1RUWK &HQWUDO )ORULGD 7KHUHIRUH FDXWLRQ VKRXOG EH H[HUFLVHG LQ VSHFLILF DWWHPSWV WR JHQHUDOL]H ILQGLQJV IURP WKLV VWXG\ WR SRSXODWLRQV RXWVLGH RI WKH VDPSOH VHOHFWHG $VVXPSWLRQV )RU WKH SXUSRVHV RI WKLV VWXG\ LW ZDV DVVXPHG WKDW HGXFDEOH PHQWDOO\ UHWDUGHG VWXGHQWV DVVLJQHG WR VSHFLDO FODVVHV DFWXDOO\ KDYH QHHGV WKDW FDQQRW EH PHW LQ SUHVHQW DGPLQLVWUDWLYH DQG LQVWUXFWLRQDO DUUDQJHPHQWV VHUYLQJ FKLOGUHQ RI QRUPDO LQWHOOLJHQFH WKDW WKH VDPSOHG VWXGHQWV UHVSRQGHG DFFRUGLQJ WR WKHLU DFWXDO LQWHUHVWV DQG QRW DFFRUGLQJ WR WKH H[SHFWDQF\ RI RWKHUV

PAGE 16

WKDW WKH UDQGRP VDPSOH RI HGXFDGOH PHQWDOO\ UHWDUGHG DGROHVFHQWV ZDV UHSUHVHQWDWLYH RI WKH JHQHUD@ SRSXODWLRQ RI HGXFDEOH PHQWDOO\ UHWDUGHG WKDW WKH GHSHQGHQW YDULDEOH ZDV PHDQLQJIXO WR WKH LQGHSHQGHQW YDULDEOH 'HILQLWLRQ RI 7HUPV 9RFDWLRQDO (YDOXDWLRQ WKH DSSUDLVDO RI WKH LQGLYLGXDOnV FDSDFLW\ LQFOXGLQJ SDWWHUQV RI ZRUN EHKDYLRU DELOLW\ WR DFTXLUH RFFXSDWLRQDO VNLOOV DQG WKH VHOHFWLRQ RI DSSURn SULDWH YRFDWLRQDO JRDOV (GXFDEOH 0HQWDOO\ 5HWDUGHG $GROHVFHQWV D VHFRQGDU\ OHYHO VFKRRO VWXGHQW EHWZHHQ WKH FKURQRORJLFDO DJHV RI DQG ZKR LV DVVLJQHG WR D VSHFLDO VHOIFRQWDLQHG FODVV IRU HGXFDEOH PHQWDOO\ UHWDUGHG VWXGHQWV LQ WKH SXEOLF VFKRROV 'LUHFWLYH UHIHUV WR D SUHVHQWDWLRQ RI WUHDWPHQW IURP WKH IUDPH RI UHIHUHQFH RI WKH JURXS OHDGHU OHDGHU PDGH VXJJHVWLRQV DQG FRPPHQWV IURP KLV EDFNJURXQG DQG SDVW H[SHULHQFH FRQFHUQLQJ FDUHHU FKRLFHV RI WKH VXEMHFWf 1RQGLUHFWLYH UHIHUV WR D SUHVHQWDWLRQ RI WUHDWPHQW IURP WKH IUDPH RI UHIHUHQFH RI WKH PHQWDOO\ UHWDUGHG VWXGHQW OHDGHU GLG QRW PDNH VXJJHVWLRQV DQG FRPPHQWV IURP KLV EDFNJURXQG RU SDVW H[SHULHQFH FRQFHUQLQJ FDUHHU FKRLFHV VXJJHVWLRQV FDPH IURP LQIRUPDWLRQ RIIHUHG E\ WKH VWXGHQWf 0DWXULW\ /HYHOV UHIHUV WR JUDGH DFKLHYHPHQW OHYHOV

PAGE 17

WKURXJK DQG WKURXJK JUDGH DFKLHYHPHQW OHYHOV WKURXJK ZHUH WKH ORZ OHYHO PDWXULW\ JURXS JUDGH DFKLHYHPHQW OHYHOV WKURXJK ZHUH WKH KLJK OHYHO PDWXULW\ JURXSf 3UHYRFDWLRQDO UHIHUV WR DQ\ FRXQVHOLQJ JXLGDQFH RU WUDLQLQJ RI DQ RFFXSDWLRQDO QDWXUH EHIRUH WKH DFWXDO HPSOR\PHQW VWDJH

PAGE 18

&+$37(5 ,, 5(9,(: 2) 5(/$7(' /,7(5$785( 7KLV UHYLHZ RI WKH OLWHUDWXUH LV GLYLGHG LQWR WKUHH VHFWLRQV SHUWDLQLQJ WR WKH WRSLF RI VWXG\ 7KH ILUVW VHFWLRQ FRQVLVWV RI D EULHI KLVWRULFDO RYHUYLHZ RI SUH YRFDWLRQDO HYDOXDWLRQ DQG WUDLQLQJ RI WKH PHQWDOO\ UHn WDUGHG 7KH VHFRQG VHFWLRQ FRQVLVWV RI D UHVHDUFK UHYLHZ RI YRFDWLRQDO HYDOXDWLRQ DSSURDFKHV DQG DVVHVVPHQW PHDVXUHV DSSURSULDWH ZLWK WKH PHQWDOO\ UHWDUGHG VWDQGDUGL]HG DELOLW\ DVVHVVPHQWV ZRUN VDPSOHV LQWHUHVWV DQG QHHGV DVVHVVPHQWVf 7KH WKLUG VHFWLRQ FRQVLVWV RI D UHYLHZ RI VWXGLHV LQ WKH DUHD RI SUHYRFDWLRQDO FRXQVHOLQJ DQG JXLGDQFH HIIRUWV ZLWK WKH PHQWDOO\ UHWDUGHG +LVWRULFDO 2YHUYLHZ 7KH GHYHORSPHQW RI SUHYRFDWLRQDO WUDLQLQJ SURJUDPV IRU WKH PHQWDOO\ UHWDUGHG KDV EHHQ D UHODWLYHO\ UHFHQW SKHQRPHQRQ LQ WKH KLVWRU\ RI $PHULFDQ HGXFDWLRQ %\ WKH V VHYHUDO VHFRQGDU\ OHYHO WUDLQLQJ SURJUDPV H[LVWHG LQ ODUJHU XUEDQ FHQWHUV +XQJHUIRUG +XQJHUIRUG t 5RVHQZHLJ 0RQHV f +XQJHUIRUG f IRUPXODWHG WKH FRQFHSW RI RFFXSDWLRQDO HGXFDWLRQ ZLWK WKH LQWHQWLRQ RI VKLIWLQJ WKH HGXFDWRUnV DOOHJLDQFH IURP

PAGE 19

DEVWUDFW DFDGHPLFV WR D YLJRURXV HPSKDVLV XSRQ WKH UHn WDUGHG LQGLYLGXDOnV DELOLW\ WR DVVXPH YRFDWLRQDO UHVSRQn VLELOLW\ 7KH ODERU GHILFLW FUHDWHG E\ WKH 6HFRQG :RUOG :DU DVVLVWHG WKH PHQWDOO\ UHWDUGHG LQ GHPRQVWUDWLQJ WKHLU HPSOR\DELOLW\ DQG PDQ\ FDSDELOLWLHV 7KH HQG RI WKH ZDU EURXJKW DERXW QHZ UHDOL]DWLRQV FRQFHUQLQJ WKH JRDOV SXUn SRVHV DQG IXQFWLRQV RI SXEOLF HGXFDWLRQ FRPPXQLW\ DJHQFLHV DQG WKH IHGHUDO JRYHUQPHQW ,QWHUHVW DPRQJ UHKDELOLWDWLRQ UHVHDUFKHUV LQ SUHGLFWn LQJ YRFDWLRQDO VXFFHVV DPRQJ WKH PHQWDOO\ UHWDUGHG EHJDQ ZKHQ &RZGHU\ f YHULILHG WKDW SHUVRQV RI OLPLWHG PHQWDO VSHFLILFDWLRQV FRXOG VXFFHVVIXOO\ WDNH SDUW LQ WKH FRQn VWUXFWLYH DQG XVHIXO ZRUN RI VRFLHW\ 8QWLO UHFHQWO\ WKH PHQWDOO\ UHWDUGHG KDG EHHQ YLVXDOL]HG DV SHUIRUPLQJ RQO\ DW XQVNLOOHG OHYHOV 'L0LFKDHO .RNDVND 2VZDOG f $ QHZ DSSUHFLDWLRQ RI WKH ZRUN SRWHQWLDO RI WKH PHQWDOO\ UHWDUGHG HYROYHG IURP DQ LQFUHDVHG XQGHUVWDQGLQJ RI WKH YRFDWLRQDO DVVHWV RI WKLV JURXS ZKLFK DUH RIWHQ QRW UHIOHFWHG LQ VWDQGDUGL]HG PHQWDO WHVWV .HQQHG\ 2n&RQQRU t 7L]DUG f 7D\ORU f FRQFOXGHG VSHn FLILFDOO\ WKDW JHQHUDO LQWHOOLJHQFH WHVWV ZLWK WKHLU HPSKDVLV RQ VFKRODVWLF VNLOOV PD\ OHDYH XQWDSSHG RWKHU YDULDEOHV RI FUXFLDO LPSRUWDQFH IRU WKH PHQWDOO\ UHWDUGHG $SSHOO :LOOLDPV DQG )LVKHOO f IRXQG WKDW YDULDEOHV

PAGE 20

VXFK DV JHQHUDO DELOLW\ YRFDWLRQDO DSSURDFK DWWLWXGLQDO WRQH OHDUQLQJ VSHHG DQG SV\FKRPRWRU DSWLWXGH UHODWHG IDFWRUV ZHUH DEOH WR VLJQLILFDQWO\ GLIIHUHQWLDWH UHSHDWHGO\ FOLHQWV IURP WKRVH ZKR EHFDPH JDLQIXOO\ HPSOR\HG ,Q WKH SURFHVV RI GLVWLQJXLVKLQJ VXFFHVVIXO IURP QRQVXFFHVVIXO UHWDUGHG FOLHQWV ZLWK UHVSHFW IRU MRE SODFHPHQW WKH DXWKRUV IRXQG WKDW QHLWKHU LQWHOOLJHQFH QRU HGXFDWLRQDO OHYHO ZHUH GLVFULPLQDWLQJ YDULDEOHV ,Q IXUWKHU LQYHVWLJDWLRQ IRU D SUHGLFWRU RI YRFDn WLRQDO VXFFHVV *UDJHQW f H[DPLQHG D JURXS RI UHWDUGHG DGXOWV DQG DGROHVFHQWV LQ D *RRGZLOO ,QGXVWULHV YRFDWLRQDO HYDOXDWLRQ DQG WUDLQLQJ SURJUDP QRWLQJ WKDW WKH LQWHOOLn JHQFH TXRWLHQW ZDV D OHVV HIIHFWLYH SUHGLFWRU RI HPSOR\n PHQW WKDQ ZHUH LQWHUYLHZV DQG SURMHFWLYH WHFKQLTXHV 6DOL DQG $PLU f VWXGLHG PHQWDO DJH DV D GRPLQDQW IDFWRU LQ WKH GHWHUPLQDWLRQ RI UHKDELOLWDWLRQ SURVSHFWV 'Hn SHQGHQW YDULDEOHV ZHUH VXFFHVV DW ZRUN DQG VSHFLILF MRE OHYHO ,QGHSHQGHQW YDULDEOHV ZHUH SHUVRQDOLW\ IDFWRUV DIIHFWLQJ WKH ZRUN RI WKH VDPSOH RI UHWDUGHG LQGLYLGXDOV RYHU DJH ,4 WR f 5HVXOWV LQGLFDWHG Df VXFn FHVV DW ZRUN ZDV PRVW FOHDUO\ UHODWHG WR PRWRU FRRUGLQDn WLRQ Ef WKH UHODWLRQVKLS EHWZHHQ ,4 DQG RXWSXW ZDV ZHDN Ff SHUVRQDOLW\ FKDUDFWHULVWLFV DQG SHUIRUPDQFH ZHUH KLJKO\ FRUUHODWHG WR f DQG Gf SHUIRUPDQFH ZDV UHODWHG WR VRFLDO DGMXVWPHQW f

PAGE 21

$OELQ f H[DPLQHG WKH UHODWLRQVKLS RI ,4 WR YRFDWLRQDO VXFFHVV DYHUDJH SURGXFWLRQ LQ WKUHH GLIIHUHQW QRQFRQVHFXWLYH ZHHNVf ZLWK HPSOR\HHV LQ D VKHOWHUHG ZRUNVKRS 7KH GDWD ZHUH DQDO\]HG ZLWK D 6SHDUPDQ FRUUHODn WLRQ DQG WKH DQDO\VLV LQGLFDWHG QR VLJQLILFDQW UHODWLRQn VKLS H[LVWHG ,PSOLFDWLRQV ZHUH WKDW ,4 GRHV QRW VHHP WR EH D SUHUHTXLVLWH RU YDOLG SUHGLFWRU IRU YRFDWLRQDO SURGXFWLRQ 7KHVH VWXGLHV WHQG WR FRQWULEXWH WR WKH JHQHUDO K\SRWKHVLV WKDW PHQWDO DJH LV QRW WKH GRPLQDQW IDFWRU LQ WKH GHWHUPLQDWLRQ RI YRFDWLRQDO SURVSHFWV IRU WKH PHQWDOO\ UHWDUGHG DQG WKDW D FRPSUHKHQVLYH DVVHVVPHQW LQ WKH IRUP RI D FRPSOHWH SUHYRFDWLRQDO HYDOXDWLRQ LV D FRQWULEXWLQJ IRUFH WR VRFLRRFFXSDWLRQDO VXFFHVV RI PHQWDOO\ UHWDUGHG LQGLYLGXDOV ,W LV WKH DVVXPSWLRQ RI PDQ\ %R]DUWK 5RVHQEHUJ f WKDW WKH HYDOXDWLRQ DQG SUHGLFWLRQ RI YRFDWLRQDO SRWHQWLDO DQGRU VXFFHVV LV D KLJKO\ GHYHORSHG SURFHVV 7KH PDLQ WKUXVW RI UHVHDUFK LQ YRFDWLRQDO HYDOXDWLRQ KDV EHHQ WR VHHN VFLHQWLILF VXSSRUW IRU WKH XVH RI SUHGLFWLYH GHYLFHV 2YHUV f ,Q D FRPSUHKHQVLYH UHYLHZ RI VWXGLHV FRQFHUQLQJ WKH SUHGLFWLYH DVVHVVPHQW RI WKH VRFLDO DQG YRFDWLRQDO DGMXVWn PHQW RI WKH UHWDUGHG &REE f FRQFOXGHG WKDW MXGJPHQWV DQG DVVHVVPHQWV PRUH RIWHQ WKDQ QRW XQGHUHVWLPDWH WKH DELOLW\ RI UHWDUGHG SHUVRQV WR VDWLVIDFWRULO\ PHHW FRPPXQLW\ DQG RFFXSDWLRQDO GHPDQGV +H VXPPDUL]HG KLV UHYLHZ DV IROORZV

PAGE 22

7KH PRVW FRQVLVWHQW DQG RXWVWDQGLQJ ILQGLQJ RI DOO IROORZXS VWXGLHV LV LQ WKH KLJK SURn SRUWLRQ RI DGXOW UHWDUGHG ZKR DFKLHYH VDWLVn IDFWRU\ DGMXVWPHQW E\ ZKDWHYHU FULWHULD DUH HPSOR\HG ,W LV PRUH DSSURSULDWH WR PDNH DQ DVVXPSWLRQ RI SRVLWLYH DGDSWDWLRQ RQ VRPH PHDQLQJIXO FULWHULD RI HPSOR\PHQW XQWLO QHJDn WLYH HYLGHQFH DSSHDUV UDWKHU WKDQ DVVXPH D SRRU GLDJQRVLV XQWLO SRVLWLYH HYLGHQFH DSSHDUV S f $V HPSKDVL]HG E\ &REE f WUDGLWLRQDOO\ ZRUN HYDOXDn WLRQ RU SUHYRFDWLRQDO H[SORUDWLRQ KDV EHHQ XWLOL]HG WR SURYLGH DQ HIIHFWLYH PHWKRG RI MXGJLQJ FOLHQW SHUIRUPDQFH ,Q UHFHQW \HDUV WKH FRQFHSW RI ZRUN HYDOXDWLRQ KDV EHHQ PRGLILHG RU EURDGHQHG WRZDUG DQ LQWHUDFWLRQ SURFHVV *HOO PDQ 0RHG f DQ H[DPLQDWLRQ RI WKH SHUVRQ WKH ZRUN VLWXDWLRQ DQG WKH LQWHUDFWLRQ EHWZHHQ WKHP ZKLFK IDFLOLWDWHV WKH WUDQVLWLRQ WR HPSOR\PHQW RI YRFDWLRQDOO\ KDQGLFDSSHG SHUVRQV &XUUHQW REMHFWLYHV IRU YRFDWLRQDO HYDOXDWLRQ VHUYLFHV DV H[SUHVVHG E\ WKH 9RFDWLRQDO (YDOXDWLRQ DQG :RUN $GMXVWn PHQW $VVRFLDWLRQ 9(:$$f 9RFDWLRQDO (YDOXDWLRQ 3URMHFW )LQDO 5HSRUW f UHIOHFW WKH WUHQG DZD\ IURP SUHGLFWLRQ RI VXFFHVV RU IDLOXUH DV D UROH RI HYDOXDWLRQ SURJUDPV 3UHYRFDWLRQDO HYDOXDWLRQ DV D FRPSUHKHQVLYH DVVHVVPHQW DV ZHOO DV D UHPHGLDO WUHDWPHQW SURFHVV VHHPV WR EH HYLGHQFHG E\ FXUUHQW UHSRUWV 9RFDWLRQDO (YDOXDWLRQ 3URMHFW )LQDO 5HSRUW f 1HII f VXJJHVWV WKH IROORZLQJ GLIIHULQJ DSSURDFKHV WR YRFDWLRQDO HYDOXDWLRQ Df VWDQGDUGL]HG SV\FKRORJLFDO WHVWLQJ DSSURDFK Ef ZRUN VDPSOH DSSURDFK Ff MRE DQDO\VLV

PAGE 23

DSSURDFK DQG Gf VLWXDWLRQDO DVVHVVPHQW DSSURDFK LQWHUHVW DQG SHUVRQDOLW\ IDFWRUVf 7KH QH[W SRUWLRQ RI WKLV UHYLHZ FRQFHUQV LWVHOI ZLWK WKH YDULRXV DSSURDFKHV WR YRFDWLRQDO HYDOXDWLRQ DQG UHVHDUFK SHUWDLQLQJ WR WKHLU XVH ZLWK PHQWDOO\ UHWDUGHG LQGLYLGXDOV 9RFDWLRQDO (YDOXDWLRQ $SSURDFKHV 6WDQGDUGL]HG 7HVWLQJ $SSURDFK 6WDQGDUGL]HG WHVWLQJ KDV KDG ZLGH XVH WKURXJK DWWHPSWV WR SUHGLFW YRFDWLRQDO VXFFHVV 2YHUV f VKDULQJ WKH DWWULEXWHV RI HDV\ DQG LQH[SHQVLYH DGPLQLVWUDWLRQ DQG REMHFWLYLW\ DQG DFFHSWDEOH UHOLDELOLW\ $FFRUGLQJ WR 1HII f WKHLU PDLQ GLVDGYDQWDJHV DUH ZLWK ORZ SUHn GLFWLYH YDOLGLW\ DUWLILFLDOLW\ DQG JHQHUDO LQDSSURSULDWHn QHVV IRU XVH ZLWK HYHQ WKH ERUGHUOLQH PHQWDOO\ UHWDUGHG $ EULHI UHYLHZ RI DUJXPHQWV SHUWDLQLQJ WR WKH XVH RI VWDQGDUGL]HG YRFDWLRQDO DELOLW\ WHVWV ZLWK WKH PHQWDOO\ UHWDUGHG \LHOGHG WKH IROORZLQJ VWDWHPHQW E\ GHILQLWLRQ UHWDUGDWLRQ DVVXPHV D VORZ UDWH RI OHDUQLQJ 6WDQGDUGL]HG WHVWV WKDW PHDVXUH RQH WULDO OHDUQLQJ ZLOO UHLQIRUFH WKH LQLWLDO GLDJQRVLV RI UHWDUGDWLRQ ZLWKRXW VXSn SO\LQJ WKH LQIRUPDWLRQ QHFHVVDU\ IRU WKH GHn WHUPLQDWLRQ RI YRFDWLRQDO IHDVLELOLW\ 7RELDV S f ,Q UHODWLRQ WR WKH PHQWDOO\ UHWDUGHG *HOOPDQ f VXJJHVWV WKDW WKH YHU\ VWUHQJWK RI VWDQGDUGL]HG WHVWV PD\ EH WKHLU JUHDWHVW ZHDNQHVV 7KH\ DUH GHVLJQHG DQG DGn PLQLVWHUHG WR PD[LPL]H SHUIRUPDQFH WKXV UHVXOWLQJ LQ D

PAGE 24

SRRU UHIOHFWLRQ RI WKH FOLHQWnV JHQHUDO RULHQWDWLRQ WRZDUG ZRUN RU ZKDW ZRXOG EH H[SHFWHG IURP KLP LQ DQ DFWXDO MRE SHUIRUPDQFH VLWXDWLRQ 7LPPHUPDQ DQG 'RFWRU f GLVn FXVV WKUHH FDXWLRQV FRQFHUQLQJ WKH XVH RI VWDQGDUG WHVWV ZLWK WKH PHQWDOO\ UHWDUGHG Df WKH VSHFLILF UHVXOWV RI VWDQGDUG WHVWV DUH WRR RIWHQ JHQHUDOL]HG WR LQGLFDWH VRPH DOOHQFRPSDVVLQJ ZRUN DELOLW\ Ef IUHTXHQWO\ LQDGHTXDWH ZRUN DWWLWXGHV DQG EHKDYLRUV FDQQRW EH DVVHVVHG GXULQJ D VKRUW VWDQGDUG WHVW LQ DQ LVRODWHG DGPLQLVWUDWLRQ VHWWLQJ DQG Ff VWDQGDUG WHVWV DUH IUHTXHQWO\ NHSW VKRUW WR HOLPn LQDWH IDWLJXH RU GHFOLQLQJ PRWLYDWLRQ WKXV LQDSSURSULDWHO\ UHIOHFWLQJ WKH HPSOR\PHQW VHWWLQJ :KLOH ZRUNLQJ ZLWK WKH PHQWDOO\ UHWDUGHG 7RELDV DQG *RUHOLFN f PDGH DQ HIIRUW WR GHWHUPLQH WKH SUHGLFn WLYH DELOLW\ RI D VLQJOH VWDQGDUGL]HG WHVW WKH 3XUGXH 3HJERDUG (LJKW\RQH FOLHQWV ZLWK ,4 UDQJHV RI UHWDUGDn WLRQ IURP WR PDGH XS WKH VDPSOH 3HUIRUPDQFH RQ WKH 3XUGXH 3HJERDUG ZDV FRUUHODWHG ZLWK TXDOLW\ RI SURn GXFWLRQ RQ WZR VHSDUDWH EHQFK RSHUDWLRQV 2QH ZDV D EDOOSRLQW SHQ DVVHPEO\ DQG WKH RWKHU D ZLUH FODPS DVVHPEO\ 6LJQLILFDQW GLIIHUHQFHV ZHUH IRXQG LQ SHUIRUPDQFH RQ WKH 3XUGXH QRW RQO\ EHWZHHQ UHWDUGHG DQG QRUPDO DGXOWV EXW DOVR EHWZHHQ GLIIHUHQW OHYHOV ZLWKLQ WKH UHWDUGHG UDQJH 7KH UHVXOWLQJ VFRUHV RQ WKH YDULRXV VXEWHVWV RI WKH 3XUGXH 3HJERDUG FRUUHODWHG KLJKO\ f ZLWK SURGXFWLRQ RXWSXW RQ WZR VHSDUDWH RSHUDWLRQV 7KH DXWKRUV VXJJHVW D

PAGE 25

VXEVWDQWLDO UHODWLRQVKLS EHWZHHQ LQWHOOHFWXDO IXQFWLRQLQJ DV PHDVXUHG E\ WKH :$,6f DQG PDQLSXODWLYH GH[WHULW\ 'HVSLWH WKHVH KLJK FRUUHODWLRQV 7LPPHUPDQ DQG 'RFWRU f FRQFOXGHG WKDW WKH 3XUGXH 3HJERDUG KDG TXHVWLRQDEOH YDOXH DQG PXVW EH XVHG FDXWLRXVO\ ZLWK WKH PHQWDOO\ UHn WDUGHG IRU WKH IROORZLQJ UHDVRQV Df WKH KLJK UHODWLRQn VKLS EHWZHHQ WKH 3XUGXH 3HJERDUG DQG ZRUNVKRS DVVHPEO\ WDVNV FDQQRW JXDUDQWHH VLJQLILFDQW FRUUHODWLRQV ZLWK RWKHU W\SHV RI WDVNV VXFK DV WKRVH LQ VHUYLFH RFFXSDWLRQV DQG PDQXDO WDVNV UHTXLULQJ GLIIHUHQW PRYHPHQWV ERWK WDVNV ZHUH DVVHPEO\ RSHUDWLRQV H[SHFWHG WR FRUUHODWH ZLWK WKH 3XUGXH 3HJERDUGf DQG Ef WKH VWXG\ GHDOW ZLWK RQO\ RQH DVSHFW RI SURGXFWLYLW\ TXDQWLW\ WR WKH H[FOXn VLRQ RI TXDOLW\ GRHV QRW PHQWLRQ ZKHWKHU RU QRW LQFRUn UHFWO\ DVVHPEOHG XQLWV ZHUH FRXQWHG DV ILQLVKHG SLHFHVf (YHQ WKRXJK D VLJQLILFDQW LPSOLFDWLRQ IURP WKH VWXG\ WKDW WKHUH DUH DSSDUHQW OHYHOV RI GH[WHULW\ FRUUHVSRQGLQJ WR OHYHOV RI UHWDUGDWLRQ LW UHPDLQV XQFOHDU DV WR ZKHWKHU RU QRW ORVV RI GH[WHULW\ LV GXH WR WKH OHYHO RI UHWDUGDn WLRQ 2WKHU IDFWRUV VXFK DV ODFN RI YRFDWLRQDO H[SRVXUH VKRXOG EH FRQVLGHUHG EHIRUH PDNLQJ WKH VZHHSLQJ FRQFOXn VLRQV DERXW WKH UHODWLRQVKLS EHWZHHQ GH[WHULW\ DQG LQWHOn OLJHQFH ZLWKLQ WKH UHWDUGHG UDQJH :DJQHU DQG +DZYHU f REWDLQHG VRPHZKDW VLPLODU UHVXOWV LQ DQ DWWHPSW WR LGHQWLI\ RQH RU PRUH WHVWV ZKLFK ZRXOG SUHGLFW ZRUNVKRS VXFFHVV IRU VHYHUHO\ UHWDUGHG DGXOWV

PAGE 26

$ VDPSOH LQFOXGHG VXEMHFWV DVVHVVHG E\ WKH 6WDQIRUG %LQHW WR KDYH DQ ,4 RI OHVV WKDQ 7KH WHVW LQVWUXPHQWV ZHUH WKH 2n&RQQRU )LQJHU DQG 7ZHH]HU 'H[WHULW\ 7HVWV VXEWHVW RI WKH 0LQQHVRWD 5DWH RI 0DQLSXODWLRQ 7HVW WKH *RFGHQRXJK +DUULV 'UDZ D 0DQ 7HVW WKH $FWLYH 6FRUH RI WKH +DQG 7HVW DQG WKH %HQGHU *HVWDOW 7HVW &OLHQWV ZHUH DGPLQLVWHUHG WKH EDWWHU\ DQG WKH LQVWUXFWRU UDQNHG DOO VXEMHFWV DFFRUGLQJ WR HVWDEOLVKHG FULWHULD VXFK DV UHVSHFW IRU DXWKRULW\ DQG GLUHFWLRQV FRPSOHWHV DVVLJQPHQWV ZLWK JRRG TXDOLW\ JHWV DORQJ ZLWK FRZRUNHUV DQG OHDUQV QHZ WDVN VNLOOV ZLWKRXW WRR PXFK GLIILFXOW\ $OO RI WKH WHVW VFRUHV UHJLVWHUHG D VLJQLILFDQW UHn ODWLRQVKLS ZLWK WKH LQVWUXFWRUnV UDQNLQJV RI FOLHQWV 7KH %HQGHU*HVWDOW GLVSOD\HG WKH KLJKHVW FRUUHODWLRQ f VXJJHVWLQJ WKH SRVVLELOLW\ RI SUHGLFWLQJ UDQNLQJV RI ZRUN SHUIRUPDQFH ZLWK VRPH GHJUHH RI DFFXUDF\ :DJQHU DQG +DZYHU f VXJJHVW WZR LPSOLFDWLRQV IURP WKHLU VWXG\ ILUVW WKDW WKHUH VHHPV WR EH D VLQJOH LQWDFWQHVV IDFWRU IRXQG LQ ORZ OHYHO PHQWDO UDQJHV RU WKDW WKHUH LV D VLQJOH IDFWRU LQ UHWDUGDWLRQ ZKLFK FDXVHV UHSUHVVLRQ LQ RWKHU DUHDV RI OHDUQLQJ DQG SHUIRUPDQFH GXH WR WKH KLJK GHJUHH RI FRUUHODWLRQ EHWZHHQ VHYHUDO GLIIHUHQW WHVWV DQG WKH FULWHULD PHDVXUHG DQG VHFRQG WKDW WKH KLJK FRUn UHODWLRQV VXSSRUW WKH YDOLGLW\ RI SV\FKRORJLFDO WHVWLQJ ZLWK D UHWDUGHG SRSXODWLRQ

PAGE 27

7LPPHUPDQ DQG 'RFWRU f FRQVLGHUHG WKH IROORZLQJ OLPLWDWLRQV WR WKH :DJQHU DQG +DZYHU f VWXG\ WKDW RI WKH GLIILFXOW\ RI JHQHUDOL]LQJ WKH UHVXOWV GXH WR VPDOO VDPSOH VL]H DQG JHRJUDSKLF DUHDV WKDW WKH UHVXOWV FRXOG RQO\ SXUSRUW WR SUHGLFW SHUIRUPDQFH LQ D VKHOWHUHG ZRUNVKRS QRQFRPSHWLWLYH VLWXDWLRQf DQG WKDW SUHGLFWLYHn QHVV DSSHDUV WR DSSO\ RQO\ WR EHQFK ZRUN DQG ZRRGZRUNLQJ WDVNV 'HVSLWH WKH OLPLWDWLRQV DQG FULWLFLVPV RI DSWLWXGH DVVHVVPHQW DV D SUHGLFWLRQ WRRO LQVWUXPHQWV VXFK DV WKH 3XUGXH 3HJERDUG DSSHDU WR EH YDOLG DQG XVHIXO LQVWUXPHQWV HVSHFLDOO\ IRU WKH ORZHU OHYHO UHWDUGHG LQGLYLGXDOV :KHQ DVVHVVLQJ WKH DELOLWLHV RI WKH PLOGO\ UHWDUGHG GLIn IHUHQW JHQHUDOL]DWLRQV ZRXOG DSSHDU WR DSSO\ DQG GLIIHUHQW DVVXPSWLRQV QHHG WR EH H[DPLQHG $Q LQVWUXPHQW KDYLQJ VRPH DSSOLFDELOLW\ IRU WKH PHQWDOO\ UHWDUGHG LV WKH *HQHUDO $SWLWXGH 7HVW %DWWHU\ *$7%f D WHVW XVHG FRPPRQO\ LQ HPSOR\HH VHOHFWLRQ 'Hn YHORSHG E\ WKH 8QLWHG 6WDWHV (PSOR\PHQW 6HUYLFH WKLV LQVWUXPHQW EDWWHU\ ZDV GHVLJQHG WR DVVHVV YRFDWLRQDOO\ VLJQLILFDQW DSWLWXGHV IRU YRFDWLRQDO FRXQVHOLQJ MRE VHOHFWLRQ DQG MRE SODFHPHQW %UROLQ f 7KH EDWWHU\ PDNHV SUHGLFWLRQV IRU DERXW RFFXSDWLRQV SULPDULO\ RI WKH XQVNLOOHG RU VHPLVNLOOHG W\SH DQG PHDVXUHV DSWLn WXGHV LQ QLQH DUHDV JHQHUDO OHDUQLQJ YHUEDO DSWLWXGH QXPHULFDO DELOLW\ VSDWLDO DSWLWXGH IRUP SHUFHSWLRQ FOHULFDO SHUFHSWLRQ PRWRU FRRUGLQDWLRQ ILQJHU GH[WHULW\ DQG PDQXDO GH[WHULW\

PAGE 28

$FFRUGLQJ WR WKH 8QLWHG 6WDWHV 'HSDUWPHQW RI /DERU f WKH *$7% LV WKH PRVW ZLGHO\ XVHG DSWLWXGH WHVW DQG RQH RI WKH EHVW YDOLGDWHG DQG PRVW FDUHIXOO\ UHVHDUFKHG SV\FKRPHWULF WHVWV LQ H[LVWHQFH $ SDUWLFXODU VWUHQJWK LV LWV FORVH LQWHJUDWLRQ ZLWK WKH 'LFWLRQDU\ RI 2FFXSDn WLRQDO 7LWOHV '27f ZKLFK FRQVWLWXWHV WKH PRVW FRPSUHKHQn VLYH WD[RQRP\ RI WKH $PHULFDQ MRE PDUNHW %DFKHU f 8QIRUWXQDWHO\ WKH *$7% UHTXLUHV D UHDGLQJ DELOLW\ IRU GLUHFWLRQ DQG FDOFXODWLRQV DW DERXW WKH VHYHQWK JUDGH OHYHO )RU WKLV UHDVRQ WKH 0DQSRZHU $GPLQLVWUDWLRQ LQLWLDWHG UHVHDUFK RQ D EDWWHU\ RI WHVWV WKDW ZRXOG PHDn VXUH WKH VDPH QLQH DSWLWXGH DUHDV DV WKH *$7% EXW ZRXOG DOVR EH VXLWDEOH IRU XVH ZLWK PDQ\ QRQUHDGLQJ DQG GLVDGn YDQWDJHG FOLHQWV 7KH UHVXOWLQJ LQVWUXPHQW WKH 1RQUHDGLQJ $SWLWXGH 7HVW %DWWHU\ 1$7%f KDV VLPLODU VXEWHVWV DV WKH *$7% DQG VXSSRVLQJO\ LV XQGHUVWRRG E\ LQGLYLGXDOV ZLWK OLPLWHG YHUEDO DELOLWLHV &DUEXKQ DQG :HOOV f DWWHPSWHG WR HVWDEOLVK FRQn FXUUHQW YDOLGLW\ RI 1$7% IDFWRUV ZKLFK ZRXOG VLJQLILFDQWO\ GLIIHUHQWLDWH WKH YRFDWLRQDO WUDLQHHV UDWHG DV JRRG DQG H[FHOOHQW ZRUNHUV IURP WKRVH UDWHG DV IDLU DQG SRRU FRPn SHWLWLYH ZRUNHUV E\ RQWKHMRE WUDLQLQJ VXSHUYLVRUV 7KHLU VDPSOH LQFOXGHG VXEMHFWV ZLWK ,4V UDQJLQJ IURP WR DQG D PHDQ UHDGLQJ DELOLW\ RI JUDGH OHYHO 7KH FULWHULRQ IRU DGPLVVLRQ WR WKH SUHSODFHPHQW SURJUDP ZDV DVVXPHG SRWHQWLDO IRU UHWXUQLQJ WR WKH FRPPXQLW\ DV

PAGE 29

VHPLLQGHSHQGHQW DQG HFRQRPLFDOO\ SURGXFWLYH LQGLYLGXDOV 3UHGLFWLYH PHDVXUHV LQFOXGHG WKH :$,6 :5$7 DQG WKH 1$7% ZKLOH FULWHULRQ PHDVXUHV ZHUH WKUHH UDWLQJV RQ D QLQHIDFWRU ILYHSRLQW UDWLQJ VFDOH RI HDFK WUDLQHHnV MRE SHUIRUPDQFH 5HVXOWV LQGLFDWHG WKDW WKH 1$7% )DFWRU VSDFH RU QRQYHUEDO LQWHOOLJHQFHf LV UHODWHG WR VXFK MRE FKDUDFWHULVWLFV DV OHDUQLQJ WLPH DQG UHVRXUFHIXOQHVV IRU IHPDOHV )DFWRU ,, SHUFHSWXDO VSHHGf ZDV UHODWHG WR OHDUQLQJ WLPH IRU PDOHV )DFWRUV ,,, DQG ,9 LQWHUQDOL]Dn WLRQ RI UXOHV DQG JHQHUDO OHDUQLQJ DELOLW\f ZHUH KLJKO\ UHODWHG WR IHPDOHVn OHDUQLQJ WLPH DELOLW\ WR KDQGOH HTXLSPHQW DQG UHVRXUFHIXOQHVV )DFWRU 9 GH[WHULW\ DQG VSHHGf ZDV WKH PRVW LPSRUWDQW FKDUDFWHULVWLF RI RYHUDOO MRE SHUIRUPDQFH IRU IHPDOHV 7KH DXWKRUV FRQFOXGHG WKDW WKH 1$7% FRXOG EH XVHG ZLWK OHYHOV RI PHQWDO UHWDUGDWLRQ DERYH D PLQLPXP RI ,4 PLOG DQG PRGHUDWH OHYHOVf %RWK &DUEXKQ DQG :HOOV f DQG %UROLQ f LQ UHYLHZV RI DSWLWXGH WHVWV UHFRPPHQGHG WKH 1$7% DV D YDOXDEOH VRXUFH RI PHDVXUHPHQW RI YRFDWLRQDO DSWLWXGH IRU KLJKHU OHYHO PHQWDOO\ UHWDUGHG LQGLYLGXDOV 'HVSLWH OLPLWDWLRQV RI PDQ\ RI WKH VWXGLHV FRQGXFWHG ZLWK VWDQGDUGL]HG YRFDWLRQDO RU SV\FKRORJLFDO LQVWUXPHQWV DQG WKH PHQWDOO\ UHWDUGHG &REE f VXJJHVWV IURP KLV UHYLHZ RI SUHGLFWLYH VWXGLHV WKDW WKH PRVW VWDEOH SUHn GLFWLYH LQGLFDWRUV KDYH EHHQ IRXQG WR EH VWDQGDUG PHDVXUHV RI PDQXDO GH[WHULW\ WR FULWHULD RI ZRUN FRPSHWHQFH S f

PAGE 30

,Q UHYLHZLQJ PDQXDO GH[WHULW\ WHVWV 2YHUV f VXJJHVWV WKDW VWDQGDUGL]HG LQVWUXPHQWV RI GH[WHULW\ SUHGLFW DV ZHOO DV RU EHWWHU WKDQ MRE VDPSOHV DQG PD\ EH DGPLQLVWHUHG LQ D PXFK VKRUWHU WLPH DW D JUHDWHU VDYLQJV WR FOLHQWV DQG VWDII $V LQGLFDWHG E\ 6XSHU DQG &ULWHV f PDQXDO GH[n WHULW\ LV QRW D VLQJXODU WUDLW 7KH DXWKRUV YLHZ LVRODWHG PRYHPHQWV DV ODFNLQJ VLJQLILFDQFH ZKHQ DQ HQWLUH SDWWHUQLQJ RI PRYHPHQWV LV UHTXLUHG E\ ZRUNHUV RQ WKH MRE ,PSOLFDn WLRQV DUH GUDZQ WKDW VWDQGDUGL]HG WHVWV RI PDQXDO GH[WHULW\ VKRXOG EH UHFRPPHQGHG IRU XVH EXW ZLWK FDXWLRQ ZKHQ ZRUNLQJ ZLWK WKH PHQWDOO\ UHWDUGHG -DFREV DQG :HLQJROG f LQGLFDWH WKDW SV\FKRORJLFDO WHVWLQJ RI WKH PHQWDOO\ UHWDUGHG KDV YDOXH LQ WKDW WKH UHVXOWV Df DOORZ IRU DQ REMHFWLYH FRPSDULVRQ RI WKH FOLHQW ZLWK KLV SHHUV Ef JLYH JHQHUDO LQGLFDWRUV DV WR DUHDV WR EH VWUHVVHG RU HOLPLQDWHG Ff DOORZ IRU DQ DVVHVVPHQW RI FKDQJLQJ FDSDELOLWLHV RI WKH FOLHQW DQG Ff PD\ VXJJHVW WKDW EHJLQQLQJ RU FRQWLQXLQJ WUDLQLQJ LV QRW IHDVLEOH IRU WKH FOLHQW %XUGHWW f YLHZHG WKH YDOXH RI SV\FKRORJLFDO H[DPLQDWLRQV RI WKH PHQWDOO\ UHn WDUGHG WR EH LQ WKH DELOLW\ RI WKH LQWHUSUHWDWLRQ RI WHVW UHVXOWV WR GLVFRYHU DUHDV RI WUDLQLQJ DV ZHOO DV JLYLQJ DQ DGHTXDWH HVWLPDWH RI PRWLYDWLRQDO DELOLWLHV SHUVRQDOLW\ DQG OHYHO RI IXQFWLRQLQJ %LWWHU f YLHZHG VWDQGDUGL]HG LQVWUXPHQWV DV YDOXDEOH LQ H[SORULQJ

PAGE 31

SHUVRQDOLW\ DQG DGMXVWPHQW GLIILFXOWLHV ZKLFK FRXOG SRWHQWLDOO\ DIIHFW MRE SHUIRUPDQFH 6HYHUDO DXWKRUV KDYH KHDYLO\ FULWL]HG VWDQGDUGL]HG WHVWLQJ DV WKH GRPLQDQW SUHGLFWRU RI YRFDWLRQDO VXFFHVV ZLWK WKH PHQWDOO\ UHWDUGHG :ROIHQVEHUJHU f LQ KLV H[WHQVLYH UHYLHZ RI OLWHUDWXUH RI YRFDWLRQDO UHn KDELOLWDWLRQ RI WKH PHQWDOO\ UHWDUGHG FULWL]HG WKH FRQn FHSWXDOL]DWLRQ RI WKH HYDOXDWLRQ SURFHVV DV WKH VDFUHG FRZ RI YRFDWLRQDO UHKDELOLWDWLRQ SURJUDPV GHVSLWH WKH ODFN RI HYLGHQFH DV WR LWV YDOXH :ROIHQVEHUJHU GLVn FXVVHG VHYHUDO FULWLFLVPV FRPPRQ ZLWK VWXGLHV XVLQJ SV\n FKRORJLFDO SUHGLFWLYH LQVWUXPHQWV +H PHQWLRQV WKH FRPPRQ SUDFWLFH RI YDOLGDWLQJ D SUHGLFWLYH WHFKQLTXH ZLWK ZRUNVKRS SHUIRUPDQFH UDWKHU WKDQ FRPSHWLWLYH HPSOR\n PHQW WKH SRRU FRQWURO DQG GHVLJQ RI VWXGLHV RIWHQ SODJXHG ZLWK VHOHFWLRQ SUREOHPV DQG WKH FRQVLVWHQW IDLOXUH WR HPSKDVL]H YDULDEOHV RWKHU WKDQ UHWDUGDWLRQ DV VLJQLILFDQW FDXVDO IDFWRUV :ROIHQVEHUJHU f DQG 1HII f DUJXH WKH QHHG IRU LPSURYHG H[SHULPHQWDO FRQGLWLRQV DQG QHZ PRUH SURn GXFWLYH HYDOXDWLRQ DSSURDFKHV WR WKH YRFDWLRQDO HYDOXDn WLRQ RI WKH PHQWDOO\ UHWDUGHG 1HII f IXUWKHU FULWL]HG WKH WUDGLWLRQDO SV\FKRORJLFDO DVVHVVPHQW DSSURDFK DV EHLQJ URXQGDERXW DQG LQGLUHFW LQ DFKLHYLQJ D PHDVXUHn PHQW RI FUXFLDO YDULDEOHV QHFHVVDU\ LQ WKH UHDO ZRUOG RI ZRUN 7KH IROORZLQJ FULWLFLVPV DUH SRVHG Df WKH WLPH

PAGE 32

VSDQ LV XVXDOO\ YHU\ VKRUW Ef DQ HQFRXUDJHPHQW WR ZRUN WRZDUG IXOO SRWHQWLDO LV JLYHQ LQ WKH WHVWLQJ VLWXDWLRQ EXW LV ODFNLQJ LQ WKH HPSOR\PHQW VLWXDWLRQ DQG Ff ODFN RI UHDOLW\ LQ DVVHVVPHQW VLWXDWLRQV 7KH KHDY\ FULWLFLVP RI VWXGLHV LQYROYLQJ VWDQGDUGL]HG SV\FKRORJLFDO RU PHQWDO DVVHVVPHQW LQVWUXPHQWV DV DQ DSSURDFK WR YRFDWLRQDO HYDOXDWLRQ +RIIPDQ 1HII 2YHUV :ROIHQVEHUJHU f KDV OHG WR HYDOXDn WLRQ DSSURDFKHV SURYLGLQJ D YDOLG DQG PRUH UHDOLVWLF SLFWXUH RI DFWXDO ZRUN FDSDELOLWLHV :RUN 6DPSOH $SSURDFK $ ZRUN VDPSOH DFFRUGLQJ WR 1HII f LV D PRFN XS D FORVH VLPXODWLRQ RI DQ DFWXDO LQGXVWULDO RSHUDWLRQ QRW GLIIHUHQW LQ LWV HVVHQWLDOV IURP WKH NLQG RI ZRUN D SRWHQWLDO HPSOR\HH ZRXOG EH UHTXLUHG WR SHUIRUP RQ DQ RUGLQDU\ MRE S f $ QXPEHU RI YRFDWLRQDO HYDOXDWRUV YLHZ ZRUN VDPSOHV DV VXSHULRU WR VWDQGDUGL]HG WHVWV HVSHFLDOO\ ZLWK D PHQWDOO\ UHWDUGHG SRSXODWLRQ *ROG +RIIPDQ 1HII f
PAGE 33

VLWXDWLRQV -(96 2YHUV 6 8VGDQH f Ef VLQFH WKH\ DUH PXFK OLNH UHDO ZRUN WKH\ PRWLYDWH FOLHQWV PRUH WKDQ VWDQGDUGL]HG WHVWV +RIIPDQ 1HII 2YHUV f DQG Ff ZRUN VDPSOHV DUH PXFK OHVV DIIHFWHG WKDQ VWDQGDUGL]HG WHVWV E\ IDFWRUV VXFK DV HGXFDWLRQDO OHYHO VSHHFK DQG KHDULQJ LPSDLUPHQWV DQG H[FHVVLYH DQ[LHW\ -(96 /XVWLJ 2YHUV f $OWKRXJK IDYRULQJ WKH LQFUHDVHG XVH RI WKH ZRUN VDPSOH DSSURDFK WR HYDOXDWLRQ 7LPPHUPDQ DQG 'RFWRU f SUHVHQW JHQHUDO FDXWLRQV DJDLQVW D KHDY\ UHn OLDQFH XSRQ WKH XVH RI ZRUN VDPSOHV ZLWK WKH PHQWDOO\ UHWDUGHG 7KHVH FDXWLRQV LQFOXGH WKH IDFW WKDW WKH UHn VHPEODQFH RI ZRUN VDPSOHV WR DFWXDO MREV GRHV QRW DVVXUH WKHLU SUHGLFWLYH YDOLGLW\ VLQFH PDQ\ MRE GLIIHUHQFHV FDQQRW EH GXSOLFDWHG LQ ZRUN VDPSOHV WKH SUREOHP RI VXEn MHFWLYH REVHUYDWLRQ DQG LQWHUSUHWDWLRQ E\ UDWHUV DQG WKH H[LVWHQFH RI D ZLGHVSUHDG ODFN RI VWDQGDUGL]DWLRQ DPRQJ ZRUN VDPSOHV ZLWK YHU\ IHZ EHLQJ YDOLGDWHG ZLWK WKH PHQWDOO\ UHWDUGHG ,Q WKH ILHOG RI ZRUN HYDOXDWLRQ IRU WKH KDQGLFDSSHG VRPH HIIRUWV KDYH EHHQ PDGH WR GHYLVH EDWWHULHV RI ZRUN VDPSOHV ZKLFK ZRXOG KDYH XQLYHUVDO VLJQLILFDQFH 2QH ZHOO DFFHSWHG ZRUN VDPSOH V\VWHP LV WKH :LGH 5DQJH (Pn SOR\PHQW 6DPSOH 7HVW :5(67f DOVR WLWOHG WKH -DVWDN .LQJ :RUN 6DPSOHV D UHFHQW GHYHORSPHQW RI -DVWDN DQG

PAGE 34

.LQJ f ,W FRQVLVWV RI D VKRUW EDWWHU\ RI ZRUN VDPSOHV GHYHORSHG DW D ZRUNVKRS IRU WKH PHQWDOO\ DQG SK\VLFDOO\ KDQGLFDSSHG 7KH SULPDU\ SXUSRVH RI WKH :5(67 DFFRUGLQJ WR LWV DXWKRUV LV WR HYDOXDWH GH[WHULW\ DQG SHUFHSWXDO DELOLWLHV LQ VPDOO JURXSV ZLWKLQ D WZR KRXU WLPH SHULRG %UROLQ f GLVFXVVHV WKH VWUHQJWKV RI WKH EDWWHU\ IRU XVH ZLWK WKH PHQWDOO\ UHWDUGHG FOLHQW DV SURYLGLQJ WKH RSSRUWXQLW\ IRU WKH FOLHQW WR SUDFWLFH H[HUFLVHV SULRU WR WLPLQJ DQG HYDOXDWLRQ RI WKH ZRUN VDPSOH E\ OLVWLQJ LQGXVWULDO QRUPV ERWK FRPSHWLWLYH DQG ZRUNVKRSf DORQJ ZLWK VKRUW DGPLQLVWUDWLRQ WLPH DQG SUHn FLVH LQVWUXFWLRQV ,Q FRQWUDVW 7LPPHUPDQ DQG 'RFWRU f UHSRUWHG WKH QRUPDWLYH GDWD IRU WKH :5(67 DV VNHWFK\ IRXQG WKH LQVWUXFWLRQV DV FRQIXVLQJ DQG REVFXUH WKXV PDNLQJ LW QHFHVVDU\ WR PRGLI\ LQVWUXFWLRQV DQG SURFHGXUHV LQ VHYHUDO DUHDV WR FORVHU DSSUR[LPDWH WKH QHHGV RI WKH PHQWDOO\ UHWDUGHG ,Q FULWLFLVP %UROLQ f DOVR QRWHG D QHHG IRU IXUWKHU DQG PRUH V\VWHPDWLF SURFHGXUHV FRQFHUQLQJ WKH REVHUYDWLRQ RI FOLHQW EHKDYLRU ZKHQ XVLQJ WKH :5(67 EDWWHU\ $ IXUWKHU SUREOHP FRQFHUQHG WKH DSSOLFDWLRQ RI UHVXOWV RI WKH HYDOXDWLRQ WR DQ\ VSHFLILF MREV RU RFFXSDn WLRQDO JURXSV VLQFH WKH EDWWHU\ GRHV QRW UHODWH LWV ZRUN VDPSOHV WR WKH '27 RU DQ\ RWKHU MRE FODVVLILFDWLRQ V\VWHP %RWWHUEXVFK f QRWHG WKDW WKHUH ZDV QR ZD\ RI NQRZLQJ ZKDW FRPSHWLWLYH MREV ZHUH UHODWHG WR VSHFLILF VDPSOHV RU

PAGE 35

WR WKH EDWWHU\ DV D ZKROH 'HVSLWH WKH REYLRXV VKRUWn FRPLQJV WKH WHVW EDWWHU\ DSSHDUV WR KDYH SRWHQWLDO IRU ERWK PLOG DQG PRGHUDWHO\ UHWDUGHG JURXSV %UROLQ 7LPPHUPDQ t 'RFWRU f 2QH RI WKH EHWWHU NQRZQ ZRUN HYDOXDWLRQ V\VWHPV LV WKH 72:(5 7HVWLQJ 2ULHQWDWLRQ DQG :RUN (YDOXDWLRQ LQ 5HKDELOLWDWLRQf GHYHORSHG LQ DW WKH ,QVWLWXWH IRU WKH &ULSSOHG DQG 'LVDEOHG ,&'f 7KH 72:(5 V\VWHP LQFOXGHV EURDG DUHDV RI ZRUN HYDOXDWLRQ DQG RYHU ZRUN VDPSOHV LQ DOO (YDOXDWLRQ WDNHV DSSUR[LPDWHO\ WKUHH ZHHNV ZLWK HDFK RFFXSDWLRQDO WDVN SUHVHQWHG LQ D JUDGXDO VHTXHQFH IURP VLPSOH WR FRPSOH[ 6RPH RI WKH DGYDQWDJHV RI WKH V\VWHP DFFRUGLQJ WR 5RVHQEHUJ f DUH WKDW VHUYLFH SURFHGXUHV DUH IOH[LEOH UDWKHU WKDQ ULJLG PDNLQJ WKH WDVNV DGDSWDEOH WR WKH PHQn WDOO\ UHWDUGHG DQG WKH WDVNV UHODWH GLUHFWO\ WR VNLOO SRWHQWLDO QHHGHG LQ YRFDWLRQDO WUDLQLQJ 5RVHQEHUJ YLHZV WKH 72:(5 DV DQ DSSURDFK WR YRFDWLRQDO HYDOXDWLRQ UDWKHU WKDQ DQ H[DFW WHVW RI WKH FOLHQWnV YRFDWLRQDO SRWHQWLDO ,Q WXUQ WKH VDPH IOH[LELOLW\ RI LQVWUXFWLRQ DQG DGPLQLVn WUDWLRQ PRGLILFDWLRQV PDGH WR FRPSHQVDWH IRU KLJK UHDGLQJ DQG FRPSUHKHQVLRQ OHYHOV SUHWLPHG SUDFWLFH WULDOV DQG DGDSWDELOLW\ WR ORFDO SODFHPHQW RSSRUWXQLWLHVf ZRXOG DSSHDU WR HIIHFW WKH DOUHDG\ ZHDN YDOLGLW\ UHSRUWV RI WKH WHVW VDPSOHV DV ZHOO DV WKH QRUPV FRQFHUQLQJ TXDOLW\ DQG TXDQWLW\ RI SHUIRUPDQFH 5DEXFKD 7LPPHUPDQ ]

PAGE 36

'RFWRU f %UROLQ f VXJJHVWV WKDW WKH 72:(5 KDV DSSOLFDELOLW\ ZLWK WKH PHQWDOO\ UHWDUGHG DW PLOG OHYHOV DW OHDVW LQ VRPH ZRUN VDPSOH DUHDV ZLWK DSSURn SULDWH PRGLILFDWLRQV 2QH RI WKH PRUH DGYDQWDJHRXV UHDVRQV IRU LWV DSSOLFDELOLW\ ZLWK WKH PHQWDOO\ UHWDUGHG LV WKH ZLGH YDULHW\ RI RFFXSDWLRQDO H[SRVXUH DYDLODEOH WKURXJK WKH ZRUN WDVN VDPSOHV $QRWKHU HYDOXDWLRQ LQVWUXPHQW WKH -HZLVK (PSOR\PHQW 9RFDWLRQDO 6HUYLFH -(96f ZDV GHYHORSHG XQGHU D FRQWUDFW ZLWK WKH 0DQSRZHU $GPLQLVWUDWLRQ 'HSDUWPHQW RI /DERU 7KH V\VWHP FRQVLVWV RI ZRUN VDPSOHV FRYHULQJ GLIn IHUHQW ZRUN DUHDV ZLWKLQ ZRUNHU WUDLW JURXSV %UROLQ f 6HYHUDO RI WKH ZRUN WDVNV KDYH EHHQ DGDSWHG 9,(:n6 f WR DVVHVV WKH ZRUN SRWHQWLDO RI SHUVRQV ZLWK OHDUQLQJ GLVDELOLWLHV DQG YDULRXV OHYHOV RI PHQWDO UHWDUGDWLRQ 7KH HYDOXDWLRQ SHULRG LQYROYLQJ DSSUR[LPDWHO\ WZR ZHHNV LV LQLWLDWHG ZLWK WKH VLPSOHVW ZRUN VDPSOHV DQG SURFHHGV WRYDUGV LQFUHDVLQJ FRPSOH[LW\ (DFK ZRUN VDPSOH LV DGn PLQLVWHUHG XQGHU DFWXDO ZRUN FRQGLWLRQV IRU WKDW VSHFLILF WDVN ZLWK WKH FOLHQW REVHUYHG WLPHG DQG UDWHG RQ D WKUHH SRLQW VFDOH WKH KLJKHU VFRUHV LQGLFDWLYH RI SHUn IRUPDQFH DQG EHKDYLRU UHTXLUHG LQ FRPSHWLWLYH HPSOR\PHQW 2QH RI WKH PDMRU VWUHQJWKV RI WKH V\VWHP %UROLQ f LV WKDW HDFK ZRUN VDPSOH LV GLUHFWO\ UHODWHG WR WKH ZRUN WUDLW JURXS DUUDQJHPHQW LQ WKH 'LFWLRQDU\ RI 2Fn FXSDWLRQDO 7LWOHV '27f DOORZLQJ IRU D GLUHFW FRPSDULVRQ

PAGE 37

EHWZHHQ DFWXDO RFFXSDWLRQDO FDWHJRULHV DQG HYDOXDWHG VNLOOV :HDNQHVVHV RI WKH -(96 V\VWHP %UROLQ 7LPPHUPDQ IF 'RFWRU f DUH WKDW WKH LQVWUXFWLRQV IRU WKH WDVNV DUH QRW VXLWHG WR UHDGLQJ OHYHOV RI WKH PHQWDOO\ UHWDUGHG WKDW WKH SUHVVXUH RI WLPH OLPLWV XSRQ WKH FOLHQW FDXVHV XQGXH DQ[LHW\ DQG PLVWDNHV WKH LQDELOLW\ RI WKH FOLHQW WR FRQFHLYH WKH ZRUN VDPSOHV DV UHODWHG WR UHDO ZRUN VLWXDWLRQV DQG WKH QHHG WR GLVDVVHPEOH WKH ZRUN VDPSOHV EHIRUH WKH\ FDQ EH XVHG DJDLQ :LWK DQ LQFUHDVLQJ XVH RI -(96 E\ VWDWH HPSOR\PHQW DJHQFLHV DQG WKH LQWURGXFWLRQ RI WKH QHZ ZRUN VDPSOHV 9,(:6 f IRU WKH PHQWDOO\ UHWDUGHG FRQVLGHUDWLRQ LV ZDUUDQWHG E\ WKRVH LQYROYHG LQ WKH YRFDWLRQDO HYDOXDWLRQ RI WKH PHQWDOO\ UHWDUGHG $QRWKHU HYDOXDWLRQ XQLW WKH 6LQJHU 9RFDWLRQDO (YDOXDn WLRQ 6\VWHP LV D ZRUNRULHQWHG VFUHHQLQJ GHYLFH GHVLJQHG WR KHOS WKH LQGLYLGXDO PDNH D YRFDWLRQDO FKRLFH WKURXJK D KDQGV RQ H[SORUDWLRQ H[SHULHQFH RI VHYHUDO MRE WDVNV 7KH V\VWHP XWLOL]HV DQ DXGLRYLVXDO DSSURDFK WR SUHVHQW SURJUDPPHG LQVWUXFWLRQ RQ WKH SHUIRUPDQFH RI VSHFLILF WDVNV VHOHFWHG LQ RFFXSDWLRQDO FOXVWHUV VHOIFRQWDLQHG ZLWKLQ ZRUN VWDWLRQV HDFK RXWILWWHG ZLWK WKH QHFHVVDU\ LQVWUXFWLRQDO WRROV 3XUSRUWHGO\ WKH V\VWHP DYRLGV WKH SLWIDOOV RI UHDGLQJ GHILFLHQFLHV WKURXJK WKH FRPELQHG ILOP VWULS DQG WDSH SUHVHQWDWLRQ DQG DOVR SHUPLWV WKH WUDLQHH WR ZRUN DW KLV RZQ SDFH 7KH KDQGV RQ QDWXUH RI WKH V\VWHP LV FRQVLGHUHG D YHU\ SRVLWLYH DVSHFW IRU

PAGE 38

SUHYRFDWLRQDO H[SORUDWLRQ DQG DZDUHQHVV SDUWLFXODUO\ IRU WKH PLOGO\ UHWDUGHG %UROLQ f \HW FDXWLRQ PXVW EH H[HUFLVHG ZKHQ XVLQJ WKH V\VWHP IRU SXUSRVHV RI HYDOXDn WLRQ 2QO\ RQH RU WZR RI WKH ZRUN VWDWLRQV DUH FRQVLGHUHG DSSURSULDWH IRU WKH PRUH VHULRXVO\ UHWDUGHG LQGLYLGXDO ZLWK LQGLFDWLRQV WKDW VRPH RI WKH ZRUN WDVNV ZHUH WRR LQYROYHG IRU WKH UHWDUGHG SRSXODWLRQ $FFRUGLQJ WR 5DEXFKD f SUREOHPV KDYH H[LVWHG LQ WKH RUDO SUHVHQWDWLRQ RI LQVWUXFWLRQV WKH VPDOO VFUHHQ VL]H DQG SRRU DXGLR ZKHQ XVHG ZLWK WKH UHWDUGHG RU KDQGLFDSSHG 7KH *RRGZLOO ,QGXVWULHV RI &KLFDJR DQG &RRN &RXQW\ ,QF SURGXFHG WKH 0XOWLGLPHQVLRQDO 2EMHFWLYH 9RFDWLRQDO (YDOXDWLRQ 029(f V\VWHP D FRPSXWHUL]HG PHWKRG RI DVn VHVVLQJ DQ LQGLYLGXDOnV YRFDWLRQDO DELOLWLHV 7KH V\VWHP FRQVLVWV RI IDFWRUSXUH WHVWV FRPSOHWHO\ REMHFWLYHf ZKLFK WDNH DSSUR[LPDWHO\ WKUHH KRXUV WR DGPLQLVWHU +HVWHU f $IWHU WHVWLQJ WKH UDZ DSWLWXGH VFRUHV DQG DGGLn WLRQDO SHUVRQDOLW\ FKDUDFWHULVWLFV RI WKH LQGLYLGXDO WKHVH DUH DOO IHG LQWR WKH 029( FRPSXWHU SURJUDP 7KH GDWD DUH DQDO\]HG E\ WKH FRPSXWHU WR GHWHUPLQH KRZ WKH LQGLYLGXDOnV DELOLWLHV UHODWH WR MREV FRQWDLQHG LQ WKH 'LFWLRQDU\ RI 2FFXSDWLRQDO 7LWOHV $FFRUGLQJ WR +HVWHU f WKH 029( V\VWHP GRHV QRW GXSOLFDWH LQIRUPDWLRQ RIIHUHG E\ WKH RWKHU PDMRU V\VWHPV WKXV LW FDQ EH XVHG LQ FRQMXQFWLRQ ZLWK -(96 DQG 72:(5 DV D IXUWKHU JXLGDQFH LQIRUPDWLRQ DLG +HVWHU VXJJHVWV WKDW EHFDXVH RI WKH

PAGE 39

IDFWRUSXUH QDWXUH RI WKH 029( WHVWV WKH V\VWHP FDQ EH XVHG ZLWK DOO OHYHOV RI SHUVRQV UHJDUGOHVV RI EDFNJURXQG RU KDQGLFDS ZLWK WKH H[FHSWLRQ RI WKH EOLQGf 'XH WR WKH UHODWLYH QHZQHVV RI WKH V\VWHP DQG WKH ODFN RI XVH RXWVLGH RI &KLFDJR RQO\ WKH RULJLQDO WHVW UHSRUWV FRXOG EH IRXQG RQ WKH V\VWHP 7KH 0DF'RQDOG 9RFDWLRQDO &DSDFLW\ 6FDOH 9&6f GHn YHORSHG DW WKH 0DF'RQDOG 7UDLQLQJ &HQWHU 3LQNDUG *LOn PRUH 5LFKHU t :LOOLDPV f FRQVLVWV RI HLJKW VHSDUDWH WHVWV RU UDWLQJV DGPLQLVWHUHG LQ DQ HYDOXDWLRQ VHWWLQJ ZKHUH REVHUYDWLRQV FDQ EH PDGH RYHU D WZR ZHHN SHULRG IRU WKH SXUSRVH RI SUHGLFWLQJ WKH YRFDWLRQDO SRWHQWLDOLWLHV RI \RXQJ UHWDUGHG DGXOWV 7KH 9&6 SXUSRUWV WR GLIIHUHQn WLDWH DPRQJ SHUVRQV QHHGLQJ FORVH DQG FRQVWDQW VXSHUYLVLRQ WKRVH RI ZRUNVKRS SRWHQWLDO DQG WKRVH FDSDEOH RI VRPH OHYHO RI FRPSHWLWLYH HPSOR\PHQW $V LQLWLDOO\ GHVLJQHG WKH 9&6 ZDV FRPSULVHG RI HLJKW PHDVXUHV Df ZRUN KDELWV Ef SK\VLFDO FDSDFLW\ Ff VRFLDO PDWXULW\ Gf JHQHUDO KHDOWK Hf PDQXDO VNLOOV If DULWKPHWLF Jf GLUHFWLRQ IROORZLQJ DQG Kf PRWLYDWLRQ )XUWKHU UHVHDUFK E\ +R f UHYHDOHG WKDW SK\VLFDO FDSDFLW\ JHQHUDO KHDOWK DQG PRWLYDWLRQ ZHUH QRW VLJQLILFDQWO\ SUHGLFWLYH WR ZDUUDQW WKHLU LQFOXVLRQ LQ WKH 9&6 %DVHG RQ DQ LQGLYLGXDOnV VFRUHV RQ HDFK RI WKH HLJKW PHDVXUHV ZKLFK KDYH YDU\LQJ ZHLJKWV WKH VFRUHV DUH UHFRUGHG DQG WDOOLHG RQ D SURILOH VKHHW DQG D SUHGLFWLRQ RI WKH LQGLYLGXDOnV YRFDWLRQDO

PAGE 40

FDSDFLW\ DV GD\ FDUH VKHOWHUHG ERUGHUOLQH RU FRPSHWLn WLYH LV DVFHUWDLQHG 7KH 9&6 ZDV GHYHORSHG HVSHFLDOO\ IRU XVH ZLWK WKH PHQWDOO\ UHWDUGHG \HW VRPH VHULRXV TXHVWLRQV KDYH EHHQ UDLVHG FRQFHUQLQJ VRPH RI WKH SURFHGXUHV LQYROYHG LQ WKH RULJLQDO VWXG\ ZLWK WKH LQVWUXPHQW :LQGOH f VXJn JHVWV WKDW WKH 0DF'RQDOG 9&6f VWXG\ FRQWDLQHG VXFK VHULRXV PHWKRGRORJLFDO VKRUWFRPLQJV WKDW LW ZDV ZRUWKOHVV DV D SUHGLFWLYH VWXG\ +H QRWHG LQ SDUWLFXODU WKDW WKH UHDVRQV IRU SODFHPHQW ZHUH QRW LQGHSHQGHQW RI WKH GHVFULSWLYH FKDUDFWHULVWLFV RI WKH FOLHQW DQG WKXV UHWURVSHFWLYHO\ UHODWHG WR SODFHPHQW VHOIIXOILOOLQJ SURSKHF\f 7KLV ZRXOG HQWDLO SODFLQJ D FOLHQW DW D FHUWDLQ OHYHO GXH WR KLV SHUIRUPDQFH RQ D WHVW EDWWHU\ DQG WKHQ XVLQJ WKH SODFHn PHQW WR HVWDEOLVK WKH YDOLGLW\ RI WKH WHVW EDWWHU\ ZLWKRXW FRQVLGHUDWLRQ DV WR ZKHWKHU WKH FOLHQW SHUIRUPHG DQ\ EHWWHU :LQGOH f DOVR FULWL]HG WKH XVH RI FKL VTXDUHV ZLWK XQHTXDO JURXS FHOOV DQG SRLQWHG RXW VHULRXV FRPSXWDn WLRQDO HUURUV LQ WKH RULJLQDO 9&6 VWXG\ 'D\DQ f LQ DQRWKHU VWXG\ DWWHPSWHG WR YDOLGDWH WKH 9&6 XVLQJ D SRSXODWLRQ RI PHQWDOO\ UHWDUGHG \RXQJ DGXOWV 7KUHH KXQGUHG DQG VL[W\VL[ LQVWLWXWLRQDO UHVLn GHQWV ZHUH HDFK DGPLQLVWHUHG WKH 9&6 WZLFH ZLWK FDUH WDNHQ WR DYRLG FRQWDPLQDWLRQ RI UDWLQJV E\ VWDII PHPEHUV 7KH 9&6 VFRUHV ZHUH QRW XVHG LQ PDNLQJ SURJUDPPLQJ GHn WHUPLQDWLRQV WKXV DYRLGLQJ WKH PDLQ REMHFWLRQ WR WKH

PAGE 41

0DF'RQDOG VWXG\ 'D\DQ LQGLFDWHG VLJQLILFDQW ILQGLQJV FRQFHUQLQJ YRFDWLRQDO FDSDFLW\ ZLWK WKH PHQWDOO\ UHWDUGHG 7KHVH LQFOXGHG OLWWOH RU QR GLVFULPLQDWLRQ DFFRUGHG WR DJH VRPH UHODWLRQVKLS H[LVWLQJ EHWZHHQ ,4 DQG WKH OHYHO RI HPSOR\PHQW DQG GLVSURSRUWLRQDWH SHUFHQWDJHV RI PLQRUn LWLHV LQ WKH FRPSHWLWLYHO\ HPSOR\HG JURXS &RPSDULVRQ EHWZHHQ WKH PHDQ VFRUHV RI HDFK 9&6 PHDn VXUH DQG YDULRXV FULWHULRQ JURXSV LQGLFDWHG WKDW GLIIHUHQFHV ZHUH DOO LQ WKH VDPH GLUHFWLRQ FRPSHWLWLYHO\ HPSOR\HG VKHOWHUHG ZRUNVKRS QRQHPSOR\HG DQG GD\ FDUH $FFRUGLQJ WR %UROLQ f DQG 7LPPHUPDQ DQG 'RFWRU f WKH 9&6 GRHV DSSHDU WR KDYH VRPH SUHGLFWLYH YDOXH IRU GLVn FULPLQDWLRQ OHYHOV RI SODFHPHQW IRU WKH PHQWDOO\ UHWDUGHG :ULJKW DQG 7URWWHU f DQG :ROIHQVEHUJHU f UHFRPn PHQGHG IXUWKHU UHVHDUFK ZLWK WKH 9&6 WR GHWHUPLQH WKH YDOLGLW\ RI WKH VFDOH E\ LQFUHDVLQJ VDPSOH VL]H IXUWKHU FURVVYDOLGDWLRQ DQG VWXGLHV RI WKH LQVWUXPHQWnV SUHGLFn WLYH YDOXH )XUWKHU UHYLVLRQ RI WKH VFDOH E\ +R f LV D VLJQLILFDQW VWDUWLQJ SRLQW LQ WKLV GLUHFWLRQ 5DWLQJ 6FDOH DQG 6LWXDWLRQDO $VVHVVPHQW $SSURDFK $FFRUGLQJ WR 'XQQ f YRFDWLRQDO HYDOXDWLRQ SUHn GLFWRUV FDQ EH H[SHFWHG WR UHDFK WKHLU JUHDWHVW YDOLGLW\ ZKHQ WKH\ FORVHO\ DSSUR[LPDWH D UHDO ZRUN VHWWLQJ 7KLV SKLORVRSK\ KDV JLYHQ ELUWK WR WKH VLWXDWLRQDO DVVHVVPHQW GHILQHG E\ 'XQQ f DV D V\VWHPDWLF SURFHGXUH IRU REn VHUYLQJ UHFRUGLQJ DQG LQWHUSUHWLQJ ZRUN EHKDYLRU

PAGE 42

DSSOLFDEOH WR D YDULHW\ RI UHDO RU VLPXODWHG ZRUN VLWXDn WLRQV S f %UROLQ f SURYLGHV VHYHUDO DGYDQWDJHV DQG GLVn DGYDQWDJHV RI WKH VLWXDWLRQDO DVVHVVPHQW DSSURDFK WR YRFDn WLRQDO HYDOXDWLRQ DVVHVVHG WKURXJK D UDWLQJ VFDOHf IRU WKH PHQWDOO\ UHWDUGHG 6RPH DGYDQWDJHV DUH Df DFWLYLW\ DSSUR[LPDWHV WKH UHDO ZRUN VLWXDWLRQ Ef HOLPLQDWHV WKH W\SLFDO WHVWLQJ VLWXDWLRQ DQ[LHW\f Ff LV SRVVLEOH WR DVVHVV PDQ\ W\SLFDO ZRUN EHKDYLRUV LQWHUSHUVRQDO UHODn WLRQVKLSV FRRSHUDWLRQ SUHVVXUHV DXWKRULW\f Gf JLYHV WKH LQGLYLGXDO PRUH WLPH WR DGMXVW WR WKH VLWXDWLRQ EHIRUH DVVHVVPHQW DQG Hf LQGLYLGXDO FDQ EH HYDOXDWHG XQGHU YDULRXV FRQGLWLRQV ZLWK VHYHUDO VXSHUYLVRUV DQG LQ VHYHUDO ZRUN VLWXDWLRQV 6RPH GLVDGYDQWDJHV DUH Df LV GHSHQGHQW XSRQ DFFXUDWH LQWHUSUHWDWLRQ RI REVHUYDWLRQV Ef SUREOHP RI YDULDQFH DPRQJ UDWHUV RQ REVHUYHG EHKDYLRU DQG Ff WKH JURXS VHWWLQJ PD\ HIIHFW WKH UDWHUfV HYDOXDn WLRQV 6HYHUDO UDWLQJ LQVWUXPHQWV RQ WKH YRFDWLRQDO EHKDYLRU RI WKH PHQWDOO\ UHWDUGHG KDYH EHHQ GHYHORSHG 'DQLHOV f GHVLJQHG WKH 9RFDWLRQDO $GMXVWPHQW 5DWLQJ 6FDOH 9$56f ZLWK LWHPV RI ZKLFK UHIOHFWHG SRVLWLYH YRFDWLRQDO DGMXVWPHQW DQG ZKLFK DUH QHJDWLYH 5Hn OLDELOLW\ ZDV DVFHUWDLQHG E\ FRPSDULQJ VFRUHV RI WKUHH UDWHUV ZKR LQGHSHQGHQWO\ DVVHVVHG HDFK JURXS RI WUDLQHHV LQ HDFK RI WKH IRXU MRE DUHDV UDWHG E\ WKH LQVWUXPHQWV

PAGE 43

7KH WHVW SRSXODWLRQ ZDV FRPSRVHG RI \RXQJ DGXOW PDOHV DJH UDQJH WR f ,4 WR f HQUROOHG LQ D UHn KDELOLWDWLRQ FHQWHU $ EDVLF ZHDNQHVV RI WKH VWXG\ DFFRUGLQJ WR 7LPPHUPDQ DQG 'RFWRU f ZDV WKDW WKH UDWLQJ VFDOH FRQVLVWHG RI RQO\ D VPDOO SDUW RI D SURMHFW RQ VHOIFRQFHSW DQG YRFDWLRQDO DGMXVWPHQW WKXV HIIRUWV WR HVWDEOLVK SUHGLFWLYH YDOLGLW\ IRU WKH VFDOH QHYHU RFFXUUHG $QRWKHU EHKDYLRU UDWLQJ VFDOH LV WKH :RUN $GMXVWPHQW 5DWLQJ )RUP :$5)f GHYHORSHG E\ %LWWHU DQG %RODQRYLFK f GHVLJQHG WR SUHGLFW MRE UHDGLQHVV RI PHQWDOO\ UHWDUGHG LQGLYLGXDOV ,W ZDV GHVLJQHG WR SURYLGH D V\Vn WHPDWLF REVHUYDWLRQ UHOHYDQFH UHOLDELOLW\ REVHUYDWLRQV DQG YHULILFDWLRQV RI EHKDYLRU SDWWHUQV $ VWXG\ E\ %LWWHU DQG %RODQRYLFK f WR YDOLGDWH WKH IRUP XVHG D VDPSOH RI FOLHQWV PHDQ DJH PHDQ ,4 f LQ OLJKW RI IRXU FULWHULD PHQWLRQHG 7KH UDWLQJV ZHUH FRPSOHWHG E\ WKUHH FRXQVHORUV DQG RQH IRUHPDQ DIWHU WKH WKLUG DQG WK ZHHNV RI WUDLQLQJ 7KH :$5) FRQWDLQV HLJKW VXEn VFDOHV FRQVLVWLQJ RI DUHDV RI ZRUN EHKDYLRU HDFK EURNHQ GRZQ LQWR ILYH OHYHOV RI SHUIRUPDQFH &RUUHODWLRQV ZHUH FRQGXFWHG EHWZHHQ FRXQVHORU UDWLQJV DIWHU ERWK WKUHH DQG ZHHNV DQG MRE VXFFHVV &RUUHODn WLRQDO VWXGLHV ZHUH DOVR FRQGXFWHG LQYROYLQJ RQH FRXQVHORU DQG WKH IRUHPDQ SHUWDLQLQJ WR HDFK VXEVFDOH RI WKH WHVW DQG HYHQWXDO MRE VXFFHVV ,Q JHQHUDO WKH :$5) UDWLQJV

PAGE 44

FRUUHVSRQGHG WR WKH FRXQVHORUnV SRROHG MXGJPHQW RI HPn SOR\DELOLW\ DW WKH HQG RI WKH ZHHN WUDLQLQJ SHULRG ZLWK DFWXDO MRE VXFFHVV GHWHUPLQHG E\ VL[ PRQWKV HPSOR\n PHQW 6LJQLILFDQW GLIIHUHQFHV LQ WKH QXPEHU RI FOLHQWV DVVHVVHG DW HDFK WLPH E\ HDFK UDWHU PDNH WKH FRUUHODWLRQV UHODWLYHO\ XVHOHVV IRU GHWHUPLQLQJ WKH HIIHFW RI WKH DGGLWLRQDO ZHHNV %LWWHU DQG %RODQRYLFK f LQGLFDWH D QHHG IRU IXUWKHU UHVHDUFK LQ WKH DUHDV RI LGHQWLILFDWLRQ RI VSHFLILF EHKDYLRUV DWWDLQPHQW RI REVHUYHU FRQVLVWHQF\ DQG WKH GHYHORSPHQW RI QRUPDWLYH LQIRUPDWLRQ $Q DVVHVVPHQW WRRO GHYHORSHG E\ /HYLQH DQG )UHHPDQ f WKH 6DQ )UDQFLVFR 9RFDWLRQDO &RPSHWHQF\ 6FDOH LV GHVFULEHG DV DVVHVVLQJ YRFDWLRQDO FRPSHWHQFH RI PHQWDOO\ UHWDUGHG DGXOWV 7KH VFDOH FRQWDLQV LWHPV UHODWLQJ WR PRWRU VNLOOV FRJQLWLRQ UHVSRQVLELOLW\ DQG VRFLDO HPRWLRQDO EHKDYLRU 7ZR SUHWHVWV HDFK LQYROYLQJ LQGLYLGXDOV ZHUH FRQGXFWHG WR SURGXFH WKH LWHPV 7KHVH ZHUH WKHQ QRUPHG XVLQJ D SRSXODWLRQ RI PHQWDOO\ UHWDUGHG LQGLYLGXDOV LQ ZRUNVKRSV UHSUHVHQWLQJ DOO JHRJUDSKLF DUHDV RI WKH 8QLWHG 6WDWHV $FFRUGLQJ WR 'RZQLH f WKH WHVW DSSHDUV WR KDYH IDFH YDOLGLW\ EXW WKH SRWHQWLDO UHVSRQVHV DUH LQ DPELJXRXV WHUPV VXFK DV KDUGO\ HYHU 5HFRPPHQGDWLRQV DUH IRU IXUWKHU VWXG\ ZLWK WKH LQVWUXPHQW WR HVWDEOLVK SUHGLFWLYH YDOLGLW\ WKURXJK DGGLWLRQDO QRUPV

PAGE 45

7KH 9RFDWLRQDO $GMXVWPHQW 5DWLQJ 6FDOH IRU WKH 5Hn WDUGHG 6RQJ t 6RQJ f LV XVHG WR PHDVXUH WKH SHUn IRUPDQFH DQG VSHFLILF EHKDYLRUV RI WKH FOLHQW LQ DOO W\SHV RI ZRUN VHWWLQJV 7KH WHVW SRSXODWLRQ ZDV FRPSRVHG RI FOLHQWV DJH UDQJH ,4 UDQJH WR f IURP D UHKDELOLWDWLRQ FHQWHU )LYH DUHDV ZRUN DELOLW\ ZRUN KDELWV ZLWKGUDZQ EHKDYLRU DJJUHVVLYH EHKDYLRU DQG EL]DUUH EHKDYLRUf ZHUH UDWHG RQ D ILYH SRLQW FRQWLQXXP UDQJLQJ IURP YHU\ SRRU WR H[FHOOHQW 7KH VDPH VXSHUn YLVRU\ SHUVRQQHO ZKR KDG FRPSOHWHG WKH UDWLQJ VFDOH DOVR FRPSOHWHG FODVVLILFDWLRQ RI WKH ZRUNHUV WKXV SRVVLEO\ FRQWDPLQDWLQJ WKH UHVXOWV DQG H[SODLQLQJ WKH KLJK FRUUHODn WLRQV WKDW ZHUH IRXQG 2Q D WZR ZHHN IROORZXS HPSOR\HUVn UDWLQJV RQO\ GLVSOD\HG SUHGLFWLYH YDOLGLW\ U f RQ VFDOH QXPEHU WZR ZRUN KDELWVf 7KH DXWKRUV EODPH WKH ODFN RI SUHGLFWLYH YDOLGLW\ DQG GLIIHUHQFHV LQ FULWHULD EHWZHHQ WKDW RI WKH ZRUNVKRS DQG WKDW UHTXLUHG RQ WKH MRE 7KH XVH RI ZRUN VDPSOHV DQG UDWLQJ VFDOHV LQ YRFDn WLRQDO HYDOXDWLRQ RI WKH UHWDUGHG KDV UHFHLYHG VKDUS FULWLn FLVP E\ DXWKRULWLHV LQ WKH ILHOG RI VSHFLDO HGXFDWLRQ :ROIHQVEHUJHU f FRQWHQGV WKDW WKH ZRUN VDPSOH KDV UDWKHU GRXEWIXO YDOXH LQ HYDOXDWLQJ WKH PHQWDOO\ UHWDUGHG EHFDXVH RI D QXPEHU RI EHKDYLRUDO DV ZHOO DV SUDFWLFDO FRQVLGHUDWLRQV +H VXJJHVWV WKDW FDXWLRQ EH WDNHQ LQ WKDW ZRUN VDPSOLQJ FDQ GHDGHQ PRWLYDWLRQ VLQFH QR UHZDUG LV JHQHUDOO\ JLYHQ IRU ZRUNLQJ RQ VDPSOHV $OVR WKH

PAGE 46

ZRUN LV RIWHQ FRQWULYHG RU PDNH EHOLHYH DQG FOLHQWV RIWHQ GR QRW H[HUW WKHLU RSWLPXP HIIRUW LQ SHUIRUPLQJ WKHVH WDVNV :ROIHQVEHUJHU DOVR LQGLFDWHG WKDW VRPH ZRUN VDPSOHV DSSHDU WR PHDVXUH VNLOOV WKDW VKRXOG EH WDXJKW IRU ORQJ SHULRGV RI WLPH EHIRUH EHLQJ HYDOXDWHG VXFK DV WKH FRn RUGLQDWHG XVH RI WRROV +H ZDUQHG WKDW LQ ZRUNLQJ WKURXJK D VHULHV RI ZRUN WDVNV FOLHQWV DUH RIWHQ VKLIWHG IURP WDVN WR WDVN DQG GR QRW UHPDLQ ORQJ HQRXJK LQ RQH DFWLYLW\ WR UHDFK WKHLU KLJKHVW OHYHO RI SHUIRUPDQFH OLPLWHG WDVN SHUIRUPDQFH PD\ QRW DFFXUDWHO\ SUHGLFW ODWHU SHUn IRUPDQFH LQ WKH UHWDUGHGf :ROIHQVEHUJHU f VXJJHVWV WKDW ZH UHGXFH HYDOXDn WLRQ WLPH DQG LQFUHDVH WUDLQLQJ H[SHULHQFHV ZLWK UHJDUG WR PHQWDOO\ UHWDUGHG FOLHQWV -DVWDN DQG .LQJ f DQG %UROLQ f HQFRXUDJH PXOWLSOH DGPLQLVWUDWLRQ RI HYDOXDWLRQ EDWWHULHV ZLWK WKH UROH RI WUDLQLQJ RU H[n SORULQJ DV DQ REMHFWLYH WR LQFUHDVH WKH FOLHQWnV IXQFn WLRQLQJ DELOLW\ 3DWWHUVRQ f SRLQWV RXW WKDW ZKDW RIWHQ LV RYHUORRNHG LQ GLVFXVVLRQV RI WKH ZRUN VDPSOLQJ DSSURDFK LV WKDW UHVHDUFK LQGLFDWHV WKDW LW LV QRW RQO\ RU PDLQO\ LQDELOLW\ RU ODFN RI PDQXDO VNLOO ZKLFK PDNHV IRU IDLOXUH LQ HPSOR\PHQW S f 7KXV WKH UHVHDUFK WHQGV WR VXSSRUW WKH QHHG WR IXUWKHU LQYHVWLJDWH WKH XVH RI SUHVHQW ZRUN VDPSOHV DV D SUHGLFWLYH RU HYDOXDWLYH LQVWUXPHQW ZLWK WKH PHQWDOO\ UHWDUGHG 5DWKHU LW PD\ EH QHFHVVDU\ WR LQYHVWLJDWH RWKHU YDULDEOHV LPSRUWDQW IRU

PAGE 47

VXFFHVVIXO HPSOR\PHQW DQG RWKHU XVHV RI WKH ZRUN VDPSOH DSSURDFK VXFK DV H[SORUDWRU\ RU WUDLQLQJ WRROV 6WXGLHV VXSSRUW WKH K\SRWKHVLV RI SRRU DGDSWDELOLW\ UDWKHU WKDQ SRRU DELOLW\ DV DQ H[SODQDWLRQ IRU ORZ VFRUHV RQ LQLWLDO WULDOV RQ PRWRU WDVNV ZLWK WKH PHQWDOO\ UHWDUGHG &ODUNH f VXJJHVWV WKDW ZLWK SUDFWLFH WKH PHQWDOO\ UHWDUGHG FORVHO\ DSSURDFK WKH SHUIRUPDQFH OHYHO RI QRUPDO LQGLYLGXDOV ZKR KDYH DOVR KDG WKH DGYDQWDJH RI SUDFWLFH *RUGDQ f DQG 5H\QROGV DQG $GDPV f IRXQG WKDW WKH GLIIHUHQFH LQ SHUIRUPDQFH OHYHO EHWZHHQ VXEMHFWV RI KLJK DQG ORZ DELOLW\ GLPLQLVKHG ZLWK LQFUHDVLQJ SUDFWLFH /RZHU UDWHV RI LPSURYHPHQW IRU QRUPDO LQGLYLGXDOV ZHUH H[SODLQHG E\ D KLJK LQLWLDO SHUIRUPDQFH DQ DFWLYDWLQJ RI D SRRO RI SUHYLRXVO\ DFTXLUHG UHOHYDQW UHVSRQVHV FDUULHG RYHU IURP SV\FKRPRWRU WDVNV HQFRXQWHUHG LQ HYHU\GD\ VLWXDn WLRQV 5H\QROGV t $GDPV S f &XUUHQW VWXGLHV UHODWLQJ WR SHUIRUPDQFH RQ ZRUN WDVNV E\ WKH PHQWDOO\ UHWDUGHG DUH EHJLQQLQJ WR VKRZ WKDW HYHQ LQGLYLGXDOV ZLWK ORZHU OHYHOV RI UHWDUGDWLRQ FDQ DFKLHYH VXFFHVV RQ KLJK OHYHO WDVNV ZLWK SURSHU WUDLQLQJ SUDFWLFH *ROG f LQ DWWHPSWLQJ WR REWDLQ EDVHUDWH GDWD RI QRQUHLQIRUFHG SHUIRUPDQFH RQ DFTXLVLWLRQ DQG SURGXFWLRQ ZLWK PRGHUDWHO\ DQG VHYHUHO\ UHWDUGHG LQGLYLGXDOV SURGXFn LQJ D FRPSOH[ DVVHPEO\ IRXQG DQ H[FHSWLRQDOO\ KLJK UDWH DQG TXDOLW\ RI SHUIRUPDQFH LQ WKH DEVHQFH RI SD\ 7KH ILQGLQJV RIIHUHG DQ LQGLFDWLRQ WKDW PRUH VKRXOG EH GRQH

PAGE 48

WR LQFUHDVH WKH OHYHO DQG YDOXH RI ZRUN ZKLFK WKH UHn WDUGHG GR LQVWHDG RI DWWHPSWLQJ WR GHYHORS ODUJHU HYDOXDn WLRQ DQG UHLQIRUFHPHQW V\VWHPV 9LWK WKH JHQHUDO LQFUHDVH LQ SURGXFWLYLW\ DQG WKH GURS LQ PHDQ HUURUV WKH DXWKRU FRQFOXGHG WKDW WKH WDVN LWVHOI KDG VWURQJ UHLQIRUFLQJ SURSHUWLHV *ROG f DOVR QRWHG D QRQVLJQLILFDQW UHODWLRQVKLS EHWZHHQ DFTXLVLWLRQ DQG SURGXFWLRQ UHMHFWLQJ WKH DVVXPSn WLRQ WKDW D VWURQJ UHODWLRQVKLS H[LVWV EHWZHHQ OHDUQLQJ DELOLW\ DQG SURGXFWLRQ DVVXPSWLRQ PDGH E\ 72:(5 f -(96 f DQG RWKHU HYDOXDWLRQ V\VWHPV DQG VWDQGDUGL]HG LQVWUXPHQWVf 7KH GDWD *ROG f LQGLFDWH WKDW FXUUHQW HYDOXDWLRQ SURFHGXUHV UHVXOW LQ D PLVOHDGLQJ XQGHUHVWLPDWH RI WKH SHUIRUPDQFH FDSDELOLWLHV RI WKH UHWDUGHG *ROG f DQG UDLVH VHULRXV TXHVWLRQV DERXW WKH OLPLWLQJ HIn IHFWV RI FXUUHQW YRFDWLRQDO GLDJQRVWLF DQG HYDOXDWLYH SURFHGXUHV IRU WKH PHQWDOO\ UHWDUGHG *ROG f VXJJHVWV WKDW IXWXUH UHVHDUFK IRFXV RQ Df SURJUDP GHYHORSPHQW UDWKHU WKDQ RUJDQL]DWLRQ Ef SURFHGXUHV RI HYDOXDWLRQ ZKLFK SURGXFH GHPRQVWUDEO\ XVHIXO LQIRUPDWLRQ UDWKHU WKDQ PRUH SUHGLFWLRQ DQG Ff GHYHORSPHQW RI XQLYHUVDO UHLQn IRUFHPHQW V\VWHPV DQG FODULILFDWLRQ RI WUDLQLQJ REMHFWLYHV $V UHIOHFWHG E\ WKH 9(:$$ 9RFDWLRQDO (YDOXDWLRQ 3URMHFW )LQDO 5HSRUW f WKH FXUUHQW REMHFWLYHV IRU YRFDWLRQDO HYDOXDWLRQ VHUYLFHV VHHP WR EH SURJUHVVLQJ WRZDUG D PRUH WUHDWPHQW RULHQWHG VHUYLFH UDWKHU WKDQ

PAGE 49

WKH VWULFW DVVHVVPHQW DQG SUHGLFWLYH SURFHGXUH 7KH FXUUHQW REMHFWLYHV DUH WR LGHQWLI\ DQ RSWLPDO RXWFRPH IRU WKH LQGLYLGXDO WR LQWHQVLI\ WKH IXQFWLRQDO FRPSHn WHQFLHV DQG IXQFWLRQDO GLVDELOLWLHV RI WKH LQGLYLGXDO PHGLFDO SV\FKRORJLFDO VRFLDO YRFDWLRQDO HGXFDWLRQDO FXOWXUDO HFRQRPLFf WR LGHQWLI\ WKRVH VHUYLFHV QHHGHG WR RYHUFRPH WKH IXQFWLRQDO GLVDELOLWLHV DQG WR UHGXFH RU HOLPLQDWH IXQFWLRQDO GLVDELOLWLHV RI WKH LQGLYLGXDO LQ D QHZ VLWXDWLRQ 7KHUH DSSHDUV WR EH HYLGHQFH WR VXJJHVW WKDW FXUUHQW DVVHVVPHQW DQG HYDOXDWLRQ V\VWHPV DIIHFW WKH FOLHQW LQ D SRVLWLYH PDQQHU *ZLOOLDQ f LGHQWLILHG VHYHUDO GLUHFW FOLHQW EHQHILWV IURP YRFDWLRQDO HYDOXDWLRQ LQFOXGLQJ JUHDWHU DZDUHQHVV RI JRDOV D EHWWHU XQGHUVWDQGLQJ RI SHUVRQDO DELOLWLHV FDSDFLW\ DQG SRWHQWLDO DORQJ ZLWK PRUH UHDOLVWLF DVSLUDWLRQV 7KH UHPDLQLQJ VHFWLRQ RI WKLV OLWHUDWXUH UHYLHZ LQGLFDWHV WKH LPSRUWDQFH RI WKHVH YDULDEOHV DZDUHQHVV LQWHUHVWV DQG DVSLUDWLRQVf LQ WKH FDUHHU GHYHORSPHQW RI WKH PHQWDOO\ UHWDUGHG LQGLYLGXDO 9RFDWLRQDO ,QWHUHVWV DQG 1HHGV $ QXPEHU RI VWXGLHV RQ YRFDWLRQDO SODFHPHQW RI WKH UHWDUGHG LQFOXGH IROORZXS DQG DQDO\VLV RI UHDVRQV IRU MRE IDLOXUH ,Q WKH PDMRULW\ RI FDVHV WKH FOLHQWV ZHUH DEOH WR PHHW WKH VNLOO DQG VWUHQJWK GHPHQGV RI WKH MRE EXW IDLOXUHV DSSHDUHG WR UHIOHFW GLIILFXOWLHV SHUWDLQLQJ WR ZRUN LQWHUHVW KDELWV PRWLYDWLRQ DQG XQGHUVWDQGLQJ

PAGE 50

RI MRE UHTXLUHPHQWV &RKHQ 'LQJHU 2n&RQQRU 3HFNKDP 3HWHUVRQ t 6PLWK :LQGOH f )RU WKH DERYH UHDVRQV WKHUH KDV EHHQ D VWHDGLO\ LQFUHDVLQJ FRQFHUQ VKRZQ LQ WKH DUHD RI YRFDWLRQDO LQn WHUHVW HYDOXDWLRQ RI WKH PHQWDOO\ UHWDUGHG 7LPPHUPDQ DQG 'RFWRU f VXJJHVW WKUHH SRVVLEOH UHDVRQV ZK\ WKH DUHD RI LQWHUHVW DVVHVVPHQW KDV ODJJHG EHKLQG RWKHU DUHDV RI YRFDWLRQDO HYDOXDWLRQ IRU WKH PHQWDOO\ UHWDUGHG 7KHVH DUH WKDW WKH PHQWDOO\ UHWDUGHG KDYH QRW EHHQ FRQn VLGHUHG DV VHULRXV FDQGLGDWHV IRU FRPSHWLWLYH HPSOR\PHQW LQ WKH SDVW WKDW GXH WR WKH PHQWDOO\ UHWDUGHG LQGLYLGXDOnV OLPLWHG VRFLDO PDWXULW\ DQG H[SRVXUH WR MRE GLIIHUHQFHV WKHLU DELOLW\ WR PDNH RFFXSDWLRQDO FKRLFHV KDV EHHQ TXHVn WLRQHG DQG ODVWO\ WKDW YHU\ IHZ LQWHUHVW DVVHVVPHQW LQVWUXPHQWV DUH DSSURSULDWH IRU WKH PHQWDOO\ UHWDUGHG SRSXODWLRQ :HOONQRZQ DQG FRPPRQO\ XVHG LQWHUHVW LQYHQWRULHV VXFK DV WKH 6WURQJ 9RFDWLRQDO ,QWHUHVW %ODQN 69,%f DQG WKH .XGHU KDYH SURYHQ WR EH XQVDWLVIDFWRU\ IRU XVH ZLWK WKH PHQWDOO\ UHWDUGHG EHFDXVH RI WKHLU GHSHQGHQFH XSRQ YHUEDO DELOLWLHV DQG D ZLGH UDQJH RI H[SHULHQFHV %UROLQ f :LWK WKH GHVLUH WR DVVHVV WKH YDOLGLW\ RI RFFXSDWLRQDO FKRLFHV DPRQJ YDULRXV OHYHOV RI WKH PHQWDOO\ UHWDUGHG SRSXODWLRQ &RKHQ DQG 5XVDOHP f FRQGXFWHG D VWXG\

PAGE 51

XVLQJ WKUHH VXEJURXSV QRUPDO QRQLQVWLWXWLRQDOL]HG PHQWDOO\ UHWDUGHG DQG LQVWLWXWLRQDOL]HG PHQWDOO\ UHn WDUGHG 7KH VWXGHQWV ZHUH SUHVHQWHG QLQH RFFXSDWLRQDO YDOXHV DGYDQFHPHQW EHQHILWV LQGHSHQGHQFH LQWHUHVWLQJ ZRUN SUHVWLJH LQWHUSHUVRQDO UHODWLRQV VDODU\ VHFXULW\ DQG ZRUNLQJ FRQGLWLRQV (DFK VWXGHQW ZDV DVNHG WR LGHQWLI\ WKH PRVW LPSRUWDQW RFFXSDWLRQDO YDOXH RUDO DQG ZULWWHQ SUHVHQWDWLRQf 7KH PHDQ UDQNV IRU WKH QLQH LWHPV ZHUH FRPSXWHG ERWK IRU WKH WKUHH PDMRU JURXSV DQG IRU VXEJURXSV RI ER\V DQG JLUOV 5HVXOWV LQGLFDWHG D JUHDWHU UHVHPEODQFH EHWZHHQ VXEJURXSV RI JLUOV WKDQ EHWZHHQ JURXSV RI ER\V ZKLFK ZDV YLHZHG DV D UHIOHFWLRQ RI JUHDWHU VLPLODULWLHV RI SHUFHLYHG UROHV DFURVV LQWHOOHFWXDO OHYHOV %RWK VH[HV DQG VXEJURXSV SODFHG KLJK YDOXH XSRQ MRE DGYDQFHn PHQW VXJJHVWLQJ WKDW GHVSLWH LQWHOOHFWXDO FDSDFLW\ VWXn GHQWV KDYH KLJK H[SHFWDWLRQV RI DFKLHYLQJ KLJKOHYHO YRFDWLRQDO REMHFWLYHV 7KH PHQWDOO\ UHWDUGHG ER\V SODFHG JUHDWHU HPSKDVLV RQ MRE EHQHILWV DQG OHVV XSRQ UHODWLRQV ZLWK RWKHUV DQG LQWHUHVWLQJ ZRUN WKXV LQGLFDWLQJ WKDW $ ER\V VHHPHG WR SUHIHU LPPHGLDWH RU HDUO\ JUDWLILFDWLRQ :KHWKHU RU QRW WKH FKRLFHV UHIOHFW OHJLWLPDWH FRQVLVWHQW LQWHUHVWV ZDV QRW UHDOO\ HVWDEOLVKHG 7KH DXWKRU LQGLFDWHG WKH QHHG IRU VLPLODU VWXGLHV ZLWK ROGHU DJH JURXSV WR HYDOXDWH LI LQWHUHVWV H[SUHVVHG DW WKH \RXQJHU DJH OHYHO UHPDLQ FRQVWDQW

PAGE 52

%XUJ DQG %XUQHWW f QRWHG WKDW ZLWK LQWHUHVW DVVHVVPHQW RI WKH PHQWDOO\ UHWDUGHG Df WHVWV UHTXLULQJ HYHQ D PLQLPXP RI UHDGLQJ DELOLW\ ZHUH XQUHDOLVWLF DQG Ef WKDW HYHQ LQ SLFWXUH WHVWV UHVSRQVH ZDV IUHTXHQWO\ WR VSHFLILF SHRSOH RU LWHPV ZLWKLQ WKH SLFWXUH UDWKHU WKDQ WR WKH RFFXSDWLRQ LOOXVWUDWHG 7R FRXQWHUDFW WKLV DIIHFW WKH DXWKRUV GHYLVHG D VHW RI YHUEDO GHVFULSWLRQV WR DFFRPSDQ\ D VWDQGDUGL]HG LQWHUHVW LQVWUXPHQW WKH *HLVW 3LFWXUH ,QWHUHVW ,QYHQWRU\ *38f ,Q VRPH FDVHV RULJLQDO WLWOHV ZHUH XVHG ZKLOH IRU WKH PDMRULW\ RI WLWOHV VLPSOLILFDWLRQV RI D PRUH GHVFULSWLYH QDWXUH ZHUH DGRSWHG 'XULQJ WKH WHVW DGPLQLVWUDWLRQ WKH H[DPLQHU UHDGV WKH TXHVWLRQV DQG GHVFULEHV HDFK SLFWXUH WKHQ DVNV WKH FOLHQW WR LQGLFDWH KLV SUHIHUHQFH E\ UDLVLQJ WKH DSSURSULDWH QXPEHU RI ILQJHUV RU SRLQWLQJ WR KHU SUHIHUHQFH 7KLV ELVHQVRU\ IRUP RI DGPLQLVWUDWLRQ IRU WKH UHWDUGHG QR HVWDEOLVKHG YDOLGLW\f \LHOGHG VFRUHV ZKLFK ZHUH FRPSDUHG ZLWK VFRUHV IURP WKH *(,67 PDQXDO &RPSDULVRQV LQGLFDWHG WKDW WKH UHWDUGHG VDPSOH REWDLQHG PHDQ VFRUHV H[FHHGLQJ WKH JHQHUDO SRSXODWLRQ LQ WKH DUHDV RI PHFKDQLFDO FOHULFDO RXWGRRU VRFLDO VHUYLFH DQG GUDPDWLF 7KH DXWKRUV GR QRW GUDZ VSHFLILF FRQFOXVLRQV IURP WKH ILQGLQJV OHDYLQJ RSHQ WKH TXHVWLRQ RI YDOLG LQWHUHVW SDWWHUQV IRU WKH PHQWDOO\ UHWDUGHG $QRWKHU LQWHUHVW LQYHQWRU\ WKH 9RFDWLRQDO ,QWHUHVW DQG 6RSKLVWLFDWLRQ $VVHVVPHQW 9,6$f GHYHORSHG DV SDUW

PAGE 53

RI WKH -RKQVWRQH 3URMHFW E\ 3DUQLFN\ DQG .DKQ f ZDV GHVLJQHG VSHFLILFDOO\ IRU D PHQWDOO\ UHWDUGHG SRSn XODWLRQ 7KH LQWHUHVW SRUWLRQ RI WKH LQVWUXPHQW FRQVLVWV RI D QXPEHU RI OLQH GUDZLQJV VKRZLQJ SHRSOH ZRUNLQJ DW YDULRXV MREV XVXDOO\ FRQVLGHUHG DV VXLWDEOH IRU WKH UHn WDUGHG PDOHV LQ WKH DUHDV RI FRQVWUXFWLRQPDLQWHQDQFH IDUPJURXQGV IRRG VHUYLFH JDUDJH LQGXVWULDO DQG ODXQGU\ IHPDOHV LQ WKH DUHDV RI EXVLQHVVFOHULFDO IRRG VFLHQFH KRXVHNHHSLQJ DQG ODXQGU\f 7KH FOLHQW LV VKRZQ HDFK GUDZLQJ DQG DVNHG LI KH ZRXOG OLNH WR SHUIRUP WKH ZRUN D ORW D OLWWOH RU QRW DW DOO SUREOHP RI UHVSRQVH VHW ZLWK WKH FOLHQWf 7KH VRSKLVWLFDWLRQ SRUWLRQ RI WKH 9,6$ FRQVLVWV RI D VHULHV RI TXHVWLRQV DVNHG DERXW SLFWXUHV WDNHQ IURP WKH LQWHUHVW SRUWLRQ RI WKH WHVW WR DVVHVV WKH VXEMHFWnV NQRZOHGJH RI WKH MREV ZKLFK DUH LOOXVWUDWHG $FFRUGLQJ WR WKH DXWKRUV WKH SUHPLVH LV WKDW IRU DQ LQGLYLGXDOnV LQWHUHVW WR KDYH KLJK YDOLGLW\ WKH LQWHUHVW VKRXOG UHn IOHFW NQRZOHGJH RI WKH MRE DUHD 3DUQLFN\ .DKQ DQG %XUGHWW f UHSRUWHG WHPSRUDO UHOLDELOLW\ GDWD IRU WHUPV RI RQH PRQWK DQG WR PRQWK WHVWUHWHVW FRUUHODWLRQ FRHIILFLHQWV 7KH PHDQ VKRUWWHUP FRUUHODWLRQ FRHIILFLHQW IRU PDOHV DQG IHPDOHV ZDV 7KH PHDQ ORQJWHUP FRUUHODWLRQ FRHIILFLHQW IRU PDOHV ZDV IRU IHPDOHV WKH U 9DOLGLW\ UHSRUWV FRQVLVWHG RI 9,6$ VFRUHV IDYRUDEO\ FRPSDUHG ZLWK

PAGE 54

UDWLQJV RI ZRUN VXSHUYLVRUV $ ODWHU UHSRUW E\ 3DUQLFN\ .DKQ DQG %XUGHWW f LQGLFDWHG WKDW LQWHUHVWV DV PHDVXUHG E\ WKH 9,6$ DSSHDU WR EH LQGHSHQGHQW RI WKH OHYHO RI LQWHOOLJHQFH &$ DQG NQRZOHGJH RI MREV 7KH VWUHQJWK RI DVVRFLDWLRQV EHWZHHQ ,4 DQG VRSKLVWLFDWLRQ VFRUHV ZHUH ORZ WR PRGHUDWH $OVR D KLJK GHJUHH RI FRQVWUXFW YDOLGLW\ ZDV IRXQG WKURXJK D IDFWRU DQDO\VLV IRU ERWK PDOH DQG IHPDOH IRUPV RI WKH LQWHUHVW VFDOHV ,Q D VWXG\ WR GHWHUPLQH WKH HIIHFWV RI YDULRXV H[n SORUDWRU\ WUHDWPHQWV XSRQ WKH VRSKLVWLFDWLRQ OHYHO RI PLOGO\ UHWDUGHG VWXGHQWV 6DORPRQH /HKPDQQ DQG *UHHQ f DGPLQLVWHUHG WKH 9,6$ ZLWK VRPH SHUFHLYHG VKRUWn FRPLQJV Df WKH SHUVRQDOLW\ RI WKH 9,6$ DGPLQLVWUDWRU PD\ DIIHFW WKH VFRUHV Ef D QHHG IRU H[WHQVLYH QRUPDWLYH GDWD RQ WKH 9,6$ ZLWK QRUPDO LQGLYLGXDOV DQG Ff WKH QHHG IRU D JUHDWHU QXPEHU RI UHDOLVWLF SRWHQWLDO MRE HPSOR\PHQW DUHDV IRU WKH UHWDUGHG 7LPPHUPDQ DQG 'RFWRU f LQ WKHLU UHYLHZ RI DVVHVVPHQW LQVWUXPHQWV IRU WKH PHQWDOO\ UHWDUGHG RIIHU VRPH SRVLWLYH DVSHFWV RI WKH 9,6$ LW JLYHV DSSURSULDWH MREV LQ VLPSOH OLQH GUDYLQJV LW DWWHPSWV WR GHDO ZLWK WKH SUREOHP RI VRSKLVWLFDWLRQ DQG LW KDV WKH DGYDQWDJH RI WKH FOLHQW QRW KDYLQJ WR PDNH IRUFHG FKRLFHV EHWZHHQ SLFWXUHV 6HYHUDO SUREOHPV ZLWK XVH RI WKH 9,6$ DUH DOVR SRLQWHG RXW E\ 7LPPHUPDQ DQG 'RFWRU f WKH VLPSOH UHVSRQVH IRUPDW RIWHQ GRHV QRW GLVFULPLQDWH EHWZHHQ MRE DUHDV VRPH RI WKH GUDZLQJV DUH

PAGE 55

DPELJXRXV DV WR ZKDW WKH MRE LV RU ZKLFK RI WKH SHUVRQV SLFWXUHG LV UHODWHG WR WKH MRE WKH VRSKLVWLFDWLRQ HOHPHQW RI WKH WHVW UHTXLUHV FRQVWDQW XSGDWLQJ DQG FKDQJH DQG LW RIIHUV D OLPLWHG YLHZ RI D VPDOO QXPEHU RI ILHOGV ZKLFK DUH LOOXVWUDWHG 7KH 5HDGLQJ )UHH 9RFDWLRQDO ,QWHUHVW ,QYHQWRU\ ZDV GHYHORSHG E\ %HFNHU f WR REMHFWLYHO\ DVVHVV D UHWDUGHG LQGLYLGXDOnV SHUIRUPDQFH IRU SDUWLFXODU ZRUN DUHDV 7KH *XLGH WR -REV IRU WKH 0HQWDOO\ 5HWDUGHG 3HWHUVRQ t -RQHV f VHUYHG DV WKH SULPDU\ VRXUFH RI ZRUN DUHDV ZKLFK WKH PHQWDOO\ UHWDUGHG KDYH GHPRQVWUDWHG SURILFLHQF\ DQG SURGXFWLYLW\ ZLWKLQ (OHYHQ PDOH MRE FOXVWHU VFDOHV DQG HLJKW IHPDOH VFDOHV ZHUH GHYHORSHG 7KH FOLHQW ZDV SUHVHQWHG ZLWK WKUHH SLFWXUHV RI MRE DUHDV DQG ZDV DOORZHG WR PDNH RQO\ RQH FKRLFH DOWKRXJK DOO WKUHH DUHDV PD\ KDYH EHHQ RI LQWHUHVW WR KLP 7KH QRUPDWLYH VDPSOH IRU WKH VWXG\ %HFNHU f FRQVLVWHG RI (05 VXEMHFWV IURP DOO JHRJUDSKLFDO VHFWLRQV RI WKH 8QLWHG 6WDWHV 1RUPV DUH DYDLODEOH IRU PDOH DQG IHPDOH VXEMHFWV LQ WKH SXEOLF VFKRROV DQG IURP LQVWLWXWLRQV 9DOLGLW\ RI WKH LQYHQWRU\ ZDV REWDLQHG ZLWK FRQFXUUHQW WHVWLQJ XVLQJ WKH *HLVW 3LFWXUH ,QWHUHVW ,QYHQWRU\ *38f VLQFH LW FRQWDLQV PDOH DQG IHPDOH IRUPV VLPLODU WR WKH 5HDGLQJ )UHH 9RFDWLRQDO ,QWHUHVW ,QYHQWRU\ &RHIILFLHQWV RI FRUUHODWLRQ ZHUH FRPSXWHG ZLWK WKH LQWHUHVW VFDOHV UHVXOWLQJ LQ D SRVLWLYH UHODWLRQVKLS 7KH WHVWUHWHVW

PAGE 56

PHWKRG RI HVWDEOLVKLQJ UHOLDELOLW\ FRHIILFLHQWV WZR ZHHN LQWHUYDOf IRU PDOHV DQG IHPDOHV \LHOGHG WKH IROORZn LQJ YDOXHV SXEOLF VFKRRO PDOHf LQVWLWXWLRQ PDOHf SXEOLF VFKRRO IHPDOHf LQVWLWXWLRQ IHPDOHf 7KHUH UHPDLQV D QHHG WR HVWDEOLVK SUHGLFWLYH YDOLGLW\ RI WKH VFDOHV IRU WKH 5HDGLQJ )UHH 9RFDWLRQDO ,QWHUHVW ,QYHQWRU\ DQG WR DVVHVV LWV YXOQHUDELOLW\ WR WKH VDPH FULWLFLVPV EURXJKW DJDLQVW WKH *HLVW DQG 9,6$ GXH WR WKH PDQ\ VLPLODULWLHV DPRQJ WKH WKUHH LQVWUXPHQWV 7KH :LGH 5DQJH ,QWHUHVW 2SLQLRQ 7HVW :5,27f LV D UHFHQW WHVW SXEOLVKHG E\ -DVWDN DQG -DVWDN f ZKLFK LV FRPSRVHG RI VHWV RI WKUHH SLFWXUHV HDFK 7KH FOLHQW LV WR UHVSRQG WR WKH PRVW DQG OHDVW OLNHG SLFWXUHV LQ HDFK VHW 1R UHDGLQJ LV UHTXLUHG RQ WKH SDUW RI WKH FOLHQW 7KH LQVWUXPHQW ZDV QRUPHG RQ WKH JHQHUDO SRSXODWLRQ DGXOWV DQG KLJK VFKRRO VWXGHQWVf LQFOXGLQJ D KLJK SHUFHQWDJH RI UHKDELOLWDWLRQ FOLHQWV 9DU\LQJ YDOLGLW\ FRHIILFLHQWV ZHUH REWDLQHG WKURXJK FRQFXUUHQW WHVWLQJ XVLQJ WKH *38 EXW WKH DXWKRUV GLVFRXQW WKH YDOXH RI YDOLGLW\ HVWLPDWHV E\ LPSO\LQJ WKDW WKH LQVWUXPHQW VKRXOG EH YDOLGDWHG HDFK WLPH LW LV XVHG 5HOLDELOLW\ FRHIILFLHQWV ZHUH REWDLQHG WKURXJK D VSOLWKDOI PHWKRG IRU PDOHV DQG IHPDOHV ZLWKLQ HDFK RI WKH FOXVWHU DUHDV 7KH OHQJWK RI WKH WHVW LV UDWKHU ORQJ HVSHFLDOO\ IRU WKRVH ZLWK OLPLWHG DWWHQWLRQ VSDQV PDNLQJ XVH RI WKH LQVWUXPHQW ZLWK WKH PHQWDOO\ UHn WDUGHG TXHVWLRQDEOH

PAGE 57

$QRWKHU LQVWUXPHQW ZKLFK IRFXVHV VSHFLILFDOO\ XSRQ FRPSHWHQFLHV QHHGHG IRU SRVWVFKRRO DGMXVWPHQW LV WKH 6RFLDO DQG 3UHYRFDWLRQDO ,QIRUPDWLRQ %DWWHU\ 63,%f ZKLFK ZDV GHYHORSHG E\ +DOSHUQ 5DIIHOG ,UYLQ DQG /LQN f WR HYDOXDWH WKH LPSDFW RI SUHYRFDWLRQDO ZRUNVWXG\f SURJUDPV XSRQ WKH PLOGO\ UHWDUGHG 7KH 63,% DVVHVVHV NQRZOHGJH LQ QLQH VNLOO DUHDV UHODWHG WR FRPPXQLW\ DGn MXVWPHQW 1RUPV ZHUH JDWKHUHG RQ WKH PLOGO\ UHWDUGHG IRU VDPSOH JURXSV IURP GLIIHUHQW UHJLRQV ZLWKLQ WKH 8QLWHG 6WDWHV 5HOLDELOLW\ HVWLPDWHV ZHUH HVWDEOLVKHG ZLWK YDU\n LQJ VDPSOH VL]HV WKURXJK WKH WHVWUHWHVW PHWKRG 9DOLGLW\ ILJXUHV ZHUH REWDLQHG E\ FRPSDULQJ WKH UHODWLRQVKLS EHWZHHQ WKH QLQH WHVWV RI WKH EDWWHU\ DQG WKH ILYH FULWHULRQ VXEn VFDOHV RYHU D RQH \HDU SHULRG 2QO\ WHQWDWLYH FRHIILFLHQWV ZHUH DYDLODEOH GXH WR WKH QHZQHVV RI WKH LQVWUXPHQW :KLOH UHSRUWLQJ ZRUN ZLWK WKH 0LQQHVRWD VWXGLHV %DZLV f LQWURGXFHG D WKHRU\ RI ZRUN DGMXVWPHQW WR DVVHVV WKH DSSURSULDWHQHVV RI MRE SODFHPHQW DFFRUGLQJ WR WKH GHJUHH RI FRUUHVSRQGHQFH EHWZHHQ DQ LQGLYLGXDOnV ZRUN SHUVRQDOLW\ DELOLWLHV DQG QHHGVf DQG WKH ZRUN HQYLURQPHQW DELOLWLHV UHTXLUHG DQG QHHGV VDWLVILDEOH E\ WKH UHLQIRUFHU V\VWHP RI WKH MREf 7R PDWFK DQ DVVHVVHG LQGLYLGXDO ZLWK DQ DVVHVVHG ZRUN HQYLURQPHQW VDWLVIDFWRULO\ DVVXPHV WKDW ZRUN HQYLURQPHQWV DQG SHUVRQDOLWLHV DUH VWDEOH DQG FRQn VLVWHQW 'DZLV f LQGLFDWHV WKDW PHQWDO DELOLW\ DQG LQWHUHVWV DUH IRXQG WR EH VWDEOH EXW DGPLWV WKDW WKH

PAGE 58

DVVXPSWLRQ LV D ZHDN SRLQW LQ WKH WKHRU\ 7KH ZRUN DGn MXVWPHQW WKHRU\ KDV EHHQ XVHG ZLWK GLVDGYDQWDJHG LQGLYLGXDOV SULPDULO\ EXW VHHPV WR KDYH LPSOLFDWLRQV IRU SUHYRFDWLRQDO SURJUDPV IRU WKH PHQWDOO\ UHWDUGHG 7KH QHHG WR LQYHVWLJDWH YDULDEOHV RU FRPELQDWLRQV RI YDULDEOHV RWKHU WKDQ MXVW YRFDWLRQDO DSWLWXGH RU DELOLW\ VHHPV WR EH ZHOO VXSSRUWHG LQ WKH OLWHUDWXUH 7KH IDLOXUH RI PDQ\ DSWLWXGH LQVWUXPHQWV WR SUHGLFW RU FRQWULEXWH WR WKH YRFDWLRQDO VXFFHVV RI WKH PHQWDOO\ UHWDUGHG KDV HQFRXUDJHG WKH GHYHORSPHQW RI VHYHUDO LQWHUHVW DQG QHHGV LQYHQWRULHV 7KLV UHYLHZ RI OLWHUDWXUH LQGLFDWHV YDU\LQJ OHYHOV RI VXFFHVV ZLWK FRQIOLFWLQJ FRQFOXVLRQV FRQFHUQLQJ DVVHVVPHQW LQ WKLV LPSRUWDQW DUHD RI VWXG\ )XUWKHU FRQn FHUQV DUH ZLWK WKH XVH RI WKH UHVXOWV RI LQWHUHVW DVVHVVn PHQWV WR EHVW KHOS WKH PHQWDOO\ UHWDUGHG LQGLYLGXDO WR DWWDLQ WKH KLJKHVW OHYHO RI FDUHHU GHYHORSPHQW SRVVLEOH 3UHYRFDWLRQDO *XLGDQFH DQG WKH 0HQWDOO\ 5HWDUGHG :LWK WKH GHYHORSPHQW RI VHYHUDO SUHYRFDWLRQDO LQWHUHVW DVVHVVPHQW LQVWUXPHQWV WKHUH LV FRQFHUQ IRU WKH FDUHHU GHYHORSPHQW OHYHOV DWWDLQDEOH E\ WKH PHQWDOO\ UHWDUGHG DQG WKH QHFHVVDU\ SUHYRFDWLRQDO JXLGDQFH SURJUDPV QHFHVVDU\ WR IDFLOLWDWH WKLV GHYHORSPHQW 5HODWHG WR WKH FXUUHQW LQWHUHVW LQ YRFDWLRQDO DGMXVWn PHQW RI WKH PHQWDOO\ UHWDUGHG VHYHUDO VWXGLHV KDYH LQn YHVWLJDWHG YDULDEOHV RI FRQFHUQ LQ SUHYRFDWLRQDO JXLGDQFH SURJUDPV *LQ]EHUJ f GHVFULEHG WKUHH SHULRGV LQ

PAGE 59

WKH GHYHORSPHQW RI RFFXSDWLRQDO FKRLFH IRU LQGLYLGXDOV RI DYHUDJH DELOLW\ 'XULQJ WKH IDQWDV\ SHULRG H[WHQGLQJ WR \HDUV RI DJH WKH LQGLYLGXDO LV PRWLYDWHG E\ KHUR ILJXUHV WKHUH LV OLWWOH UHDOLW\ WHVWLQJ RI WKH DSSURn SULDWHQHVV RI KLV FKRLFH ZKLFK RIWHQ ODFNV VWDELOLW\ 'XULQJ WKH VHFRQG VWDJH WR \HDUV WKH LQGLYLGXDO EHFRPHV DZDUH RI WKH TXDOLILFDWLRQV DELOLW\ DQG WUDLQLQJf DQG SHUVRQDO YDOXHV VXSSRUWLQJ KLV FKRLFHV ZLWK PRUH UHDOLVWLF FRQFHSWLRQV RI WKH SUHSDUDWLRQV QHHGHG WR DWWDLQ WKH MRE $ UHYLHZ RI VHOHFWHG VWXGLHV LQGLFDWHV WKDW WKH PLOGO\ UHWDUGHG DGROHVFHQW PD\ KDYH SUREOHPV ZKLFK KLQGHU KLP LQ WKH GHYHORSPHQW RI UHDOLVWLF YRFDWLRQDO JRDOV .XKQ f LQWHUYLHZHG PLOGO\ UHWDUGHG MXQLRU KLJK VFKRRO VWXGHQWV DQG QRUPDO &$ DQG 0$ FRQWUROV UHSRUWLQJ WKDW WKH PLOGO\ UHWDUGHG VWXGHQWV KDG D VLJQLILFDQWO\ ORZHU OHYHO RI XQGHUVWDQGLQJ RI WKHLU VHOHFWHG IXWXUH RFFXSDn WLRQV WKDQ WKH RWKHU JURXSV .DW]HQ f ZRUNLQJ ZLWK WKH PLOGO\ UHWDUGHG LQ D VSHFLDO FODVV IRXQG WKDW WKH GHJUHH RI UHDOLVP LQ YRFDWLRQDO SUHIHUHQFH ZDV UHODWHG WR ,4 DQG GHJUHH RI VHOIDFFHSWDQFH 7KH DXWKRU DOVR IRXQG WKDW IRU DOO VXEMHFWV WKHUH ZDV DQ LQYHUVH UHODWLRQn VKLS EHWZHHQ VHOIDFFHSWDQFH DQG UHDOLVP RI YRFDWLRQDO FKRLFH EXW IRU PDOH JURXSV WKH GHJUHH RI UHDOLVP ZDV DOVR D IXQFWLRQ RI ,4

PAGE 60

-RQHV f LQYHVWLJDWHG WKH UHODWLRQVKLS EHWZHHQ YRFDWLRQDO RXWORRN DQG W\SH RI SURJUDP ZLWK VSHFLDO FODVV VWXGHQWV 8VLQJ D VHOIFRQVWUXFWHG LQVWUXPHQW WKH ILQGLQJV LQGLFDWHG WKDW WKH DSSURSULDWHQHVV RI YRFDn WLRQDO SUHIHUHQFH LQFUHDVHG ZLWK ,4 IRU ERWK VH[HV LQ JHQHUDO IHPDOHV ZHUH OHVV UHDOLVWLF LQ WKHLU FKRLFHV WKDQ PDOHV WKH GHJUHH RI UHDOLVP RI YRFDWLRQDO FKRLFH DOVR LQFUHDVHG ZLWK FKURQRORJLFDO DJH DQG ZKLWH VWXGHQWV ZHUH PRUH UHDOLVWLF WKDQ QRQZKLWH )RLPDQ DQG %XGRII f DOVR VWXGLHG WKH UHODWLRQVKLS RI ,4 WR WKH GHYHORSPHQW RI UHDOLVWLF YRFDWLRQDO FKRLFH LQWHUHVWV DQG DVSLUDWLRQVf 7KH UHWDUGHG SRSXODWLRQ ZDV IXUWKHU GLYLGHG LQWR JURXSLQJV RI KLJK VFRUHUV RU JDLQHUV DQG QRQJDLQHUV GHWHUPLQHG E\ VFRUHV RQ WKH .RKVn %ORFN 'HVLJQ 7HVWf )LQGLQJV FRQVLVWHG RI WKH IROORZLQJ Df WKHUH ZHUH IHZ RU QR GLIIHUHQFHV EHWZHHQ UHWDUGHG DQG QRQUHWDUGHG DGROHVFHQWV LQ YRFDWLRQDO GHYHORSPHQW Ef WKH KLJK OHDUQLQJ SRWHQWLDO UHWDUGHG VWXGHQWV DVSLUHG IRU DQG H[SHFWHG WR REWDLQ ORZHU OHYHO MREV PLQLPL]HG WKH ULVN RI IDLOXUHf EXW GLVSOD\HG UHDOLVWLF DZDUHQHVV DQG LQYROYHPHQW LQ WKH IXWXUH MRE FKRLFH Ff WKH QRQJDLQLQJ UHWDUGHG UHVSRQGHG YHU\ SRVLWLYHO\ LQLWLDOO\ FRQFHUQLQJ YRFDWLRQDO DVSLUDWLRQV ZLVKIXO WKLQNLQJf EXW FRXOG QRW RIIHU DQ\ VROLG HYLGHQFH DERXW WKH UHDOLW\ RI KLV UHVSRQVHV DSSHDUHG WR EH LQ WKH IDQWDV\ SHULRGf DQG Gf WKH KLJK OHYHO UHWDUGHG WHQG WR H[SUHVV WKH VDPH RXWFRPHV DV WKHLU


xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID EMAKXROTJ_NC12A5 INGEST_TIME 2014-10-07T01:09:07Z PACKAGE AA00025754_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES