Citation
The selection of administrators in public community colleges : guidelines from the research

Material Information

Title:
The selection of administrators in public community colleges : guidelines from the research
Creator:
Eldridge, Betty-June Hauenstein, 1947-
Publication Date:
Language:
English
Physical Description:
ix, 222 leaves : ; 28 cm.

Subjects

Subjects / Keywords:
Attribution theory ( jstor )
Correlation coefficients ( jstor )
Educational research ( jstor )
Hiring ( jstor )
Human resources management ( jstor )
Psychological research ( jstor )
Psychology ( jstor )
Research methods ( jstor )
Research studies ( jstor )
Support personnel ( jstor )
College administrators -- Selection and appointment ( lcsh )
Community colleges -- Personnel management ( lcsh )
Dissertations, Academic -- Educational Leadership -- UF
Educational Leadership thesis Ph. D
Personnel management -- Research -- Statistical methods ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Thesis:
Thesis (Ph. D.)--University of Florida, 1988.
Bibliography:
Includes bibliographical references.
Additional Physical Form:
Also available online.
General Note:
Typescript.
General Note:
Vita.
Statement of Responsibility:
by Betty-June Hauenstein Eldridge.

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:
Copyright [name of dissertation author]. Permission granted to the University of Florida to digitize, archive and distribute this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
Resource Identifier:
024825677 ( ALEPH )
20071228 ( OCLC )

Downloads

This item has the following downloads:


Full Text
















THE SELECTION OF ADMINISTRATORS IN
PUBLIC COMMUNITY COLLEGES:
GUIDELINES FROM THE RESEARCH








By

BETTY-JUNE HAUENSTEIN ELDRIDGE


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL
FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY





UNIVERSITY OF FLORIDA


1988






























This study is dedicated

to the memory of my first teacher --

my mother















ACKNOWLEDGMENTS


I recognize with gratitude the support given by those

who contributed to this study. I wish to express my sincere

gratitude to Dr. Phillip Clark, Dr. James Wattenbarger, and

Dr. James Hensel, chairman and members of my doctoral

committee, whose guidance and encouragement made possible the

successful conduct of this study. A special thank you is

given to Dr. Michael Nunnery for helping me in the initial

stage of choosing a topic and for introducing me to meta-

analysis.

I could not have completed this study without the

steadfast support of friends and family. Thanks go to Leila

Cantara for her proofreading and unending supply of red ink.

Appreciation goes to Judie Swan and Dora Proctor for

untangling my attempts at word processing. I thank my father

for his moral and financial support. Warmest regards go to

John, Everett, and Sara Eldridge, who "helped mommy study."

My greatest appreciation goes to my husband, Everett.

Without his continual encouragement and unflagging belief in

my ability this study could never have been accomplished.















TABLE OF CONTENTS

page

ACKNOWLEDGEMENTS ...................................... iii

LIST OF TABLES ........................................ .. vi

ABSTRACT .............................................. vii

CHAPTERS


I INTRODUCTION ...................................... 1

Statement of the Problem ......................... 8
Justification for the Study ........................ 8
Delimitations ..................................... 10
Limitations .............................................12
Definition of Terms ........ ......................... 13
Assumptions ........................................... 14
Procedures ...................................... .. 14
Organization of the Dissertation ................. 16

II BACKGROUND LITERATURE ............................. 17

Introduction ... ................................. .... 17
History of Personnel Selection .................... 17
Objectivity, Reliability, and Validity
in Personnel Selection Research ............... 24
Review of the Research on Selection Methods
for Administrative Personnel ................... 33
Summary and Critique ............................. 95

III PROCEDURES ............................................ 97

Introduction ....................................... 97
Purpose of the Study ............................. 97
Procedures for the Meta-analysis ................. 108
Conclusion ......................................... 114

IV PRESENTATION OF THE META-ANALYSIS OF THE DATA...... 116

Introduction ...................................... 116
A General View of the Research ................... 118
Administrative Personnel Selection Methods ........ 122









Selection Methods Significantly Related to
Administrative Achievement: A Meta-analysis .... 132
Relationships Between Selection Methods
and Separate Measures of Administrative
Achievement .................................... 137
Prediction to Successively Higher Levels
of Administration .............................. 139
Validity of the Assessment Center Approach ....... 141
Correlation of Types of Interactions Within
Selection Methods and Measures of
Administrative Achievement ..................... 144
Analysis of the Research Methods Used in
Administrative Personnel Selection Research .... 144
Conclusions of the Meta-analysis of the
Content and Methodologies of Personnel
Selection Research ............................. 149

V GUIDELINES FOR SELECTION OF ADMINISTRATIVE
PERSONNEL IN PUBLIC COMMUNITY COLLEGES ........... 155

Introduction ..................................... 155
The Need for Valid Personnel Selection Methods .... 155
Guidelines for Selecting Community College
Administrators ................................. 160
Recommendations for Research ..................... 166
Summary and Conclusion ........................... 168

APPENDIX ............................................. 170

REFERENCES ........................................... 210

BIOGRAPHICAL SKETCH ................................... 222















LIST OF TABLES


page
Table

1. Frequency of Independent Variable
Classifications of Correlations by
Reporting Year ................................ 119

2. Mean Correlations, Medians, Standard
Deviations, and Ranges of Uncorrected Data .... 122

3. Rankings of Validities of Selection Methods
With and Without Weighting for Sample Size .... 128

4. Selection Method Frequency of Use Compared
with Mean Effect Size ......................... 131

5. Selection Method Frequency of Use Compared
with Mean Effect Size Derived Through
Meta-analysis ................................. 133

6. Mean Correlations, Sample Size Weighted
Correlations, and Corrected Correlations of
Selection Methods ............................. 134

7. Selection Method Correlation to Outcome
Criteria ....................................... 139

8. Correlations of Selection Methods Measured Within
and Without Assessment Center Processes ....... 142

9. Effects of Range Restriction on Outcome
Measures of Administrative Achievement ........ 147















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy


THE SELECTION OF ADMINISTRATORS IN
PUBLIC COMMUNITY COLLEGES:
GUIDELINES FROM THE RESEARCH

By

Betty-June Hauenstein Eldridge

December 1988

Chairman: Phillip A. Clark
Major Department: Educational Leadership

The purpose of this study was to develop a set of

guidelines for improving the selection of administrators of

public community colleges. Fifty-two research studies were

found which met the parameters of this study. Only those

studies which concerned selection of administrative personnel,

and were published in the United States from 1962 through

1985, inclusive, were used.

Predictor variables consisted of nine personnel

selection methods: (a) aptitude and intelligence measures,

(b) personal interviews, (c) job-related skills indicators,

(d) psychological attribute indicators, (e) value systems

assessments, (f) biographical information, (g) peer ratings,

(h) self-appraisals, and (i) assessment center processes.


vii









Outcome criteria consisted of several specific measures of on-

the-job achievement in administrative positions.

The statistical procedure of meta-analysis was applied

to the 128 validity coefficients found in these studies. The

use of this procedure allowed an objective look into cross

sections of the research data. For each cross section of the

data, mean correlations, weighted by sample size, were

computed. Sampling error, the difference between the

population parameter and the sample statistic, was corrected

for in each of the resulting mean correlations. The

reliabilities of the independent and dependent variables were

computed according to reliability data reported in the

research. These data were used to adjust the mean

correlations for error of measurement. Thus, new

correlations, closer to the true score correlations, were

derived.

Major findings were that (a) all personnel selection

methods considered, with the exception of self-appraisals,

were valid predictors of administrative achievement, (b) the

methods which showed the highest validity, overall assessment

ratings and job-related skills indicators, were the least used

in personnel selection, and (c) those methods used in the

selection of community college administrators, biographical

information, personal interviews, and peer ratings, were

moderately valid. Ten guidelines for administrative

personnel selection in public community colleges were derived.


viii









Legal constraints placed upon personnel selection in public

organizations were considered. These guidelines included (a)

establishing assessment centers for selection and training,

(b) increasing the job-relatedness of selection procedures,

and (c) using statistical methods to simulate overall

assessment ratings.















CHAPTER I
INTRODUCTION

The purpose of this dissertation was to develop a set of

guidelines for improving the selection of administrators of

public community colleges. The research method used was a

meta-analysis of administrative personnel selection research

done in the United States in the past 24 years.

Administrators in our public community colleges must possess

both extensive knowledge and sound judgment to meet the

complex challenges of the years ahead. The extent to which

a community or junior college meets its challenges largely

depends upon the qualities of its future leaders. Thus, these

integral people must be chosen carefully if they are to be

properly matched to the demands of their positions. Those

concerned with the placement of administrators in higher

education can no longer afford the luxury of using arbitrary

selection methods. This is true in times of surplus, and even

more true in times of shortage, budget cuts, and retrenchment

(Mandell, 1964). Rapid change itself has mandated a

continuous reevaluation of personnel selection procedures.

Administrators, well-qualified in the past, may not be

prepared to face challenges of the job in the late 1980s or

1990s. According to Sharp (1984), community colleges grew so









2

rapidly in the years between 1960 and 1980 that education

writers spoke of the "community college movement" (p. 12).

During this time enrollments swelled from 400,000 to

4,000,000. These institutions called forth new and different

types of leaders. As Lynch, former member of the Wall Street

Journal news staff (in Sharp, 1984), stated, "gone are the

days when college presidents were tweedy, pipe-smoking types

who ran things at a leisurely pace as money rolled in and new

buildings were rising all over their campuses" (1983, p. 61).

These changes have impacted upon the levels of administrators

subordinate to the college president.

Initially, many community college administrators were

recruited from the ranks of secondary school administrators

(Sharp, 1984). A proven executive ability, rather than a

distinguished teaching career, became an increasingly

important qualification for appointment. Those candidates

possessing management training and experience or advanced

degrees in higher education became available in increasing

numbers. By the mid-1970s, the selection process, responding

to equal opportunity laws and affirmative action commitments,

acquired a public character and produced results quite

different from those of previous generations. Slowly, the

profile of the college administrator changed as women and

minorities took office. Sunshine laws further complicated

selection in several states and appeared to some observers to











be counterproductive in procuring effective leaders (Sharp,

1984).

Achievement in these administrative positions can be

influenced by complex factors beyond a candidate's basic

physical and mental qualifications for the job.

Administrators are called upon to carry out ministerial

actions which require skills that can be measured empirically.

But, they also must make decisions that involve judgment.

Successful administrative performance results from an

interaction of at least ability, personality, motivation, and

situational factors. Consequently, it would seem that these

factors should be included in the selection methods used to

determine managerial talent (Rawls & Rawls, 1974).

Researchers have tested the validity of personnel

selection methods since the turn of the century. Munsterberg,

a Harvard researcher in the field of applied psychology,

developed rudimentary tests for the selection of personnel in

various manual occupations. His research laid the basis for

the military and industrial personnel testing done during

World War I (Ghiselli, 1973). A post-war surge in personnel

selection studies focused upon the identification of the

essential characteristics people needed for success in

specific occupations requiring predominantly physical skills.

After World War II, increasing amounts of personnel

selection research were undertaken. This effort was put forth









4

mainly by private corporations. During this period, selection

methods were explored that would help employers hire people

for positions that required administrative skills. The

predominant researchers of this period, including Ghiselli

(1955), Halpin (1954), and Hemphill (1960), found that it was

insufficient to attempt to match a person's aptitudes and

personality traits to a generic administrative job

description. To them, defining the job specifically was an

important part of finding the right person for the job. Many

researchers agreed that one single test or examination method

probably did not yield sufficient information about a

candidate to ensure a proper person-job fit, especially for

an administrative position.

Determination of the predictive validity of a test or

selection method is fairly easy when the outcome criterion,

job performance, can be quantified. The presence or absence

of physical and some mental skills is easily seen. But the

ability of a person to lead others, to perform well under

stress, or even the quality of a person's judgment, often

eludes measurement.

Managerial performance came to be viewed as a product of

many interacting variables (Bray & Moses, 1972). Thus, a

movement away from the old model of single criterion measures

toward a systems view of selection was undertaken. As a

result, comprehensive systematic models which took into

account individual, job, and organization variables were









5

adopted. In so doing, the selection process encompassed the

personal characteristics, training needs, and organizational

climate factors that fostered managerial achievement.

Adoption of this more complicated view involving complex

interactions between ability, motivation, and opportunity

variables led to a number of changes that affected managerial

selection (Dunnette, 1963).

One of these changes was the development of the assessment

center approach. During the mid-1950s, American Telephone and

Telegraph (AT&T) began the Management Progress Study. For

years this longitudinal study remained the most valid and

scholarly research done in administrative personnel selection

(Crooks, 1973). The assessment center concept was developed

from this study and it gained rapid acceptance by the business

community. In the original assessment center approach, job

candidates were subjected to multiple tests of personality,

ability, and motivation. During each candidate's three-day

evaluation he was subjected to the traditional tests for

knowledge and experience, and also to newer methods such as

job simulations, role-playing, and leaderless group

discussions. Trained observers recorded their judgments of

each candidate's performance during the entire assessment

process. Hiring personnel hoped to select the most suitable

candidate for each position by studying the results of the

tests and the observer's judgments. In the final evaluation

of each candidate all observer's ratings were considered









6

collectively. A low score on one part of the examination

could be offset or disregarded due to adequate performance on

other parts. This approach replaced the previously used

"successive hurdles" method in which a failing score on any

one part of an selection procedure would disqualify a

candidate from further consideration (Shoop, 1974).

By the 1960s, many companies and public administration

organizations had implemented the assessment center approach.

The results of the AT&T study were so convincing that many

copied all or part of the original process. There was little

additional research done to test either the findings of the

original study or its validity in a new situation (Crooks,

1973). During this time, the selection of administrators in

public junior and community colleges has continued to follow

the traditional methods of reviewing a candidate's education,

experience, references, and conducting a personal interview.

The exercises completed in hiring college and university

administrators are frequently anything but a process. Too

often they are a set of actions which leave selection

committees dissatisfied with what they have done and

candidates dissatisfied with what they have undergone. No

universal process exists for improving what often is an inept

or discouraging attempt to fill an administrative vacancy with

the best available person.

The problem is magnified by the increasing number of

searches underway in institutions of higher education. In









7

many colleges and universities more searches are underway for

administrators than for faculty. In addition, affirmative

action and equal opportunity require that administrative

searches be more public, more extensive, and more expensive

than ever before. All this expenditure of time and resources

demands that more attention be paid to the selection process

(Kelly & Nelson, 1978).

Much has been written about the problems of selecting

administrators in higher education, however, little research

has been done on the topic. A change in the procedures now

followed may be warranted. The results of many years of

research in personnel selection methods in other fields have

been usefully applied in this case. The conclusions of this

research helped to determine guidelines for improving the

selection of administrators for our public junior and

community colleges.

Research in administrative personnel selection methods has

been done in corporations, the military services, public

administrative agencies, and in education. The present study

was needed in order to synthesize this research and apply it

to the selection of community college administrators. The

conclusions of the research, when applied to this problem,

have indicated those selection methods which are useful in

predicting administrative achievement.











Statement of the Problem

The problem addressed in this study was to develop

guidelines for improving the selection of administrators of

public junior and community colleges.

The following questions were addressed:

1. What methods for the selection of administrative

personnel are commonly used by corporations, public

administration agencies, the military services, and

institutions of higher education?

2. What methods for the selection of administrative

personnel, derived from a meta-analysis of selection research,

are significantly related to achievement in administrative

positions?

3. What information, derived from a meta-analysis of the

methodologies of selection research, is useful in interpreting

the findings of this study?

4. What guidelines can be derived from this research for

improving personnel selection methods for administrators of

public community colleges?



Justification for the Study

Radical changes have occurred in the public workplace

in the last two decades. These changes have required

management to become more flexible in its thinking and to

reorient itself to new conditions. In higher education the

advancements in tenure, unionization, equal opportunity









9

legislation, and increased protection of individual rights

present community college administrators with more complex

situations than ever before. However, selection procedures

for these positions have rarely been reviewed to determine

their continued validity.

McIntyre (1966) stated that the process used in selecting

higher education administrators is less than optimal.

Of all the rituals encumbering the selection process,
interviewing is undoubtedly the hoariest--and the
sorriest. Nothing in the research on selection
methodology is so completely established and
repeatedly verified as is the unreliability of short
interviews as they are usually conducted.
Unfortunately, the record of letters of recommendation
is as dismal as that of interviewing. Although the
subject has not been researched to any great extent,
all available evidence indicates that the reading of
letters of recommendation is approximately as
enlightening as the reading of tea leaves. Rating
scales vary considerably in usefulness, but the usual
scale is little if any better than the usual letter
recommendation. The traits to be rated are often of
limited relevance, the points on the scale are seldom
clearly defined, and leniency is so rampant that only
the upper end of the scale is ordinarily used. (pp.
7-8)

In seeking to improve selection methods, every phase of

the process should be investigated and reevaluated. Much can

be accomplished when those in control of an organization take

a broader view of the problem of selection of administrators

and allow new knowledge to influence their thinking.

Occasionally innovations in procedures occur serendipitously,

but more often they are the result of long hard thinking,

experimentation, and evaluation. Purposeful changes in









10

organizational methods usually grow out of an intensive

investigation of data. Sands (1963) stated that

further study is especially needed in the area of
predicting successful managerial functioning. The
forecast for success can be improved through
correlation studies using data already on hand. All
that is necessary is the willingness to search, the
ability to analyze, and the imagination to interpret
the relationships discovered. (p. 188)

The conclusions of research must be presented in a usable

form. Two decades of research in selection of administrative

personnel is not useful to those selecting administrators of

community colleges unless it is analyzed and aimed toward

solving that specific problem. Silvern (1971) examined 15

research studies by education systems from January 1965

through June 1967. He found that, although the published

material on education systems had been proliferate,

applications of this material were found in only 1% of all of

the literature. The present study attempts to fill a void by

presenting the results of 24 years of administrative selection

research in a usable format.



Delimitations

The delimitations set restrict the research used in this

study to a specific locale, time period, phase of the

selection process, and type of position considered.

1. The research reviewed in this study pertained to the

following areas: (a) corporations, (b) public administration,

(c) military services, and (d) education.








11

2. The study dealt with personnel selection methods rather

than with individual selection tests. When only one

particular test was administered, such as a Strong Vocational

Interest Blank, this test was recorded by the type of

selection method it represented.

3. The methods of personnel selection, the independent

variable, used in this study included (a) aptitude and

intelligence measures, (b) personal interviews, (c) job-

related skills indicators, (d) psychological attribute

indicators, (e) value systems assessments, (f) biographical

information, (g) peer ratings, (h) self-appraisals, and (i)

assessment center processes.

4. The outcome measures of administrative achievement, the

dependent variable, used in this study included (a)

administrative level achieved, (b) salary level attained, (c)

supervisor ratings, (d) number of years serving in an

administrative position within the same organization, (e)

achievement of tenure, (f) objective performance data, and (g)

promotion rate.

5. Research studies which defined administrative

achievement as measured by (a) admissions personnel ratings,

(b) success in being hired, or (c) performance success in

subsequent training programs, were not used in this study.

These criteria were judged to be too far removed from any

measurement of actual performance on the job.









12

6. The research studies analyzed were published in the

United States from January 1, 1962 through December 31, 1985.



Limitations

The following confinements were observed in the

investigation:

1. Performance ratings and promotions are based in part

upon subjective ratings given by an employee's supervisor.

As such, they may not be totally objective measures of job

performance.

2. A lack of independence between the predictor and the

criteria variables may occur when personnel selection

examination results are made available to an employee's

supervisor. The possible effects of this criterion

contamination, where it was identified, were addressed in this

study.

3. Restriction of range occurred when the original

population or sample limited itself, or was limited by,

factors which related to the selection and subsequent

employment process. Thus, if 50% of those interviewed were

hired, and the dependent variable, administrative achievement,

was determined only for those hired, the effective sample

variance for the study was cut in half. This affects the

subsequent correlation between that selection method and any

measure of administrative achievement. For this reason, the









13

possible effects of range restriction, where they were

identified, were addressed in this study.



Definition of Terms

Administrator. An administrator is any person who manages

or directs the affairs of an institution or any major part

thereof. As pertains to higher education, anyone above the

level of professor, such as department chairman or

professional central administrative personnel, is considered

an administrator.

Community college. A community college is a two-year

college, offering academic, general education, vocational

training, terminal, and transfer programs. It can also be

known as a junior college.

Corporation. A corporation is any company or related

group of companies which produces goods and/or services for

profit. Usually this type of organization has a quantifiable

measure of success by which to judge the effectiveness of the

administrators and employees, such as number of units produced

or amount of profit made. Alternative terms for "corporation"

in this study include those such as the corporate world,

business and industry, or companies.

Higher education. Higher education consists of public and

private community colleges, four-year colleges, and

universities in the United States.









14

Meta-analysis. The statistical analysis of a large

collection of analysis results from individual studies for the

purpose of integrating the findings.

Military services. Military services are national-level

military organizations in the United States.

Public administration. Any agency which provides public

services at the federal, state, county, city, district, or

regional level is considered a public administrative agency.

Supervisor. A supervisor is a person who oversees,

directs, or manages work or workers.



Assumptions

The following assumptions have been made in this study:

1. It has been assumed that a meta-analysis of the

research on administrative personnel selection was an

appropriate methodology for determining those methods which

may, when properly implemented, improve the selection of

administrators of community colleges.

2. It has been assumed that the research included in this

study was guarded against restriction of range and criterion

contamination wherever possible.



Procedures

The purpose of this study was to develop a set of

guidelines for improving the selection of administrators of

public community colleges. The research method used was a









15

meta-analysis of administrative personnel selection research

done in the United States in the past 24 years. Procedures

used in this study follow those described by Glass (1977) and

in Hunter, Schmidt, and Jackson (1982) for the meta-analysis

of a sample of independent correlational studies. The steps

involved included the following:

1. Identification of the research studies.

2. Classification of each study dependent variable,

administrative achievement, according to the Delimitations

section of this chapter.

3. Classification of each study independent variable,

selection method, according to the Delimitations section of

this chapter.

4. Calculation of frequencies of selection methods used

to answer the questions: (a) What were the most commonly used

selection methods across all types of organizations, and (b)

what were the most commonly used selection methods in each

type of organization.

5. Calculations of the means and standard deviations from

the means in selected cross-sections of the data were used to

determine which selection methods were significantly related

to achievement in administrative positions.

6. Determination of what information, derived from the

meta-analysis of the methodologies of the research studies

used, can be useful in interpreting the findings of the above

analysis of selection methods.









16

7. Analyzation of the findings of the above questions to

determine how the frequency of use of each selection method

related to its ability to predict administrative achievement.

8. Derivation of guidelines useful in improving personnel

selection methods for administrators of public community

colleges.

9. Presention of the data in textual form supported by

frequency tables and figures, and supported by tables and

figures reporting means and standard deviations of the

correlational data.



Organization of the Dissertation

A review of the literature pertaining to administrative

personnel selection methods is provided in Chapter Two.

Chapter Three presents a detailed discussion of the meta-

analytic research procedures used in this study, such as

methodology, sources of data, data analysis techniques, and

presentation of the data. A presentation of the findings of

the meta-analysis of the research studies used is contained

in Chapter Four, thus answering the questions posed in the

Statement of the Problem. Chapter Five contains the

guidelines derived from the meta-analysis of the research

studies and the presentation of these guidelines for

application to the selection of community college

administrators.















CHAPTER II
BACKGROUND LITERATURE


Introduction

The responsibilities inherent in administrative positions

have undergone a complex evolution since the turn of the

century. In many organizations, however, the selection

process for administrative personnel has not seen appreciable

change. In this review, the focus is on the research

literature pertaining to the usefulness of various selection

procedures in the prediction of administrative success. The

review is organized into the following sections: the history

of personnel selection; a discussion of objectivity,

reliability, and validity in personnel selection research; a

review of the personnel selection methods used as independent

variables in personnel selection; introduction of the research

studies reviewed; and a summary and critique.

History of Personnel Selection

Traditionally, Munsterberg' s 1911 experiment with motormen

has been viewed as the beginning of research in the use of

examinations for personnel selection (Ghiselli, 1973).

Nevertheless, anecdotal evidence leads one to suggest that,

even before 1910, other psychologists conducted similar

studies with tests, but these were small in scope and went









18

unpublished. Under the impetus of the scientific management

movement of the early 1900s, some efficiency experts of that

time were using simple exams for evaluating applicants for

jobs. They reported fragmentary evidence of validity in the

attempt to justify their activities (Ghiselli, 1973). The

earliest review of an industrial application was in 1915 when

Scott (in Arvey & Campion, 1982, p. 283) reported low

reliability for evaluations given by six personnel managers

who had interviewed the same 36 sales applicants.

During World War I, the large-scale testing both of

soldiers and industrial workers provided stimulation,

methodology, and respectability to the examination of the

utility of examinations in the assessment of occupational

aptitude. In 1923, Freyd (in Guion, 1987) published a 10-step

outline for personnel selection. This outline was so complete

that it differs little from the procedures used in personnel

selection research today. This all led to a post-war surge

of systematic research in personnel testing.

Selection Processes in Business and Industry

Prior to World War I, most businesses were owner-managed

by the individuals who had founded them. But the growth of

the economy since the 1920s brought changes which have

replaced these colorful characters with impersonal

corporations. The function of the administrator had become

such that it required many skills and much knowledge, so that

it often took a number of administrative personnel to assist









19

in decision making. By the 1960s, corporate management

required a knowledge of a variety of fields and operations

rarely needed in business a few years earlier. Corporations

* were growing, combining, and expanding so fast that they

needed to hire competent executives at a pace which could not

be fulfilled by promotion from within the ranks (Sands, 1963,

p. 3).

One particular study was more comprehensive than much of

the personnel selection research of the 1950s and 1960s. This

was the American Telephone and Telegraph (AT&T) (Bray & Grant,

1966) study from which came the assessment center method for

the selection of administrative personnel. Called the

Management Progress Study, initiated in 1956, it is the first

known corporate research and application of the assessment

center method. The 422 men brought into the study were

followed up annually and reassessed at 7-year intervals in a

effort to keep track of their professional development. The

assessment center results were not revealed to the higher AT&T

management personnel, so that the progress of the men would

not be affected by the assessment findings. The results of

this study have been well documented. Between 1961 and 1967,

only this AT&T study and two studies done by the armed forces

on assessment centers were reported in the literature

(Crooks, 1973, p. 1).











Selection Processes in Public Administration

When public administration systems were initiated, it was

assumed that only people of the highest caliber would apply

for service. Once reality dashed this assumption, public

administrative personnel selection systems were built upon the

cornerstone of the competitive examination, a method of

deselecting inferior job candidates (Stahl, 1976, p. 129).

From the beginning, this system has placed special emphasis

upon formal selection procedures such as testing for specific

knowledge and skills related to the particular job the person

was applying for. The starting points for the selection

process are basically two: a determination of the objective

for selection, which may be a given position, occupation,

program, or service-wide career, and a setting of basic

standards for selection, the skills and knowledge that are

necessary to meet the preceding objective. During the early-

to mid-1960s, some public administration organizations

realized a need to ensure at least an adequate intake of high

caliber people at all levels so that there would be no

shortage of talent when movement upward or outward took place.

Thus, some services began to give more attention to the

selection of persons who possessed a capability of growth and

development. To this end, public administrative organizations

began to implement the assessment center approach.

However, most public administrative organizations select

a person to fill a particular job vacancy, and thus their









21

testing is for a specific competency in that area, not an

overall assessment of the applicant's potential. Most of

these agencies expect personnel to have learned their skills

previous to application and do not intend to take in an

inexperienced person of promise and then train him or her for

an administrative position.

Most of the background studies cited as a basis for

personnel selection in public administration were actually

conducted by private corporations. Many of the selection

practices have been borrowed and adapted from business

research. AT&T's original industrial development of the

assessment center method was so well grounded in research, it

was often casually implied that therefore any assessment

center would pick the "right" person (Ross, 1979, p. 41).

Selection Processes in the Military Services

Although many fields acquire administrators already

trained for the positions they are hired to fill, the military

services often must select people of officer quality and then

pay the additional cost of training them. Thus, the basic

objective in the selection process is to identify measures

which would result in officers entering the force with a high

probability of success. The services must select personnel

not only to fill a particular position as it becomes

available, but also to identify people who have the ability

to be trained repeatedly, sometimes every three years, for a









22

variety of different positions (Akman & Nordhauser, 1974, p.

3).

Often, the services must select officer-quality personnel

to fill positions which have no equivalent in the nonmilitary

world. In these cases, either very global criteria must be

set to determine if the candidate has the intelligence or

aptitude to learn the job-required skills and then be

successful at it, or very specific abilities or natural

characteristics must be measured to ensure the candidate will

be well suited for an important aspect of the position.

Because a large number of people must be tested and screened

for placement, the military services have developed many

paper-and-pencil tests to measure both general and specific

aptitudes. They have relied heavily upon past achievement,

such as college degrees and the results of officer candidacy

tests. The importance of test scores and military class

standings is shown by the fact that often one's scores and

future attainment of rank are positively correlated.

Selection methods used in the military services and in

public administration are often classified as "successive

hurdles" techniques. A lower than acceptable score on any one

particular screening test, or part of a test, will either

disqualify the candidate from that service or from particular

jobs within the service (Shoop, 1974, p. 341).










Selection Processes in Higher Education

In higher education, search committees are often used for

recruiting administrative personnel. These committees provide

for maximum participation in the selection process by a

variety of constituencies within the institution. The

membership of the search committee depends upon the vacancy

to be filled. The committee may include faculty and staff

members, administrators, and, in some cases, students. A

member of the central personnel staff is often an ex-officio

committee member in order to orient the committee to the

proper and legal selection procedures (Sprunger & Bergquist,

1978, p. 116). Usually, the search committee recruits,

screens candidates, checks references, participates in

preliminary interviewing, and recommends a fixed number of

candidates to a designated administrator, who makes the final

selection (Fortunato & Waddell, 1981, p. 107).

The initial screening of candidates has usually been done

by a review of the vitae and job applications. Supplemental

information has usually been obtained by talking to references

and others who knew the individuals and their work. The on-

campus interview has been one of the most important steps in

the selection process. The manner in which this interview is

conducted has usually been critical to the success of the

recruitment-selection process (Sprunger & Berquist, 1978, p.

119). Webster (cited in Grove, 1981, p. 56) emphasized that

the interviewer must understand what behavior is required of









24

the person finally selected to fill the position. He stressed

such practices as reviewing the application form with the

applicant, giving the applicant an opportunity to talk about

himself or herself in a way that helps that individual to

present relevant evidence, making the interview more

systematic and probing, examining test scores and other

information, and weighting the various types of information

in order to reach an intelligent decision. Webster

recommended taking time following each interview to clarify

impressions and to formulate a judgment. He suggested making

use of multiple independent evaluations in which two or more

interviewers record decisions independently and subsequently

reconcile differences.

Objectivity. Reliability, and Validity in Personnel
Selection Research

These three primary considerations enter into the

effectiveness and propriety of examination processes for

employment selection: objectivity, reliability, and validity.

Objectivity is even more important in the 1980s than it was

previously due to the legal demand for testing to have a job-

specific relation. Reliability is important but is often not

reported in personnel selection research. Validity is the

major concern of this dissertation, and the research studies

reviewed here pertain to the validity of each selection method

in predicting administrative achievement.










Objectivity

According to Stahl (1976), one of the prime reasons for

professionalizing all steps in the selection process is to

ensure thoroughgoing objectivity (p. 131). Only those methods

which disregard extraneous factors such as race, religion,

politics, sex, residence, and age can be considered thoroughly

objective. An objective selection method should identify

those characteristics of mind and skill necessary, and only

those necessary, to the given purpose, whether the purpose is

to fill a particular position or to begin a career.

Objectivity is not only desirable but is mandated in

hiring for positions in the public sector. Throughout almost

all of American history, legislation has been enacted to

facilitate equal treatment in employment. An abundance of

case law has built up around civil rights legislation, and

rulings have occurred which have broadened the impact of such

legislation. This broadening has had the effect of placing

more and more constraints on what employers are and are not

allowed to do in terms of employee selection (Burrington,

1982, p. 55).

Two landmark cases have set the serious tone with which

violations of civil rights in hiring will be met. In the U.S.

Supreme Court case of Griggqqs vs. Duke Power (401 U.S. 424,

1971), an employer rejected black job applicants on the basis

of lack of completion of high school or on the results of a

general intelligence test. Evidence showed that employees who









26

had not completed high school or taken the tests had continued

to perform satisfactorily and made progress in departments for

which the high school and test criteria were not used

(Burrington, 1982). It was ruled that employment standards

and tests which are not significantly related to job

performance violate the Civil Rights Act of 1964, Title VII

(Alexander, 1980, p. 515).

The second case, Edward L. Kirkland et al.. Plaintiffs v.

New York State Department of Correctional Services et al..

Defendants (73 LIV. 1548, 1974), was brought under the Fifth

and Fourteenth Amendments to the U.S. Constitution and under

the Civil Rights Acts of 1866 and 1871. The case concerned

a Correction Sergeant (male) written examination which had the

impact of allowing only 1.9% of blacks, and no Hispanics, to

be eligible for promotion. No recourse was made to the Civil

Rights Act of 1964, so the Equal Employment Opportunity

Commission (EEOC) Guidelines, required under that act, were

not binding. However, the U.S. District Court, Southern

District, New York, considered these Guidelines in evaluating

the job relatedness of the examination. A number of

implications can be deduced from this case for personnel

selection in general and the examination process in particular

(Wisner, 1975, pp. 266-267).

1. In rendering a decision, the court is likely to

follow the EEOC Guidelines even though the case may not have

been brought under the Guidelines.









27

2. A job analysis should be performed prior to the

development of each examination.

3. Those critical elements, or more important

characteristics of the job, which distinguish between those

likely to be successful and those less likely to be

successful, must be included in the examination.

4. Subject matter experts, to be considered qualified,

must be of an appropriate rank and sufficiently experienced

in the current skills, knowledge, and abilities required in

the job for which the test is developed.

5. Discriminatory test results cannot be refuted by

demonstrating that the results occurred because of certain

environmental conditions or cultural characteristics related

to the protected group population unless it can be shown that

the examination is job-related.

6. The fact that specific parts of the test are not

found to be discriminatory does not serve to support the use

of an examination whose overall results are found to be

discriminatory.

7. All subtests included in the examination must be

weighted in accord with their importance to the job.

8. Data should be obtained concerning the examination

results immediately after administering the exam to determine

the examination's impact on protected groups.

9. Every effort should be made to set passing scores at

a point considered to separate the candidates most likely to









28

be successful from those considered to have the least chance

of success.

10. Be prepared to answer the question "why was this

method of measurement considered the best means to employ?"

11. In court, do not rely too heavily on your

professional witness to attest to the job relatedness of your

examination.

Thus, the public employer has but one choice--to validate

the employee selection procedures. Fortunately, the choice

is the best one to follow from the standpoint of good

personnel management.

Reliability

The consistency with which a method measures what it is

expected to measure is called reliability (Ary, Jacobs, &

Razavieh, 1979, p. 206). In order to achieve consistency, a

selection test or method must leave little room for chance in

the subject's final score or rating. If a test is reliable,

a person taking it at two different times should make

substantially the same score or be ranked in approximately the

same position each time. Test reliability depends also on the

nature of the variable being measured. For example, the trait

of academic achievement can be measured more consistently than

that of personality. Although all measurements of human

qualities contain some error, no test or method of personnel

selection is of value unless it has a high degree of

reliability (Stahl, 1976, p. 133).









29
Reliability can be affected by varied administration of

a selection procedure and by the scoring or judging of an

applicant's performance. Inexperienced persons or committees

may deviate from standardized or prescribed procedures in

testing an applicant. Vague observational techniques or

scoring of an applicant's behavior or responses leads to an

unreliability which has nothing to do with the competency of

the candidate. The difficulty of any selection method or

examination affects its reliability. When a method is

difficult, the subjects guess on most of the questions and a

low reliability results. Conversely, if the type of

examination is easy, all subjects have correct responses on

most of the items and only a few more difficult items

discriminate among subjects. This again results in a low

reliability. Thus, any selection committee must keep in mind

the fact that they must adjust the substance of questions

asked in order to discriminate subtle differences between

similar candidates on traits that are difficult to measure.

The achievement of a high degree of reliability in

selection procedures for administrative positions in a

community college may be difficult. In addition to the

problems inherent in assessing traits which elude measurement

by persons who are often inexperienced in the task,

reliability can be affected by the lack of heterogeneity of

the group of applicants. The greater the homogeneity of the

group of subjects being considered, the lower the reliability








30

of any selection method in differentiating among those

subjects. Candidates for these administrative positions are

likely to be quite similar in the traits being measured.

Therefore, it is more difficult to rank them than it would be

to rank the same number of subjects chosen randomly from the

entire population. This restriction in range is a problem in

most personnel selection research as very seldom is the sample

drawn from the population at large.

Validity

Validity is defined as the extent to which an instrument

or method measures what it is intended to measure. Unlike the

physical sciences, where there is a direct means of measuring

the outcome variable, in personnel selection indirect means

must be used to measure complex attributes.

The type of validity addressed in this study is criterion-

related validity. Criterion-related validity refers to the

relationship between the scores on a measuring instrument and

an independent external variable (criterion) believed to

measure directly the behavior or characteristic in question

(Ary, Jacobs, & Razavieh, 1979). In this type of validity the

emphasis is on the criterion rather than on the instrument

itself. One is primarily interested in what the instrument

can predict rather than in the test content.

There are several characteristics that a criterion measure

should possess. One must judge whether the criterion chosen

as the dependent variable really represents successful









31

performance of the behavior in question. If the criterion

does not reflect the attribute under study, it would be

meaningless to use it as a basis for validating another

instrument. In the case of this study, the outcome criteria

of supervisor's ratings, longevity in a position, and

salary/position or rank attained were deemed to be adequate

indicators of success in an administrative position.

A second characteristic is that a criterion must be

reliable. The criterion must be a consistent measure of the

attribute over time or from situation to situation. The

criteria chosen for this study, rank, position, and salary

achieved, have generally been considered to be related to

success. Length of time in an administrative position within

one organization may have been a mark of high achievement in

the past more than it is today. Now many administrators look

upon great longevity as an indicator of stagnation rather than

of achievement. So this indicator may not be as relevant to

later studies as to earlier ones.

Finally, a valid criterion should be free from bias. The

scoring should not be influenced by any factors other than

actual performance. The main problem in relation to personnel

selection research is a form of bias called criterion

contamination. This occurs when an individual's score on the

criterion, such as supervisor rating, is influenced by the

scorer's knowledge of the subject's predictor score.

Contamination of the criterion can be prevented by not









32

permitting the person who grades or rates the criterion to see

the predictor scores.

Although perfect criterion-related predictive validity is

practically impossible to achieve, a reasonable amount of

validity is possible. Selection upon the basis of tests or

methods which have no known validity may be little different

from selection upon the basis of a turn of a card. Yet some

personnel agencies have chosen to use methods which have not

been subjected to validity study.

The process of determining the validity of a selection

method involves statistical correlation between the various

examination results and some criteria of performance on the

job. Where the worker is engaged in production activities in

which it is easy to measure output, the question of the

criterion presents no problem. But the simplicity of the

problem disappears when one tries to find adequate criteria

of performance for an administrator. In these cases, the

researcher must rely upon appraisals by those closely

acquainted with the work of the particular administrator

(Stahl, 1976, p. 132).

Kirchner and Reisberg (1962) pointed out the problem of

determining an adequate criteria of managerial performance (p.

301). Supervisors' ratings of the performance of

administrative personnel were found to be subjective and

widely varied. Successful supervisors preferred subordinate

administrators who showed initiative toward organizational









33

goals, whereas supervisors considered to be less successful

rated their subordinate administrators more highly who

followed orders and got along with others.

There are two main procedures by which the validity of an

examination or method for selection is best determined. Each

is supplementary to the other. In the first instance,

concurrent validity, the exam may be given to administrators

of known ability already on the job. If those administrators

who have been predetermined to be most successful score

highest in the evaluation, while the least successful ones

score the lowest, the evaluation method is said to have

evidence of validity. The second procedure consists of long-

term follow-up studies of the performance of those employees

who have been selected for administrative positions (Stahl,

1976, p. 132).

Review of the Research on Selection Methods
for Administrative Personnel

Methods for personnel evaluation have been described in

as many different ways as there have been researchers to

describe them. According to Northcott (1960, p. 289), the

procedures in standard use have fallen into three groups:

questionnaires, tests, and interviews. Stahl (1976, p. 136)

stated that the forms which examination can take may be

classified into five categories: (a) a systematic evaluation

of education and experience, (b) oral tests, (c) standardized

qualification inquiries, (d) written tests, and (e) tests of








34

personality traits. According to the "Uniform Guidelines on

Employee Selection Procedures," issued by the Equal Employment

Opportunity Commission, the United States Department of

Justice, the United States Civil Service Commission, and the

United States Department of Labor in 1978, selection

procedures are defined as including "the full range of

assessment techniques from traditional paper and pencil tests,

performance tests, training programs, or probationary periods

and physical, educational, and work experience requirements

through informal or casual interviews and unscored application

forms" (Quaintance, 1980, p. 126).

The selection methods reviewed in this study have been

delineated into the following types: (a) aptitude and

intelligence measures, (b) personal interviews, (c) job-

related skills indicators, (d) psychological attribute

indicators, (e) value systems assessments, (f) biographical

information, (g) peer ratings, (h) self appraisals, and (i)

assessment center processes. These methods are the

independent, or predictor variables of the administrative

personnel selection research studies used in this meta-

analysis. Statistical data and methodological details of each

of the research studies reviewed for this dissertation are

found listed alphabetically by researcher's name(s) in the

Appendix.










Aptitude and Intelligence Measures as a Selection Method

In a large proportion of the research, at least one type

of intelligence or mental aptitude test has been used. Most

of the tests utilized measured verbal abilities, but some

measured critical thinking, or mathematical or mechanical

aptitudes.

In a review of 12 research studies aimed at relating a

measure of aptitude or I.Q. to later measures of

administrative achievement, Korman (1968) found that the

"typical managerial applicant population is already highly

preselected on abilities and is relatively homogeneous on

these variables" (p. 301). Thus, the tests of ability do not

tend to discriminate finely among such a population in which

preselection has already taken place. Korman states that

"verbal ability tests seem to show little usefulness for

predicting managerial performance above the first-line

supervisory level" (p. 297).

Dating back into the 1940s, cognitive ability research

showed mixed results. Williams and Leavitt (1947), using

Marine Corps officers as subjects, showed no significant

relationship with supervisor ratings. However, Knauft (1949)

found a significant relationship with later ratings. In the

1950's Handyside and Duncan (1954) and Meyer (1956), each

using supervisors as subjects, found significant relationships

with ratings, but Comrey and High (1955), Ricciuti (1955), and

Thorndike and Hagen (1959) did not.









36

Research used in this study. A total of 15 research

studies were found which used aptitude and intelligence

measures as a predictor variable and met the other

delimitations of this dissertation. These studies yielded 21

correlation coefficients which ranged in strength from r = .01

to r = .41. Samples ranged in size from 26 to 8,885 persons.

Hicks and Stone (1962), while working for Aerojet-General

Corporation and California State Polytechnic College,

respectively, evaluated the effectiveness of a test battery

in discriminating between successful and unsuccessful

managers. One part of this concurrent study related the

mental ability of managers, supervisors, shop foremen, and

engineering supervisors, to the outcome measure, supervisor

rating. Results were not significant. The researchers

concluded that mental abilities did not show a large

relationship to managerial success probably as the result of

sample attenuation through preselection on education and

career achievement.

Williams and Harrell (1964), while at San Francisco State

College and Stanford University, respectively, did a follow-

up study of Stanford MBA graduates. The purpose was to find

predictors which were significantly correlated with on-the-

job achievement. While the grade point averages for

undergraduate courses and for required graduate courses fell

short of significant correlations with the success criterion,

there was a significant correlation between grades on elective









37

graduate courses and salary. The researchers suggested that

research attention might be directed at some figure which

would concentrate on the elective course area.

Dicken and Black (1965), while at Stanford University,

explored the validity of clinical interpretations of an

objective test battery in two different corporate settings.

Subjects were first-level supervisors in a manufacturing

organization and in an insurance company. Only the highest

of the derived correlations was significant. The researchers

acknowledged the restricted range of their samples and its

effect on predictability, stating that "measures of ability

and interest cannot be expected to make fine discrimination"

(p. 46).

In 1965, Tenopyr and Ruch (in Campbell, Dunnette, Lawler,

& Weick, 1970) studied the relationship between intelligence

test results and the salary level attained by corporate

production managers. The resulting correlation coefficient

in this concurrent study was significant.

Bray and Grant (1966), while with American Telephone and

Telegraph Company, sought to study the assessment center

process. This study has become the landmark study, to date,

in assessment center development. A sample of low-level

management employees of AT&T were tested in this carefully

designed assessment center. This was one of the few studies

in personnel selection research where no criterion

contamination was allowed. A follow-up of the subjects eight








38

years after testing showed a significant correlation between

three measures of mental aptitude and the outcome variable of

salary level achieved.

In favor of situational tests over mental ability tests,

the researchers stated that when mental ability as measured

by a paper-and-pencil test was partialed out of judged

ability, reliable variance still remained. They found that

overall and combined assessment ratings did tend to be higher

than the correlations for any individual technique (Bray &

Grant, 1966).

In 1967, Bentz (in Campbell et al., 1970) summarized the

results of the Sears, Roebuck Company psychological testing

program. In relating aptitude and intelligence measures to

the prediction of executive effectiveness, a significant

correlation was found. Bentz stated that the successful

administrators were superior in intellectual endowment.

Kraut (1969), while working for IBM Corporation, sought

to determine the relationship between high level managers'

promotional success and measures of their intellectual

ability. No significant relationships were found. The

researcher cited problems with the predictor tests and range

restriction of the sample as possible reasons for the results

obtained.

Wollowick and McNamara (1969), while with IBM Corporation,

sought to determine the validity of an assessment center

approach in predicting management potential and to determine









39

the relative value of the components of the program. Within

this study, they found that two measures of aptitude and

intelligence were not significantly related to the outcome

variable. The researchers suggested that "it may be possible

to consider eliminating the paper-and-pencil tests not

contributing to the predictive validity" (p. 352).

Campbell et al. (1970), in a study involving the Standard

Oil Company of New Jersey, attempted to discover how employees

who possess the potential to be successful in management could

be identified early in their careers. Although the

intelligence and aptitude measures used were not significantly

related to the outcome criterion, Laurent (in Campbell et al.,

1970) concluded that successful managers "have shown a total

life pattern of successful endeavors" (p. 169).

Moses (1972) studied the relationship between assessment

and subsequent progress in management for personnel in the

Bell Telephone System. This study was part of the AT&T

assessment center process, but not a part of the original

study (Bray & Grant, 1966). A significant relationship was

found between aptitude and intelligence measures. The study

involved a large sample of nonmanagement personnel.

Moses and Boehm (1975) replicated the study done by Moses

in 1972, but the sample consisted of nonmanagement females.

A significant relationship was found in this study. The

researchers stated that "the assessment process predicts the









40

future performance of women as accurately as it does that of

men" (p. 529).

Grimsley and Jarrett (1975), while with the University of

Southern California and California State University,

respectively, studied the relationship between past

achievement test scores and criterion measures obtained in

administrative positions. The significant results found in

this concurrent validity study led the researchers to conclude

that "the differences in test scores of more or less

successful managers result from fundamental differences in

mental ability and personality rather than the influence of

on-the-job experience" (p. 226).

Huck and Bray (1976) studied a sample of non-management

females derived from the AT&T assessment center process.

Their purpose was to test the validity of an assessment center

process on a population different from the population of the

Management Progress Study (Bray & Grant, 1966). Although the

relationship between aptitude and intelligence measures and

the criterion variable did not prove to be significant, the

researchers found that "judgments made by assessment center

staffs are good predictors of later performance" (p. 27).

Gantz, Erickson, and Stephenson (1977) sought to determine

why some persons in a research and development population

achieve promotion, particularly into supervisory or managerial

positions, and other persons do not. Measures of aptitude and

intelligence were not significantly related to either









41

promotion rate or to supervisor ratings in this concurrent

study.

Hinrichs (1978) evaluated the predictive validity of the

AT&T assessment center process and compared the predictive

accuracy of the assessment center with the naturalistic

management evaluation. No significant findings were obtained

between aptitude and intelligence measures and administrative

level achieved. The researchers found that "prediction based

upon managerial review of the personnel files did as well as

the assessment center after 8 years" (p. 600).

Conclusions. Restriction of range was acknowledged as the

biggest drawback to the predictive ability of aptitude and

intelligence measures. Preselection of the samples by

management tended to reduce predictability (Dicken & Black,

1965). Intelligence, as measured typically by verbal ability

tests, is a fair predictor of first-line supervisory

performance but not of higher-level managerial performance

(Korman, 1968). The conclusions drawn by most researchers in

explaining the general lack of a significant relationship

between aptitude and intelligence measures and measures of

administrative achievement would support the statement that

"measures of ability and interest cannot be expected to make

fine discrimination" (Dicken & Black, 1965, p. 46).

Personal Interviews as a Selection Method

"The use of the interview as a device for appraising

applicants for a job is generally regarded with a good deal









42

of suspicion and distrust by industrial psychologists"

(Ghiselli, 1966, p. 389). A factor that very likely has

produced disaffection with the employment interview is its

nebulous and intangible character. The interview involves a

social interaction between the interviewer and the applicant,

thus it varies substantially in form and content from one

applicant to another for one and the same interviewer.

In attempting to improve the selection interview,
industrial psychologists have sought to develop sets
of rules to follow in order to improve the reliability
and validity of the interview. To some extent these
rules are based upon scattered empirical evidence, but
because research on the interview is so very limited
they are more commonly based upon professional
judgment or sheer common sense. Mayfield's (1964)
attempt to integrate the pertinent research can only
be described as heroic. Yet the conclusions and
generalizations he arrives at are but weakly founded.
(Ghiselli, 1966, p. 390)

Early studies done on the personal interview method by

Binet and Scott (in Arvey & Campion, 1982) showed low

interrater reliabilities. Wagner (1949), in a summary of

interview research, noted low validities in most cases. In

1964, Mayfield reviewed all interview research done since

Wagner (1949) and found a median validity correlation

coefficient of r = .27. He stated that this value was not

"particularly high" (Arvey & Campion, 1982, p. 284).

Research used in this study. A total of 6 research

studies were found that used personal interviews as a

predictor variable and met the other delimitations of this

dissertation. These studies yielded 7 correlation









43

coefficients which ranged in strength from r = .03 to r = .35.

Samples ranged in size from 122 to 8,885 persons.

Ghiselli (1966), while at the University of California,

attempted to evaluate the validity of data gathered during

personal interviews in predicting administrative job

proficiency. The independent variable was the personnel

interview used in a corporation over a 17-year period in

hiring account executives. The outcome criterion was survival

with the company for a 3-year period. The interview was

relatively unstructured and included no questions that were

of a highly personal nature. However, some questions that

would not be legal today in a public employment interview were

used. A significant correlation was found between the two

variables in this highly range-restricted sample. Ghiselli

thus found the interview to have "at least a moderately

substantial validity" (p. 394).

Grant and Bray (1969) studied the contributions made by

the interview to the assessment center process, and also, the

relationships between interview variables and the progress

criterion of salary level attained after 9 years. These

relatively unstructured interviews covered job-related and

personal topics. The interviewers were professional

psychologists. For the purpose of the study, the researchers

divided the interview into 18 different personality variables

and related each variable to the outcome criterion. The

significant relationship found showed that "the interview did









44

produce reliable ratings of managerial qualities which related

significantly with ratings made on the basis of several other

techniques and with advancement" (p. 34).

Carleton (1970), while with the Standard Oil Company, Ohio

(SOHIO) assessment program, studied the relationships of

several personnel selection methods to the outcome criterion

of supervisor rating. A period of 2.5 to 5 years elapsed

between measurement of variables. The researcher concluded

that "results of the interview report appeared particularly

impressive in light of the usual criticism of the interview

as an assessment technique" (p. 566).

Campbell et al. (1970), while at the University of

Minnesota and Yale University (Lawler), in the Standard Oil

Company of New Jersey assessment center study, reviewed the

relationship between the personal interview and later measures

of administrative achievement. The outcome criterion

consisted of a combination of performance ratings to include

salary level attained, supervisor ratings, and administrative

level achieved.

Moses (1972) conducted a study of 8,885 nonmanagement

males. These men, while employed by the Bell System, were

interviewed as part of the assessment center process. A

significant correlation was found with the criterion of

administrative level achieved.

Turnage and Muchinsky (1984) conducted a study based upon

data from a one-day supervisory selection program (assessment









45

center) developed by a large manufacturing firm. The subjects

were 799 nonmanagement employees who were subsequently

promoted to supervisory positions. The individual interview

was based largely on the candidate's questionnaire responses,

in which the candidate was required to play the role of a

supervisor faced with discharging an employee. In this

respect, this interview method overlaps the method of job-

related skills indicators. The predictor variable, interview,

was correlated separately with three criteria: salary level

attained, supervisor ratings, and promotion rate. Only one

resulting coefficient was significant. Based on these, and

other findings in the study, the researchers concluded that

there appeared to be no appreciable relationship between how

one is evaluated in an assessment center and how one performs

on the job.

A number of reasons were surmised by the researchers for

the lack of correlation. Methodological factors such as low

criterion reliability, low predictor reliability, severe

restriction of range, marked skew in the data, procedural

inconsistencies, lack of comparability across assessment

groups, and errors in data collection, were cited as possible

problems within this concurrent study.

Conclusions. The structured personal interview, in which

the interviewer, or interviewers, follow a set schedule of

questions, has consistently shown higher reliabilities than

the unstructured interview (Wright, 1969). In addition,









46
interviews conducted by a board or panel appear to be

promising as a means of enhancing reliability and validity

(Arvey & Campion, 1982). Greater validities may be found if

researchers first decide the purpose the interview is intended

to serve. It has been suggested that the personal interview

is useful in determining interpersonal skills and motivation

(Schmitt, 1976), and in imparting job information from the

interviewer to the applicant (Arvey & Campion, 1982).

The employment interview continues to be used although

organizational psychologists are aware of the findings

concerning the method's limited reliability and validity.

Arvey and Campion stated that "interviewers ignore base rate

information, do not pay attention to disconfirming evidence,

and over-depend on case specific information in making their

judgments" (Arvey & Campion, 1982, p. 316).

Job-related Skills Indicators as a Selection Method

Situational methods offer the potential of adding to the

scope of human characteristics that can be evaluated. The

rationale behind using situational exercises is that they

simulate the type of work to which the candidate will be

exposed and allow his or her performance to be observed under

somewhat realistic conditions. Situational tests measure more

complex or dynamic behavior rather than aptitudes or traits

isolated by more traditional psychometric tests. The whole

personality is observed in interaction with simulations of the

future job environment (Howard, 1974).








47

The most commonly used situational tests of job-related

skills for administrative positions are examined here. The

in-basket exercise is one of those most frequently used in

assessment centers. The candidate is faced with an

accumulation of memos, reports, notes of incoming telephone

calls, letters, and other materials supposedly collected in

the in-basket of the job he or she is to take over. The

candidate is asked to dispose of these materials in the most

appropriate manner by writing letters, notes, self-reminders,

agenda for meetings, etc. Ratings of performance range from

subjective evaluations to highly standardized checklists.

In the leaderless group discussion, the participants are

given a discussion question and are instructed to arrive at

a group decision. Topics may include such things as promotion

decisions, disciplinary actions, or business expansion

problems. Sometimes participants are given a particular point

of view to defend. Personality dimensions such as

interpersonal skills, acceptance by the group, individual

influence, and leadership abilities can be evaluated in this

way.

Management games, such as stock market tasks,

manufacturing exercises, and merger negotiations are common

job-related selection exercises. Participants are asked to

solve problems, either cooperatively or competitively. These

games often bring out leadership, organizational abilities,

and interpersonal skills. Some games also permit observations









48

under stress, especially when conditions suddenly change or

when competition stiffens.

Research used in this study. A total of 10 research

studies were found that used job-related skills indicators as

a predictor variable and met the other delimitations of this

dissertation. These 10 studies yielded 12 correlation

coefficients which ranged in strength from r = .02 to r = .49.

Samples ranged in size from 31 to 8,885 persons.

In 1965, Tenopyr and Ruch (in Campbell et al., 1970)

studied the relationship between job-related skills and salary

level attained in a concurrent validity study. Subjects were

production managers at North American Aviation. They were

given a test designed to assess a supervisor's ability to

handle human relations problems. Although a significant

finding resulted, the researchers concluded that this

correlation was "useful" but not "overwhelming" (Campbell et

al., 1970, p. 193).

Dicken and Black (1965) studied the validity of clinical

interpretations of an objective test battery in an industrial

setting. A sample of first-line supervisors in a

manufacturing company was given tests of job-related skills.

These tests were administered to the participants individually

and in groups. A significant correlation was found between

job-related skills indicators and salary level attained after

3.5 years.









49

Bray and Grant (1966) found that job-related, or

situational, selection methods had considerable influence on

the judgments of the assessor. Job-related skills indicators

of a sample of managers were related to salary level attained

8 years later. A significant correlation was found. The

situational exercises used consisted of an in-basket exercise,

a simulated manufacturing problem, and a leaderless group

discussion. The researchers concluded that the job-related

situational exercises contributed enough to predictivity to

warrant their relatively high costs in time and money.

Wollowick and McNamara (1969) studied the validity of an

assessment center approach in predicting management potential

and to determine the relative values of the components of the

program. One component of the assessment center consisted of

both written and performance tests of job-related skills.

These tests were administered, individually and in groups, to

a sample of low- and mid-management men. The results of these

tests were significantly related, approximately three years

later, to administrative level achieved.

Meyer (1970), while at General Electric Company, studied

the relationship between a job-related skills indicator, the

in-basket exercise, and an outcome criterion of supervisor

ratings. A demonstrated inability of a general aptitude test

to determine which employees would become successful unit

managers led to the use of this method. This exercise was

developed by John Hemphill, then of the Educational Testing









50
Service. A comparison of overall score on the skills

indicator test and the outcome variable yielded a significant

correlation in this concurrent study. Meyer concluded that

the "performance style one exhibits in handling carefully

selected, but true-to-life 'in-basket' items does correlate

with demonstrated on-the-job performance of a managerial

position, especially the ability to handle the planning and

administrative aspects of the job" (p. 307).

Campbell et al. (1970) studied the relationship between

job-related skills indicators and a criterion measure of

administrative achievement. The outcome criterion was a

combination of performance ratings, to include salary level

attained, supervisor ratings, and administrative level

achieved. This concurrent validity study was a component of

the Early Identification of Management Potential (EIMP)

assessment program of the Standard Oil Company of New Jersey.

A subject group of mid- to high-level managers showed a

significant correlation between written measures of job-

related skills indicators and the outcome criterion.

The researchers found that job-related skills indicators,

in the form of a management judgment test, were among the four

best measures of achievement. However, they surmised that the

high correlation was, in part, attributable to possible

criterion contamination due to the managers' perceptions of

their own career success.









51

Moses (1972) studied the relationship between job-related

skills indicators and administrative level achieved seven

years later. The job-related skills indicators were

performance-type tests by which assessors determined a

subject's organizing, planning, decision-making, and

leadership skills. A large sample of nonmanagement men showed

a significant correlation coefficient between the two

variables.

Moses and Boehm (1975) studied the relationship between

several aspects of performance at an assessment center and

subsequent administrative level achieved. A large sample of

nonmanagement women was assessed between 1963 and 1971. A

significant correlation was found in this predictive study.

This study was not part of the landmark AT&T study by Bray and

Grant (1966) and some criterion contamination may have

occurred. The researchers concluded that the job-related

skills indicators of leadership, decisionmaking, organizing,

and planning related highly with the outcome criterion.

Huck and Bray (1976), while with the Wickes Corporation

and AT&T, respectively, studied the power of the AT&T

assessment center to predict future job performance of female

employees. Subjects were non-management personnel of Michigan

Bell Telephone. The independent variable of job-related

skills indicators consisted of performance tests of

leadership, decision-making, planning, and organizing. Some

criterion contamination was acknowledged. The researchers









52
concluded that the assessment center process was as valid a

selection method for women as for men.

Turnage and Muchinsky (1984), while at Iowa State

University, conducted a predictive study based upon data from

a one-day supervisory selection program (assessment center)

developed by a large manufacturing firm. The subjects were

employees who were subsequently promoted to supervisory

positions. An overall significant correlation was found

between job-related skills indicators and these methods of

measuring subsequent job performance combined. This was one

of the few significant results of this study.

Conclusions. Though much more expensive and time

consuming to administer than paper-and-pencil tests and

questionnaires, the need to find ways of evaluating

characteristics not covered by the latter is sufficient to

warrant extensive experimentation with relatively elaborate

techniques (Bray & Grant, 1966). There has been relatively

little research on the relationship between how a manager

behaves in a game and his behavior in an actual decision

situation. The behavior of a manager in a business game may

be quite different from his behavior on the job where the

rewards and punishments are much larger. Further, business

games tend to de-emphasize the interpersonal dimension in

managerial performance, whereas many management jobs appear

to emphasize it heavily (Lawler, 1967, p. 370).










Psychological Attribute Indicators as a Selection Method

McGregor (1960) reported that the greatest single factor

that apparently influences superior and inferior performance

by supervisory people was to be found in the area of

personality variables. According to Nash (1966),

it is reasonable to expect that the vocational
interests of a manager might be related to the
effectiveness of his job performance. His enthusiasm,
effort, and level of job satisfaction may be largely
determined by how interested he is in his work and
associates. (p. 250)

According to Edel (1968), for executive and managerial

positions, personality characteristics may well be more

important to success than skill or technical know-how (p.

231). To point up this importance, Micherson (in Edel, 1968)

surveyed 79 large and small business corporations and reported

that, of those executives who failed, over 70% did so because

of some flaw in their personality rather than from a lack of

ability.

A wide variety of research on personality tests of various

types has been conducted. This work has been successful in

defining problems and has contributed to the overall

understanding of personality adjustment. Three problems in

the use of personality tests in occupational prediction

deserve special mention: (a) the ease with which test scores

can be distorted by a test-wise applicant to portray the type

of personality desired, (b) the lack of reliability displayed

by many personality measures, and (c) the failure to design









54

the tests specifically for purposes of occupational prediction

(Edel, 1968).

Early studies gave mixed validity results. Knauft (1949)

studied a sample of 33 bakery shop managers and a correlation

of r = .39 was found, but the personality test used was later

withdrawn from the market (Korman, 1968). Comrey and High

(1955) tested the validity of some ability and interest

scores. Using a sample of over 200 production supervisors,

scores on several preference records and vocational scales

were compared to objective performance data. All correlations

except one were insignificant. La Gaipa (1960) studied the

validity of certain personality traits of over 400 naval

officer candidates compared to later supervisor ratings. Only

one of several correlations was significant. MacKinney and

Wolins (1960) studied the relationship between two interest

measures and several later measures of performance using three

overlapping samples of supervisors. Results were inconsistent

random patterns of significant and insignificant correlations.

Mixed results were also found by Robbins and King (1961) in

four samples of sales managers.

Research used in this study. A total of 22 research

studies were found which used psychological attribute

indicators as a predictor variable and met the other

delimitations of this dissertation. These studies yielded 37

correlation coefficients which ranged in strength from r = .01

to r = .47. Samples ranged in size from 20 to 1,375 persons.









55
Fleishman and Peters (1962), while at Yale University,

studied the relationships between psychological attribute

indicators and managerial achievement. The resulting

correlation was not significant. A small sample was used in

this concurrent study. The researchers found that (a)

managers who rated highly on "conformity" were less valued by

senior raters, (b) the leadership attitudes of consideration

and structure were not mutually exclusive, and (c) top

management tended to identify the effectiveness of subordinate

managers with the effectiveness of their superiors.

Hicks and Stone (1962) explored the relationship between

psychological attribute indicators and supervisors' ratings

in this concurrent study. A nonsignificant correlation was

found for the sample of shop foremen and engineering

supervisors. The researchers concluded that there may be

certain basic characteristics which the successful managers

possess regardless of their areas of specialization. They

described the personality of the successful manager as one of

emotional strength in a person who views things from a broad,

theoretical point of view, avoiding over involvement in

detail.

Goodstein and Schrader (1963), while with the University

of Iowa and the Civilian Personnel Field Agency, Ordnance

Field Activity, United States Army, respectively, studied the

validity of a personality inventory in identifying those

personality characteristics associated with managerial and








56

supervisory success in a large military industrial

organization. In their concurrent study of male civilian

managers working for the Army, the variable of psychological

attribute indicators was compared to the criterion of

administrative level achieved. A significant relationship was

found.

The researchers stated that the results suggested there

may be significant differences in personality characteristics

not only between managers and nonmanagers but also among

managers at different levels of responsibility. They

purported that success in first-line supervision may be

determined mainly by technical skill and knowledge which is

relatively independent of personality factors whereas, at the

upper levels of management, such personality-related variables

as organizing, directing, planning, and decision making become

important.

Williams & Harrell (1964) attempted to discover which

personality factors were related to business success. In

their predictive study they used the salary level attained

and administrative level achieved of Stanford MBA graduates

15 to 31 years after graduation. The overall comparison of

psychological attributes to later achievement was

insignificant. However, there was a significant positive

relationship between "success and the score on the

Masculinity-Femininity scale {of the Strong Vocational

Interest Blank), indicating that those individuals with the









57

stronger masculine interests have a somewhat better chance of

success" (Williams & Harrell, 1964, p. 167).

Dicken and Black (1965) studied the predictive validity

of psychometric evaluation for the selection of supervisors.

Predictor variables were two well-known personality

inventories. Two criteria, salary level attained and

promotion rate, both measured 3.5 to 7 years after testing,

were used. Insignificant correlation coefficients were found

for each of the two samples. The researchers cited

restriction of range as a problem in studies of this type by

preselection of the samples by management. They stated that

"measures of ability and interest cannot be expected to make

fine discrimination" (p. 46).

Bray and Grant (1966) explored the relationship between

psychological attribute indicators and salary level attained

8 years later. Subjects were low-level managers. Although

an overall nonsignificant correlation was found between the

variables, several personality traits such as lack of

passivity and control of feelings were positively correlated

with success in management.

Nash (1966), while at the University of Maryland, sought

to discover the relationships between a manager's vocational

interests and the effectiveness of his job performance. This

concurrent validity study used corporation managers as

subjects. The correlation coefficient between psychological

attribute indicators and supervisor ratings was found to be









58

significant. Nash concluded that "effectiveness is

significantly related to the vocational interest patterns of

managers" (p. 254).

In 1967, Bentz (in Campbell et al., 1970) summarized a

series of studies which he had conducted within the

Psychological Research and Services Section of Sears, Roebuck

Company. In this study a significant relationship was found

between psychological attribute indicators and promotion rate

within Sears. Bentz's conclusion was that the Sears managers

accomplish their goals because they are superior "in

intellectual endowment, social competence, and emotional

stamina" (Campbell, et al. 1970, p. 187). Campbell et al.

(1970) stated that there was truth to Bentz's statement, but

that it was not strongly supported by the findings of his

concurrent validity study.

Cummin (1967), while at Harvard, sought to discover if the

"more successful" executives would display motivation toward

achievement, power, and autonomy, and the "less successful"

executives would show motivation toward affiliation,

aggression, and deference in a personality inventory. His

concurrent validity study of business persons compared

psychological attribute indicators to current salary level

attained. He found a significant correlation. Cummin

concluded that "the successful executive is one who is

determined to maintain a high standard of excellence in his

work, and to assume greater responsibilities and more control








59

over his environment as he advances within the organization"

(p. 81).

Grant, Katkovsky, and Bray (1967), while at AT&T and

Fordham University (Katkovsky), studied the contributions of

projective techniques to the assessment of management

potential. The study used data collected in the Bell System

Management Progress Study (Bray, 1964). A comparison of

psychological attribute indicators to salary level attained

eight years later showed a nonsignificant correlation. The

researchers found that within this method several of the

projective variables were reliably related to salary progress.

Those variables were dependence and subordinate roles (with

negative relationships), achievement motivation, and

leadership role.

In a concurrent validity study, Ghiselli (1968) evaluated

motivational factors in the success of managers. He

administered a personality inventory to several samples of

corporation middle management personnel. Psychological

attribute indicators related significantly with supervisor

ratings.

As compared with the employed population as a whole,

Ghiselli found that persons in middle management positions

appeared to have a substantially lower desire for security and

for financial reward and a higher desire for self-

actualization. They did not differ from the employed

population in the desire for power over others.









60

Edel (1968), while working for the Department of Defense,

studied the relationship between a psychological "need for

success" and managerial performance. He found a significant

correlation in this concurrent study. Edel's study used one

test to measure need for success. Reliability data had not

yet been provided for this test.

In 1968, Miner (in Miner, 1977), while at Georgia State

University, studied the relationship between measures of

psychological attributes of school administrators and

separate, concurrent, outcome criteria of supervisor ratings

and salary level attained. He found no significant

correlations. He concluded that managerial motivation was not

rewarded in that school system.

Campbell et al., (1970) conducted the Early Identification

of Management Potential study for Standard Oil Company of New

Jersey. Within this study they compared measures of

psychological attributes to a combination of concurrent

outcome criteria to include salary level attained, supervisor

ratings, and administrative level achieved. Subjects were

mid- to high-level corporation managers. An insignificant

correlation was found. The researchers concluded that these

indicators of temperament showed no useful relationship with

any of the effectiveness measures.

Harrell and Harrell (1974) conducted a predictive study

in order to determine predictors of administrative

achievement. While working for the Office of Naval Research,









61

they found a significant correlation between psychological

attribute indicators and salary level attained 10 years later.

Subjects were former MBA graduates who had achieved management

positions in corporations.

Grimsley and Jarrett (1975) studied the relationship

between managerial achievement and test measures obtained in

the employment situation. This concurrent study used two

samples of mid- and top-level corporation managers. The

results of several measures of psychological attributes were

compared to the outcome criteria of administrative level

achieved. Insignificant correlation coefficients were found

with each of the two samples. The researchers concluded that

"the differences in test scores of more or less successful

managers result from fundamental differences in mental ability

and personality rather than the influence of on-the-job

experience" (p. 226).

Gantz, Erickson, and Stephenson (in Miner, 1977) sought

to determine if certain psychological attributes could be

causes of managerial success. Using a sample of research

scientists and engineers in a federally funded laboratory,

several measures of psychological attributes were

administered. These proved to be insignificantly related to

both supervisors' ratings and promotion rate. The researchers

concluded that perceived creativity was rewarded by promotion

into leadership roles, including formal managerial roles where









62
managerial skill was needed, and actual or potential

leadership ability was not rewarded.

Two consequent studies by Gantz, Erickson, and Stephenson,

were also reported in Miner (1977). In the first, a sample

of former research scientists was followed up 5.3 years after

testing and showed a significant correlation between measures

of psychological attributes related to motivation to manage

and promotion rate. As some of the subjects may have

experienced grade level changes based in part on research

competence, the success index, promotion rate, may not have

been based entirely on managerial competence. The researchers

concluded that the borderline level of the correlation

indicated that further analyses dealing with the predictive

power of the role-motivation theory ought to be conducted.

Using a second sample consisting of top sales persons and

marketing managers, the researchers conducted a similar study

with four years intervening between variables. The

correlation was significant. The researchers stated that,

within this department of the same company, "promotion into

the higher grade levels was based entirely on managerial

competence" (Miner, 1977, p. 31). The conclusion drawn was

that the motives measured by the particular psychological

attributes indicator used did serve as a cause of subsequent

managerial accomplishment (Miner, 1977).

Miner (1977), in a concurrent validity study, sought to

discover the relationships between different measures of









63

psychological attributes and two measures of administrative

achievement. Using a sample of business personnel managers,

he found insignificant relationships with both administrative

level achieved and with salary level attained. Although

neither correlation was significant, Miner stated that "the

overall measures of motivation to manage are quite

consistently related to the occupational success indexes" (p.

74).

In a separate study by Miner (1977), faculty members and

administrators in three different business schools were given

an inventory of psychological attributes. Concurrently, each

subject's score was compared with his or her administrative

level attained. Miner reported no significant relationship,

and "if anything, the administrators have less motivation to

manage than the regular faculty, although the difference does

not approach significance" (p. 51).

McClelland and Boyatzis (1982), while at Harvard

University and McBer & Company, respectively, studied the

relationship between psychological attribute indicators and

long-term achievement in management. Subjects were entry-

level managers who were part of the AT&T assessment center

study. A significant correlation was found between the

predictor variable and administrative level achieved after 16

intervening years. The researchers concluded that a high need

for achievement was associated with managerial success at

lower levels of management, while at higher levels of









64

management persons with a need for influencing others

dominated.

Stahl (1983), while at Clemson University, used a measure

of psychological attributes to test the hypothesis that high

managerial motivation consisted of high personal needs for

achievement and power. He found a significant correlation

with each of two outcome criterion variables, supervisor

ratings and administrative level achieved. The researcher's

conclusions were drawn from this and other data collected from

other samples using the same methodology. He concluded that

"there was a higher proportion of subjects with high

managerial motivation among the managers than among the non-

managers," and that "there was a higher proportion of managers

with high managerial motivation among the promoted managers

than among the non-promoted managers" (p. 786). Stahl

stressed the need for researchers to implement longitudinal

validation studies to confirm these results.

Turnage and Muchinsky (1984) compared the predictive

validity of assessment center evaluations versus traditional

measures in forecasting supervisory job performance. A sample

of employees who were subsequently promoted to supervisory

positions within a large manufacturing firm was used.

Correlation coefficients between findings of psychological

attribute indicators and three separate outcome variables

yielded no significant relationships. The researchers cited

many possible methodological shortcomings of the study that









65

could have contributed to the general lack of significant

findings.

Conclusions. Results obtained by the studies reviewed

herein suggest that there is evidence supporting the

proposition that effective managers have identifiable

interests which distinguish them from less effective managers.

However, most motivation studies have used concurrent validity

as the research method. Predictive validity studies that

assess the dynamic nature of personality may be needed.

According to Miner (1978), "the greatest research needs seem

to be for additional longitudinal studies of the relationships

between motivation to manage...and success" (p. 751).

Another limitation of interest measures is that they have

been demonstrated to be fakeable. Nash suggested that future

research should focus on the impact of fakability on the

actual use of measured interests in selection programs (1965,

p.34).

Campbell et al. (1970) addressed achievement and power for

managers. Concerning job/task analysis of what effective

managers actually do, they listed the frequency of behavior

aimed at influencing others (power) and the frequency of

behavior concerned with setting and accomplishing goals

(achievement). They remarked that "better managers tend to

show a lifetime pattern of high achievement, power, and

economic motivation" (p. 361).








66

One of the most recent and extensive treatments of the

relationships of achievement to power is presented in Veroff's

book of readings in honor of McClelland.

Achievement motivation directs people to meeting
socialized standards of excellent performance and thus
to highly efficient task-centered strivings, whereas
power motivation directs people to doing whatever
draws most attention to their own effect on the world.
The two motives seem to be fused in instances where
the standard of excellence is to win in a social
competitive activity or to solve a problem that will
be given a great deal of recognition. (Veroff, 1982,
p. 100)

Value Systems Assessments as a Selection Method

The selection method of values systems assessments

measures the strength of a person's economic, aesthetic,

social, political, and religious concerns (Hinrichs, 1978).

According to England and Lee (1974),

a personal value system is viewed as a relatively
permanent perceptual framework which shapes and
influences the general nature of an individual's
behavior. Values are similar to attitudes but are
more ingrained, permanent, and stable in nature; they
are also more general and less tied to any specific
referent than is the case with many attitudes.
(p. 412)

Most personnel selection research studies which dealt with

the independent variable of psychological attribute indicators

verged on or overlapped the selection methods using values as

an indicator. The studies used in this section were those

that this researcher determined were predominantly concerned

with values as defined above.

Research used in this study. A total of 4 research

studies were found which used value systems assessments as a








67

predictor variable and met the other delimitations of this

dissertation. These studies yielded 5 correlation

coefficients which ranged in strength from r = 10 to r = .32.

Samples ranged in size from 30 to 1,375 persons.

In 1944, the Psychological Research and Services Section

of Sears, Roebuck Company established a psychological testing

program. Over a number of years researchers Within this

program conducted a series of investigations into the

prediction of executive effectiveness in the Sears

organization. A predictive study was accomplished by Bentz

in 1967 (in Campbell et al., 1970). A span of 11 to 17 years

elapsed between the measurement of values and collection of

the criterion variable, promotion rate. A significant

correlation was found.

England and Lee (1974), while at the University of

Minnesota, studied the relationship between managerial values

and managerial success in several countries. The portion of

their study that dealt with managers in the United States was

used in this dissertation. Managerial success was defined as

a measure of salary level attained in this concurrent study.

The correlation of this measure with a written survey of

values was significant for a sample of corporation directors

and high level executives. The researchers concluded that the

value patterns were predictive of managerial success and could

be used in selection and placement decisions. The general

pattern that emerged from the study indicated that more









68

successful managers appeared to have pragmatic, dynamic,

achievement oriented values, whereas less successful managers

had more static and passive values, corresponding to a desire

for organizational stasis rather than an organization in flux

(England & Lee, 1974).

Grimsley and Jarrett (1975) studied the "effectiveness of

a particular methodological approach which can be used in

analyzing data gathered in the process of assessing managerial

applicants in the employment situation" (p. 215). Two sample

groups of mid- and top-level corporate managers were used in

this concurrent study. These managers came from more than 100

different companies and many industries in various parts of

the United States. All were administered a "study of values"

test and test scores were compared with administrative level

achieved. The results were not significant. The findings of

this study led the researchers to conclude that the

"differences in test scores of more or less successful

managers result from fundamental differences in mental ability

and personality rather than the influence of on-the-job

experience" (p. 226).

Hinrichs (1978), while with Management Decision Systems,

Inc., conducted an 8-year followup study of the IBM

Corporation assessment center. Within this study, the results

of a study of values were correlated to the outcome variable

of administrative level achieved. The resulting correlation

coefficient of r = .26 was not significant. High scores on








69

political and economic subsets of the study of values were

positively related to progression in formal business

hierarchy, while scores on the religious and social scales

related negatively.

Conclusions. The few research findings available on

values as a selection method consistently show that certain

economic and political beliefs correlate positively, and

strong social and religious beliefs correlate negatively, with

high administrative achievement (Grimsley & Jarrett, 1975).

England and Lee (1974) found that "more successful" managers

favored an achievement orientation and preferred an active

role in interaction with other individuals instrumental to

achievement of the manager's organizational goals, while "less

successful" managers had values associated with a static and

protected environment in which they took relatively passive

roles (pp. 418-419). According to Hinrichs (1978), those

traits or characteristics that seem to reflect a degree of

social awareness tend to be detrimental to success in

administration. "The flavor is one of the 'hard charging,'

perhaps somewhat socially insensitive and upward mobile

individual" (p. 600).

Biographical Information as a Selection Method

Biographical information, personal history data about an

employee, is most often furnished an employer in the form of

an application blank or a resume. According to Levine and

Flory (1975), the most widely used selection technique is the








70

evaluation of the job application blank or the resume. These

are used to determine whether an individual meets minimum

qualifications for a position. If these qualifications are

not met, the applicant is no longer considered. In some cases

this is the only information an employer receives about a

prospective employee, but in most cases the application is

followed by an interview, written test, or some additional

means of assessment.

To provide some idea of the frequency of use of

applications, Carlson (in Levine & Flory, 1975) posited that

if an arbitrary assumption were made that 50% of those filing

applications or resumes are ultimately interviewed, then over

1,000,000,000 applications and resumes per year are prepared

and screened in the United States. Yet, despite its

universality, the evaluation of the validity of biographical

information for personnel selection and placement has not been

researched in a systematic fashion to any great extent.

Research has been done on the empirically weighted

application, usually for predicting turnover. More recent

research on application evaluation has focused on the factors

affecting the rater's quality of evaluation of the information

rather than the quality of the method of evaluation per se.

According to Spencer and Worthington (1952) and Peck and

Parsons (1956), the few studies dealing with method of

evaluation other than empirical weighing have demonstrated








71

that projective evaluation of application blank information

could validly predict performance and tenure.

A trend in selection research in recent years has been the

increasing use of personal background variables in the

prediction of occupational success. Such variables are

considered to have several advantages for this purpose. The

most frequently cited advantages in utilizing these kinds of

variables are that (a) they are less threatening than the

items on typical personality inventories and thus are less

subject to "facade" effects, and (b) the behaviors described

by these items are often reflections of attitudinal and

personality variables.

Results of past studies of the relationship between

biographical information and administrative achievement have

been mixed. Vernon (1950), using civil service managers, and

La Gaipa (1960), using Naval officer candidates, found no

significant relationships. The findings of Riccuiti (1955),

with U.S. Naval officers, Meyer (1956), with firstline

supervisors, and MacKinney and Wolins (1960) with supervisors,

found no consistent relationships. Significant relationships

were found by Haggerty (1953), at the U.S. Military Academy,

Soar (1956), using service station managers, and Scollay

(1957), with promotion managers.

Childs and Klimoski (1986), in a study which involved

employees in both management and nonmanagement positions

investigated the validity of a biographical inventory in the








72

prediction of occupational success. They found that positive

social orientation, interpersonal confidence, and educational

achievement were positively related to their outcome measures

of "job and career success" (p. 7).

Research used in this study. A total of 8 research

studies were found which used biographical information as a

predictor variable and met the other delimitations of this

dissertation. These studies yielded 11 correlation

coefficients which ranged in strength from r = .05 to r = .57.

Samples ranged in size from 30 to 799 persons.

ln 1963, Haggerty (in Mayfield, 1970) studied the

relationships between certain predictor variables and

achievement of U.S. Army officers. In this predictive study

using a sample of officer cadets, the researchers found a

nonsignificant correlation between biographical information

and a combination of achievement measures.

Williams and Harrell (1964) sought to determine which, if

any, of a number of factors were related to business success.

The independent variable in this predictive study was that of

biographical information available to an employer at the time

of a student's graduation from business school. Salary level

attained, and administrative level achieved, measured many

years after graduation, were the criteria of success.

Insignificant correlations were found for this sample of

Stanford MBA graduates. A subset of this information that was

significantly and positively related to later achievement was








73

a person's previous participation as a leader in organizations

on campus.

Kotula and Haggerty in 1966 (in Korman, 1968) studied the

relationship between biographical information given on written

personal history blanks and supervisor ratings. Two groups

of Army officers were studied. Only one coefficient was

reported, r = .17, which was significant at the .05 level.

Campbell et al. (1970) attempted to discover how to

identify, early in their careers, those employees who possess

the potential to be successful in management. The assessment

procedure in this concurrent study contained a background

biographical survey the scores from which were correlated to

an "Overall Success Index", a combination of dependent

variables. A sample of mid- to high-level corporation

managers was used. A significant correlation was found

between biographical information and the outcome criterion of

achievement. Laurent, one of the researchers involved with

this study, concluded that "successful executives in the

Standard Oil of New Jersey organization have shown a total

life pattern of successful endeavors. They were good in

college, are active in taking advantage of leadership

opportunities, and see themselves as forceful, dominant,

assertive, and confident" (Campbell et al., 1970).

Harrell and Harrell (1974) conducted a predictive study

of MBA graduates in order to determine predictors of








74

management success. They found a significant relationship

between biographical information and salary level attained 10

years later.

Gantz, Erickson, and Stephenson (in Miner, 1977) conducted

a concurrent study of research scientists and engineers for

the purpose of gaining "more theoretical understanding of why

some persons in a research and development population achieve

promotion, particularly into formally designated supervisory

or managerial positions, and other persons do not" (p. 18).

The organization was a federally funded public laboratory.

The independent variable, the selection method of biographical

information, was found to be not related to either promotion

rate or supervisor ratings.

Hinrichs (1978) found a significant relationship between

biographical information and administrative level achieved

eight years later for a sample of 30 managers. The resulting

correlation coefficient was slightly higher than that for the

Overall Assessment Rating.

Turnage and Muchinsky (1984) sought to examine the

predictability of assessment center evaluation versus

traditional measures in forecasting job success. As part of

the study, they separately compared biographical information

with supervisor ratings and salary level attained up to

several years later. These variables were insignificantly

related for the mostly nonmanagement personnel tested. The








75

researchers concluded that certain background data predict

criteria as well as assessment center data.

Conclusions. Compared to the interview method, Levine and

Flory (1975) found the application blank information to be

superior in accuracy of information. However, they noted that

"sizable inaccuracies" had been found in each method (p. 384).

Childs and Klimoski (1986) found that an outgoing and self-

confident personality preordains success in both job- and non-

job-related situation. They also found that success in one's

educational history may aid the attainment of success in one's

career, regardless of whether that job is personally

satisfying. Most studies that showed significant positive

correlations between biographical information and

administrative achievement attributed the relationship to

personality characteristics which were also positive

attributes of leadership ability.

Peer Ratings as a Selection Method

Peer ratings of supervisory potential achievement consist

of impressions gained from interactions in an equal, non-

supervisor-subordinate, nature. They are predictions of how

well a peer will do in a supervisory position, should he or

she be placed in one (Korman, 1968).

Among the more consistent findings concerning peer ratings

is the significant validity these afford in predicting later

performance. Many studies conducted in military settings

indicated that peer evaluations during officer training








76

successfully predicted later criteria of performance.

Williams and Leavitt (1947) found that peer ratings were

better predictors of long-term success in the Marine Corps

than were superiors' ratings. Much military research was

contributed by Hollander (1954, 1956, 1957, 1964, 1965) and

the evidence was consistently favorable. La Gaipa (1960), in

the Development of the Officer Candidate Biographical

Information Blank, sampled Naval officers in both shore duty

and fleet duty. He found a significant correlation between

peer ratings and later performance criterion.

There has also been evidence of the utility of peer

ratings from other spheres of activity, as Weitz found a

relationship of r = .40 between peer nominations and later

ratings of life insurance agents in a supervisory position

(1958). As Hollander has stated, "when employed with

discrimination, it [the peer rating method] can provide a

unique contribution to evaluation" (1965, p. 434).

Research used in this study. A total of 8 research

studies were found which used peer ratings as a predictor

variable and met the other delimitations of this dissertation.

These studies yielded 10 correlation coefficients which ranged

in strength from r = .04 to r = .53. Samples ranged in size

from 40 to 799 persons.

Haggerty (1963), in two separate predictive studies,

explored the predictive ability of peer ratings obtained on

Military Academy Cadets. The first group of men showed a








77

significant correlation with the outcome variable of

supervisors' ratings. The second group of cadets showed a

significant correlation with a later combination of overall

ratings.

Roadman's (1964) predictive study compared peer ratings,

taken in a management school setting at IBM, to promotion

rates of a sample of graduates who had received promotions at

least two years later. A significant correlation was found.

He concluded that a careful and comprehensive peer rating

administered in a middle manager training program can identify

those who later move into senior executive positions.

Hollander (1965), while at the State University of New

York at Buffalo, did a predictive study comparing the

relationship between peer assessments and supervisor ratings.

The study was begun in 1955 at the Naval Officer Candidate

School (OCS) in Newport, Rhode Island. An entire OCS class

was made available for this investigation. Four forms, each

setting out different qualities to be rated, were utilized;

these dealt with leadership, motivation for naval service,

probability of success in OCS training, and success as a

future officer. Results of part of the study were reported

by Hollander (1956) and showed that very early in the training

program, students were able to accurately determine which of

their peers would do well after training.

In the follow-up study (Hollander, 1965), supervisor

ratings of former trainees, who had become officers, were used









78

to test the validity of peer prediction of future success.

This follow-up of officers at least 3 years after graduation

showed a significant correlation. Hollander concluded that

peer nominations used early in training can make a distinctive

contribution to the prediction of a long-range criterion of

performance after training.

Lawler (1967), while working for the Department of

Administrative Sciences of Yale University, explored the

relationship between peer ratings and supervisors' ratings.

In this concurrent validity study, a sample of mid-and top-

level managers in a manufacturing company were rated by

several peer raters on three traits. Lawler found that

personnel selection "decisions (which include peer ratings)

will be of a higher quality than if just superiors' ratings

are relied upon" (p. 378).

Mayfield (1970), while employed with the Life Insurance

Agency Management Association, studied the relationship

between peer ratings and supervisors' ratings 2.5 years later.

The purpose of this predictive study was to investigate the

value of the buddy nomination procedure in the selection of

assistant managers for life insurance companies. The scores

were not used in promotion decisions to prevent criterion

contamination.

Two of the three samples showed significant correlations

between peer and supervisor ratings. Mayfield concluded that








79

peer ratings can be made in a "realistic" administrative

setting and retain their predictiveness.

Mitchel (1975), while at Bowling Green State University,

studied the predictive abilities of parts of the Standard Oil

Company of Ohio's assessment center program. The relationship

between peer ratings and a later measure of salary level

attained proved significant. He concluded that "peer and

assessor ratings, as well as combinations of variables, were

predictive of a salary criterion of managerial success"

(Mitchel, 1975, p. 578).

Gantz, Erickson, and Stephenson (in Miner, 1977) sought

to determine why some persons in a research and development

population achieve promotion into formally designated

supervisory or managerial positions and whether those who were

promoted were the ones who should have been promoted. A study

of research scientists and engineers in a large, federally-

funded laboratory showed a significant correlation between

peer ratings and promotion rate. These peer ratings of

overall ability and perceived creativity were statistically

significant at the .01 level.

Turnage and Muchinsky (1984), examined the ability of

assessment center evaluations to predict actual job

performance criteria, and to compare the predictability of

assessment center evaluations versus traditional measures in

forecasting job success. Their data came from a one-day

supervisory selection program developed by a large









80

manufacturing firm. The outcome data were gathered from 1

month to several years after assessment from a sample of

previous assessees. None of the peer ratings was significant.

Several possible explanations for the low validity

coefficients were given. These included poor experimental

control, insufficiently trained assessors, aggregation of data

over a 4-year period, and the effects of intervening

variables.

Conclusions. There appears to be a reasonable basis to

contend that peer nominations do provide distinctive

prediction of performance at a considerably later time. Peer

evaluations are relevant because peers are situated to

evaluate how a person performs in terms of the lateral

relationships in working toward organizational goals.

Further, peers often see the worker at times when his superior

is not viewing his behavior and, therefore, they may see

aspects of his behavior of which the superior is not aware

(Lawler, 1967). However, attention must be paid to the

question of the willingness of participants to rate each other

accurately if they know full use will be made of peer ratings

for administrative purposes (Roadman, 1964). A problem Lawler

(1967) acknowledged was that if the peer rater knew that his

or her opinion was "going to count" in an employment setting,

ratings may lose their validity, "particularly if a situation

exists where an individual's self-interests might be best

served by distortion of the peer ratings" (p. 379). According








81

to Korman (1968) there is a need for the initiation of

research on the selection method of peer ratings concerning

both its predictive validity and the general characteristics

that correlate with peer ratings.

Self-Appraisals as a Selection Method

Self-ratings of administrative ability are relevant in

selection because the individual has more information about

his own behavior than anyone else and because self-perceptions

are important determinants of an individual's future behavior

(Lawler, 1967, p. 371). A negative view of self-selection for

administrative positions has apparently been shared by those

behavioral scientists who study, develop, and validate

personnel selection procedures. The reason for this appears

to be rooted in a Theory X view of people as they are expected

to behave in a personnel selection setting. In these settings

people are assumed to lack objectivity in assessing their own

performance or personal attributes. They can be expected to

overestimate their performance, skills, knowledge, and

abilities to improve their chances for appointment (Levine,

1978).

Published studies on self-assessment of skills, abilities,

knowledge, and other applicant attributes as predictors of

administrative job performance have been virtually nonexistent

until recently (Levine, 1978, p. 230). Studies on self-

selection, done before 1962, have dealt mainly with

objectively measurable traits (Nickels & Renzaglia, 1958).








82

Research used in this study. A total of 8 research

studies were found which used self-appraisals as a predictor

variable and met the other delimitations of this dissertation.

These studies yielded 9 correlation coefficients which ranged

in strength from r = .01 to r = .26. Samples ranged in size

from 30 to 799 persons.

Prien and Liske (1962), while at Case-Western Reserve

University, explored the relationship between supervisor

ratings of job performance and incumbent self-ratings of job

performance on tasks which were intangible in nature. In

their concurrent validity study of employees of various

corporations, significant correlations were found between

self-ratings and first-level supervisor ratings, and

insignificant correlations were found between self-ratings and

second-level supervisor ratings. The preponderance of studies

showed, as this one did, that individuals rate themselves

higher than they are rated by comparison groups.

Lawler (1967) while with the Department of Administrative

Sciences of Yale University, studied mid- and top-level

managers in a manufacturing organization. Four coefficients

of the relationship between self-appraisal and supervisory

job performance ratings showed an average nonsignificant

correlation. This concurrent validity study also showed

evidence of self over-estimation of perceived administrative

abilities.









83

Thornton (1968), while a summer associate at the firm of

Rohrer, Hibler and Replogle in Chicago, investigated the

relationship between supervisory perceptions and incumbent

self-perceptions of the performance of executive personnel.

A sample of high-level managers in a large corporation were

included in this study. An average of coefficients of 27

traits, or behavioral characteristics, studied showed a

significant correlation between self-ratings and supervisor

perceptions of performance ratings. The tendency was for

self-evaluations of performance to be higher than supervisory

perception and, in this study, those executives who overrated

themselves were considered least promotable.

Campbell et al. (1970), in a long-term staff study,

reviewed the relationship between self-assessment and a

combination of performance ratings to include salary,

supervisory ratings, and administrative level attained. This

was part of the Early Identification of Management Potential

(EIMP) study carried out by the Standard Oil Company of New

Jersey (SONJ). A sample of managers of SONJ was used. The

resulting correlation coefficient was significant.

Contrary to previous evidence, Heneman (1974) found a

tendency for self-ratings to be less lenient than supervisory

ratings, with a significant correlation. His concurrent study

compared self- and supervisor ratings of job performance of

former MBA graduates of Indiana University, 7 to 10 years

after graduation. He suggested that future research on









84

managerial performance should include self-ratings where it

is made clear that these ratings will be used for research

purposes only.

Hinrichs (1978) completed one of the few predictive

validity studies of self-assessment of administrative

performance. The predictions of the assessment center were

not used administratively to prevent criterion contamination.

This study is included in an 8-year followup of a management

assessment center at International Business Machines (IBM).

Self-ratings of marketing personnel correlated significantly

with the outcome variable of administrative level achieved.

Steel and Ovalle (1984), while at the Air Force Institute

of Technology, sought to compare the relative validity of

self-ratings for predicting objective criteria of managerial

job performance. A concurrent validity study at a large

lending institution revealed an insignificant correlation

between self-rating and supervisor rating.

Turnage and Muchinsky (1984) conducted a predictive study

in a large manufacturing firm. The subjects were employees

who were subsequently promoted to supervisory positions.

Self-evaluations of job-related personal characteristics, such

as ability to withstand stress, intellectual abilities, and

interpersonal skill, made up the independent variable. Self-

evaluations related insignificantly and negatively with both

the outcome variables of salary level attained and supervisor

ratings. The problem of criterion contamination (i.e., the








85

availability of assessment scores to superiors) may have been

a factor in this study.

Conclusions. It has been suggested by Levine (1978) that,

as predictors of job performance, self-assessments should at

least supplement other information. Self-assessments may

replace more traditional selection methods, especially in

measuring psychological attributes that may be relatively

inaccessible by other means. He suggested that self-

assessments might have a motivational impact on those

applicants who are hired, as these people will strive to be

consistent with their self-perceived competencies. Levine

admits that research is needed to determine what applicant

attributes are most validly, or least validly, self-assessed.

Assessment Center Processes as a Selection Method

A highlighting feature of the assessment center is that

candidates are evaluated not on what they have done in present

or past jobs but on how they are likely to cope with a new

type of position. This involves using various situational

tests as well as incorporating some of the more classic

selection procedures, such as aptitude tests and interviews.

Assessments are conducted at least partially in groups, which

permits observing group interactions as well as obtaining peer

ratings (Howard, 1974).

The following dimensions are most often assessed: (a)

leadership, (b) organizing and planning, (c) decision making,

(d) oral and written communications skills, (e) initiative,








86

(f) energy, (g) analytical ability, (h) resistance to stress,

(i) use of delegation, (j) behavior flexibility, (k) human

relations competence, (1) originality, (m) controlling, (n)

self-direction, and (o) overall potential (Howard, 1974).

The origin of the use of multiple assessment procedures

on a large scale is credited to German military psychologists.

The British adapted the procedures to the screening of officer

candidates, and the United States Office of Strategic Services

(OSS) took over the approach from the British during World War

II (Bray & Grant, 1966). Howard (1974) stated that the first

industrial use of an assessment center has been generally

attributed to AT&T beginning in 1956.

Research used in this study. A total of 14 research

studies were found that used Overall Assessment Ratings (OAR)

of an assessment center process as a predictor variable and

met the other delimitations of this dissertation. These

studies yielded 16 correlation coefficients which ranged in

strength from r = .00 to r = .65. Samples ranged in size from

25 to 5,943 persons.

The first industrial use of an assessment center is

generally attributed to the American Telephone and Telegraph

Company. Other centers have been variations on AT&T's theme.

The AT&T experimentation was research-oriented and designed

to follow the development of managerial personnel for many

years after assessment (Howard, 1974).









87

Bray and Grant (1966) measured personal characteristics

hypothesized to be of importance either in developmental

change in early adulthood or success in business management.

The overall assessment rating was significantly correlated

with both salary level attained and administrative level

achieved 8 years after assessment. This was one of few

studies in which criterion contamination was eliminated. The

researchers concluded that "no single characteristic

determines progress in management" (Bray & Grant, 1966).

Moses (1972) studied the relationship between assessment

center processes and managerial achievement. Using a large

sample of assessees of an AT&T one-day assessment center,

Moses found a significant relationship between overall

assessment rating and administrative level achieved.

Moses and Boehm (1975) studied the relationship between

the assessment center process and subsequent progress in

management for women in the Bell System. Assessment center

data were obtained on 4,846 nonmanagement women from 1963

until 1971. The conclusion the researchers drew in this study

was that the assessment center process predicted the future

performance of women as accurately as it did that of men.

Huck and Bray (1976) tested the validity of an assessment

center process on a population different from the population

of the original Management Progress Study. Using a relatively

small sample of females who had been assessed previously, the









88

researchers found a significant correlation between overall

assessment rating and later administrative achievement.

The International Business Machines (IBM) Corporation

assessment center was patterned after the AT&T model and

included 2.5 days of assessment activity followed by 2 days

of developmental activity. Several studies have reported

results of the IBM assessment center (Hinrichs, 1978; Kraut

& Scott, 1972; Roadman, 1964; and Wollowick & McNamara, 1969).

Wollowick and McNamara (1969) studied the progress of low-

and mid-management males as subjects. The purpose of the

study was to determine the validity of an assessment center

approach in predicting management potential and to determine

the relative value of each of the assessment center

components. An overall assessment rating was determined for

each subject at the conclusion of the two-day assessment

session. The four observers who formed the assessment staff,

and who were operating management personnel at least two

levels above the participants, assigned each subject an

overall assessment rating taking into consideration all of the

variables in the program. According to the overall assessment

rating, each subject was ranked on a 5-point scale rating

potential for advancement within the company (Wollowick &

McNamara, 1969).

These overall assessment ratings were compared, 2.5 to 3.2

years later, to the subjects' administrative levels achieved.

This resulted in a significant correlation. It was evident









89

to the researchers that the subjectively derived overall

assessment rating utilized in this assessment program was a

valid predictor of management success.

Kraut and Scott (1972) studied the data provided by the

IBM assessment center on the progress of non-management males

into administrative positions. The purpose of the study was

to examine the validity of the assessment program in

predicting administrative achievement. Overall assessment

ratings were correlated with administrative level achieved

from 1 to 5 years later using the same 5-point rating system

that was used in the Wollowick and McNamara (1969) study. The

correlation between overall assessment rating and later

achievement was significant. This assessment program was one

of the few selection methods which showed validity in

predicting success beyond the first-level promotion after

assessment.

Hinrichs (1978) completed a predictive study using a small

sample of IBM managers. A significant relationship was found

between assessment center overall assessment ratings and

administrative level achieved eight years later. However,

Hinrichs also found that the relatively simple prediction

based upon managerial review of the personnel files did as

well in prediction as the assessment center.

Researchers in the IBM studies reviewed here concluded

that large-scale assessment programs appear useful in making

discrimination of management potential which are later









90

confirmed by the rate of promotions, as well as demotions

(Kraut & Scott, 1972). The studies contained criterion

contamination as the subjects' results were provided to their

supervisors. However, it was observed that the "relationship

of ratings to first promotions is moderate enough to reduce

fears of 'crown prince' or 'kiss of death' effects" (p. 124).

According to Kraut & Scott (1972),

compared to the normal promotional system in most
companies, the program typically has some obvious
advantages in reliability and validly measuring
management potential. Instead of judgments by one's
immediate manager which may be more or less
subjective, evaluations in the program are made by
several managers (raters) who are likely to be much
more objective. Further, they are not making
judgments about the individual's management potential
from his performance in a non-management job; instead
they evaluate all candidates against a common
yardstick comprised of standardized management-type
tasks. (p.124)

The Standard Oil Company, Ohio (SOHIO), created an

assessment center in 1963. This Formal Assessment of

Corporate Talents (FACT) Program was modeled after the AT&T

assessment process (Carleton, 1970). Most data from the SOHIO

program pointed to the superiority of the overall assessment

ratings but did not eliminate any single category of

assessment components (Howard, 1974).

Carleton (1970) sought to determine the relationships

between test and rating data and later measures of

administrative achievement. A significant correlation was

found between the overall assessment rating and subsequent

supervisor ratings. Carleton concluded that both test and









91

rating data generated in a multiple technique assessment

center held up well as predictors of behavioral ratings made

several years later.

Finley (1970) studied the predictive validity of

projective tests in the SOHIO management assessment center.

Using a sample derived from the FACT Program, he found a

significant relationship between overall assessment ratings

and subsequent supervisor ratings.

Mitchel (1975) studied a sample of managers from SOHIO's

FACT Program. An insignificant correlation was found between

overall assessment ratings, minus assessor's ratings of

potential, and salary level attained 5 years after assessment.

The overall rating was not the most valid predictor in this

study, nor did it appear in any of the stepwise equations done

by the researcher.

The SOHIO studies involved both trained and untrained

assessors. Evidence for discriminant validity was found for

the two trained assessor groups, but not for the group in

which assessors consisted of untrained supervisors. It was

concluded that the assessment center method was able to

predict multiple criteria fairly well, but that the poor

quality of criterion measures probably reduced the convergent

validity coefficients (Howard, 1974).

One of the most extensive validations of the assessment

center method against measures of management effectiveness was

the Early Identification of Management Potential (EIMP) study




Full Text
124
systems, job-related skills, and overall assessment center
ratings were less often used.
One of the most commonly used administrative personnel
selection methods was the biographical information method.
This information is presented in the form of application
blanks and resumes (Levine & Flory, 1975). Its popularity
stemmed from the accepted notion that the best predictor of
future behavior is past behavior (Childs & Klimoski, 1986) .
In most cases, biographical information has been used to help
determine if an individual meets the minimum qualifications
for a particular job. Most often this has been the way in
which a large group of job applicants was reduced to a
manageable number for further consideration.
Another often used method has been the personal
interview. According to Carlson (in Levine & Flory, 1975)
approximately 50% of those persons filing applications or
resumes are ultimately interviewed. Spriegel and James (in
Ulrich & Trumbo, 1965) revealed that 99% of 852 firms surveyed
had interviewed applicants before hiring. This method may be
combined with tests of aptitude or job-related skills, but at
the administrative level often it is not. According to Jablin
(1975) ,
the interview should serve as the key predictor of
the prospective employee's ability to fit into the
organizational environment. In other words, the
interview would not evaluate the candidate's
occupational skills and aptitudes but rather his/her


16
7. Analyzation of the findings of the above questions to
determine how the frequency of use of each selection method
related to its ability to predict administrative achievement.
8. Derivation of guidelines useful in improving personnel
selection methods for administrators of public community
colleges.
9. Presention of the data in textual form supported by
frequency tables and figures, and supported by tables and
figures reporting means and standard deviations of the
correlational data.
Organization of the Dissertation
A review of the literature pertaining to administrative
personnel selection methods is provided in Chapter Two.
Chapter Three presents a detailed discussion of the meta-
analytic research procedures used in this study, such as
methodology, sources of data, data analysis techniques, and
presentation of the data. A presentation of the findings of
the meta-analysis of the research studies used is contained
in Chapter Four, thus answering the questions posed in the
Statement of the Problem. Chapter Five contains the
guidelines derived from the meta-analysis of the research
studies and the presentation of these guidelines for
application to the selection of community college
administrators.


158
biographical information, peer ratings, and personal
interviews. These three methods showed medium effect sizes
after meta-analysis, meaning that they can be valid predictors
of administrative achievement.
Two selection methods, overall assessment rating and job-
related skills indicators, were ranked high in effect size
ratings, but are seldom used in selection for administration
in higher education. Only one use of an assessment center
process for higher education was identified by this researcher
(Brubaker, 1983) It was not used for selection, but for
career development.
Three additional selection methods, psychological
attributes indicators, aptitude and intelligence measures, and
value systems assessements, also showed medium effect sizes,
but the use of these methods in a public institution may lead
to legal problems if the hiring institution cannot prove that
such tests are specifically job related.
One method, self-appraisal, showed a low effect size.
This was the only one of the nine selection methods that was
proven to be invalid in the prediction of administrative
achievement.
Legal Constraints Upon Personnel Selection
In selecting administrators in public community colleges,
the persons responsible for the search must comply with legal
constraints which affect their ability to apply conclusions
drawn directly from the research. Although personal


130
must be considered, however, before such use is justified" (p.
418) .
Ranked seventh was the method of psychological attribute
indicators with a mean correlation of r = .19 (a = .11) This
correlation was lowered to r = .15 when weighted for sample
size. Thus, this method dropped to eighth place out of the
nine methods. Measures of interests have not been used
extensively in managerial selection (Nash, 1965). Often it
was not used past the first-level of supervisory selection.
The intelligence and aptitude measures selection method
originally displayed a mean correlation coefficient of r = .19
(a = .11) which was strengthened to r = .21 by weighting for
sample size. Its ranking was raised from eighth to fifth
place by this increase in strength. The use of this method
has become increasingly infreguent as the higher levels of
management are reached.
Ninth in original and final rankings was the method of
self-appraisals with a mean correlation coefficient of r = .17
(a = .10). Predictive validity was dropped to r = .10 by
weighting for sample size. Self-appraisal remained the lowest
ranked administrative personnel selection method. This method
has been freguently seen in the form of some resumes and in
the cover letters to resumes.
In Table 4 the frequency of use of each selection method
compared to the relative strength of the mean effect size of


59
over his environment as he advances within the organization"
(p. 81) .
Grant, Katkovsky, and Bray (1967) while at AT&T and
Fordham University (Katkovsky), studied the contributions of
projective techniques to the assessment of management
potential. The study used data collected in the Bell System
Management Progress Study (Bray, 1964). A comparison of
psychological attribute indicators to salary level attained
eight years later showed a nonsignificant correlation. The
researchers found that within this method several of the
projective variables were reliably related to salary progress.
Those variables were dependence and subordinate roles (with
negative relationships), achievement motivation, and
leadership role.
In a concurrent validity study, Ghiselli (1968) evaluated
motivational factors in the success of managers. He
administered a personality inventory to several samples of
corporation middle management personnel. Psychological
attribute indicators related significantly with supervisor
ratings.
As compared with the employed population as a whole,
Ghiselli found that persons in middle management positions
appeared to have a substantially lower desire for security and
for financial reward and a higher desire for self-
actualization. They did not differ from the employed
population in the desire for power over others.


88
researchers found a significant correlation between overall
assessment rating and later administrative achievement.
The International Business Machines (IBM) Corporation
assessment center was patterned after the AT&T model and
included 2.5 days of assessment activity followed by 2 days
of developmental activity. Several studies have reported
results of the IBM assessment center (Hinrichs, 1978; Kraut
& Scott, 1972; Roadman, 1964; and Wollowick & McNamara, 1969).
Wollowick and McNamara (1969) studied the progress of low-
and mid-management males as subjects. The purpose of the
study was to determine the validity of an assessment center
approach in predicting management potential and to determine
the relative value of each of the assessment center
components. An overall assessment rating was determined for
each subject at the conclusion of the two-day assessment
session. The four observers who formed the assessment staff,
and who were operating management personnel at least two
levels above the participants, assigned each subject an
overall assessment rating taking into consideration all of the
variables in the program. According to the overall assessment
rating, each subject was ranked on a 5-point scale rating
potential for advancement within the company (Wollowick &
McNamara, 1969).
These overall assessment ratings were compared, 2.5 to 3.2
years later, to the subjects' administrative levels achieved.
This resulted in a significant correlation. It was evident


173
d.
Predictor:
Job-related skills
indicators
Criterion:
Combination.
Correlation
coefficient (r):
0.49
e.
Predictor:
Personal interviews.
Criterion:
Combination.
Correlation
coefficient (r):
0.20
f.
Predictor:
Biographical information.
Criterion:
Combination.
Correlation
coefficient (r):
0.57
g-
Predictor:
OAR
Criterion:
Combination.
Correlation
coefficient (r):
0.30
4. Carleton, F. 0. (1970). Relationships between follow-up
evaluations and information developed in a management
assessment center. Proceedings of the 78th Annual
Convention of the American Psychological Association, pp.
565-568.
Purpose: To determine if test and rating data generated in
a multiple-technique assessment center hold up well as
predictors of behavioral ratings made several years later.
Type organization: Corporation.
Subjects: 122 managers.
Type validity: Predictive.
Years between variables: 2.5 to 5
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: Yes.
Results:
a* Predictor: Personal interviews.


29
Reliability can be affected by varied administration of
a selection procedure and by the scoring or judging of an
applicant's performance. Inexperienced persons or committees
may deviate from standardized or prescribed procedures in
testing an applicant. Vague observational techniques or
scoring of an applicant's behavior or responses leads to an
unreliability which has nothing to do with the competency of
the candidate. The difficulty of any selection method or
examination affects its reliability. When a method is
difficult, the subjects guess on most of the questions and a
low reliability results. Conversely, if the type of
examination is easy, all subjects have correct responses on
most of the items and only a few more difficult items
discriminate among subjects. This again results in a low
reliability. Thus, any selection committee must keep in mind
the fact that they must adjust the substance of questions
asked in order to discriminate subtle differences between
similar candidates on traits that are difficult to measure.
The achievement of a high degree of reliability in
selection procedures for administrative positions in a
community college may be difficult. In addition to the
problems inherent in assessing traits which elude measurement
by persons who are often inexperienced in the task,
reliability can be affected by the lack of heterogeneity of
the group of applicants. The greater the homogeneity of the
group of subjects being considered, the lower the reliability


107
ratings, (b) turnover, (c) objective performance data, (d)
overall ratings, (e) promotion rate, (f) administrative level,
(g) tenure, (h) suggestion plan activity, (i) pay, (j)
mobility, (k) position level changes, (1) termination, (m)
critical incidents, and (n) combat ratings.
For the purpose of this study these have been condensed
to the following seven measures:
1. administrative level achieved,
2. salary level attained,
3. supervisor ratings,
4. number of years serving in an administrative
position within the same organization,
5. achievement of tenure,
6. objective performance data, and
7. promotion rate.
Research reviewed that defined administrative
achievement as measured by (a) admissions personnel ratings,
(b) success in being hired, or (c) performance success in
subsequent training programs were not used in this study.
These criterion were judged to be too far removed from any
measurement of actual on-the-job performance.
Additional variables. Additional variables were
classified to aid in the meta-analyses of both the content and
the methodology of the research studies used in this
dissertation. In order to understand better the validities


160
required by state law to conduct all search business in
public. The names, resumes, and letters of recommendation of
all candidates must be available to the press and general
public and all committee votes are a matter of public record
(McLaughlin, 1985). Thus, the problem of the reliability of
the data provided within some selection methods may be
suspect, even if that method itself has been proven a valid
predictor of administrative achievement.
Guidelines for Selecting Community College Administrators
In deriving guidelines from the research, it must first
be determined whether such guidelines can be generalized to
aid in the selection of community college administrators.
Hammons and Ivery (1987) found statistically significant
differences between the tasks of five presidents of community
colleges and five chief executive officers of corporations.
Bare (1986) identified two unlike "cores" in college and
university administration. He found that work in the
administrative core was organized in a mechanistic manner with
an emphasis on systematic approaches to problem solving.
Bare found that a more organic form of work organization
prevailed within the technical core. Departmental
chairpersons employed the more consultative leadership style
appropriate to academic decision-making. He stated that
researchers must be aware that the core, rather than the
institution, may be the most fruitful level for theoretical
analysis. Thus, it may be the case that the personnel


21
testing is for a specific competency in that area, not an
overall assessment of the applicant's potential. Most of
these agencies expect personnel to have learned their skills
previous to application and do not intend to take in an
inexperienced person of promise and then train him or her for
an administrative position.
Most of the background studies cited as a basis for
personnel selection in public administration were actually
conducted by private corporations. Many of the selection
practices have been borrowed and adapted from business
research. AT&T's original industrial development of the
assessment center method was so well grounded in research, it
was often casually implied that therefore any assessment
center would pick the "right" person (Ross, 1979, p. 41).
Selection Processes in the Military Services
Although many fields acquire administrators already
trained for the positions they are hired to fill, the military
services often must select people of officer quality and then
pay the additional cost of training them. Thus, the basic
objective in the selection process is to identify measures
which would result in officers entering the force with a high
probability of success. The services must select personnel
not only to fill a particular position as it becomes
available, but also to identify people who have the ability
to be trained repeatedly, sometimes every three years, for a


118
128 correlation coefficients rather than discussing the
studies individually.
Selection methods are abbreviated in the tables as
follows: (a) aptitude and intelligence measures (APT), (b)
personal interviews (INTER), (c) job-related skills indicators
(JOB), (d) psychological attribute indicators (PSYCH), (e)
value systems assessments (VAL), (f) biographical information
(BIO), (g) peer ratings (PEER), (h) self-appraisals (SELF),
and (i) overall assessment ratings (OAR). Outcome criteria
are abbreviated in the tables as follows: (a) supervisor
ratings (SUPRAT), (b) administrative level achieved (ADLEV),
(c) promotion rate (PRO), (d) salary level attained (SAL), (e)
number of years serving in an administrative position within
the same organization (NUMYR), and (f) a combination of any
of the above criteria (COMB).
A General View of the Research
In Tables 1 and 2, descriptive analyses of the research
are displayed. The frequency of reporting of correlations by
year and selection method is indicated in Table 1. The nine
independent variable classifications are those used throughout
this dissertation. The greatest number of correlations (16)
was reported in 1984, while 14 were gathered in 1977, 13 in
1965, and 12 in 1970. Other years yielded an average of four


APPENDIX
ABSTRACTS OF SELECTED STUDIES
1. Bentz, V. J. (1967). Sears, Roebuck studies. In J. P.
Campbell, M. D. Dunnette, E. E. Lawler, & K. E. Weick, Jr.,
Measuring executive effectiveness (pp. 184-187). New York:
Appleton-Century-Crofts.
Purpose: To develop a psychological testing program for the
prediction of executive effectiveness in the Sears
organization.
Type organization: Corporation.
Subjects: 1,375 managers.
Type validity: Predictive.
Years between variables: 11 through 17.
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: Assumed yes.
Results:
a. Predictor: Aptitude and intelligence measures.
Criterion:
Promotion rate.
Correlation
coefficient (r):
0.21
Predictor:
Value systems assessments.
Criterion:
Promotion rate.
Correlation
coefficient (r):
0.22
Predictor:
Psychological attribute indicators.
Criterion:
Promotion rate.
Correlation
coefficient (r):
0.20
170


CHAPTER V
GUIDELINES FOR SELECTION OF ADMINISTRATIVE PERSONNEL
IN PUBLIC COMMUNITY COLLEGES
Introduction
The purpose of this study was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. The findings of research on
administrative personnel selection methods were surveyed and
a meta-analysis was conducted on the data. In this chapter
the results of this meta-analysis are applied to the selection
of administrative personnel in public community colleges. The
results are discussed and guidelines for improving personnel
selection methods for administrators of public community
colleges are presented.
The Need for Valid Personnel Selection Methods
The history of the two-year college movement was
punctuated by a tremendous growth spurt in the 1960s and
1970s, which brought about an increased demand for
administrators to staff the new and growing colleges. During
this period, the two-year colleges did not always have time
to "grow their own" administrators. This historical fact
probably helped to contribute to a relatively open labor
market for two-year college administrators (Twombly, 1987, p.
14) .
155


Range restriction: Low.
Criterion contamination?:
208
Yes.
Results:
a. Predictor: Aptitude and intelligence measures.
Criterion: Salary level attained.
Correlation coefficient (r): -0.01
b. Predictor: Aptitude and intelligence measures.
Criterion: Adminstrative level achieved.
Correlation coefficient fr): -0.01 (N=116)
c. Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.08
d. Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.09 (N=116)
e. Predictor: Biographical information.
Criterion: Salary level attained.
Correlation coefficient (r): 0.17
f. Predictor: Biographical information.
Criterion: Administrative level achieved.
Correlation coefficient fr): 0.11 (N=116)
52. Wollowick, H. B., & McNamara, W. J. (1969).
Relationship of the components of an assessment center to
management success. Journal of Applied Psychology. 53. 348-
352.
Purpose: To determine the validity of an assessment center
approach in predicting management potential and to determine
the relative value of the components of the program.


18
unpublished. Under the impetus of the scientific management
movement of the early 1900s, some efficiency experts of that
time were using simple exams for evaluating applicants for
jobs. They reported fragmentary evidence of validity in the
attempt to justify their activities (Ghiselli, 1973). The
earliest review of an industrial application was in 1915 when
Scott (in Arvey & Campion, 1982, p. 283) reported low
reliability for evaluations given by six personnel managers
who had interviewed the same 36 sales applicants.
During World War I, the large-scale testing both of
soldiers and industrial workers provided stimulation,
methodology, and respectability to the examination of the
utility of examinations in the assessment of occupational
aptitude. In 1923, Freyd (in Guin, 1987) published a 10-step
outline for personnel selection. This outline was so complete
that it differs little from the procedures used in personnel
selection research today. This all led to a post-war surge
of systematic research in personnel testing.
Selection Processes in Business and Industry
Prior to World War I, most businesses were owner-managed
by the individuals who had founded them. But the growth of
the economy since the 1920s brought changes which have
replaced these colorful characters with impersonal
corporations. The function of the administrator had become
such that it required many skills and much knowledge, so that
it often took a number of administrative personnel to assist


23
Selection Processes in Higher Education
In higher education, search committees are often used for
recruiting administrative personnel. These committees provide
for maximum participation in the selection process by a
variety of constituencies within the institution. The
membership of the search committee depends upon the vacancy
to be filled. The committee may include faculty and staff
members, administrators, and, in some cases, students. A
member of the central personnel staff is often an ex-officio
committee member in order to orient the committee to the
proper and legal selection procedures (Sprunger & Bergquist,
1978, p. 116). Usually, the search committee recruits,
screens candidates, checks references, participates in
preliminary interviewing, and recommends a fixed number of
candidates to a designated administrator, who makes the final
selection (Fortunato & Waddell, 1981, p. 107).
The initial screening of candidates has usually been done
by a review of the vitae and job applications. Supplemental
information has usually been obtained by talking to references
and others who knew the individuals and their work. The on-
campus interview has been one of the most important steps in
the selection process. The manner in which this interview is
conducted has usually been critical to the success of the
recruitment-selection process (Sprunger & Berquist, 1978, p.
119). Webster (cited in Grove, 1981, p. 56) emphasized that
the interviewer must understand what behavior is required of


48
under stress, especially when conditions suddenly change or
when competition stiffens.
Research used in this study. A total of 10 research
studies were found that used job-related skills indicators as
a predictor variable and met the other delimitations of this
dissertation. These 10 studies yielded 12 correlation
coefficients which ranged in strength from r = .02 to r = .49.
Samples ranged in size from 31 to 8,885 persons.
In 1965, Tenopyr and Ruch (in Campbell et al., 1970)
studied the relationship between job-related skills and salary
level attained in a concurrent validity study. Subjects were
production managers at North American Aviation. They were
given a test designed to assess a supervisor's ability to
handle human relations problems. Although a significant
finding resulted, the researchers concluded that this
correlation was "useful" but not "overwhelming" (Campbell et
al., 1970, p. 193) .
Dicken and Black (1965) studied the validity of clinical
interpretations of an objective test battery in an industrial
setting. A sample of first-line supervisors in a
manufacturing company was given tests of job-related skills.
These tests were administered to the participants individually
and in groups. A significant correlation was found between
job-related skills indicators and salary level attained after
3.5 years.


74
management success. They found a significant relationship
between biographical information and salary level attained 10
years later.
Gantz, Erickson, and Stephenson (in Miner, 1977) conducted
a concurrent study of research scientists and engineers for
the purpose of gaining "more theoretical understanding of why
some persons in a research and development population achieve
promotion, particularly into formally designated supervisory
or managerial positions, and other persons do not" (p. 18).
The organization was a federally funded public laboratory.
The independent variable, the selection method of biographical
information, was found to be not related to either promotion
rate or supervisor ratings.
Hinrichs (1978) found a significant relationship between
biographical information and administrative level achieved
eight years later for a sample of 30 managers. The resulting
correlation coefficient was slightly higher than that for the
Overall Assessment Rating.
Turnage and Muchinsky (1984) sought to examine the
predictability of assessment center evaluation versus
traditional measures in forecasting job success. As part of
the study, they separately compared biographical information
with supervisor ratings and salary level attained up to
several years later. These variables were insignificantly
related for the mostly nonmanagement personnel tested. The


217
McIntyre, K. E. (1966). Selection of Administrators.
Columbus, OH: University Council for Educational
Administration.
McLaughlin, J. B. (1985). From secrecy to sunshine: An
overview of presidential search practice. Research in
Higher Education. 22.(2), 195-209.
Meyer, H. H. (1956). An evaluation of a supervisory
selection program. Personnel Psychology. 9, 499-513.
Meyer, H. H. (1970). The validity of the in-basket test as
a measure of managerial performance. Personnel
Psychology. 23. 297-307.
Miner, J. B. (1977). Motivation to manage: A ten year
update on the "studies in management education" research.
Atlanta: Organizational Measurements Systems Press.
Miner, J. B. (1978). Twenty years of research on role-
motivation theory of managerial effectiveness. Personnel
Psychology. 31. 739-760.
Mitchel, J. O. (1975). Assessment center validity: A
longitudinal study. Journal of Applied Psychology. 60,
573-579.
Moses, J. L. (1972). Assessment center performance and
management progress. Studies in Personnel Psychology.
4(1), 7-12.
Moses, J. L., & Boehm, V.R. (1975). Relationship of
assessment center performance to management progress of
women. Journal of Applied Psychology. 60, 527-529.
Nash, A. N. (1965). Vocational interests of effective
managers: A review of the literature. Personnel
Psychology. 18. 21-37.
Nash, A. N. (1966). Development of an SVIB key for
selecting managers. Journal of Applied Psychology. 50,
250-254.
Nickels, J. B., & Renzaglia, G. A. (1958). Additional data
on the relationships between expressed and measured
values. Journal of Applied Psychology. 42, 99-104.
Northcott, C. H. (1960). Personnel management: Principles
and practices. London: Sir Isaac Pitman.


157
responsible for the search described the position to be filled
and wrote an appropriate position description. The position
opening was often advertised in the Chronicle of Higher
Education. Whatever the method used, the rule has been for
each applicant to complete an application form, supply
transcripts, and send three letters of recommendation
(Scigliano, 1979). Thus, the "first cut" made by the search
committee has been based upon the selection methods of
biographical information and peer ratings in the form of the
completed application blank and recommendations. Those
applicants who passed this stage were asked to supply previous
employer experience affidavits, copies of all licenses or
certificates, and transcripts from all institutions attended
(Scigliano, 1979).
According to Huegli and Eich (1979), the most crucial
decisions reached in the search process were those which
rested on the results of on-site interviews with candidates.
Tremendous weight has traditionally been placed on the results
of the interview. The candidate pool has usually been pared
down to only a handful of applicants and great pressure was
on to select one of them or begin the whole process over
again. After this final screening, the recommendation was
made by the committee to the chief administrative officer and
board (Huegli & Eich, 1979).
Thus, according to the findings of the present study, the
search process mainly involves the selection methods of


203
Criterion contamination?: Yes.
Results:
Predictor: OAR.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.24.
46. Stahl, M. J. (1983). Achievement, power, and
managerial motivation: Selecting managerial talent with the
job choice exercise. Personnel Psychology. 36, 775-789.
Purpose: To determine if high managerial motivation is
related to high needs for achievement and power.
Type organization: Corporation.
Subjects: 45 mid-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.36
47. Steel, R. P., & Ovalle, N. K. 2d. (1984). Self
appraisal based upon supervisory feedback. Personnel
Psychology. 37, 667-685.
Purpose: To examine the relationship between self
appraisals and objective criteria of job performance.
Type organization: Corporation.
Subjects: 401 mid-level managers.
Type validity: Concurrent.


136
Aptitude and intelligence measures. A total of 17,482
subjects were included in 15 studies which yielded 21
correlation coefficients involving aptitude and intelligence
measures. The mean correlation of r = .19 (a = .11) was
raised to r = .21 when weighted according to sample size.
Correction for unreliability caused the weighted mean
correlation to be raised from r = .21 to r = .30.
Peer ratings. A total of 3,413 subjects were included
in 8 studies which yielded 10 correlation coefficients
involving peer ratings. The original mean correlation of r
= .28 (a = .14), weighted for sample size, was r = .20. The
adjustments for error of measurement caused the mean
correlation to be raised from r = .20 to .28.
Biographical information. A total of 3,326 subjects were
included in 8 studies which yielded 11 correlation
coefficients involving biographical information. The mean
correlation of r = .23 (a = .16) was depressed to r = .18 when
weighted for sample size. Correction for unreliability caused
the mean correlation to be raised from r = .18 to r = .26.
Psychological attribute indicators. A total of 7,503
subjects were included in 22 studies which yielded 37
correlation coefficients involving psychological attribute
indicators. The mean correlation of r = .19 (o = .11) was
depressed to r = .15 when weighted for sample size.


95
Summary and Critique
The job of the administrator has changed considerably.
It is perceived as more complex than ever before, requiring
multiple skills and abilities, and is affected by many outside
forces. Some selection methods of administrative personnel
are used in sequence as screening devices, whereas other
methods consider many different personal qualities in concert
in the evaluation of a job candidate.
The use of one or several selection methods is a function
of the importance of the job, the money and the time available
for selection, and the accuracy of any one method compared
with a combination of methods. Research studies can be cited
that arrive at opposite conclusions. In some cases, one
selection method can be superior to several methods. In other
cases a combination of methods is superior to any one.
Theoretically, several methods can never be inferior to one
if they are combined on a statistically sound basis, but this
is rarely done. A valid method can be reduced in
effectiveness by the addition of an inferior one, except when
the less valid method measures attributes not otherwise
detected (Mandell, 1964, p. 22).
The quality of validity, addressed in this study, has been
studied in personnel selection research and literature for
decades. Often a method was "sold" based upon what it was
claimed to measure, without any evidence that the claims were
justified. In some cases, the endorsement of a user who had


152
Korman (1968) found that peer ratings could be highly
predictive of later promotion rate. The present study showed
that peer ratings were significantly related to later
administrative achievement. According to Korman (1968), "when
one sees how consistently this is an effective predictor of
officer success, it is surprising that the method has not been
studied more thoroughly in the industrial context" (p. 313) .
Biographical information ranked third from the lowest,
but was still significant, as a selection method. Hinrichs
(1969) concluded that using biographical information was as
effective as using an assessment center process in identifying
management potential. This study does not bear this
conclusion out, however, in light of the possibility of
obtaining higher correlation, further research for moderator
variables within this method may be warranted.
The selection method of psychological attribute
indicators was significant in predicting administrative
success. However, it was the lowest ranked significant
method. This was a poor showing considering that 29% of the
research effort has been put toward understanding this method.
Thus, this study substantiated Korman's (1968) view, 20 years
later, that "there seems little reason for thinking that we
have learned much about the psychological variables indicative
of managerial behavior insofar as these variables are
determinable by objective personality inventories" (p. 302).


150
pertains to the predictive validity of each method. Although
a sufficient number of studies and correlation coefficients
were found to gain an accurate assessment of the predictive
validities of the nine selection methods, insufficient
research/reporting was found on details within each study
needed to develop moderator variables for in-depth research.
Sufficient data were provided to complete the review of
several methodological concerns. A summary of these results
is presented here.
Content
The overall assessment rating displayed a decidedly
higher correlation than all other methods. This correlation
was derived from a combination of both clinical judgments by
trained assessors or psychologists, and statistical synthesis
of these judgments (Korman, 1968). Thus, this method has
validity-enhancing aspects rarely found in other methods.
Job-related skills indicators, although lower in mean
correlation than the overall assessment rating, have been
attributed with contributing a great deal to the overall
assessment rating in terms of validity (Howard, 1974). This
study supported this view.
Values systems assessments showed a relatively high
correlation with administrative success. The results of this
study supported the findings of England and Lee (1974) that
"values patterns were significantly predictive of managerial


7
many colleges and universities more searches are underway for
administrators than for faculty. In addition, affirmative
action and equal opportunity require that administrative
searches be more public, more extensive, and more expensive
than ever before. All this expenditure of time and resources
demands that more attention be paid to the selection process
(Kelly & Nelson, 1978) .
Much has been written about the problems of selecting
administrators in higher education, however, little research
has been done on the topic. A change in the procedures now
followed may be warranted. The results of many years of
research in personnel selection methods in other fields have
been usefully applied in this case. The conclusions of this
research helped to determine guidelines for improving the
selection of administrators for our public junior and
community colleges.
Research in administrative personnel selection methods has
been done in corporations, the military services, public
administrative agencies, and in education. The present study
was needed in order to synthesize this research and apply it
to the selection of community college administrators. The
conclusions of the research, when applied to this problem,
have indicated those selection methods which are useful in
predicting administrative achievement.


213
Ghiselli, E. E. (1973). The validity of aptitude tests in
personnel selection, Personnel Psychology. 26. 461-477.
Glass, G. (1976). Primary, secondary, and meta-analysis of
research. Educational Researcher. 5, 3-8.
Glass, G. (1977). Integrating findings: The meta-analysis
of research. Review of Research in Education. 5,
351-379.
Glass, G. McGaw, B., & Smith, M. L. (1981). Meta-analvsis
in social research. Beverly Hills: Sage.
Goodstein, L. D., & Schrader, W. J. (1963). An empirically-
derived managerial key for the California Psychological
Inventory. Journal of Applied Psychology. 47. 42-45.
Grant, D. L., & Bray, D. W. (1969). Contributions of the
interview to assessment of management potential. Journal
of Applied Psychology. 53. 24-34.
Grant, D. L., Katkovsky, W., & Bray, D. W. (1967).
Contributions of projective techniques to assessment of
management potential. Journal of Applied Psychology. 51,
226-232.
Grimsley, G., & Jarrett, H. F. (1975). The relation of past
managerial achievement to test measures obtained in the
employment situation: Methodology and Results II.
Personnel Psychology. 28., 215-231.
Grove, D. A. (1981). A behavioral consistency approach to
decision making in employment selection. Personnel
Psychology. 34. 55-64.
Guin, R. M. (1987). Changing views for personnel selection
research. Personnel Psychology, 40, 199-213.
Haggerty, H. R. (1953). Personnel research for the U.S.
Military Academy 1942-1953. U.S. Army Personnel Research
Board Technical Research Report 1077. Wash., D.C.: U.S.
Army Personnel Research Office.
Haggerty, H. R. (1963). Status report on research for the
U.S. Military Academy (Cadet Leaders Task). Wash., D.C.:
U.S. Army Personnel Research Office
Halpin, A. W. (1954). The leadership behavior and combat
performance of airplane commanders. Journal of Abnormal
and Social Psychology. 49. 19-22.


66
One of the most recent and extensive treatments of the
relationships of achievement to power is presented in Veroff's
book of readings in honor of McClelland.
Achievement motivation directs people to meeting
socialized standards of excellent performance and thus
to highly efficient task-centered strivings, whereas
power motivation directs people to doing whatever
draws most attention to their own effect on the world.
The two motives seem to be fused in instances where
the standard of excellence is to win in a social
competitive activity or to solve a problem that will
be given a great deal of recognition. (Veroff, 1982,
p. 100)
Value Systems Assessments as a Selection Method
The selection method of values systems assessments
measures the strength of a person's economic, aesthetic,
social, political, and religious concerns (Hinrichs, 1978).
According to England and Lee (1974),
a personal value system is viewed as a relatively
permanent perceptual framework which shapes and
influences the general nature of an individual's
behavior. Values are similar to attitudes but are
more ingrained, permanent, and stable in nature; they
are also more general and less tied to any specific
referent than is the case with many attitudes.
(p. 412)
Most personnel selection research studies which dealt with
the independent variable of psychological attribute indicators
verged on or overlapped the selection methods using values as
an indicator. The studies used in this section were those
that this researcher determined were predominantly concerned
with values as defined above.
Research used in this study. A total of 4 research
studies were found which used value systems assessments as a


76
successfully predicted later criteria of performance.
Williams and Leavitt (1947) found that peer ratings were
better predictors of long-term success in the Marine Corps
than were superiors' ratings. Much military research was
contributed by Hollander (1954, 1956, 1957, 1964, 1965) and
the evidence was consistently favorable. La Gaipa (1960), in
the Development of the Officer Candidate Biographical
Information Blank, sampled Naval officers in both shore duty
and fleet duty. He found a significant correlation between
peer ratings and later performance criterion.
There has also been evidence of the utility of peer
ratings from other spheres of activity, as Weitz found a
relationship of r = .40 between peer nominations and later
ratings of life insurance agents in a supervisory position
(1958). As Hollander has stated, "when employed with
discrimination, it [the peer rating method] can provide a
unique contribution to evaluation" (1965, p. 434).
Research used in this study. A total of 8 research
studies were found which used peer ratings as a predictor
variable and met the other delimitations of this dissertation.
These studies yielded 10 correlation coefficients which ranged
in strength from r = .04 to r = .53. Samples ranged in size
from 40 to 799 persons.
Haggerty (1963), in two separate predictive studies,
explored the predictive ability of peer ratings obtained on
Military Academy Cadets. The first group of men showed a


129
been rarely used outside an assessment center program. The
measurement of job-related skills indicators usually involves
rating a performance on tests of job skills, and this involves
costs in both time and money in setting up reliable
performance tests of administrative skills.
Biographical information correlation coefficients showed
a decrease in predictive strength from r = .23 (a = .16) tor
= .18 after weighting for sample size. Thus, it slipped from
fourth place to seventh place in predictive validity. This
is a mediocre ranking for such a frequently used selection
method.
The selection method of personal interviews, was
strengthened when weighted for sample size. Originally ranked
fifth with a correlation of r = .20 (a = .10), it was raised
to r = .24, fourth place, when weighted. Thus, at least one
of the three most frequently used selection methods ranked in
the top half in predictive validity.
The selection method of values systems assessments was
strengthened when weighted for sample size. The mean
correlation coefficient was raised from r = .20 (a = .09) to
i = .25. Thus, this method rose from sixth to third in
ranking by predictive strength. According to England and Lee
(1974), "it is evident that value patterns are predictive of
managerial success and could be used in selection and
placement decisions. A number of legal and ethical issues


BIOGRAPHICAL SKETCH
Betty-June Eldridge was born in Maryland on June 17,
1947. She attended public schools in Montgomery County,
Maryland, from elementary school through junior college. She
completed her undergraduate studies at West Chester State
College, Pennsylvania, in 1970. She received an M.Ed. degree
from the University of Florida in 1983, and has pursued her
doctoral program there since that date.
Most of her professional career has been spent as an
intelligence analyst with the Central Intelligence Agency,
serving at the National Photographic Interpretation Center in
Washington, D.C. She is currently employed by the Defense
Intelligence Agency as an intelligence operations specialist
in the United States Special Operations Command, Tampa,
Florida. She and her family reside in Brandon, Florida.
222


195
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Job-related skills indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.29
34. Miner, J. B., & Miner, M. G. (1977). Motivation to
manage: A ten year update on the "Studies in Management
Education" research. Atlanta: Organizational Measurement
Systems Press.
Purpose: To study the relationship between two
psychological attribute measures and indexes of management
success.
Type organization: Corporation.
Subjects: 101 high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a- Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.10
b. Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.13


Legal constraints placed upon personnel selection in public
organizations were considered. These guidelines included (a)
establishing assessment centers for selection and training,
(b) increasing the job-relatedness of selection procedures,
and (c) using statistical methods to simulate overall
assessment ratings.
IX


substitute literary exposition for quantitative
rigor. The proper integration of research
requires the same statistical methods that are
applied in primary data analysis. (1976, p. 6)


181
Years between variables: 4
Assessment center?: No.
Range restriction: Medium.
Criterion contamination?; No.
Results:
Predictor; Psychological attribute indicators.
Criterion; Promotion rate.
Correlation coefficient (r); 0.39
14. Ghiselli, E. E. (1966). The validity of a personnel
interview. Personnel Psychology. 19. 389-394.
Purpose; To examine the proposition that superficial data
pertaining to what an individual has done has little power
in predicting job "success".
Type organization: Corporation.
Subjects: 275 applicants for low-level executive positions.
Type validity; Predictive.
Years between variables: 3
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
Predictor: Personal interviews.
Criterion: Number of years serving in an
administrative position within the same
organization.
Correlation coefficient (r): 0.35
15. Ghiselli, E. E. (1968). Some motivational factors in
the success of managers. Personnel Psychology. 21. 431-440.


132
showed a medium effect size. However, the validities of these
selection methods have not been corrected for sampling error
or reliability, and these corrections, done in the meta
analysis, were found to increase the effect sizes of the mean
correlations.
Selection Methods Significantly Related to Administrative
Achievement: A Meta-analvsis
The results presented in this section were derived by a
meta-analysis of the 128 correlation coefficients sorted by
independent variable, selection method. Formulas (4) through
(8), presented in Chapter III, were used.
Eight of the nine personnel selection methods were
significantly related to achievement in administrative
positions. The one method which was not significantly related
was self-appraisals. The rankings of the validities of the
selection methods remained the same after these corrections
for sampling error and reliability were completed in the meta
analysis (see Table 5). The fact that the relative rankings
of the selection methods remained the same after these
corrections were made is understandable. Correction for
sampling error had little effect due to the facts that (a) the
amount of sampling error is inversely proportional to the
sample sizes, and (b) the sample sizes, when combined across
studies, were large. Adjustments for reliability of
independent and dependent variables effected the strengths of


Type validity: Predictive.
Years between variables: 10
187
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a. Predictor; Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.19
b. Predictor: Biographical information.
Criterion: Salary level attained.
Correlation coefficient (r^: 0.28
22. Heneman, H. G. Ill (1974). Comparisons of self- and
superior ratings of managerial performance. Journal of
Applied Psychology. 59, 638-642.
Purpose: To identify broad dimensions of managerial
performance, obtain self- and superior performance ratings,
and assess these ratings in terms of leniency, restriction
of range, halo, and convergent-discriminant validity.
Type organization: Combination.
Subjects: 102 MBA graduates.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Self-appraisals.


159
interviews probably will continue to be the most important
part of the process (Webb, 1983), testing has often been the
most controversial part. From Griggs v. Duke Power Co., 1971
(in Wisner, 1975) several rules for testing in personnel
selection were derived. Tests must be used only as
supplements to other selection methods and all tests must be
validated within the organization in order to be considered
job-related (Webb, 1983).
The case of Edward L. Kirkland et al.. Plaintiffs V. New
York State Department of Correctional Services et al..
Defendants (73 LIV. 1548, 1974) strengthened the requirement
for job-relatedness of selection tests. The court mandated
that every measure available be used to assure that tests
would not discriminate against any group of persons. Through
its institution, in 1972, of the Higher Education Guidelines,
the federal government called for an extension to recruitment
and hiring activities beyond the generally perceived network
approach. Dingerson, Rodman, and Burns (1985) sought to
assess what changes have occurred in hiring procedures since
implementation of the Higher Education Guidelines. They found
that there has been no substantial change in the hiring of
candidates from underrepresented populations for nationally
advertised positions. There is simply no evidence that change
is underway (Dingerson, Rodman, & Burns, 1985).
Sunshine laws are another source of constraint in hiring
in public community colleges. Some search committees are


121
overall assessment rating at least one measure of job-related
skills was also recorded.
The remaining administrative personnel selection methods
were represented relatively egually in the research with
between 5 and 11 correlation coefficients each recorded in
this study. Interestingly, these "least studied" methods
contain all three of the methods which are used most
freguently in actual administrative personnel selection.
A review of the data derived from the research studies
revealed that 91% of the administrative personnel selection
research was done within corporations, none within public
administration agencies, 6% within the military services, and
3% within educational institutions.
In Table 2, descriptive data are presented for the
distributions of the validity coefficients before weighting
for sample sizes or correcting for sampling error and
reliability. The mean correlation, median, standard
deviation, and range are shown for each selection method. The
mean and standard deviation were calculated using formulas (1)
and (2) respectively from Chapter III.
It is clear that the correlations which make up each mean
correlation have a wide range. Typically the standard
deviations from the mean egual approximately 50% of the mean.
For most selection methods, the distributions were fairly


CHAPTER I
INTRODUCTION
The purpose of this dissertation was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. The research method used was a
meta-analysis of administrative personnel selection research
done in the United States in the past 24 years.
Administrators in our public community colleges must possess
both extensive knowledge and sound judgment to meet the
complex challenges of the years ahead. The extent to which
a community or junior college meets its challenges largely
depends upon the qualities of its future leaders. Thus, these
integral people must be chosen carefully if they are to be
properly matched to the demands of their positions. Those
concerned with the placement of administrators in higher
education can no longer afford the luxury of using arbitrary
selection methods. This is true in times of surplus, and even
more true in times of shortage, budget cuts, and retrenchment
(Mandell, 1964). Rapid change itself has mandated a
continuous reevaluation of personnel selection procedures.
Administrators, well-qualified in the past, may not be
prepared to face challenges of the job in the late 1980s or
1990s. According to Sharp (1984), community colleges grew so
1


31
performance of the behavior in question. If the criterion
does not reflect the attribute under study, it would be
meaningless to use it as a basis for validating another
instrument. In the case of this study, the outcome criteria
of supervisor's ratings, longevity in a position, and
salary/position or rank attained were deemed to be adequate
indicators of success in an administrative position.
A second characteristic is that a criterion must be
reliable. The criterion must be a consistent measure of the
attribute over time or from situation to situation. The
criteria chosen for this study, rank, position, and salary
achieved, have generally been considered to be related to
success. Length of time in an administrative position within
one organization may have been a mark of high achievement in
the past more than it is today. Now many administrators look
upon great longevity as an indicator of stagnation rather than
of achievement. So this indicator may not be as relevant to
later studies as to earlier ones.
Finally, a valid criterion should be free from bias. The
scoring should not be influenced by any factors other than
actual performance. The main problem in relation to personnel
selection research is a form of bias called criterion
contamination. This occurs when an individual's score on the
criterion, such as supervisor rating, is influenced by the
scorer's knowledge of the subject's predictor score.
Contamination of the criterion can be prevented by not


192
Subjects: 365 mid- and high-level managers.
Type validity: Predictive.
Years between variables: 4 through 7
Assessment center?: No.
Range restriction: High.
Criterion contamination?: Yes.
Results:
Predictor: Aptitude and intelligence measures.
Criterion: Promotion rate.
Correlation coefficient fr): .02
29. Kraut, A. I., & Scott, G. J. (1972). Validity of an
operational management assessment program. Journal of
Applied Psychology. 56, 124-129.
Purpose: To study the predictive validity of an assessment
center.
Type organization: Corporation.
Subjects: 432 nonmanagement employees.
Type validity: Predictive.
Years between variables: 1 through 5
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
Predictor: OAR
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.25
30. Lawler, E. E. Ill (1967). The multitrait-multirater
approach to measuring managerial job performance. Journal
of Applied Psychology. 51, 369-381.


93
Turnage and Muchinsky (1984) conducted a study based upon
data from a one-day assessment center developed by an
unidentified large manufacturing firm. The subjects were 799
nonmanagement employees who were subsequently promoted to
supervisory positions. Two overall assessment ratings were
provided using two separate outcome criteria, salary level
attained and supervisor ratings. Based on these, and other
findings in the study, the researchers concluded that there
appeared to be no appreciable relationship between how one is
evaluated in an assessment center and how one performs on the
job. They stated that "there is not a strong relationship
between job performance and promotion" (p. 600).
A number of reasons were surmised by the researchers for
the lack of correlation. Methodological factors such as low
criterion reliability, low predictor reliability, severe
restriction of range, marked skew in the data, procedural
inconsistencies, lack of comparability across assessment
groups, and errors in data collection were cited as problems
within this study (Turnage & Muchinsky, 1984).
Rankin (1981) conducted a predictive validity study in an
assessment center program in a military organization.
Subjects were low-level management civilian employees. No
relationship was found between overall assessment ratings and
a combination of outcome measures. In fact, Rankin (1981)
stated that,
the most significant correlation between overall
ratings was found within the group of assessees with


216
Kraut, A. I. (1969). Intellectual ability and promotional
success among high level manager. Personnel Psychology.
22, 281-290.
Kraut, A. I., & Scott, G. J. (1972). Validity of an
operational management assessment program. Journal of
Applied Psychology. 56, 124-129.
La Gaipa, J. J. (1960). Development of Officer Candidate
Biographical Information Blank: I. Validation of some
experimental inventories. Bureau of Naval Personnel
Technical Bulletin 60-9. June.
Lawler, E. E., III. (1967). The multitrait-multirater
approach to measuring managerial job performance, Journal
of Applied Psychology. 51, 369-381.
Levine, E. L. (1978). Self-assessment for personnel
selection: Bane or boon? Public Personnel Management.
7(4), 230-235.
Levine, E. L., & Flory, III, A. (1975). Evaluation of job
applications A conceptual framework, Public Personnel
Management. 4(6), 378-385.
Light, R. J., & Smith, P. (1971). Accumulating evidence:
Procedures for resolving contradictions among different
research studies, Harvard Educational Review. 41. 429-
471.
MacKinney, A., & Wolins, L. (1960). Validity information
exchange. Personnel Psychology. 13., 443-447.
Mandell, M. M. (1964). The selection process: Choosing the
right man for the job. New York: American Management
Association.
Mayfield, E. C. (1970). Management selection: Buddy
nominations revisited. Personnel Psychology. 23. 377-
391.
McClelland, D.C., & Boyatis, R.E. (1982). Leadership motive
pattern and long-term success in management. Journal of
Applied Psychology. 67. 737-743.
McGregor, D. (1960). The human side of enterprise. New
York: McGraw-Hill.


123
commonly used in personnel selection overall and in each
specific areas of (a) corporations, (b) public administration,
(c) military services, and (d) education. The frequency of
use of each selection method was compared to its ability to
predict administrative success. The statistical data in this
section were derived by calculating mean correlations across
selection methods and by then weighting these correlations by
sample size using formula (3) presented in Chapter III. The
complete meta-analysis of each method, in which corrections
were made for sampling error and reliability, is presented in
a later section.
Most Commonly Used Administrative Personnel Selection Methods
The most commonly used selection methods for
administrative personnel remain essentially the same
throughout all types of organizations considered.
Biographical information was used to weed out those who did
not meet minimum
qualifications
for
the
position.
Self
appraisals were
often provided
for
administrative
level
positions in the
form of cover
letters
to resumes.
The
military services and public administrative agencies were
likely to follow up with tests of aptitude and intelligence.
Almost all organizations included a personal interview. Peer
ratings were often provided in the form of references when
required. Measures of psychological attributes, values


92
carried out by the Standard Oil Company of New Jersey (SONJ).
The long-term staff study was extensively reported in Campbell
et al. (1970).
Through this study, the management of SONJ sought to
determine how employees who possess the potential to be
successful in management can be identified early in their
careers. Management personnel of SONJ were more hopeful of
identifying young men with potential for general management
positions and were less concerned with pinpointing those with
potential narrowly oriented toward specific functional areas.
A significant relationship was found between overall
assessment ratings and a combination of outcome variables in
the concurrent study.
Schmitt, Noe, Meritt, and Fitzgerald (1984) sought to
evaluate the use of the assessment center approach when
applied in an educational setting to select secondary and
elementary school administrators. Using an assessment center
designed by the National Association of Secondary School
Principals (NASSP) and the American Psychological Association,
they found a significant relationship between overall
assessment rating and subsequent supervisor ratings. However,
the number of years between the measurement of variables was
limited due to the fact that average job tenure of the
promoted participants was only 13 months. This study
represented the first validation of the use of the assessment
center process in the selection of school administrators.


87
Bray and Grant (1966) measured personal characteristics
hypothesized to be of importance either in developmental
change in early adulthood or success in business management.
The overall assessment rating was significantly correlated
with both salary level attained and administrative level
achieved 8 years after assessment. This was one of few
studies in which criterion contamination was eliminated. The
researchers concluded that "no single characteristic
determines progress in management" (Bray & Grant, 1966).
Moses (1972) studied the relationship between assessment
center processes and managerial achievement. Using a large
sample of assessees of an AT&T one-day assessment center,
Moses found a significant relationship between overall
assessment rating and administrative level achieved.
Moses and Boehm (1975) studied the relationship between
the assessment center process and subsequent progress in
management for women in the Bell System. Assessment center
data were obtained on 4,846 nonmanagement women from 1963
until 1971. The conclusion the researchers drew in this study
was that the assessment center process predicted the future
performance of women as accurately as it did that of men.
Huck and Bray (1976) tested the validity of an assessment
center process on a population different from the population
of the original Management Progress Study. Using a relatively
small sample of females who had been assessed previously, the


182
Purpose: To study the relationship between the strength of
desire for self-actualization and performance among
managers.
Type organization: Corporation.
Subjects: 233 mid-level male managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient fr): 0.22 (N=89)
b. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient fr): 0.30 (N=21)
c. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r^: 0.47 (N=20)
d. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.22 (N=22)
e. Predictor: Psychological attribute indicators.
Criterion: Number of years serving in an
administrative position within the same
organization.
Correlation coefficient (r): 0.19 (N=81)


201
Type organization: Corporation.
Subjects: 96 supervisory position incumbents.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Self-appraisals.
Criterion: Supervisor ratings.
Correlation coefficient fr): 0.25
43. Rankin, K. K. (1981). A predictive validity study of
an assessment center for research and development superiors.
Wright-Patterson AFB, OH: Air Force Institute of Technology.
Purpose: To study the relationships between assessment
center ratings and several measures of administrative
achievement.
Type organization: Military.
Subjects: 67 civilian nonmanagement personnel.
Type validity; Predictive.
Years between variables: 6
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
Predictor: OAR
Criterion: Combination.


205
49. Thornton, G. C. (1968). The relationship between
supervisory- and self-appraisals of executive performanace.
Personnel Psychology. 21, 441-455.
Purpose: To investigate the relationship between
supervisory-perceptions and incumbent self-perceptions of
the performance of executive personnel.
Type organization: Corporation.
Subjects: 64 high-level administrators.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Self-appraisals.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.24
50. Turnage, J. J., & Muchinsky, P. M. (1984). A
comparison of the predictive validity of assessment center
evaluations versus traditional measures in forecasting
supervisory job performance: Interpretive implications of
criterion distortion for the assessment paradigm. Journal
of Applied Psychology. 69. 595-602.
Purpose: To examine the ability of assessment center
evaluations to predict actual job perfomance criteria and to
compare the predictability of assessment center evaluations
versus traditional measures in forecasting job success.
Type organization: Corporation.
Subjects: 799 mostly nonmanagement employees.
Type validity: Predictive.
Years between variables: 1 month to several years.
Assessment center?: Yes.


61
they found a significant correlation between psychological
attribute indicators and salary level attained 10 years later.
Subjects were former MBA graduates who had achieved management
positions in corporations.
Grimsley and Jarrett (1975) studied the relationship
between managerial achievement and test measures obtained in
the employment situation. This concurrent study used two
samples of mid- and top-level corporation managers. The
results of several measures of psychological attributes were
compared to the outcome criteria of administrative level
achieved. Insignificant correlation coefficients were found
with each of the two samples. The researchers concluded that
"the differences in test scores of more or less successful
managers result from fundamental differences in mental ability
and personality rather than the influence of on-the-job
experience" (p. 226).
Gantz, Erickson, and Stephenson (in Miner, 1977) sought
to determine if certain psychological attributes could be
causes of managerial success. Using a sample of research
scientists and engineers in a federally funded laboratory,
several measures of psychological attributes were
administered. These proved to be insignificantly related to
both supervisors' ratings and promotion rate. The researchers
concluded that perceived creativity was rewarded by promotion
into leadership roles, including formal managerial roles where


102
the same variable. Thus, "variation in reliability
contributes to the variation in correlations across studies
if the review covers all studies using a given predictor
rather than a fixed test" (p. 91).
Although hundreds of studies exist and many claims are
made, the results of existing administrative personnel
selection research are unsettlingly discrepant. Fifty-two
studies met the fairly strict delimitations set forth in this
research. Several studies yielded more than one correlation
coefficient. A total of 128 correlation coefficients were
derived from the 52 studies. Care was taken to maintain
independence of samples within and between studies. Where
subsamples were analyzed within a study, and the relationship
between the same variables was tested in each subsample, these
subsamples were combined to yield an average correlation
coefficient. The few negative correlations found were treated
as positive for the reason that this study deals with the
strengths, not the directions, of the methods to predict
achievement. The strengths of correlations between selection
methods and administrative achievement in the research studies
reviewed ranged from r = .00 to r = .65. Their mean
correlation was r = .22 which increased to r = .25 when the
correlations were weighted for sample size according to
formula (3) in this chapter.
The intent of meta-analysis is to help understand what
causes variance in previous studies and to provide a much more


80
manufacturing firm. The outcome data were gathered from 1
month to several years after assessment from a sample of
previous assessees. None of the peer ratings was significant.
Several possible explanations for the low validity
coefficients were given. These included poor experimental
control, insufficiently trained assessors, aggregation of data
over a 4-year period, and the effects of intervening
variables.
Conclusions. There appears to be a reasonable basis to
contend that peer nominations do provide distinctive
prediction of performance at a considerably later time. Peer
evaluations are relevant because peers are situated to
evaluate how a person performs in terms of the lateral
relationships in working toward organizational goals.
Further, peers often see the worker at times when his superior
is not viewing his behavior and, therefore, they may see
aspects of his behavior of which the superior is not aware
(Lawler, 1967) However, attention must be paid to the
question of the willingness of participants to rate each other
accurately if they know full use will be made of peer ratings
for administrative purposes (Roadman, 1964). A problem Lawler
(1967) acknowledged was that if the peer rater knew that his
or her opinion was "going to count" in an employment setting,
ratings may lose their validity, "particularly if a situation
exists where an individual's self-interests might be best
served by distortion of the peer ratings" (p. 379) According


128
Table 3
Rankinas
of Validities of Selection
Methods With
and Without
Weiahtincr for Sample
Size
Rank
Method
Unweighted r
Method
Weighted r
1
OAR
. 32
OAR
. 37
2
PEER
.28
JOB
.30
3
JOB
.25
VAL
.25
4
BIO
.23
INTER
.24
5
INTER
.20
APT
.21
6
VAL
.20
PEER
.20
7
PSYCH
. 19
BIO
. 18
8
APT
. 19
PSYCH
. 15
9
SELF
. 17
SELF
. 10
truthfulness in references might bring, there may be a large
gap between the validity of this method when used in research
as opposed to its use in actual hiring.
Job-related skills indicators were ranked third in
strength of validity. The correlation coefficient of this
method was strengthened from i = .25 (o = .13) to r = .30 by
weighting for sample size. Thus, it was ultimately ranked as
the second most valid method after weighting. This method has


51
Moses (1972) studied the relationship between job-related
skills indicators and administrative level achieved seven
years later. The job-related skills indicators were
performance-type tests by which assessors determined a
subject's organizing, planning, decision-making, and
leadership skills. A large sample of nonmanagement men showed
a significant correlation coefficient between the two
variables.
Moses and Boehm (1975) studied the relationship between
several aspects of performance at an assessment center and
subsequent administrative level achieved. A large sample of
nonmanagement women was assessed between 1963 and 1971. A
significant correlation was found in this predictive study.
This study was not part of the landmark AT&T study by Bray and
Grant (1966) and some criterion contamination may have
occurred. The researchers concluded that the job-related
skills indicators of leadership, decisionmaking, organizing,
and planning related highly with the outcome criterion.
Huck and Bray (1976), while with the Wickes Corporation
and AT&T, respectively, studied the power of the AT&T
assessment center to predict future job performance of female
employees. Subjects were non-management personnel of Michigan
Bell Telephone. The independent variable of job-related
skills indicators consisted of performance tests of
leadership, decision-making, planning, and organizing. Some
criterion contamination was acknowledged. The researchers


2
rapidly in the years between 1960 and 1980 that education
writers spoke of the "community college movement" (p. 12) .
During this time enrollments swelled from 400,000 to
4,000,000. These institutions called forth new and different
types of leaders. As Lynch, former member of the Wall Street
Journal news staff (in Sharp, 1984), stated, "gone are the
days when college presidents were tweedy, pipe-smoking types
who ran things at a leisurely pace as money rolled in and new
buildings were rising all over their campuses" (1983, p. 61).
These changes have impacted upon the levels of administrators
subordinate to the college president.
Initially, many community college administrators were
recruited from the ranks of secondary school administrators
(Sharp, 1984). A proven executive ability, rather than a
distinguished teaching career, became an increasingly
important qualification for appointment. Those candidates
possessing management training and experience or advanced
degrees in higher education became available in increasing
numbers. By the mid-1970s, the selection process, responding
to equal opportunity laws and affirmative action commitments,
acquired a public character and produced results quite
different from those of previous generations. Slowly, the
profile of the college administrator changed as women and
minorities took office. Sunshine laws further complicated
selection in several states and appeared to some observers to


LIST OF TABLES
page
Table
1. Frequency of Independent Variable
Classifications of Correlations by
Reporting Year 119
2. Mean Correlations, Medians, Standard
Deviations, and Ranges of Uncorrected Data .... 122
3. Rankings of Validities of Selection Methods
With and Without Weighting for Sample Size .... 128
4. Selection Method Frequency of Use Compared
with Mean Effect Size 131
5. Selection Method Frequency of Use Compared
with Mean Effect Size Derived Through
Meta-analysis 133
6. Mean Correlations, Sample Size Weighted
Correlations, and Corrected Correlations of
Selection Methods 134
7. Selection Method Correlation to Outcome
Criteria 139
8. Correlations of Selection Methods Measured Within
and Without Assessment Center Processes 142
9. Effects of Range Restriction on Outcome
Measures of Administrative Achievement 147
vi


CHAPTER II
BACKGROUND LITERATURE
Introduction
The responsibilities inherent in administrative positions
have undergone a complex evolution since the turn of the
century. In many organizations, however, the selection
process for administrative personnel has not seen appreciable
change. In this review, the focus is on the research
literature pertaining to the usefulness of various selection
procedures in the prediction of administrative success. The
review is organized into the following sections: the history
of personnel selection; a discussion of objectivity,
reliability, and validity in personnel selection research; a
review of the personnel selection methods used as independent
variables in personnel selection; introduction of the research
studies reviewed; and a summary and critique.
History of Personnel Selection
Traditionally, Munsterberg's 1911 experiment with motormen
has been viewed as the beginning of research in the use of
examinations for personnel selection (Ghiselli, 1973).
Nevertheless, anecdotal evidence leads one to suggest that,
even before 1910, other psychologists conducted similar
studies with tests, but these were small in scope and went
17


TABLE OF CONTENTS
page
ACKNOWLEDGEMENTS iii
LIST OF TABLES vi
ABSTRACT vii
CHAPTERS
I INTRODUCTION 1
Statement of the Problem 8
Justification for the Study 8
Delimitations 10
Limitations 12
Definition of Terms 13
Assumptions 14
Procedures 14
Organization of the Dissertation 16
II BACKGROUND LITERATURE 17
Introduction 17
History of Personnel Selection 17
Objectivity, Reliability, and Validity
in Personnel Selection Research 24
Review of the Research on Selection Methods
for Administrative Personnel 33
Summary and Critique 95
III PROCEDURES 97
Introduction 97
Purpose of the Study 97
Procedures for the Meta-analysis 108
Conclusion 114
IV PRESENTATION OF THE META-ANALYSIS OF THE DATA 116
Introduction 116
A General View of the Research 118
Administrative Personnel Selection Methods 122
iv


78
to test the validity of peer prediction of future success.
This follow-up of officers at least 3 years after graduation
showed a significant correlation. Hollander concluded that
peer nominations used early in training can make a distinctive
contribution to the prediction of a long-range criterion of
performance after training.
Lawler (1967), while working for the Department of
Administrative Sciences of Yale University, explored the
relationship between peer ratings and supervisors' ratings.
In this concurrent validity study, a sample of mid-and top-
level managers in a manufacturing company were rated by
several peer raters on three traits. Lawler found that
personnel selection "decisions (which include peer ratings)
will be of a higher quality than if just superiors' ratings
are relied upon" (p. 378).
Mayfield (1970), while employed with the Life Insurance
Agency Management Association, studied the relationship
between peer ratings and supervisors' ratings 2.5 years later.
The purpose of this predictive study was to investigate the
value of the buddy nomination procedure in the selection of
assistant managers for life insurance companies. The scores
were not used in promotion decisions to prevent criterion
contamination.
Two of the three samples showed significant correlations
between peer and supervisor ratings. Mayfield concluded that


101
conflicting interpretations of the evidence are not uncommon
(Wolf, 1986, p. 10).
The meta-analytic method permits quantitative reviews
and syntheses of the research literature. "What is needed
are methods that will integrate results from existing studies
to reveal patterns of relatively invariant underlying
relations and causalities, the establishment of which will
constitute general principles and cumulative knowledge"
(Hunter et al., 1982, p. 26). In the meta-analytic approach,
"the findings of multiple studies should be regarded as a
complex data set, no more comprehensible without statistical
analysis than would hundreds of data points in one study"
(Glass et al., 1981, p. 12).
The Application to Administrative Personnel Selection Research
The meta-analytic approach is warranted in the case of
administrative personnel selection research. Wolf (1986)
suggested that the method for synthesizing the results of
correlational studies was to take the average of the
correlations between the two variables that examine the same
research question across separate research studies. According
to Hunter, Schmidt, and Jackson (1982), "cumulation of results
can be used whenever there are at least two studies with data
bearing on the same relation" (p. 28) They specifically
addressed the use of meta-analysis in synthesizing personnel
selection research. They recognized that in personnel
selection research different tests were often used to measure


20
Selection Processes in Public Administration
When public administration systems were initiated, it was
assumed that only people of the highest caliber would apply
for service. Once reality dashed this assumption, public
administrative personnel selection systems were built upon the
cornerstone of the competitive examination, a method of
deselecting inferior job candidates (Stahl, 1976, p. 129).
From the beginning, this system has placed special emphasis
upon formal selection procedures such as testing for specific
knowledge and skills related to the particular job the person
was applying for. The starting points for the selection
process are basically two: a determination of the objective
for selection, which may be a given position, occupation,
program, or service-wide career, and a setting of basic
standards for selection, the skills and knowledge that are
necessary to meet the preceding objective. During the early-
to mid-1960s, some public administration organizations
realized a need to ensure at least an adequate intake of high
caliber people at all levels so that there would be no
shortage of talent when movement upward or outward took place.
Thus, some services began to give more attention to the
selection of persons who possessed a capability of growth and
development. To this end, public administrative organizations
began to implement the assessment center approach.
However, most public administrative organizations select
a person to fill a particular job vacancy, and thus their


72
prediction of occupational success. They found that positive
social orientation, interpersonal confidence, and educational
achievement were positively related to their outcome measures
of "job and career success" (p. 7) .
Research used in this study. A total of 8 research
studies were found which used biographical information as a
predictor variable and met the other delimitations of this
dissertation. These studies yielded 11 correlation
coefficients which ranged in strength from r = .05 to r = .57.
Samples ranged in size from 30 to 799 persons.
In 1963, Haggerty (in Mayfield, 1970) studied the
relationships between certain predictor variables and
achievement of U.S. Army officers. In this predictive study
using a sample of officer cadets, the researchers found a
nonsignificant correlation between biographical information
and a combination of achievement measures.
Williams and Harrell (1964) sought to determine which, if
any, of a number of factors were related to business success.
The independent variable in this predictive study was that of
biographical information available to an employer at the time
of a student's graduation from business school. Salary level
attained, and administrative level achieved, measured many
years after graduation, were the criteria of success.
Insignificant correlations were found for this sample of
Stanford MBA graduates. A subset of this information that was
significantly and positively related to later achievement was


Type validity: Predictive.
Years between variables; (a) 3.5, (b) 7
Assessment center?; No.
Range restriction: Medium.
Criterion contamination?; Yes.
Results:
a. Predictor: Aptitude and intelligence measures
Criterion: Salary level attained.
Correlation coefficient (r): 0.41 (N=31)
b. Predictor: Psychological attribute indicators
Criterion: Salary level attained.
Correlation coefficient (r): 0.15 (N=31)
c. Predictor: Job-related skills indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.35 (N=31)
d. Predictor: Aptitude and intelligence measures
Criterion: Promotion rate.
Correlation coefficient fr): 0.21 (N=31)
e. Predictor: Psychological attribute indicators
Criterion: Promotion rate.
Correlation coefficient m: 0.11 (N=31)
f- Predictor: Job-related skills indicators.
Criterion: Promotion rate.
Correlation coefficient (r): 0.11 (N=31)
<3 Predictor: Aptitude and intelligence measures
Criterion: Salary level attained.


108
involved in administrative personnel research the following
aspects of each study were recorded and coded:
1. the year the study was published (where the year
was not given, the year of the report was used),
2. the type organization the sample was derived from
(corporations, educational institutions, military
services, or public administration),
3. the number of subjects in the sample,
4. the types of subjects used, their job levels and
sexes,
5. the number of years between measurement of the
independent and dependent variables,
6. an ordinal measure of range restriction,
7. the type validity, predictive or concurrent,
8. the presence or absence of criterion
contamination, and
9. whether the independent variable was derived from
an assessment center.
Procedures for the Meta-Analysis
The methods for synthesizing the results of
correlational studies that report on the relationship between
two variables are rather straightforward. Essentially, the
average of the correlations between the two variables that
examine the same research question across separate research
studies is obtained and analyzed. Reports of administrative
selection methods usually expressed findings in the form of


75
researchers concluded that certain background data predict
criteria as well as assessment center data.
Conclusions. Compared to the interview method, Levine and
Flory (1975) found the application blank information to be
superior in accuracy of information. However, they noted that
"sizable inaccuracies" had been found in each method (p. 384) .
Childs and Klimoski (1986) found that an outgoing and self-
confident personality preordains success in both job- and non
job-related situation. They also found that success in one's
educational history may aid the attainment of success in one's
career, regardless of whether that job is personally
satisfying. Most studies that showed significant positive
correlations between biographical information and
administrative achievement attributed the relationship to
personality characteristics which were also positive
attributes of leadership ability.
Peer Ratings as a Selection Method
Peer ratings of supervisory potential achievement consist
of impressions gained from interactions in an equal, non
supervisor-subordinate, nature. They are predictions of how
well a peer will do in a supervisory position, should he or
she be placed in one (Korman, 1968).
Among the more consistent findings concerning peer ratings
is the significant validity these afford in predicting later
performance. Many studies conducted in military settings
indicated that peer evaluations during officer training


9
legislation, and increased protection of individual rights
present community college administrators with more complex
situations than ever before. However, selection procedures
for these positions have rarely been reviewed to determine
their continued validity.
McIntyre (1966) stated that the process used in selecting
higher education administrators is less than optimal.
Of all the rituals encumbering the selection process,
interviewing is undoubtedly the hoariestand the
sorriest. Nothing in the research on selection
methodology is so completely established and
repeatedly verified as is the unreliability of short
interviews as they are usually conducted.
Unfortunately, the record of letters of recommendation
is as dismal as that of interviewing. Although the
subject has not been researched to any great extent,
all available evidence indicates that the reading of
letters of recommendation is approximately as
enlightening as the reading of tea leaves. Rating
scales vary considerably in usefulness, but the usual
scale is little if any better than the usual letter
recommendation. The traits to be rated are often of
limited relevance, the points on the scale are seldom
clearly defined, and leniency is so rampant that only
the upper end of the scale is ordinarily used. (pp.
7-8)
In seeking to improve selection methods, every phase of
the process should be investigated and reevaluated. Much can
be accomplished when those in control of an organization take
a broader view of the problem of selection of administrators
and allow new knowledge to influence their thinking.
Occasionally innovations in procedures occur serendipitously,
but more often they are the result of long hard thinking,
experimentation, and evaluation. Purposeful changes in


60
Edel (1968), while working for the Department of Defense,
studied the relationship between a psychological "need for
success" and managerial performance. He found a significant
correlation in this concurrent study. Edel's study used one
test to measure need for success. Reliability data had not
yet been provided for this test.
In 1968, Miner (in Miner, 1977), while at Georgia State
University, studied the relationship between measures of
psychological attributes of school administrators and
separate, concurrent, outcome criteria of supervisor ratings
and salary level attained. He found no significant
correlations. He concluded that managerial motivation was not
rewarded in that school system.
Campbell et al., (1970) conducted the Early Identification
of Management Potential study for Standard Oil Company of New
Jersey. Within this study they compared measures of
psychological attributes to a combination of concurrent
outcome criteria to include salary level attained, supervisor
ratings, and administrative level achieved. Subjects were
mid- to high-level corporation managers. An insignificant
correlation was found. The researchers concluded that these
indicators of temperament showed no useful relationship with
any of the effectiveness measures.
Harrell and Harrell (1974) conducted a predictive study
in order to determine predictors of administrative
achievement. While working for the Office of Naval Research,


19
in decision making. By the 1960s, corporate management
required a knowledge of a variety of fields and operations
rarely needed in business a few years earlier. Corporations
were growing, combining, and expanding so fast that they
needed to hire competent executives at a pace which could not
be fulfilled by promotion from within the ranks (Sands, 1963,
p. 3 ) .
One particular study was more comprehensive than much of
the personnel selection research of the 1950s and 1960s. This
was the American Telephone and Telegraph (AT&T) (Bray & Grant,
1966) study from which came the assessment center method for
the selection of administrative personnel. Called the
Management Progress Study, initiated in 1956, it is the first
known corporate research and application of the assessment
center method. The 422 men brought into the study were
followed up annually and reassessed at 7-year intervals in a
effort to keep track of their professional development. The
assessment center results were not revealed to the higher AT&T
management personnel, so that the progress of the men would
not be affected by the assessment findings. The results of
this study have been well documented. Between 1961 and 1967,
only this AT&T study and two studies done by the armed forces
on assessment centers were reported in the literature
(Crooks, 1973, p. 1).
i


41
promotion rate or to supervisor ratings in this concurrent
study.
Hinrichs (1978) evaluated the predictive validity of the
AT&T assessment center process and compared the predictive
accuracy of the assessment center with the naturalistic
management evaluation. No significant findings were obtained
between aptitude and intelligence measures and administrative
level achieved. The researchers found that "prediction based
upon managerial review of the personnel files did as well as
the assessment center after 8 years" (p. 600).
Conclusions. Restriction of range was acknowledged as the
biggest drawback to the predictive ability of aptitude and
intelligence measures. Preselection of the samples by
management tended to reduce predictability (Dicken & Black,
1965). Intelligence, as measured typically by verbal ability
tests, is a fair predictor of first-line supervisory
performance but not of higher-level managerial performance
(Korman, 1968). The conclusions drawn by most researchers in
explaining the general lack of a significant relationship
between aptitude and intelligence measures and measures of
administrative achievement would support the statement that
"measures of ability and interest cannot be expected to make
fine discriminations" (Dicken & Black, 1965, p. 46).
Personal Interviews as a Selection Method
"The use of the interview as a device for appraising
applicants for a job is generally regarded with a good deal


3
be counterproductive in procuring effective leaders (Sharp,
1984) .
Achievement in these administrative positions can be
influenced by complex factors beyond a candidate's basic
physical and mental qualifications for the job.
Administrators are called upon to carry out ministerial
actions which require skills that can be measured empirically.
But, they also must make decisions that involve judgment.
Successful administrative performance results from an
interaction of at least ability, personality, motivation, and
situational factors. Consequently, it would seem that these
factors should be included in the selection methods used to
determine managerial talent (Rawls & Rawls, 1974).
Researchers have tested the validity of personnel
selection methods since the turn of the century. Munsterberg,
a Harvard researcher in the field of applied psychology,
developed rudimentary tests for the selection of personnel in
various manual occupations. His research laid the basis for
the military and industrial personnel testing done during
World War I (Ghiselli, 1973). A post-war surge in personnel
selection studies focused upon the identification of the
essential characteristics people needed for success in
specific occupations requiring predominantly physical skills.
After World War II, increasing amounts of personnel
selection research were undertaken. This effort was put forth


199
Criterion; Administrative level achieved.
Correlation coefficient (r): 0.27
b. Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.21
c. Predictor: Job-related skills indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.35
d. Predictor: OAR
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.44 (N=5f943)
40. Moses, J. L., & Boehm, V. R. (1975). Relationship of
assessment-center performance to management progress of
women. Journal of Applied Psychology. 60, 527-529.
Purpose: To study the relationship between assessment and
subsequent progress in management for women in the Bell
System.
Type organization: Corporation.
Subjects: 4,846 nonmanagement women.
Type validity: Predictive.
Years between variables: 2 through 10
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a* Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.25


194
Results:
Predictor: Peer ratings.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.23
32. McClelland, D. C., & Boyatzis, R. E. (1982).
Leadership motive pattern and long-term success in
management. Journal of Applied Psychology. 67, 737-743.
Purpose: To determine whether specific personality
characteristics relate to success in management.
Type organization: Corporation.
Subjects: 237 male, entry-level managers.
Type validity: Predictive.
Years between variables: 16
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: No.
Results:
Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.33
33. Meyer, H. H. (1970). The validity of the in-basket
test as a measure of managerial performance. Personnel
Psychology. 23. 297-307.
Purpose: To determine to what degree performance on an in-
basket test actually does correspond to observed performance
of an actual managerial job.
Type organization: Corporation.
Subjects: 81 male, low-level managers.
Type validity: Concurrent.
Years between variables: 0


67
predictor variable and met the other delimitations of this
dissertation. These studies yielded 5 correlation
coefficients which ranged in strength from r = 10 to r = .32.
Samples ranged in size from 30 to 1,375 persons.
In 1944, the Psychological Research and Services Section
of Sears, Roebuck Company established a psychological testing
program. Over a number of years researchers within this
program conducted a series of investigations into the
prediction of executive effectiveness in the Sears
organization. A predictive study was accomplished by Bentz
in 1967 (in Campbell et al., 1970). A span of 11 to 17 years
elapsed between the measurement of values and collection of
the criterion variable, promotion rate. A significant
correlation was found.
England and Lee (1974), while at the University of
Minnesota, studied the relationship between managerial values
and managerial success in several countries. The portion of
their study that dealt with managers in the United States was
used in this dissertation. Managerial success was defined as
a measure of salary level attained in this concurrent study.
The correlation of this measure with a written survey of
values was significant for a sample of corporation directors
and high level executives. The researchers concluded that the
value patterns were predictive of managerial success and could
be used in selection and placement decisions. The general
pattern that emerged from the study indicated that more


156
During the 1980s, following this period of unprecedented
growth, community college administrators faced a new set of
challenges. Enrollments plateaued; state governments replaced
local communities as the colleges' principal source of
funding; steady state enrollments and greater fiscal
accountability meant that, for the first time, community
colleges had to look at educational outcomes (Bernstein,
1988) .
The diversity of the student body of the 1980s added to
the complexity the community college administrator faced
(Wattenbarger, Haynes, & Smith, 1982). Wolf (1985) stated
that a community college is an organization in which "goals
are unclear, technology is ambiguous, and participation is
fluid among faculty and administrators" (p. 55) Thus, he
suggested that the best way to achieve organizational outcomes
was to place emphasis on hiring intelligent people as
administrators. As Etzioni pointed out, the more selective
the organization, the less control it needs, and the fewer
problems its administrators face (in Webb, 1983).
Present Administrative Personnel Selection Procedures
In the past, the process of hiring college administrators
has usually been conducted in one of the following ways: (a)
the implementation of a search committee (Kelly & Nelson,
1978), (b) by placing a blind ad in the Chronicle of Higher
Education, or by (c) using an employment agency or executive
search firm (Mottram, 1983). The committee or person


94
military supervisors (N = 19) In a comparison of
OVERAC (OAR), the overall 1981 rating, a correlation
of .40 was achieved, representing a significance of
.045. This could be considered to be a good result
were it not for the fact that it is a negative
coefficient. This implies that military supervisors
rate their personnel opposite from that of the
assessment center. For example, if the assessment
center rated an individual poorly, these results
indicate that chances are that this individual's
military supervisor would rate him highly. (p. 67)
Conclusions. In most cases, the ratings of candidates
based on the totality of assessment procedures seemed to have
validity superior to any of the specific components. Since
the situational tests represented a unique contribution to the
process, and were relied upon heavily, it was usually assumed
that these were what made the difference, although specific
data often were not reported. The in-basket's contribution
usually was considered critical, although it mainly tapped
administrative skills and was thus more narrow in scope than
some of the other exercises. Other situational tests were so
varied, and data were so seldom reported on them, that
conclusions were difficult to draw. The mental ability tests
seemed to work for some companies but not for others;
personality tests showed moderate to little success, but they
continued to be used. The projectives and interviews were
more successful than expected from their past reputations.
More research evidently is needed on the various components
of assessment centers and their integration (Howard, 1974).


178
Results:
Predictor: OAR
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.33
10. Fleishman, E. A., & Peters, D. R. (1962).
Interpersonal values, leadership attitudes, and managerial
"success". Personnel Psychology. 15, 127-143.
Purpose: To study the interrelationships among criteria of
leadership effectiveness, value dimensions, and leader
behavior and attitudes.
Type organization: Corporation.
Subjects: 25 Managers
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.19
11. Gantz, B. S., Erickson, C. 0., & Stephenson, R. W.
Some determinants of promotion in a research and development
population. In J. B. Miner, & M. G. Miner (1977).
Motivation to manage: A ten year update on the "studies in
management education" research (pp. 18-22). Atlanta:
Organizational Measurement Systems Press.
Purpose: To gain more theoretical understanding of why some
persons in a research and development population achieve
promotion into managerial positions and others do not.
Type organization: Corporation.
Subjects: 117 engineers and research scientists.


33
goals, whereas supervisors considered to be less successful
rated their subordinate administrators more highly who
followed orders and got along with others.
There are two main procedures by which the validity of an
examination or method for selection is best determined. Each
is supplementary to the other. In the first instance,
concurrent validity, the exam may be given to administrators
of known ability already on the job. If those administrators
who have been predetermined to be most successful score
highest in the evaluation, while the least successful ones
score the lowest, the evaluation method is said to have
evidence of validity. The second procedure consists of long
term follow-up studies of the performance of those employees
who have been selected for administrative positions (Stahl,
1976, p. 132).
Review of the Research on Selection Methods
for Administrative Personnel
Methods for personnel evaluation have been described in
as many different ways as there have been researchers to
describe them. According to Northcott (1960, p. 289), the
procedures in standard use have fallen into three groups:
questionnaires, tests, and interviews. Stahl (1976, p. 136)
stated that the forms which examination can take may be
classified into five categories: (a) a systematic evaluation
of education and experience, (b) oral tests, (c) standardized
qualification inquiries, (d) written tests, and (e) tests of


26
had not completed high school or taken the tests had continued
to perform satisfactorily and made progress in departments for
which the high school and test criteria were not used
(Burrington, 1982). It was ruled that employment standards
and tests which are not significantly related to job
performance violate the Civil Rights Act of 1964, Title VII
(Alexander, 1980, p. 515).
The second case, Edward L. Kirkland et al.. Plaintiffs v.
New York State Department of Correctional Services et al..
Defendants (73 LIV. 1548, 1974), was brought under the Fifth
and Fourteenth Amendments to the U.S. Constitution and under
the Civil Rights Acts of 1866 and 1871. The case concerned
a Correction Sergeant (male) written examination which had the
impact of allowing only 1.9% of blacks, and no Hispanics, to
be eligible for promotion. No recourse was made to the Civil
Rights Act of 1964, so the Equal Employment Opportunity
Commission (EEOC) Guidelines, required under that act, were
not binding. However, the U.S. District Court, Southern
District, New York, considered these Guidelines in evaluating
the job relatedness of the examination. A number of
implications can be deduced from this case for personnel
selection in general and the examination process in particular
(Wisner, 1975, pp. 266-267).
1. In rendering a decision, the court is likely to
follow the EEOC Guidelines even though the case may not have
been brought under the Guidelines.


86
(f) energy, (g) analytical ability, (h) resistance to stress,
(i) use of delegation, (j) behavior flexibility, (k) human
relations competence, (1) originality, (m) controlling, (n)
self-direction, and (o) overall potential (Howard, 1974).
The origin of the use of multiple assessment procedures
on a large scale is credited to German military psychologists.
The British adapted the procedures to the screening of officer
candidates, and the United States Office of Strategic Services
(OSS) took over the approach from the British during World War
II (Bray & Grant, 1966). Howard (1974) stated that the first
industrial use of an assessment center has been generally
attributed to AT&T beginning in 1956.
Research used in this study. A total of 14 research
studies were found that used Overall Assessment Ratings (OAR)
of an assessment center process as a predictor variable and
met the other delimitations of this dissertation. These
studies yielded 16 correlation coefficients which ranged in
strength from r = .00 to r = .65. Samples ranged in size from
25 to 5,943 persons.
The first industrial use of an assessment center is
generally attributed to the American Telephone and Telegraph
Company. Other centers have been variations on AT&T's theme.
The AT&T experimentation was research-oriented and designed
to follow the development of managerial personnel for many
years after assessment (Howard, 1974).


THE SELECTION OF ADMINISTRATORS IN
PUBLIC COMMUNITY COLLEGES:
GUIDELINES FROM THE RESEARCH
By
BETTY-JUNE HAUENSTEIN ELDRIDGE
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL
FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1988
fgrpin F LIBRARIES


73
a person's previous participation as a leader in organizations
on campus.
Kotula and Haggerty in 1966 (in Korman, 1968) studied the
relationship between biographical information given on written
personal history blanks and supervisor ratings. Two groups
of Army officers were studied. Only one coefficient was
reported, r = .17, which was significant at the .05 level.
Campbell et al. (1970) attempted to discover how to
identify, early in their careers, those employees who possess
the potential to be successful in management. The assessment
procedure in this concurrent study contained a background
biographical survey the scores from which were correlated to
an "Overall Success Index", a combination of dependent
variables. A sample of mid- to high-level corporation
managers was used. A significant correlation was found
between biographical information and the outcome criterion of
achievement. Laurent, one of the researchers involved with
this study, concluded that "successful executives in the
Standard Oil of New Jersey organization have shown a total
life pattern of successful endeavors. They were good in
college, are active in taking advantage of leadership
opportunities, and see themselves as forceful, dominant,
assertive, and confident" (Campbell et al., 1970).
Harrell and Harrell (1974) conducted a predictive study
of MBA graduates in order to determine predictors of


22
variety of different positions (Akman & Nordhauser, 1974, p.
3) .
Often, the services must select officer-quality personnel
to fill positions which have no equivalent in the nonmilitary
world. In these cases, either very qlobal criteria must be
set to determine if the candidate has the intelligence or
aptitude to learn the job-required skills and then be
successful at it, or very specific abilities or natural
characteristics must be measured to ensure the candidate will
be well suited for an important aspect of the position.
Because a large number of people must be tested and screened
for placement, the military services have developed many
paper-and-pencil tests to measure both general and specific
aptitudes. They have relied heavily upon past achievement,
such as college degrees and the results of officer candidacy
tests. The importance of test scores and military class
standings is shown by the fact that often one's scores and
future attainment of rank are positively correlated.
Selection methods used in the military services and in
public administration are often classified as "successive
hurdles" techniques. A lower than acceptable score on any one
particular screening test, or part of a test, will either
disqualify the candidate from that service or from particular
jobs within the service (Shoop, 1974, p. 341).


5
adopted. In so doing, the selection process encompassed the
personal characteristics, training needs, and organizational
climate factors that fostered managerial achievement.
Adoption of this more complicated view involving complex
interactions between ability, motivation, and opportunity
variables led to a number of changes that affected managerial
selection (Dunnette, 1963).
One of these changes was the development of the assessment
center approach. During the mid-1950s, American Telephone and
Telegraph (AT&T) began the Management Progress Study. For
years this longitudinal study remained the most valid and
scholarly research done in administrative personnel selection
(Crooks, 1973). The assessment center concept was developed
from this study and it gained rapid acceptance by the business
community. In the original assessment center approach, job
candidates were subjected to multiple tests of personality,
ability, and motivation. During each candidate's three-day
evaluation he was subjected to the traditional tests for
knowledge and experience, and also to newer methods such as
job simulations, role-playing, and leaderless group
discussions. Trained observers recorded their judgments of
each candidate's performance during the entire assessment
process. Hiring personnel hoped to select the most suitable
candidate for each position by studying the results of the
tests and the observer's judgments. In the final evaluation
of each candidate all observer's ratings were considered


172
e. Predictor: OAR.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.57 (N= 422)
3. Campbell, J. P., Dunnette, M. D., Lawler, E. E., &
Weick, K. E. (1970). Managerial behavior, performance, and
effectiveness. New York: McGraw Hill.
Purpose: To determine how employees who possess the
potential to be successful in management can be identified
early in their careers.
Type organization: Corporation.
Subjects: 443 mid- to high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A.
Results:
a* Predictor: Aptitude and intelligence measures.
Criterion: Combination.
Correlation coefficient (r): 0.16
b. Predictor: Psychological attribute indicators.
Criterion: Combination.
Correlation coefficient (r) : 0.08
c. Predictor: Self-appraisals.
Criterion: Combination.
Correlation coefficient (r^:
0.24


171
2. Bray, D. W., & Grant, D. L. (1966). The assessment
center in the measurement of potential for business
management. Psychological Monographs. 80(17, Whole No.
625) .
Purpose; To measure personal characteristics hypothesized
to be of importance either in developmental change in early
adulthood or success in business management.
Type organization; Corporation.
Subjects; 422 male, low-level managers.
Type validity; Predictive.
Years between variables; 8
Assessment center?; Yes.
Range restriction; Medium.
Criterion contamination?; No.
Results;
a. Predictor: Job-related skills indicators.
Criterion: Salary level attained.
Correlation coefficient (r^: 0.26 (N=269)
b. Predictor: Aptitude and intelligence measures.
Criterion: Salary level attained.
Correlation coefficient (r): 0.27 (N=269)
c. Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.16 (N=269)
d. Predictor: OAR.
Criterion: Salary level attained.
Correlation coefficient frl: 0.47 (N=203)


I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality,
as a dissertation for the deqree of Doctor of Philosophy.
Phili/A. Clark, Chair
Professor of Educational
Leadership
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality,
as a dissertation for the degree of Doctor of Philosophy.
James L.
professor of Educational
/Leadership
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality,
as a dissertation for the degree of Doctor of Philosophy.
, 1 J 7 ^
J iJames W. Hensl, Professor
of Educational Leadership
This dissertation was submitted to the Graduate
Faculty of the College of Education and to the Graduate
School and was accepted as partial fulfillment of the
requirements for the degree of Doctor of Philosophy.
December 1988
yrr~u
Dean, College of Education
Dean, Graduate School


xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID EIM507U0G_IEW8TM INGEST_TIME 2014-10-06T21:39:04Z PACKAGE AA00025749_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES


186
20. Haggerty, H. R. (1963). Status report on research for
the U.S. Military Academy (Cadet Leaders Task). Washington,
DC: U.S. Army Personnel Research Office.
Purpose: To study the relationships between certain
predictor variables and achievement of Army officers.
Type organization: Military.
Subjects: 2 samples of 78 and 420 U.S. Military Academy
Cadets.
Type validity: Predictive.
Years between variables: Unknown.
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a. Predictor: Peer ratings.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.53 (N=78)
b. Predictor: Peer ratings.
Criterion: Combination.
Correlation coefficient (r): 0.33 (N=420)
c. Predictor: Biographical information.
Criterion: Combination.
Correlation coefficient (r): 0.18 (N=78)
21. Harrell, T. W., & Harrell, M. S. (1974). Predictors
of management success (Technical Report No. 3). Arlington,
VA: Office of Naval Research.
Purpose: To determine predictors of management success.
Type organization: Corporation.
Subjects: 205 MBA graduates.


55
Fleishman and Peters (1962), while at Yale University,
studied the relationships between psychological attribute
indicators and managerial achievement. The resulting
correlation was not significant. A small sample was used in
this concurrent study. The researchers found that (a)
managers who rated highly on "conformity" were less valued by
senior raters, (b) the leadership attitudes of consideration
and structure were not mutually exclusive, and (c) top
management tended to identify the effectiveness of subordinate
managers with the effectiveness of their superiors.
Hicks and Stone (1962) explored the relationship between
psychological attribute indicators and supervisors' ratings
in this concurrent study. A nonsignificant correlation was
found for the sample of shop foremen and engineering
supervisors. The researchers concluded that there may be
certain basic characteristics which the successful managers
possess regardless of their areas of specialization. They
described the personality of the successful manager as one of
emotional strength in a person who views things from a broad,
theoretical point of view, avoiding over involvement in
detail.
Goodstein and Schrader (1963), while with the University
of Iowa and the Civilian Personnel Field Agency, Ordnance
Field Activity, United States Army, respectively, studied the
validity of a personality inventory in identifying those
personality characteristics associated with managerial and


91
rating data generated in a multiple technique assessment
center held up well as predictors of behavioral ratings made
several years later.
Finley (1970) studied the predictive validity of
projective tests in the SOHIO management assessment center.
Using a sample derived from the FACT Program, he found a
significant relationship between overall assessment ratings
and subsequent supervisor ratings.
Mitchel (1975) studied a sample of managers from SOHIO's
FACT Program. An insignificant correlation was found between
overall assessment ratings, minus assessor's ratings of
potential, and salary level attained 5 years after assessment.
The overall rating was not the most valid predictor in this
study, nor did it appear in any of the stepwise equations done
by the researcher.
The SOHIO studies involved both trained and untrained
assessors. Evidence for discriminant validity was found for
the two trained assessor groups, but not for the group in
which assessors consisted of untrained supervisors. It was
concluded that the assessment center method was able to
predict multiple criteria fairly well, but that the poor
quality of criterion measures probably reduced the convergent
validity coefficients (Howard, 1974).
One of the most extensive validations of the assessment
center method against measures of management effectiveness was
the Early Identification of Management Potential (EIMP) study


212
Dunnette, M. D. (1963). A modified model for test
validation and selection research. Journal of Applied
Psychology. 47., 317-323.
Edel, E. G. (1968). "Need for success" as a predictor of
managerial performance. Personnel Psychology. 21.
231-240.
England, G. W., & Lee, R. (1974). The relationship between
managerial values and managerial success in the United
States, Japan, India, and Australia. Journal of Applied
Psychology. 39. 411-419.
Farquahar, R. H., & Piele, P. K. (1972). Preparing
educational leaders: A review of recent literature,
ERIC/CEM/State-of-the Knowledge Series, No. 14. Fourteen.
UCEA Monograph Series. No 1. Columbus, Ohio: University
Council for Educational Administration.
Finley, R. M., Jr. (1970). Evaluation of behavior
predictions from projective tests given in a management
assessment center. Proceedings of the 78th Annual
Convention of the American Psychological Association. 2,
567-568.
Fleishman, E. A., & Peters, D. R. (1962). Interpersonal
values, leadership attitudes, and managerial "success".
Personnel Psychology. 15, 127-143.
Fortunato, R. T., & Waddell, D. G. (1981). Personnel
administration in higher education. San Francisco:
Jossey-Bass.
Gantz, B. S., Erickson, C. O., & Stephenson, R. W. (1977).
Some determinants of promotion in a research and
development population. In J. B. Miner & M. G. Miner,
Motivation to manage: A ten year update on the "studies
in management education" research (pp. 18-22). Atlanta:
Organizational Measurement Systems Press.
Ghiselli, E. E. (1955). The measurement of occupational
aptitude. Berkley: University of California.
Ghiselli, E. E. (1966). The validity of a personnel
interview. Personnel Psychology. 19. 389-394.
Ghiselli, E. E. (1968). Some motivational factors in the
success of managers. Personnel Psychology. 21, 431-440.


169
This objective view of the research, applied to the
selection of administrative personnel in public community
colleges, has led to the following conclusions. Use
assessment centers in higher education for the purposes of
both personnel selection and career development. Use job-
related skills indicators to aid in defining the position to
be filled, to enhance predictive validity in selection, and
to avoid legal problems. Avoid any testing that is not job-
related, such as value system assessments and psychological
attribute indicators.
Use aptitude and intelligence measures to improve the
labor pool feeding educational administrative positions. Look
within the selection methods of personal interviews, peer
ratings, and biographical information for the moderator
variables which increase the predictive validities of these
methods. Use computational methods to combine selection
information of a job candidate in order to simulate the
validity of an assessment center overall assessment rating.


143
The validity of the job-related skills indicators used
in every assessment center process has been suggested as a
cause of the high validities of overall assessment ratings.
Howard (1974) stated that "since the situational tests
represented a unigue contribution to the process and were
relied upon heavily, it was usually assumed that these were
what made the difference, although specific data often were
not reported" (p. 126). The results of this study bear this
out. Thus, the job-related skills indicators contributed a
great deal to the overall assessment rating validity.
A final reason for the high validity of the assessment
center process concerns the statistical combination of the
data derived from each selection method within the assessment
center process. A stepwise multiple regression, used to form
a composite of the results of all selection methods within the
assessment center, has been used to almost double the validity
between the selection process and the outcome criterion. For
example, Wollowick and McNamara (1969) found an overall
assessment rating of r = .37 in the IBM assessment center
study. Using all the assessment data in an actuarial fashion
produced a multiple correlation of R = .62, accounting for 38%
of the variance (Howard, 1974). Thus, using the most common
selection methods, and combining them statistically, is an
efficient way of increasing validity.


References
Akman, A. & Nordhauser, F. (1974). A conceptual view of the
officer procurement model (TOPOPS) (Report No.
AFHRL-TR-73-73). Silver Spring, MD: System Automation.
(ERIC Document Reproduction Service No. ED 105 174)
Alexander, S. K. (1980). School law. St. Paul: West.
Appleby, S. A. & Nunnery, M. Y. (1980). Power motivation,
leadership style, and managerial effectiveness of
community college department heads. Community/Junior
College Research Quarterly. 5, 1-10.
Arvey, R. D., & Campion, J. E. (1982). The employment
interview: A summary and review of recent research.
Personnel Psychology. 35, 281-322.
Arvey, R. D., Miller, H. E., Gould, R., & Burch, P. (1987).
Interview validity for selecting sales clerks. Personnel
Psychology. 40. 1-12.
Ary, D., Jacobs, L. C., & Razavieh, A. (1979). Introduction
to research in education. New York: Holt, Rinehart and
Winston.
Bare, A. C. (1986). Managerial behavior of college
chairpersons and administrators. Research in Higher
Education. 24.(2), 128-138.
Bentz, V. J. (1967). Sears, Roebuck studies. In J. P.
Campbell, B. D. Dunnette, E. E. Lawler, & K. E. Weick,
Jr., Measuring executive effectiveness (pp. 184-187).
New York: Appleton-Century-Crofts.
Bernstein, A. (1988). Community colleges: Coming of age.
Change. 2^(1) 4.
Bray, D. W. (1964). The management progress study, American
Psychologist. 19. 419-420.
Bray, D. W., & Grant, D. L. (1966). The assessment center
in the measurement of potential for business management.
Psychological Monographs. 80(17), (Whole No. 625).
210


25
Objectivity
According to Stahl (1976), one of the prime reasons for
professionalizing all steps in the selection process is to
ensure thoroughgoing objectivity (p. 131). Only those methods
which disregard extraneous factors such as race, religion,
politics, sex, residence, and age can be considered thoroughly
objective. An objective selection method should identify
those characteristics of mind and skill necessary, and only
those necessary, to the given purpose, whether the purpose is
to fill a particular position or to begin a career.
Objectivity is not only desirable but is mandated in
hiring for positions in the public sector. Throughout almost
all of American history, legislation has been enacted to
facilitate equal treatment in employment. An abundance of
case law has built up around civil rights legislation, and
rulings have occurred which have broadened the impact of such
legislation. This broadening has had the effect of placing
more and more constraints on what employers are and are not
allowed to do in terms of employee selection (Burrington,
1982, p. 55).
Two landmark cases have set the serious tone with which
violations of civil rights in hiring will be met. In the U.S.
Supreme Court case of Griggs vs. Duke Power (401 U.S. 424,
1971), an employer rejected black job applicants on the basis
of lack of completion of high school or on the results of a
general intelligence test. Evidence showed that employees who


15
meta-analysis of administrative personnel selection research
done in the United States in the past 24 years. Procedures
used in this study follow those described by Glass (1977) and
in Hunter, Schmidt, and Jackson (1982) for the meta-analysis
of a sample of independent correlational studies. The steps
involved included the following:
1. Identification of the research studies.
2. Classification of each study dependent variable,
administrative achievement, according to the Delimitations
section of this chapter.
3. Classification of each study independent variable,
selection method, according to the Delimitations section of
this chapter.
4. Calculation of frequencies of selection methods used
to answer the questions: (a) What were the most commonly used
selection methods across all types of organizations, and (b)
what were the most commonly used selection methods in each
type of organization.
5. Calculations of the means and standard deviations from
the means in selected cross-sections of the data were used to
determine which selection methods were significantly related
to achievement in administrative positions.
6. Determination of what information, derived from the
meta-analysis of the methodologies of the research studies
used, can be useful in interpreting the findings of the above
analysis of selection methods.


113
(1 r2) 2K
est a2 = S2 (6)
r N
J
a = estimated variance in population correlations
S2 = estimate of variance of r
r
r = mean correlation
K = number of studies
N = number of persons in study
(Hunter, Schmidt, & Jackson, 1982, p. 44)
Adjusting for error of measurement. To eliminate the
artifacts due to error of measurement, reliabilities of the
variables are computed according to reliability data given in
the research. The formulas for adjusting for error of
measurement are:
s7r
a = (7)
N
a = mean product variable
R = measure of reliability
N = number of persons in study
(Hunter, Schmidt, & Jackson, 1982, p. 82)
Then a correlation closer to the true score can be calculated
by:
P
TU
*y
a b
(8)


This study is dedicated
the memory of my first teacher
my mother


CHAPTER IV
PRESENTATION OF THE META-ANALYSIS OF THE DATA
Introduction
The Korman (1968) study combined with comments by Howard
(1974) provided a framework for this study of selection
methods for administrative personnel, their predictive
validities, and guidelines for administrative personnel
selection in public community colleges. Hunter, Schmitt, and
Jackson (1982) supplied the statistical technique, meta
analysis, by which one could quantify the answers to the
questions.
In the present study answers to the following questions
were sought:
1. What methods for the selection of administrative
personnel are commonly used by corporations, public
administration agencies, the military services, and
institutions of higher education?
2. What methods for the selection of administrative
personnel, derived from a meta-analysis of selection research,
are significantly related to achievement in administrative
positions?
116


69
political and economic subsets of the study of values were
positively related to progression in formal business
hierarchy, while scores on the religious and social scales
related negatively.
Conclusions. The few research findings available on
values as a selection method consistently show that certain
economic and political beliefs correlate positively, and
strong social and religious beliefs correlate negatively, with
high administrative achievement (Grimsley & Jarrett, 1975).
England and Lee (1974) found that "more successful" managers
favored an achievement orientation and preferred an active
role in interaction with other individuals instrumental to
achievement of the manager's organizational goals, while "less
successful" managers had values associated with a static and
protected environment in which they took relatively passive
roles (pp. 418-419). According to Hinrichs (1978) those
traits or characteristics that seem to reflect a degree of
social awareness tend to be detrimental to success in
administration. "The flavor is one of the 'hard charging,'
perhaps somewhat socially insensitive and upward mobile
individual" (p. 600).
Biographical Information as a Selection Method
Biographical information, personal history data about an
employee, is most often furnished an employer in the form of
an application blank or a resume. According to Levine and
Flory (1975), the most widely used selection technique is the


77
significant correlation with the outcome variable of
supervisors' ratings. The second group of cadets showed a
significant correlation with a later combination of overall
ratings.
Roadman's (1964) predictive study compared peer ratings,
taken in a management school setting at IBM, to promotion
rates of a sample of graduates who had received promotions at
least two years later. A significant correlation was found.
He concluded that a careful and comprehensive peer rating
administered in a middle manager training program can identify
those who later move into senior executive positions.
Hollander (1965), while at the State University of New
York at Buffalo, did a predictive study comparing the
relationship between peer assessments and supervisor ratings.
The study was begun in 1955 at the Naval Officer Candidate
School (OCS) in Newport, Rhode Island. An entire OCS class
was made available for this investigation. Four forms, each
setting out different qualities to be rated, were utilized;
these dealt with leadership, motivation for naval service,
probability of success in OCS training, and success as a
future officer. Results of part of the study were reported
by Hollander (1956) and showed that very early in the training
program, students were able to accurately determine which of
their peers would do well after training.
In the follow-up study (Hollander, 1965), supervisor
ratings of former trainees, who had become officers, were used


Criterion: Supervisor ratings.
Correlation coefficient (r): 0.22
188
23. Hicks, J. A., & Stone, J. B. (1962). The
identification of traits related to managerial success.
Journal of Applied Psychology. 46. 428-432.
Purpose: To determine if a broad battery of tests covering
aptitudes, temperament, and creativity could be used to
identify certain basic characteristics for selection,
promotion, and training purposes.
Type organization: Corporation.
Subjects: 76 low-level supervisors.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.12
b. Predictor: Aptitude and intelligence measures.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.17
24. Hinrichs, J. R. (1978). An eight-year follow-up of a
management assessment center. Journal of Applied
Psychology. 63., 596-601.
Purpose: To evaluate the predictive validity of the
assessment center process and to compare the predictive
accuracy of the assessment center with the naturalistic
management evaluation.
Type organization: Corporation.


125
potential to fit into the social-functional
environment of the job and organization. (p. 5)
Until recently, evidence for the validity of the interview
was particularly low (Arvey, Miller, Gould, & Burch, 1986).
Peer ratings have been frequently collected in the form
of references and recommendations. Research on peer ratings
may be generalized to include references given by an
applicant's peers or co-workers. According to Korman, (1968)
peer ratings are "impressions gathered from interactions of
an equal, non-supervisor-subordinate nature" (p. 313). No
research was found which addressed references per se.
The remaining selection methods were used less frequently
than the ones mentioned above. At progressively higher levels
of management, fewer inquiries have been made into an
applicant's values, motivations, and intelligence other than
what can be gathered in the most commonly used methods. Job-
skills indicators and assessment center processes have been
used only in organizations which process a fairly large amount
of applicants.
The selection methods, and the order in which they are
employed, vary slightly from one type organization to another.
Research done in corporations pervades the literature. Most
of this research has been conducted by university-based
researchers who either have been hired to do research for the
company or have been doing personnel selection research for


191
b. Predictor; Job-related skills indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.27
c. Predictor: OAR
Criterion; Supervisor ratings.
Correlation coefficient (r): 0.38
27. Kotula, L. J., & Haggerty, H. R. (1966). Research in
the selection of officer candidates and cadets. Washington,
DC: U.S. Army Personnel Research Office.
Purpose: To determine the predictive validity of the
personal history blank.
Type organization: Military.
Subjects: 426 Army officers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Biographical information.
Criterion: Supervisor ratings.
Correlation coefficient fr): 0.17
28. Kraut, A. I. (1969). Intellectual ability and
promotional success among high level managers. Personnel
Psychology. 22., 281-290.
Purpose: To study the relationship of high level managers'
promotional success to two tests of intellectual
functioning.
Type organization: Corporation.


44
produce reliable ratings of managerial qualities which related
significantly with ratings made on the basis of several other
techniques and with advancement" (p. 34).
Carleton (1970), while with the Standard Oil Company, Ohio
(SOHIO) assessment program, studied the relationships of
several personnel selection methods to the outcome criterion
of supervisor rating. A period of 2.5 to 5 years elapsed
between measurement of variables. The researcher concluded
that "results of the interview report appeared particularly
impressive in light of the usual criticism of the interview
as an assessment technique" (p. 566).
Campbell et al. (1970), while at the University of
Minnesota and Yale University (Lawler), in the Standard Oil
Company of New Jersey assessment center study, reviewed the
relationship between the personal interview and later measures
of administrative achievement. The outcome criterion
consisted of a combination of performance ratings to include
salary level attained, supervisor ratings, and administrative
level achieved.
Moses (1972) conducted a study of 8,885 nonmanagement
males. These men, while employed by the Bell System, were
interviewed as part of the assessment center process. A
significant correlation was found with the criterion of
administrative level achieved.
Turnage and Muchinsky (1984) conducted a study based upon
data from a one-day supervisory selection program (assessment


177
8. England, G. W., & Lee, R. (1974). The relationship
between managerial values and managerial success in the
United States, Japan, India, and Australia. Journal of
Applied Psychology. 39., 411-419.
Purpose: To investigate the relationship between values of
managers and their success as managers in four countries.
Type organization: Corporation.
Subjects: 878 high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Value systems assessments.
Criterion: Salary level attained.
Correlation coefficient (r): 0.32
9. Finley, R. M., Jr. (1970). Predictions from projective
tests given in a management assessment center. In A. Howard
(1974). An assessment of assessment centers, Academy of
Management Journal. 17, 123.
Purpose: To study the predictive validity of projective
tests in a management assessment center.
Type organization: Corporation.
Subjects: 109 subjects.
Type validity: Predictive.
Years between variables: .75 through 5
Assessment center?: Yes.
Range restriction: Assumed medium.
Criterion contamination?:
Yes.


122
Table 2
Mean Correlations. Medians. Standard Deviations, and Ranges
of Uncorrected Data
Selection Mean correlations, medians, standard
method deviations, and ranges
OAR [ ( X + ) ]
JOB [ ( X+ ) ]
VAL [ ( X + ) ]
INTER [ ( X+ )-]
APT [ ( +X ) ]
PEER [ ( X + ) ]
BIO [ ( + X ) ]
PSYCH [-( +X ) ]
SELF [-( X +])
-.10 .00 .10 .20 .30 .40 .50 .60 .70
Note. [ ] = range, X = mean, + = median, ( ) = SD.
symmetric. For self-appraisals, however, the distribution is
severely skewed to the left.
Administrative Personnel Selection Methods
In this section, administrative personnel selection
methods are discussed in terms of which methods have been most


219
Schmitt, N., Noe, R. A., Meritt, R., & Fitzgerald, M. P.
(1984). Validity of assessment center ratings for the
prediction of performance ratings and school climate of
school administrators, Journal of Applied Psychology. 69,
207-213.
Scigliano, V. S. (1979). The search committee: Don't call
us, we'll call you. Journal of the College & University
Personnel Association. 30(3), 36-40.
Scollay, R. W. (1957). Personal history data as a predictor
of success. Personnel Psychology. 10. 23-26.
Sharp P. F. (1984). American college presidents since World
War II. Educational Record. 65(3), 11-16.
Shoop, R. (1974). Public personnel selection: A matter of
consistency. Public Personnel Management. 3, 341- 343.
Silvern, L. C. (1971). Systems engineering of education:
The evolution of systems thinking in education. Los
Angeles: Education and Training Consultants.
Soar, R. S. (1956). Personal history data as a predictor of
success in service station management. Journal of
Applied Psychology. 40, 383-385.
Spencer, G. J., & Worthington, (1952). Validity of a
projective technique in predicting sales effectiveness,
Personnel Psychology. 5, 125-144.
Sprunger, B. E., & Bergquist, W. H. (1978). Handbook for
college administration. Washington, D.C.: Council for
the Advancement of Small Colleges.
Stahl, M. J. (1983). Achievement, power, and managerial
motivation: Selecting managerial talent with the job
choice exercise. Personnel Psychology. 36. 775-789.
Stahl, O. G. (1976). Public personnel administration. (7th
ed.). New York: Harper & Row.
Steel, R. P., & Ovalle, N. K. 2d. (1984). Self-appraisal
based upon supervisory feedback. Personnel Psychology.
37. 667-685.
Tenopyr, M. L., & Ruch, W. W. (1970). In J. P. Campbell, M.
D. Dunnette, E. E. Lawler, & K. E. Weick, Managerial
behavior, performance and effectiveness (p. 193). New
York: McGraw Hill.


45
center) developed by a large manufacturing firm. The subjects
were 799 nonmanagement employees who were subsequently
promoted to supervisory positions. The individual interview
was based largely on the candidate's questionnaire responses,
in which the candidate was required to play the role of a
supervisor faced with discharging an employee. In this
respect, this interview method overlaps the method of job-
related skills indicators. The predictor variable, interview,
was correlated separately with three criteria: salary level
attained, supervisor ratings, and promotion rate. Only one
resulting coefficient was significant. Based on these, and
other findings in the study, the researchers concluded that
there appeared to be no appreciable relationship between how
one is evaluated in an assessment center and how one performs
on the j ob.
A number of reasons were surmised by the researchers for
the lack of correlation. Methodological factors such as low
criterion reliability, low predictor reliability, severe
restriction of range, marked skew in the data, procedural
inconsistencies, lack of comparability across assessment
groups, and errors in data collection, were cited as possible
problems within this concurrent study.
Conclusions. The structured personal interview, in which
the interviewer, or interviewers, follow a set schedule of
questions, has consistently shown higher reliabilities than
the unstructured interview (Wright, 1969). In addition,


38
years after testing showed a significant correlation between
three measures of mental aptitude and the outcome variable of
salary level achieved.
In favor of situational tests over mental ability tests,
the researchers stated that when mental ability as measured
by a paper-and-pencil test was partialed out of judged
ability, reliable variance still remained. They found that
overall and combined assessment ratings did tend to be higher
than the correlations for any individual technique (Bray &
Grant, 1966).
In 1967, Bentz (in Campbell et al., 1970) summarized the
results of the Sears, Roebuck Company psychological testing
program. In relating aptitude and intelligence measures to
the prediction of executive effectiveness, a significant
correlation was found. Bentz stated that the successful
administrators were superior in intellectual endowment.
Kraut (1969), while working for IBM Corporation, sought
to determine the relationship between high level managers'
promotional success and measures of their intellectual
ability. No significant relationships were found. The
researcher cited problems with the predictor tests and range
restriction of the sample as possible reasons for the results
obtained.
Wollowick and McNamara (1969), while with IBM Corporation,
sought to determine the validity of an assessment center
approach in predicting management potential and to determine


180
Correlation coefficient (r): 0.26.
12. Gantz, B. S., Erickson, C. O., & Stephenson, R. W. The
prediction of managerial and research success. In J. B.
Miner, & M. G. Miner (1977). Motivation to manage: A ten
year update on the "studies in management education"
research (pp. 28-34). Atlanta: Organizational Measurement
Systems Press.
Purpose: To determine whether motives measured by a
psychological attribute indicators are the cause of
managerial success.
Type organization: Corporation.
Subjects: 49 male research scientists.
Type validity: Predictive.
Years between variables: 5.3
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: Assumed yes.
Results:
Predictor: Psychological attribute indicators.
Criterion: Promotion rate.
Correlation coefficient (r): 0.25
13. Gantz, B. S., Erickson, C. 0., & Stephenson, R. W. The
marketing department follow-up study. In J. B. Miner, & M.
G. Miner (1977). Motivation to manage: A ten year update
on the "studies in management education11 research (pp. 28-
34). Atlanta: Organizational Measurement Systems Press.
Purpose: To determine whether motives measured by a
psychological attribute indicators are the cause of
managerial success.
Type organization: Corporation.
Subjects: 81 top salesmen and marketing managers.
Type validity: Predictive.


114
PTU = mean true score
= mean product variable
b = mean product variable
(Hunter, Schmidt, & Jackson, 1982, p. 83)
Reliability data were not found reported in all of the
research studies reviewed. Reliability data were presented
in several studies for the independent variables used (Bray
& Grant, 1966; Dicken & Black, 1965; Grant & Bray, 1969;
Grant, Katkovsky, & Bray, 1967; Hollander, 1965). The
reliability coefficient of .82, used in this study, was
derived by averaging the reliability findings of these five
research studies.
One reference was found for reliability data on the
outcome variable of supervisor ratings. This coefficient was
.60 (King, Hunter, & Schmidt, 1980). This coefficient has
been used for the reliability coefficient of the combination
of outcome variables in this study.
Conclusion
The results of the application of the above procedures
to the data derived from the 52 research studies reviewed are
presented in Chapter IV to answer the questions posed in the
Statement of the Problem. As Glass has stated:
Most of us were trained to analyze complex
relationships among variables in the primary
analysis of research data. But at the higher
level, where variance, nonuniformity and
uncertainty are no less evident, we too often


4
mainly by private corporations. During this period, selection
methods were explored that would help employers hire people
for positions that required administrative skills. The
predominant researchers of this period, including Ghiselli
(1955), Halpin (1954), and Hemphill (1960), found that it was
insufficient to attempt to match a person's aptitudes and
personality traits to a generic administrative job
description. To them, defining the job specifically was an
important part of finding the right person for the job. Many
researchers agreed that one single test or examination method
probably did not yield sufficient information about a
candidate to ensure a proper person-job fit, especially for
an administrative position.
Determination of the predictive validity of a test or
selection method is fairly easy when the outcome criterion,
job performance, can be quantified. The presence or absence
of physical and some mental skills is easily seen. But the
ability of a person to lead others, to perform well under
stress, or even the quality of a person's judgment, often
eludes measurement.
Managerial performance came to be viewed as a product of
many interacting variables (Bray & Moses, 1972). Thus, a
movement away from the old model of single criterion measures
toward a systems view of selection was undertaken. As a
result, comprehensive systematic models which took into
account individual, job, and organization variables were


53
Psychological Attribute Indicators as a Selection Method
McGregor (1960) reported that the greatest single factor
that apparently influences superior and inferior performance
by supervisory people was to be found in the area of
personality variables. According to Nash (1966),
it is reasonable to expect that the vocational
interests of a manager might be related to the
effectiveness of his job performance. His enthusiasm,
effort, and level of job satisfaction may be largely
determined by how interested he is in his work and
associates. (p. 250)
According to Edel (1968), for executive and managerial
positions, personality characteristics may well be more
important to success than skill or technical know-how (p.
231) To point up this importance, Micherson (in Edel, 1968)
surveyed 79 large and small business corporations and reported
that, of those executives who failed, over 70% did so because
of some flaw in their personality rather than from a lack of
ability.
A wide variety of research on personality tests of various
types has been conducted. This work has been successful in
defining problems and has contributed to the overall
understanding of personality adjustment. Three problems in
the use of personality tests in occupational prediction
deserve special mention: (a) the ease with which test scores
can be distorted by a test-wise applicant to portray the type
of personality desired, (b) the lack of reliability displayed
by many personality measures, and (c) the failure to design


106
C. Superior and Faculty Rating
A look at the research studies Korman (1968) reviewed
revealed 13 selection methods that were most commonly used.
Of these, nine appeared to be mutually exclusive and have been
used as classifications of the independent variables for the
purpose of this study:
1. aptitude and intelligence measures,
2. personal interviews,
3. job-related skills indicators,
4. psychological attribute indicators,
5. values systems assessments,
6. biographical information,
7. peer ratings,
8. self-appraisals, and
9. assessment center processes.
For the purpose of this study, the numerous correlations
contained in each report of an assessment center process have
been separated and used independently, by selection method,
where possible. In addition, the overall assessment ratings
(OAR) have been included in order to allow for a comparison
between this unique conglomeration of data and individual
selection methods.
Dependent variables. All dependent variables reflected
a measure of achievement at an administrative level. In a
review of selection research, Korman (1968) used at least
fourteen different measures of achievement: (a) supervisor


37
graduate courses and salary. The researchers suggested that
research attention might be directed at some figure which
would concentrate on the elective course area.
Dicken and Black (1965), while at Stanford University,
explored the validity of clinical interpretations of an
objective test battery in two different corporate settings.
Subjects were first-level supervisors in a manufacturing
organization and in an insurance company. Only the highest
of the derived correlations was significant. The researchers
acknowledged the restricted range of their samples and its
effect on predictability, stating that "measures of ability
and interest cannot be expected to make fine discriminations"
(p. 46).
In 1965, Tenopyr and Ruch (in Campbell, Dunnette, Lawler,
& Weick, 1970) studied the relationship between intelligence
test results and the salary level attained by corporate
production managers. The resulting correlation coefficient
in this concurrent study was significant.
Bray and Grant (1966), while with American Telephone and
Telegraph Company, sought to study the assessment center
process. This study has become the landmark study, to date,
in assessment center development. A sample of low-level
management employees of AT&T were tested in this carefully
designed assessment center. This was one of the few studies
in personnel selection research where no criterion
contamination was allowed. A follow-up of the subjects eight


14
Meta-analysis. The statistical analysis of a large
collection of analysis results from individual studies for the
purpose of integrating the findings.
Military services. Military services are national-level
military organizations in the United States.
Public administration. Any agency which provides public
services at the federal, state, county, city, district, or
regional level is considered a public administrative agency.
Supervisor. A supervisor is a person who oversees,
directs, or manages work or workers.
Assumptions
The following assumptions have been made in this study:
1. It has been assumed that a meta-analysis of the
research on administrative personnel selection was an
appropriate methodology for determining those methods which
may, when properly implemented, improve the selection of
administrators of community colleges.
2. It has been assumed that the research included in this
study was guarded against restriction of range and criterion
contamination wherever possible.
Procedures
The purpose of this study was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. The research method used was a


10
organizational methods usually grow out of an intensive
investigation of data. Sands (1963) stated that
further study is especially needed in the area of
predicting successful managerial functioning. The
forecast for success can be improved through
correlation studies using data already on hand. All
that is necessary is the willingness to search, the
ability to analyze, and the imagination to interpret
the relationships discovered. (p. 188)
The conclusions of research must be presented in a usable
form. Two decades of research in selection of administrative
personnel is not useful to those selecting administrators of
community colleges unless it is analyzed and aimed toward
solving that specific problem. Silvern (1971) examined 15
research studies by education systems from January 1965
through June 1967. He found that, although the published
material on education systems had been proliferate,
applications of this material were found in only 1% of all of
the literature. The present study attempts to fill a void by
presenting the results of 24 years of administrative selection
research in a usable format.
Delimitations
The delimitations set restrict the research used in this
study to a specific locale, time period, phase of the
selection process, and type of position considered.
1. The research reviewed in this study pertained to the
following areas: (a) corporations, (b) public administration,
(c) military services, and (d) education.


30
of any selection method in differentiating among those
subjects. Candidates for these administrative positions are
likely to be quite similar in the traits being measured.
Therefore, it is more difficult to rank them than it would be
to rank the same number of subjects chosen randomly from the
entire population. This restriction in range is a problem in
most personnel selection research as very seldom is the sample
drawn from the population at large.
Validity
Validity is defined as the extent to which an instrument
or method measures what it is intended to measure. Unlike the
physical sciences, where there is a direct means of measuring
the outcome variable, in personnel selection indirect means
must be used to measure complex attributes.
The type of validity addressed in this study is criterion-
related validity. Criterion-related validity refers to the
relationship between the scores on a measuring instrument and
an independent external variable (criterion) believed to
measure directly the behavior or characteristic in question
(Ary, Jacobs, & Razavieh, 1979) In this type of validity the
emphasis is on the criterion rather than on the instrument
itself. One is primarily interested in what the instrument
can predict rather than in the test content.
There are several characteristics that a criterion measure
should possess. One must judge whether the criterion chosen
as the dependent variable really represents successful


Selection Methods Significantly Related to
Administrative Achievement: A Meta-analysis .... 132
Relationships Between Selection Methods
and Separate Measures of Administrative
Achievement 137
Prediction to Successively Higher Levels
of Administration 139
Validity of the Assessment Center Approach 141
Correlation of Types of Interactions Within
Selection Methods and Measures of
Administrative Achievement 144
Analysis of the Research Methods Used in
Administrative Personnel Selection Research .... 144
Conclusions of the Meta-analysis of the
Content and Methodologies of Personnel
Selection Research 149
V GUIDELINES FOR SELECTION OF ADMINISTRATIVE
PERSONNEL IN PUBLIC COMMUNITY COLLEGES 155
Introduction 155
The Need for Valid Personnel Selection Methods .... 155
Guidelines for Selecting Community College
Administrators 160
Recommendations for Research 166
Summary and Conclusion 168
APPENDIX 170
REFERENCES 210
BIOGRAPHICAL SKETCH 222
V


105
1. verbal ability
2. mechanical aptitude
3. adaptability
4. critical thinking
5. clerical ability
B.Objective Personality and Interest Inventory
1.
preferences
2.
vocational
interest
3 .
personality
inventory
4 .
temperament
inventory
5.
cooperation
test
6.
risk scale
C. Leadership Ability Test
1. leadership opinion
2. sentence completion
3. practical judgment
D. Personal History Data
1. personal history blank
2. age, education
3. academic grades
II. Judgmental Prediction
A. Executive Assessments (assessment centers)
B. Peer Rating
1. overall rating
2. critical incident
3.combat rating


43
coefficients which ranged in strength from r = .03 to r = .35.
Samples ranged in size from 122 to 8,885 persons.
Ghiselli (1966), while at the University of California,
attempted to evaluate the validity of data gathered during
personal interviews in predicting administrative job
proficiency. The independent variable was the personnel
interview used in a corporation over a 17-year period in
hiring account executives. The outcome criterion was survival
with the company for a 3-year period. The interview was
relatively unstructured and included no questions that were
of a highly personal nature. However, some questions that
would not be legal today in a public employment interview were
used. A significant correlation was found between the two
variables in this highly range-restricted sample. Ghiselli
thus found the interview to have "at least a moderately
substantial validity" (p. 394).
Grant and Bray (1969) studied the contributions made by
the interview to the assessment center process, and also, the
relationships between interview variables and the progress
criterion of salary level attained after 9 years. These
relatively unstructured interviews covered job-related and
personal topics. The interviewers were professional
psychologists. For the purpose of the study, the researchers
divided the interview into 18 different personality variables
and related each variable to the outcome criterion. The
significant relationship found showed that "the interview did


153
Self-appraisals were not found to be significantly
associated with administrative achievement. With the
exception of Heneman (1974), most researchers found that
"individuals rate themselves higher than they are rated by
comparison groups" as such they may lead to "inflated
statements of qualification" (Thornton, 1980, p. 265, p. 269).
This study substantiated this view.
Methodologies
The findings of the meta-analysis of the research
methodologies used in personnel selection research revealed
that the predictive validities of the predictor variables were
affected very little by the research methodologies. The
presence of criterion contamination and range restriction
produced minimal variation across studies. The results of
concurrent validity studies and predictive validity studies
similarly showed little deviation from one another.
These findings prompt this researcher to suggest that
personnel selection researchers might review the need for the
tightly controlled, long-term studies which are often called
for at the conclusion of concurrent validity studies. Is
there empirical evidence that such studies result in "more
valid" conclusions than concurrent studies, or is this an
assumption that has been perpetuated?
In Chapter V the processes currently used for the
selection of administrative personnel will be discussed. The


167
all of the university programs combined.
(Richardson, 1987, p. 39)
Hammons and Ivery (1987) found significant differences
between the tasks of corporation executives and community
college leaders. Thus, it is possible that the results of
administrative personnel selection research, which has been
done predominately within corporations, cannot be generalized
to the field of education. If this is the case, such research
is needed within the field of higher education.
The results of this study show that concurrent validity
studies may serve as well as predictive studies in determining
the validity of administrative personnel selections methods.
Thus, the information which could be obtained from concurrent
validity studies conducted within public community colleges
may add to our knowledge of the validities of administrative
personnel selection in the near term.
Three methodological practices are suggested as a result
of this study: (a) use a combination of two or more outcome
measures to add to validity in measuring administrative
achievement, (b) plan for the length of time allowed to elapse
between collection of the independent and dependent variables
to be at least three years but not greater than ten years, and
(c) report details of research conducted so that researchers
can determine what moderator variables within each method
contribute to validity.


90
confirmed by the rate of promotions, as well as demotions
(Kraut & Scott, 1972). The studies contained criterion
contamination as the subjects' results were provided to their
supervisors. However, it was observed that the "relationship
of ratings to first promotions is moderate enough to reduce
fears of 'crown prince' or 'kiss of death' effects" (p. 124).
According to Kraut & Scott (1972),
compared to the normal promotional system in most
companies, the program typically has some obvious
advantages in reliability and validly measuring
management potential. Instead of judgments by one's
immediate manager which may be more or less
subjective, evaluations in the program are made by
several managers (raters) who are likely to be much
more objective. Further, they are not making
judgments about the individual's management potential
from his performance in a non-management job; instead
they evaluate all candidates against a common
yardstick comprised of standardized management-type
tasks. (p.124)
The Standard Oil Company, Ohio (SOHIO), created an
assessment center in 1963. This Formal Assessment of
Corporate Talents (FACT) Program was modeled after the AT&T
assessment process (Carleton, 1970) Most data from the SOHIO
program pointed to the superiority of the overall assessment
ratings but did not eliminate any single category of
assessment components (Howard, 1974).
Carleton (1970) sought to determine the relationships
between test and rating data and later measures of
administrative achievement. A significant correlation was
found between the overall assessment rating and subsequent
supervisor ratings. Carleton concluded that both test and


98
administration agencies, the military services, and
institutions of higher education?
a. What are the most commonly used administrative
selection methods across all types of organizations?
b. What are the most commonly used administrative
selection methods in each type of organization?
c. How does the frequency of use of each selection
method relate to its ability to predict administrative
achievement?
2. What does research demonstrate are the methods which are
significantly related to achievement in administrative
positions?
a. To what extent do all selection methods, taken
inclusively, correlate with administrative achievement?
b. To what extent do different selection methods
correlate with administrative achievement?
c. To what extent do selection methods, taken both
inclusively and separately, correlate with different measures
of administrative achievement?
d. To what extent do selection methods predict
achievement in successively higher levels of administration?
e. To what extent do selection results of the assessment
center approach correlate with administrative achievement?
f. To what extent does the type of interaction (i.e.,
written, verbal, or performance) required by an applicant in


215
Hollander, E. P. (1965). Validity of peer nominations in
predicting a distant performance criterion. Journal of
Applied Psychology. 49. 434-438.
Howard, A. (1974). An assessment of assessment centers,
Academy of Management Journal. 17, 115-134.
Huck, J. R., & Bray, D. W. (1976). Management assessment
center evaluations and subsequent job performance of
white and black females. Personnel Psychology. 29.
13-30.
Huegli, J. M., & Eich, K. E. (1979). The administrator
search process: Managing the interview for desired
results. Journal of the College & University Personnel
Association. 3.0(2), 1-12.
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982).
Meta-analysis: Cumulating research findings across
studies. Beverly Hills: Sage.
Jablin, F. (1975). The selection interview: Contingency
theory and beyond. Human Resource Management. 14(1) .
2-9.
Kelly, S. P., & Nelson, D. F. (1978). Other quarters: The
administrative hiring process. Journal of the College
and University Personnel Association. 2.9(3), 51-58.
King, L. M., Hunter, J. E., & Schmitt, F. L. (1980). Halo
in a multidimensional forced-choice performance
evaluation scale. Journal of Applied Psychology. 65,
507-516.
Kirchner, W. K., & Reisberg, D. J. (1962). Differences
between better and less-effective supervisors in
appraisal of subordinates. Personnel Psychology. 15.
301.
Korman, A. K. (1968). The prediction of managerial
performance: A review. Personnel Psychology. 21. 295-
322 .
Kotula, L. J., & Haggerty, H. R. (1966). Research in the
selection of officer candidates and cadets. Wash., D.C.:
U.S. Army Personnel Research Office.
Knauft, E. C. (1949). A selection battery for bake shop
managers. Journal of Applied Psychology. 32. 434-438.


54
the tests specifically for purposes of occupational prediction
(Edel, 1968).
Early studies gave mixed validity results. Knauft (1949)
studied a sample of 33 bakery shop managers and a correlation
of r = .39 was found, but the personality test used was later
withdrawn from the market (Korman, 1968). Comrey and High
(1955) tested the validity of some ability and interest
scores. Using a sample of over 200 production supervisors,
scores on several preference records and vocational scales
were compared to objective performance data. All correlations
except one were insignificant. La Gaipa (1960) studied the
validity of certain personality traits of over 400 naval
officer candidates compared to later supervisor ratings. Only
one of several correlations was significant. MacKinney and
Wolins (1960) studied the relationship between two interest
measures and several later measures of performance using three
overlapping samples of supervisors. Results were inconsistent
random patterns of significant and insignificant correlations.
Mixed results were also found by Robbins and King (1961) in
four samples of sales managers.
Research used in this study. A total of 22 research
studies were found which used psychological attribute
indicators as a predictor variable and met the other
delimitations of this dissertation. These studies yielded 37
correlation coefficients which ranged in strength from r = .01
to r = .47. Samples ranged in size from 20 to 1,375 persons.


8
Statement of the Problem
The problem addressed in this study was to develop
guidelines for improving the selection of administrators of
public junior and community colleges.
The following questions were addressed:
1. What methods for the selection of administrative
personnel are commonly used by corporations, public
administration agencies, the military services, and
institutions of higher education?
2. What methods for the selection of administrative
personnel, derived from a meta-analysis of selection research,
are significantly related to achievement in administrative
positions?
3. What information, derived from a meta-analysis of the
methodologies of selection research, is useful in interpreting
the findings of this study?
4. What guidelines can be derived from this research for
improving personnel selection methods for administrators of
public community colleges?
Justification for the Study
Radical changes have occurred in the public workplace
in the last two decades. These changes have required
management to become more flexible in its thinking and to
reorient itself to new conditions. In higher education the
advancements in tenure, unionization, equal opportunity


34
personality traits. According to the "Uniform Guidelines on
Employee Selection Procedures," issued by the Equal Employment
Opportunity Commission, the United States Department of
Justice, the United States Civil Service Commission, and the
United States Department of Labor in 1978, selection
procedures are defined as including "the full range of
assessment techniques from traditional paper and pencil tests,
performance tests, training programs, or probationary periods
and physical, educational, and work experience requirements
through informal or casual interviews and unscored application
forms" (Quaintance, 1980, p. 126).
The selection methods reviewed in this study have been
delineated into the following types: (a) aptitude and
intelligence measures, (b) personal interviews, (c) job-
related skills indicators, (d) psychological attribute
indicators, (e) value systems assessments, (f) biographical
information, (g) peer ratings, (h) self appraisals, and (i)
assessment center processes. These methods are the
independent, or predictor variables of the administrative
personnel selection research studies used in this meta
analysis. Statistical data and methodological details of each
of the research studies reviewed for this dissertation are
found listed alphabetically by researcher's name(s) in the
Appendix.


198
Type organization: Corporation.
Subjects: 254 low-level managers.
Type validity: Predictive.
Years between variables: 5
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: Yes.
Results:
a. Predictor: Peer ratings.
Criterion: Salary level attained.
Correlation coefficient (r): 0.23
b. Predictor: OAR
Criterion: Salary level attained.
Correlation coefficient (r): 0.14
39. Moses, J. L. (1972). Assessment center performance
and management progress. Studies in Personnel Psychology.
4(1), 7-12.
Purpose: To study the relationship between assessment
center processes and managerial achievement.
Type organization: Corporation.
Subjects: 8,885 nonmanagement males.
Type validity: Predictive.
Years between variables: 7
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a. Predictor: Personal interviews.


57
stronger masculine interests have a somewhat better chance of
success" (Williams & Harrell, 1964, p. 167).
Dicken and Black (1965) studied the predictive validity
of psychometric evaluation for the selection of supervisors.
Predictor variables were two well-known personality
inventories. Two criteria, salary level attained and
promotion rate, both measured 3.5 to 7 years after testing,
were used. Insignificant correlation coefficients were found
for each of the two samples. The researchers cited
restriction of range as a problem in studies of this type by
preselection of the samples by management. They stated that
"measures of ability and interest cannot be expected to make
fine discriminations" (p. 46).
Bray and Grant (1966) explored the relationship between
psychological attribute indicators and salary level attained
8 years later. Subjects were low-level managers. Although
an overall nonsignificant correlation was found between the
variables, several personality traits such as lack of
passivity and control of feelings were positively correlated
with success in management.
Nash (1966), while at the University of Maryland, sought
to discover the relationships between a manager's vocational
interests and the effectiveness of his job performance. This
concurrent validity study used corporation managers as
subjects. The correlation coefficient between psychological
attribute indicators and supervisor ratings was found to be


112
Sampling error causes the variance across studies to
increase relative to the variance of the population
correlations. The effect of sampling error on the variance
is to add a known constant, which is called the sampling error
variance. To eliminate the effect of sampling error from a
meta-analysis the measure of sampling error variance is
subtracted from the observed variance. This is done using the
following formulas for finding the variance of correlations
across samples:
Z[N¡(r, ir)2]
S =
S2 = estimate of variance of r
r
Ni = number of persons in study i
r. = correlation coefficient in study i
r = mean correlation
(4)
e
(1 r2) 2K
EN
(5)
a2 = sampling error
e
r = mean correlation
K = number of studies
N = number of persons in study
Thus, from formulas (4) and (5) we see that the variance of
the population correlations is estimated by:


50
Service. A comparison of overall score on the skills
indicator test and the outcome variable yielded a significant
correlation in this concurrent study. Meyer concluded that
the "performance style one exhibits in handling carefully
selected, but true-to-life 'in-basket' items does correlate
with demonstrated on-the-job performance of a managerial
position, especially the ability to handle the planning and
administrative aspects of the job" (p. 307).
Campbell et al. (197 0) studied the relationship between
job-related skills indicators and a criterion measure of
administrative achievement. The outcome criterion was a
combination of performance ratings, to include salary level
attained, supervisor ratings, and administrative level
achieved. This concurrent validity study was a component of
the Early Identification of Management Potential (EIMP)
assessment program of the Standard Oil Company of New Jersey.
A subject group of mid- to high-level managers showed a
significant correlation between written measures of job-
related skills indicators and the outcome criterion.
The researchers found that job-related skills indicators,
in the form of a management judgment test, were among the four
best measures of achievement. However, they surmised that the
high correlation was, in part, attributable to possible
criterion contamination due to the managers' perceptions of
their own career success.


Type validity: Concurrent.
Years between variables: 0
Assessment center?; No.
Range restriction: High.
Criterion contamination?: N/A
Results;
a.
Predictor:
Aptitude and intelligence measures
Criterion:
Promotion rate.
Correlation
coefficient ir): 0.06
b.
Predictor:
Aptitude and intelligence measures
Criterion:
Supervisor ratings.
Correlation
coefficient (r): 0.07
c.
Predictor:
Psychological attribute indicators
Criterion:
Promotion rate.
Correlation
coefficient (r): 0.07
d.
Predictor:
Psychological attribute indicators
Criterion:
Supervisor ratings.
Correlation
coefficient fri: 0.05
e.
Predictor:
Biographical information.
Criterion:
Promotion rate.
Correlation
coefficient (r): 0.17
f.
Predictor:
Biographical information.
Criterion:
Supervisor ratings.
Correlation
coefficient (r): 0.15
g-
Predictor:
Peer ratings.
Criterion:
Promotion rate.


46
interviews conducted by a board or panel appear to be
promising as a means of enhancing reliability and validity
(Arvey & Campion, 1982). Greater validities may be found if
researchers first decide the purpose the interview is intended
to serve. It has been suggested that the personal interview
is useful in determining interpersonal skills and motivation
(Schmitt, 1976), and in imparting job information from the
interviewer to the applicant (Arvey & Campion, 1982).
The employment interview continues to be used although
organizational psychologists are aware of the findings
concerning the method's limited reliability and validity.
Arvey and Campion stated that "interviewers ignore base rate
information, do not pay attention to disconfirming evidence,
and over-depend on case specific information in making their
judgments" (Arvey & Campion, 1982, p. 316).
Job-related Skills Indicators as a Selection Method
Situational methods offer the potential of adding to the
scope of human characteristics that can be evaluated. The
rationale behind using situational exercises is that they
simulate the type of work to which the candidate will be
exposed and allow his or her performance to be observed under
somewhat realistic conditions. Situational tests measure more
complex or dynamic behavior rather than aptitudes or traits
isolated by more traditional psychometric tests. The whole
personality is observed in interaction with simulations of the
future job environment (Howard, 1974).


218
Peck, R. F., & Parsons, J. W. (1956). Personality factors
in work output: Four studies of factory workers.
Personnel Psychology. 9, 49-79.
Prien, E. P., & Liske, R. E. (1962). Assessments of
higher-level personnel: III. Rating criteria: A
comparative analysis of supervisor ratings and incumbent
self-ratings of job performance. Personnel Psychology.
15, 187-194.
Quaintance, M. K. (1980). The impact of the uniform
selection guidelines on public merit systems. Public
Personnel Management Journal. 9, 125-132.
Rankin, K. K. (1981). A predictive validity study of an
assessment center for research and development superiors.
Wright-Patterson AFB, OH: Air Force Institute of
Technology.
Rawls, J. R., & Rawls, D. J. (1974). Recent trends in
management selection. Personnel Journal. 53. 104-109.
Ricciuti, H. (1955). Ratings of leadership potential at the
U.S. Naval Academy and subsequent officer performance.
Journal of Applied Psychology. 39. 194- 199.
Richardson, R. Jr. (1987). A question of quality:
University programs for community college leaders.
Community, Technical, and Junior College Journal. 57(4),
39-41.
Roadman, H. E. (1964). An industrial use of peer ratings.
Journal of Applied Psychology. 48. 211-214.
Robbins, J. & King, D. (1961). Validity information
exchange. Personnel Psychology. 13. 217-219.
Ross, J. D. (1979). A current review of public sector
assessment centers: Cause for concern. Public Personnel
Management. 8, 41-46.
Sands, E. (1963). How to select executive personnel. New
York: Reinhold.
Schmitt, N. (1976). Social and situational determinants of
interview decision: Implications for the employment
interview. Personnel Psychology. 29, 79-101.


52
concluded that the assessment center process was as valid a
selection method for women as for men.
Turnage and Muchinsky (1984), while at Iowa State
University, conducted a predictive study based upon data from
a one-day supervisory selection program (assessment center)
developed by a large manufacturing firm. The subjects were
employees who were subseguently promoted to supervisory
positions. An overall significant correlation was found
between job-related skills indicators and these methods of
measuring subsequent job performance combined. This was one
of the few significant results of this study.
Conclusions. Though much more expensive and time
consuming to administer than paper-and-pencil tests and
questionnaires, the need to find ways of evaluating
characteristics not covered by the latter is sufficient to
warrant extensive experimentation with relatively elaborate
techniques (Bray & Grant, 1966) There has been relatively
little research on the relationship between how a manager
behaves in a game and his behavior in an actual decision
situation. The behavior of a manager in a business game may
be quite different from his behavior on the job where the
rewards and punishments are much larger. Further, business
games tend to de-emphasize the interpersonal dimension in
managerial performance, whereas many management jobs appear
to emphasize it heavily (Lawler, 1967, p. 370).


145
and concurrent validity studies are discussed. The results
of an analysis of the merits of predictive versus concurrent
validity are shown. Results are given for an analysis of the
relative advantages of various time intervals between the
measurement of the independent and dependent variables in the
predictive studies used.
Criterion Contamination
The extent to which criterion contamination effects
hiring and promotion of administrative personnel has been a
constant concern of researchers. Howard (1974) stated that
problems of criterion contamination have confounded predictive
validity studies when personnel decisions based upon selection
ratings were used in determining future promotions. However,
she thought that even where these data were used to aid in
initial hiring or promotion decisions, the effect on later
promotions was not large.
The results of the present meta-analysis showed that
criterion contamination had no effect on the combined
validities of the independent variables. When weighted by
sample size, predictive validity studies containing criterion
contamination showed a mean correlation of r = .25.
Predictive validity studies for which the researchers reported
that no criterion contamination was allowed showed a mean
correlation of r = .26. Thus, it appears that the fears of
researchers, that subsequent measures of administrative


162
success in competing for entry-level and mid-level management
positions in the university system (Brubaker, 1983) .
2. Implement a statistical combination of the data derived
from other selection processes. Multiple correlations of
scores derived from several selection methods increased the
validity of the methods to predict administrative achievement
(Howard, 1974). The results of this research have shown that,
where selection methods were used independently of any
assessment center, they were as valid as the same methods used
within an assessment center. Thus, the assessment center
process does not enhance the validity of the specific
selection methods, but the computational synthesis of the
combined data does enhance validity.
3. Write a detailed, task-oriented job description for any
administrative position to be filled. This process is
necessary (a) to provide selectors with specific skills,
traits, and capabilities to be sought within all selection
processes used, and (b) to satisfy the legal constraints on
job-relatedness of personnel selection procedures. Job-
related skills indicators ranked second in overall validity,
also having a confidence range above the significance level.
4. Use job-related skills indicators wherever possible
within the selection process. As indicated by the results of
this study, a job-related selection method can only add to the
validity of the overall selection process. The process of
developing job-related skill indicators assures that the legal


79
peer ratings can be made in a "realistic" administrative
setting and retain their predictiveness.
Mitchel (1975), while at Bowling Green State University,
studied the predictive abilities of parts of the Standard Oil
Company of Ohio's assessment center program. The relationship
between peer ratings and a later measure of salary level
attained proved significant. He concluded that "peer and
assessor ratings, as well as combinations of variables, were
predictive of a salary criterion of managerial success"
(Mitchel, 1975, p. 578).
Gantz, Erickson, and Stephenson (in Miner, 1977) sought
to determine why some persons in a research and development
population achieve promotion into formally designated
supervisory or managerial positions and whether those who were
promoted were the ones who should have been promoted. A study
of research scientists and engineers in a large, federally-
funded laboratory showed a significant correlation between
peer ratings and promotion rate. These peer ratings of
overall ability and perceived creativity were statistically
significant at the .01 level.
Turnage and Muchinsky (1984), examined the ability of
assessment center evaluations to predict actual job
performance criteria, and to compare the predictability of
assessment center evaluations versus traditional measures in
forecasting job success. Their data came from a one-day
supervisory selection program developed by a large


Outcome criteria consisted of several specific measures of on-
the-job achievement in administrative positions.
The statistical procedure of meta-analysis was applied
to the 128 validity coefficients found in these studies. The
use of this procedure allowed an objective look into cross
sections of the research data. For each cross section of the
data, mean correlations, weighted by sample size, were
computed. Sampling error, the difference between the
population parameter and the sample statistic, was corrected
for in each of the resulting mean correlations. The
reliabilities of the independent and dependent variables were
computed according
to
reliability data
reported in
the
research.
These
data
were used to
adjust the
mean
correlations
for
error of measurement. Thus,
new
correlations,
closer to
the true score
correlations,
were
derived.
Major findings were that (a) all personnel selection
methods considered, with the exception of self-appraisals,
were valid predictors of administrative achievement, (b) the
methods which showed the highest validity, overall assessment
ratings and job-related skills indicators, were the least used
in personnel selection, and (c) those methods used in the
selection of community college administrators, biographical
information, personal interviews, and peer ratings, were
moderately valid. Ten guidelines for administrative
personnel selection in public community colleges were derived.
vm


Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy
THE SELECTION OF ADMINISTRATORS IN
PUBLIC COMMUNITY COLLEGES:
GUIDELINES FROM THE RESEARCH
By
Betty-June Hauenstein Eldridge
December 1988
Chairman: Phillip A. Clark
Major Department: Educational Leadership
The purpose of this study was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. Fifty-two research studies were
found which met the parameters of this study. Only those
studies which concerned selection of administrative personnel,
and were published in the United States from 1962 through
1985, inclusive, were used.
Predictor variables consisted of nine personnel
selection methods: (a) aptitude and intelligence measures,
(b) personal interviews, (c) job-related skills indicators,
(d) psychological attribute indicators, (e) value systems
assessments, (f) biographical information, (g) peer ratings,
(h) self-appraisals, and (i) assessment center processes.
vii


189
Subjects; 30 male nonmanagement and low-level management
personnel.
Type validity: Predictive.
Years between variables: 8
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: No.
Results:
a.
Predictor:
Value systems assessments.
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.26
b.
Predictor:
Self-appraisals.
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.26
c.
Predictor:
Aptitude and intelligence measures.
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.21
d.
Predictor:
OAR
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.46
e.
Predictor:
Biographical information.
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.55
25. Hollander, E. P. (1965). Validity of peer nominations
in predicting a distant performance criterion. Journal of
Applied Psvcholoav. 49, 434-438.
Purpose: To determine the validity of peer ratings in
predicting future administrative achievement.


131
each method is presented. The mean correlation coefficient
is referred to as an estimate of "effect size" (Hunter et al.,
1982, p. 145). Effect size ranges were based upon definitions
for small, medium, and large effect sizes as stated for point
biserial correlations in Cohen (1977). Thus, as shown in
Table 4, the two selection methods which showed medium effect
sizes, overall assessment ratings and job-related skills
indicators, were also the least used.
Table 4
Selection Method Frequency of Use Compared With Mean Effect
Size
Frequency
Mean effect size
Low
Medium
High
Frequent
PEER
INTER
BIO
Moderate
SELF
APT
PSYCH
Seldom
VAL
OAR
JOB
Two of the three most frequently used methods,
biographical information and peer ratings, showed low effect
sizes. Only one frequently used method, personal interviews,


70
evaluation of the job application blank or the resume. These
are used to determine whether an individual meets minimum
qualifications for a position. If these qualifications are
not met, the applicant is no longer considered. In some cases
this is the only information an employer receives about a
prospective employee, but in most cases the application is
followed by an interview, written test, or some additional
means of assessment.
To provide some idea of the frequency of use of
applications, Carlson (in Levine & Flory, 1975) posited that
if an arbitrary assumption were made that 50% of those filing
applications or resumes are ultimately interviewed, then over
1,000,000,000 applications and resumes per year are prepared
and screened in the United States. Yet, despite its
universality, the evaluation of the validity of biographical
information for personnel selection and placement has not been
researched in a systematic fashion to any great extent.
Research has been done on the empirically weighted
application, usually for predicting turnover. More recent
research on application evaluation has focused on the factors
affecting the rater's quality of evaluation of the information
rather than the quality of the method of evaluation per se.
According to Spencer and Worthington (1952) and Peck and
Parsons (1956), the few studies dealing with method of
evaluation other than empirical weighing have demonstrated


202
Correlation coefficient (r): 0.00
44. Roadman, H. E. (1964). An industrial use of peer
ratings. Journal of Applied Psychology. 48. 211-214.
Purpose: To determine if peer ratings can identify managers
who later are promoted rapidly in a large corporation.
Type organization: Corporation.
Subjects; 40 mid-level managers.
Type validity; Predictive.
Years between variables: 2
Assessment center?: No.
Range restriction: High.
Criterion contamination?: No.
Results:
Predictor: Peer ratings.
Criterion: Promotion rate.
Correlation coefficient fr): 0.31.
45. Schmitt, N., Noe, R. A., Meritt, R., & Fitzgerald, M.
P. (1984). Validity of assessment center ratings for the
prediction of performance ratings and school climate of
school administrators, Journal of Applied Psychology. 69,
pp. 207-213.
Purpose: Evaluates the use of the assessment center
approach applied in an educational setting to select
secondary and elementary school administrators.
Type organization: Education.
Subjects: 153 low-level administrators.
Type validity; Predictive.
Years between variables: 1
Assessment center?: Yes.
Range restriction: Low.


135
figures were obtained using formulas (1) through (3) in
Chapter III. The correction for unreliability caused the
weighted mean correlation to be raised from r = .37 to r
= .52, which was calculated according to formula (8) in
Chapter III.
Job-related skills indicators. A total of 16,517
subjects were included in 10 studies which yielded 12
correlation coefficients involving job-related skills
indicators. The mean correlation of r = .25 (a = .13) was
raised to r = .30 when weighted according to sample size.
Correction for unreliability caused the weighted mean
correlation to be raised from r = .30 to r = .43.
Value systems assessments. A total of 2,483 subjects
were included in the 4 studies which yielded 5 correlation
coefficients involving value systems assessments. The mean
correlation was r = .25 (o = .09) when weighted for sample
size. Correction for unreliability caused the mean
correlation to be raised from r = .25 to .35.
Personal interviews. A total of 11,526 subjects were
included in 6 studies which yielded 7 correlation coefficients
involving personal interviews. The original mean correlation,
.20 (a = .10), weighted for sample size, was r = .24. The
correction for the error of measurement caused the mean
correlation to be raised from r = .24 to .34.


140
employees would be successful junior or middle managers. In
such tests, the mean correlation with mental ability was
definitely higher for top level executives than for middle
managers, but the large amount of overlap between the
distribution of scores reduced any ability to make predictions
from the tests.
Korman (1968) also found that verbal ability tests showed
little usefulness for predicting managerial performance above
the first-line supervisory level, because the typical
managerial applicant population was already highly preselected
on abilities. Biographical information, likewise, showed
"some predictive value" for first-line supervisors only
(Korman, 1968, p. 308).
There were not sufficient correlation coefficients
present in this study to do a complete meta-analysis in which
each selection method was averaged by each level of management
achieved. However, by combining all selection methods, and
comparing them to the different administrative levels
achieved, the following conclusions were found.
Selection methods correlated r = .36 with attainment of
the first-level of supervisory achievement. These same
methods correlated r = .27 and r = .28, respectively, with
the mid- and high-levels of administrative achievement. Thus,
the results confirmed the literature findings that selection
methods can more accurately predict achievement of the


64
management persons with a need for influencing others
dominated.
Stahl (1983), while at Clemson University, used a measure
of psychological attributes to test the hypothesis that high
managerial motivation consisted of high personal needs for
achievement and power. He found a significant correlation
with each of two outcome criterion variables, supervisor
ratings and administrative level achieved. The researcher's
conclusions were drawn from this and other data collected from
other samples using the same methodology. He concluded that
"there was a higher proportion of subjects with high
managerial motivation among the managers than among the non
managers," and that "there was a higher proportion of managers
with high managerial motivation among the promoted managers
than among the non-promoted managers" (p. 786). Stahl
stressed the need for researchers to implement longitudinal
validation studies to confirm these results.
Turnage and Muchinsky (1984) compared the predictive
validity of assessment center evaluations versus traditional
measures in forecasting supervisory job performance. A sample
of employees who were subsequently promoted to supervisory
positions within a large manufacturing firm was used.
Correlation coefficients between findings of psychological
attribute indicators and three separate outcome variables
yielded no significant relationships. The researchers cited
many possible methodological shortcomings of the study that


27
2. A job analysis should be performed prior to the
development of each examination.
3. Those critical elements, or more important
characteristics of the job, which distinguish between those
likely to be successful and those less likely to be
successful, must be included in the examination.
4. Subject matter experts, to be considered qualified,
must be of an appropriate rank and sufficiently experienced
in the current skills, knowledges, and abilities required in
the job for which the test is developed.
5. Discriminatory test results cannot be refuted by
demonstrating that the results occurred because of certain
environmental conditions or cultural characteristics related
to the protected group population unless it can be shown that
the examination is job-related.
6. The fact that specific parts of the test are not
found to be discriminatory does not serve to support the use
of an examination whose overall results are found to be
discriminatory.
7. All subtests included in the examination must be
weighted in accord with their importance to the job.
8. Data should be obtained concerning the examination
results immediately after administering the exam to determine
the examination's impact on protected groups.
9. Every effort should be made to set passing scores at
a point considered to separate the candidates most likely to


138
achievement (Huck & Bray, 1976; Korman, 1968; Steel & Ovalle,
1984) What is called for is an outcome measure that
accurately and objectively assesses an administrator's level
of achievement on the job, rather than someone's subjective
opinion of such achievement. No plausible solutions to this
dilemma were suggested in the literature. In fact,
discrepancies in inter-dependent variable validities were
found in the literature. The relationships between selection
methods, both inclusively and separately, with the six
different measures of dependent variables used in this study
are shown in Table 7.
The outcome criterion which showed the highest mean
correlation was derived from an average of only two widely
disparate correlation coefficients and, thus, may not have
been a reliable measure of achievement. However, the measure
of administrative achievement which was a combination of two
or more of the other outcome criteria had a relatively high
correlation, r = .26, and was based upon a larger sample of
correlations. This finding substantiates the Campbell et. al
(1970) view that such an "overall success index" (p. 166),
while not explicitly identifying or quantifying actual
managerial behavioral dimensions, did identify managers
differing substantially in their ability to utilize
organizational resources effectively.


47
The most commonly used situational tests of job-related
skills for administrative positions are examined here. The
in-basket exercise is one of those most frequently used in
assessment centers. The candidate is faced with an
accumulation of memos, reports, notes of incoming telephone
calls, letters, and other materials supposedly collected in
the in-basket of the job he or she is to take over. The
candidate is asked to dispose of these materials in the most
appropriate manner by writing letters, notes, self-reminders,
agenda for meetings, etc. Ratings of performance range from
subjective evaluations to highly standardized checklists.
In the leaderless group discussion, the participants are
given a discussion question and are instructed to arrive at
a group decision. Topics may include such things as promotion
decisions, disciplinary actions, or business expansion
problems. Sometimes participants are given a particular point
of view to defend. Personality dimensions such as
interpersonal skills, acceptance by the group, individual
influence, and leadership abilities can be evaluated in this
way.
Management games, such as stock market tasks,
manufacturing exercises, and merger negotiations are common
job-related selection exercises. Participants are asked to
solve problems, either cooperatively or competitively. These
games often bring out leadership, organizational abilities,
and interpersonal skills. Some games also permit observations


65
could have contributed to the general lack of significant
findings.
Conclusions. Results obtained by the studies reviewed
herein suggest that there is evidence supporting the
proposition that effective managers have identifiable
interests which distinguish them from less effective managers.
However, most motivation studies have used concurrent validity
as the research method. Predictive validity studies that
assess the dynamic nature of personality may be needed.
According to Miner (1978), "the greatest research needs seem
to be for additional longitudinal studies of the relationships
between motivation to manage...and success" (p. 751).
Another limitation of interest measures is that they have
been demonstrated to be fakeable. Nash suggested that future
research should focus on the impact of fakability on the
actual use of measured interests in selection programs (1965,
P-34) .
Campbell et al. (1970) addressed achievement and power for
managers. Concerning job/task analysis of what effective
managers actually do, they listed the frequency of behavior
aimed at influencing others (power) and the frequency of
behavior concerned with setting and accomplishing goals
(achievement). They remarked that "better managers tend to
show a lifetime pattern of high achievement, power, and
economic motivation" (p. 361).


24
the person finally selected to fill the position. He stressed
such practices as reviewing the application form with the
applicant, giving the applicant an opportunity to talk about
himself or herself in a way that helps that individual to
present relevant evidence, making the interview more
systematic and probing, examining test scores and other
information, and weighting the various types of information
in order to reach an intelligent decision. Webster
recommended taking time following each interview to clarify
impressions and to formulate a judgment. He suggested making
use of multiple independent evaluations in which two or more
interviewers record decisions independently and subsequently
reconcile differences.
Objectivity, Reliability, and Validity in Personnel
Selection Research
These three primary considerations enter into the
effectiveness and propriety of examination processes for
employment selection: objectivity, reliability, and validity.
Objectivity is even more important in the 1980s than it was
previously due to the legal demand for testing to have a job-
specific relation. Reliability is important but is often not
reported in personnel selection research. Validity is the
major concern of this dissertation, and the research studies
reviewed here pertain to the validity of each selection method
in predicting administrative achievement.


220
Thorndike, R., & Hagen, E. (1959). Ten thousand careers.
New York: John Wiley & Sons.
Thornton, G. C. (1968). The relationship between
supervisory- and self-appraisals of executive
performance. Personnel Psychology. 21. 441-455.
Thornton, G. C. (1980). Psychometric properties of self
appraisals of job performance. Personnel Psychology. 33,
263-271.
Turnage, J. J., & Muchinsky, P. M. (1984). A comparison of
the predictive validity of assessment center evaluations
versus traditional measures in forecasting supervisory
job performance: Interpretive implications of criterion
distortion for the assessment paradigm. Journal of
Applied Psychology. 69., 595-602.
Twombly, S. B. (1987). The importance of beginnings: The
relationship of entry positions to career outcomes for
two-year college presidents, Community College
Review.15(2). 14-21.
Ulrich, L., & Trumbo, D. (1965). The selection interview
since 1949. Psychological Bulletin. 63., 100-116.
Vernon, P. (1950). The validation of civil service
selection board procedures. Occupational Psychology. 24.,
75-95.
Veroff, J. (1982). Assertive motivations: Achievement
versus power. In A. J. Stewart (Ed.), Motivation and
society: A volume in honor of David C. McClelland. San
Francisco: Jossey-Bass, 99-132.
Wagner, R. (1949). The employment interview: A critical
summary. Personnel Psychology. 2, 17-46.
Wattenbarger, J. L., & Haynes, F. T., Jr. (1982). Coping
with complexity: Another viewpoint for community
colleges. Community College Review. .10(2) 3-11.
Webb, M. (1983) Employment administration: Elements and
effects. The Journal of College and University
Personnel. 12.(1), 17-22.
Weitz, J. (1958). Selecting supervisors with peer ratings.
Personnel Psychology. 11, 25-35.
Williams, F.J., & Harrell, T.W. (1964). Predicting success
in business. Journal of Applied Psychology. 48, 164-167.


110
size. Research studies reviewed here which were not
originally reported using this correlation were all converted
to their equivalent Pearson product-moment correlations using
the formulas given in Wolf (1986).
Averaging correlations across studies and variables.
From the correlation coefficients derived by each query, means
and standard deviations were calculated for the relationships
between selection methods and administrative achievement using
formulas (1) and (2). According to Wolf "the average of the
correlations between the two variables that examine the same
research question across separate research studies is
obtained" (1986, p. 28). Means are typically calculated by
averaging the raw Pearson product-moment correlation
coefficients (r) using the formula:
Zr
r = (1)
n
r = mean correlation
r = Pearson correlation
n = number of correlation coefficients combined
(Wolf, 1986, p.29)
The standard deviations of the means were then
calculated using the formula:
a
square root of
Zr£ -
(Sr)2
N
N
(2)
a = standard deviation


99
completing a selection test correlate with administrative
achievement?
3. What information, derived from a meta-analysis of the
methodologies of selection research, is useful in interpreting
the findings of this study?
a. To what extent does criterion contamination effect
the correlation between selection method results and
administrative achievement?
b. To what extent does restriction of range effect the
correlation between selection method results and
administrative achievement?
4. What guidelines can be derived from this research for
improving personnel selection methods for administrators of
public community colleges?
a. To what extent do the methods now used for the
selection of administrative personnel in public community
colleges correspond with those methods which show the highest
correlation to administrative achievement?
b. What selection methods, indicated by the results of
the meta-analysis, could be added to or dropped from the
methods currently in use to make the selection process more
accurate and efficient?
The Rationale for the Meta-analytic Approach
In his presidential address to the 1976 meeting of the
American Educational Research Association (AERA) Glass (1976)
introduced the concept of meta-analysis. In distinguishing


ACKNOWLEDGMENTS
I recognize with gratitude the support given by those
who contributed to this study. I wish to express my sincere
gratitude to Dr. Phillip Clark, Dr. James Wattenbarger, and
Dr. James Hensel, chairman and members of my doctoral
committee, whose guidance and encouragement made possible the
successful conduct of this study. A special thank you is
given to Dr. Michael Nunnery for helping me in the initial
stage of choosing a topic and for introducing me to meta
analysis .
I could not have completed this study without the
steadfast support of friends and family. Thanks go to Leila
Cantara for her proofreading and unending supply of red ink.
Appreciation goes to Judie Swan and Dora Proctor for
untangling my attempts at word processing. I thank my father
for his moral and financial support. Warmest regards go to
John, Everett, and Sara Eldridge, who "helped mommy study."
My greatest appreciation goes to my husband, Everett.
Without his continual encouragement and unflagging belief in
my ability this study could never have been accomplished.
iii


28
be successful from those considered to have the least chance
of success.
10. Be prepared to answer the question "why was this
method of measurement considered the best means to employ?"
11. In court, do not rely too heavily on your
professional witness to attest to the job relatedness of your
examination.
Thus, the public employer has but one choiceto validate
the employee selection procedures. Fortunately, the choice
is the best one to follow from the standpoint of good
personnel management.
Reliability
The consistency with which a method measures what it is
expected to measure is called reliability (Ary, Jacobs, &
Razavieh, 1979, p. 206). In order to achieve consistency, a
selection test or method must leave little room for chance in
the subject's final score or rating. If a test is reliable,
a person taking it at two different times should make
substantially the same score or be ranked in approximately the
same position each time. Test reliability depends also on the
nature of the variable being measured. For example, the trait
of academic achievement can be measured more consistently than
that of personality. Although all measurements of human
qualities contain some error, no test or method of personnel
selection is of value unless it has a high degree of
reliability (Stahl, 1976, p. 133).


119
Table 1
Frequency of Independent Variable Classifications of
Correlations by Reporting Year
Selection method
Year
APT
INTER
JOB
PSYCH VAL
BIO
PEER
SELF
OAR
1962
1
2
1
1963
1
1
2
1964
2
2
2
1
1965
5
3
4
1
1966
1
1
1
2
1
2
1967
1
3 1
1
1
1968
6
1
1969
2
1
1
1
1970
1
2
2
1
1
1
1
3
1971
1972
1
1
1
2
1973
1974
1 1
1
1
1975
3
1
2 2
1
2
1976
1
1
1
1977
2
9
2
1
1978
1
1
1
1
1
1979
1980
1981
1
1982
1
1983
1
1984
2
2
2
2
2
3
3
1985


141
first-level of administration, but have less validity in the
prediction of those applicants who will succeed at higher
administrative levels.
Validity of the Assessment Center Approach
Howard (1974) stated that, because of the expense
involved, psychologists tried to develop assessment centers
in more than a haphazard way. She surmised that projective
tests, interviews, and job-related exercises were designed
with care toward reliability and validity for each particular
organization. The results of this study, however, showed
little overall difference between the validities of each and
all selection methods when used within an assessment center
versus when used independent of any assessment center.
Table 8 shows that, of the eight selection methods (OAR
was not included), half displayed higher correlations within
an assessment center and half without. The mean correlations
of each group, r = .20 for the methods within an assessment
center, and r = .21 for the methods taken independent of any
assessment center, were almost identical.
Thus, it does not seem that the assessment center
approach added any value to each separate selection method.
Still the assessment center overall assessment rating was, in
11 out of 14 assessment center studies, significantly
correlated with administrative achievement. Several reasons
have been suggested for this phenomenon. Turnage and


CHAPTER III
PROCEDURES
Introduction
In this chapter, the purpose of the study, the rationale
for the use of the meta-analytic approach, the application to
selection research for administrative personnel, the
identification of the research studies used, the
classification of variables, and procedures for the meta
analysis are discussed. These procedures include: conversion
to a common metric; averaging correlations across studies and
variables; elimination of sampling error; and adjustments for
error of measurement.
The Purpose of the Study
The purpose of this study was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. Toward this end, the questions
posed in the Statement of the Problem in Chapter I have been
defined here in greater detail. In Chapter IV, the
presentation of the results of the meta-analysis, the
following questions are answered.
1. What methods for the selection of administrative
personnel are commonly used by corporations, public
97


85
availability of assessment scores to superiors) may have been
a factor in this study.
Conclusions. It has been suggested by Levine (1978) that,
as predictors of job performance, self-assessments should at
least supplement other information. Self-assessments may
replace more traditional selection methods, especially in
measuring psychological attributes that may be relatively
inaccessible by other means. He suggested that self-
assessments might have a motivational impact on those
applicants who are hired, as these people will strive to be
consistent with their self-perceived competencies. Levine
admits that research is needed to determine what applicant
attributes are most validly, or least validly, self-assessed.
Assessment Center Processes as a Selection Method
A highlighting feature of the assessment center is that
candidates are evaluated not on what they have done in present
or past jobs but on how they are likely to cope with a new
type of position. This involves using various situational
tests as well as incorporating some of the more classic
selection procedures, such as aptitude tests and interviews.
Assessments are conducted at least partially in groups, which
permits observing group interactions as well as obtaining peer
ratings (Howard, 1974).
The following dimensions are most often assessed: (a)
leadership, (b) organizing and planning, (c) decision making,
(d) oral and written communications skills, (e) initiative,


161
selection guidelines from the research can be applied with
differing validity within different levels and cores of
community college administration.
From the conclusions of the research conducted in the
present study, it was found that eight of the personnel
selection methods considered are valid predictors of
administrative achievement. These methods are the overall
assessment rating derived from assessment center processes,
job-related skills indicators, value systems assessments,
personal interviews, aptitude and intelligence measures, peer
ratings, biographical information, and psychological attribute
indicators.
This meta-analysis has provided an objective view of the
accumulated findings of the 52 research studies involved. The
following are specific guidelines derived from the meta
analysis of the research. Taken into consideration were the
legal challenges presented.
1. Set up an assessment center for the identification of
persons who might be selected as community college
administrators. The assessment center overall assessment
rating showed the highest validity of all the selection
methods. To offset the costs involved, use the center also
for management training. Expected outcomes of the University
of California assessment center method are the improved
performance by participants in their present jobs and improved


193
Purpose: To study the advantages of the multitrait-
multirater approach to measuring managerial job performance.
Type organization: Corporation.
Subjects: 113 mid- to high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a. Predictor: Self-appraisals.
Criterion: Supervisor ratings.
Correlation coefficient fr): Approximately 0.07
b. Predictor: Peer ratings.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.43
31. Mayfield, E. C. (1970). Management selection: Buddy
nominations revisited. Personnel Psychology. 23. 377-391.
Purpose: To investigate the value of the buddy nomination
procedure in the selection of assistant managers.
Type organization: Corporation.
Subjects: 154 real estate agents.
Type validity: Predictive.
Years between variables: 2.5
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: No.


N = number of cases
r = correlation coefficient
111
However, with the widely varying sample sizes found in
the research studies reviewed, from 20 to 8,885 subjects, it
was deemed necessary to weight all mean correlations by sample
size. This was done according to the formula:
S[N,-r,]
r = (3)
zNj
r(. = correlation coefficient in study i
Nj = number of persons in study i
(Hunter, Schmidt, & Jackson, 1982, p. 41)
Elimination of sampling error. When an inference is
made from a sample to a population a certain amount of error
is involved because random samples can be expected to vary
from one to another. Sampling error is defined as the
difference between a population parameter and a sample
statistic. Researchers depend upon sample statistics to
estimate population parameters, thus, the knowledge of how
samples are expected to vary from populations is a basic
element in inferential statistics (Ary, Jacobs & Razavieh,
1979, p. 136).


40
future performance of women as accurately as it does that of
men" (p. 529) .
Grimsley and Jarrett (1975), while with the University of
Southern California and California State University,
respectively, studied the relationship between past
achievement test scores and criterion measures obtained in
administrative positions. The significant results found in
this concurrent validity study led the researchers to conclude
that "the differences in test scores of more or less
successful managers result from fundamental differences in
mental ability and personality rather than the influence of
on-the-job experience" (p. 226).
Huck and Bray (1976) studied a sample of non-management
females derived from the AT&T assessment center process.
Their purpose was to test the validity of an assessment center
process on a population different from the population of the
Management Progress Study (Bray & Grant, 1966). Although the
relationship between aptitude and intelligence measures and
the criterion variable did not prove to be significant, the
researchers found that "judgments made by assessment center
staffs are good predictors of later performance" (p. 27).
Gantz, Erickson, and Stephenson (1977) sought to determine
why some persons in a research and development population
achieve promotion, particularly into supervisory or managerial
positions, and other persons do not. Measures of aptitude and
intelligence were not significantly related to either


THE SELECTION OF ADMINISTRATORS IN
PUBLIC COMMUNITY COLLEGES:
GUIDELINES FROM THE RESEARCH
By
BETTY-JUNE HAUENSTEIN ELDRIDGE
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL
FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
1988
fgrpin F LIBRARIES

This study is dedicated
the memory of my first teacher
my mother

ACKNOWLEDGMENTS
I recognize with gratitude the support given by those
who contributed to this study. I wish to express my sincere
gratitude to Dr. Phillip Clark, Dr. James Wattenbarger, and
Dr. James Hensel, chairman and members of my doctoral
committee, whose guidance and encouragement made possible the
successful conduct of this study. A special thank you is
given to Dr. Michael Nunnery for helping me in the initial
stage of choosing a topic and for introducing me to meta
analysis .
I could not have completed this study without the
steadfast support of friends and family. Thanks go to Leila
Cantara for her proofreading and unending supply of red ink.
Appreciation goes to Judie Swan and Dora Proctor for
untangling my attempts at word processing. I thank my father
for his moral and financial support. Warmest regards go to
John, Everett, and Sara Eldridge, who "helped mommy study."
My greatest appreciation goes to my husband, Everett.
Without his continual encouragement and unflagging belief in
my ability this study could never have been accomplished.
iii

TABLE OF CONTENTS
page
ACKNOWLEDGEMENTS iii
LIST OF TABLES vi
ABSTRACT vii
CHAPTERS
I INTRODUCTION 1
Statement of the Problem 8
Justification for the Study 8
Delimitations 10
Limitations 12
Definition of Terms 13
Assumptions 14
Procedures 14
Organization of the Dissertation 16
II BACKGROUND LITERATURE 17
Introduction 17
History of Personnel Selection 17
Objectivity, Reliability, and Validity
in Personnel Selection Research 24
Review of the Research on Selection Methods
for Administrative Personnel 33
Summary and Critique 95
III PROCEDURES 97
Introduction 97
Purpose of the Study 97
Procedures for the Meta-analysis 108
Conclusion 114
IV PRESENTATION OF THE META-ANALYSIS OF THE DATA 116
Introduction 116
A General View of the Research 118
Administrative Personnel Selection Methods 122
iv

Selection Methods Significantly Related to
Administrative Achievement: A Meta-analysis .... 132
Relationships Between Selection Methods
and Separate Measures of Administrative
Achievement 137
Prediction to Successively Higher Levels
of Administration 139
Validity of the Assessment Center Approach 141
Correlation of Types of Interactions Within
Selection Methods and Measures of
Administrative Achievement 144
Analysis of the Research Methods Used in
Administrative Personnel Selection Research .... 144
Conclusions of the Meta-analysis of the
Content and Methodologies of Personnel
Selection Research 149
V GUIDELINES FOR SELECTION OF ADMINISTRATIVE
PERSONNEL IN PUBLIC COMMUNITY COLLEGES 155
Introduction 155
The Need for Valid Personnel Selection Methods .... 155
Guidelines for Selecting Community College
Administrators 160
Recommendations for Research 166
Summary and Conclusion 168
APPENDIX 170
REFERENCES 210
BIOGRAPHICAL SKETCH 222
V

LIST OF TABLES
page
Table
1. Frequency of Independent Variable
Classifications of Correlations by
Reporting Year 119
2. Mean Correlations, Medians, Standard
Deviations, and Ranges of Uncorrected Data .... 122
3. Rankings of Validities of Selection Methods
With and Without Weighting for Sample Size .... 128
4. Selection Method Frequency of Use Compared
with Mean Effect Size 131
5. Selection Method Frequency of Use Compared
with Mean Effect Size Derived Through
Meta-analysis 133
6. Mean Correlations, Sample Size Weighted
Correlations, and Corrected Correlations of
Selection Methods 134
7. Selection Method Correlation to Outcome
Criteria 139
8. Correlations of Selection Methods Measured Within
and Without Assessment Center Processes 142
9. Effects of Range Restriction on Outcome
Measures of Administrative Achievement 147
vi

Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy
THE SELECTION OF ADMINISTRATORS IN
PUBLIC COMMUNITY COLLEGES:
GUIDELINES FROM THE RESEARCH
By
Betty-June Hauenstein Eldridge
December 1988
Chairman: Phillip A. Clark
Major Department: Educational Leadership
The purpose of this study was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. Fifty-two research studies were
found which met the parameters of this study. Only those
studies which concerned selection of administrative personnel,
and were published in the United States from 1962 through
1985, inclusive, were used.
Predictor variables consisted of nine personnel
selection methods: (a) aptitude and intelligence measures,
(b) personal interviews, (c) job-related skills indicators,
(d) psychological attribute indicators, (e) value systems
assessments, (f) biographical information, (g) peer ratings,
(h) self-appraisals, and (i) assessment center processes.
vii

Outcome criteria consisted of several specific measures of on-
the-job achievement in administrative positions.
The statistical procedure of meta-analysis was applied
to the 128 validity coefficients found in these studies. The
use of this procedure allowed an objective look into cross
sections of the research data. For each cross section of the
data, mean correlations, weighted by sample size, were
computed. Sampling error, the difference between the
population parameter and the sample statistic, was corrected
for in each of the resulting mean correlations. The
reliabilities of the independent and dependent variables were
computed according
to
reliability data
reported in
the
research.
These
data
were used to
adjust the
mean
correlations
for
error of measurement. Thus,
new
correlations,
closer to
the true score
correlations,
were
derived.
Major findings were that (a) all personnel selection
methods considered, with the exception of self-appraisals,
were valid predictors of administrative achievement, (b) the
methods which showed the highest validity, overall assessment
ratings and job-related skills indicators, were the least used
in personnel selection, and (c) those methods used in the
selection of community college administrators, biographical
information, personal interviews, and peer ratings, were
moderately valid. Ten guidelines for administrative
personnel selection in public community colleges were derived.
vm

Legal constraints placed upon personnel selection in public
organizations were considered. These guidelines included (a)
establishing assessment centers for selection and training,
(b) increasing the job-relatedness of selection procedures,
and (c) using statistical methods to simulate overall
assessment ratings.
IX

CHAPTER I
INTRODUCTION
The purpose of this dissertation was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. The research method used was a
meta-analysis of administrative personnel selection research
done in the United States in the past 24 years.
Administrators in our public community colleges must possess
both extensive knowledge and sound judgment to meet the
complex challenges of the years ahead. The extent to which
a community or junior college meets its challenges largely
depends upon the qualities of its future leaders. Thus, these
integral people must be chosen carefully if they are to be
properly matched to the demands of their positions. Those
concerned with the placement of administrators in higher
education can no longer afford the luxury of using arbitrary
selection methods. This is true in times of surplus, and even
more true in times of shortage, budget cuts, and retrenchment
(Mandell, 1964). Rapid change itself has mandated a
continuous reevaluation of personnel selection procedures.
Administrators, well-qualified in the past, may not be
prepared to face challenges of the job in the late 1980s or
1990s. According to Sharp (1984), community colleges grew so
1

2
rapidly in the years between 1960 and 1980 that education
writers spoke of the "community college movement" (p. 12) .
During this time enrollments swelled from 400,000 to
4,000,000. These institutions called forth new and different
types of leaders. As Lynch, former member of the Wall Street
Journal news staff (in Sharp, 1984), stated, "gone are the
days when college presidents were tweedy, pipe-smoking types
who ran things at a leisurely pace as money rolled in and new
buildings were rising all over their campuses" (1983, p. 61).
These changes have impacted upon the levels of administrators
subordinate to the college president.
Initially, many community college administrators were
recruited from the ranks of secondary school administrators
(Sharp, 1984). A proven executive ability, rather than a
distinguished teaching career, became an increasingly
important qualification for appointment. Those candidates
possessing management training and experience or advanced
degrees in higher education became available in increasing
numbers. By the mid-1970s, the selection process, responding
to equal opportunity laws and affirmative action commitments,
acquired a public character and produced results quite
different from those of previous generations. Slowly, the
profile of the college administrator changed as women and
minorities took office. Sunshine laws further complicated
selection in several states and appeared to some observers to

3
be counterproductive in procuring effective leaders (Sharp,
1984) .
Achievement in these administrative positions can be
influenced by complex factors beyond a candidate's basic
physical and mental qualifications for the job.
Administrators are called upon to carry out ministerial
actions which require skills that can be measured empirically.
But, they also must make decisions that involve judgment.
Successful administrative performance results from an
interaction of at least ability, personality, motivation, and
situational factors. Consequently, it would seem that these
factors should be included in the selection methods used to
determine managerial talent (Rawls & Rawls, 1974).
Researchers have tested the validity of personnel
selection methods since the turn of the century. Munsterberg,
a Harvard researcher in the field of applied psychology,
developed rudimentary tests for the selection of personnel in
various manual occupations. His research laid the basis for
the military and industrial personnel testing done during
World War I (Ghiselli, 1973). A post-war surge in personnel
selection studies focused upon the identification of the
essential characteristics people needed for success in
specific occupations requiring predominantly physical skills.
After World War II, increasing amounts of personnel
selection research were undertaken. This effort was put forth

4
mainly by private corporations. During this period, selection
methods were explored that would help employers hire people
for positions that required administrative skills. The
predominant researchers of this period, including Ghiselli
(1955), Halpin (1954), and Hemphill (1960), found that it was
insufficient to attempt to match a person's aptitudes and
personality traits to a generic administrative job
description. To them, defining the job specifically was an
important part of finding the right person for the job. Many
researchers agreed that one single test or examination method
probably did not yield sufficient information about a
candidate to ensure a proper person-job fit, especially for
an administrative position.
Determination of the predictive validity of a test or
selection method is fairly easy when the outcome criterion,
job performance, can be quantified. The presence or absence
of physical and some mental skills is easily seen. But the
ability of a person to lead others, to perform well under
stress, or even the quality of a person's judgment, often
eludes measurement.
Managerial performance came to be viewed as a product of
many interacting variables (Bray & Moses, 1972). Thus, a
movement away from the old model of single criterion measures
toward a systems view of selection was undertaken. As a
result, comprehensive systematic models which took into
account individual, job, and organization variables were

5
adopted. In so doing, the selection process encompassed the
personal characteristics, training needs, and organizational
climate factors that fostered managerial achievement.
Adoption of this more complicated view involving complex
interactions between ability, motivation, and opportunity
variables led to a number of changes that affected managerial
selection (Dunnette, 1963).
One of these changes was the development of the assessment
center approach. During the mid-1950s, American Telephone and
Telegraph (AT&T) began the Management Progress Study. For
years this longitudinal study remained the most valid and
scholarly research done in administrative personnel selection
(Crooks, 1973). The assessment center concept was developed
from this study and it gained rapid acceptance by the business
community. In the original assessment center approach, job
candidates were subjected to multiple tests of personality,
ability, and motivation. During each candidate's three-day
evaluation he was subjected to the traditional tests for
knowledge and experience, and also to newer methods such as
job simulations, role-playing, and leaderless group
discussions. Trained observers recorded their judgments of
each candidate's performance during the entire assessment
process. Hiring personnel hoped to select the most suitable
candidate for each position by studying the results of the
tests and the observer's judgments. In the final evaluation
of each candidate all observer's ratings were considered

6
collectively. A low score on one part of the examination
could be offset or disregarded due to adequate performance on
other parts. This approach replaced the previously used
"successive hurdles" method in which a failing score on any
one part of an selection procedure would disqualify a
candidate from further consideration (Shoop, 1974).
By the 1960s, many companies and public administration
organizations had implemented the assessment center approach.
The results of the AT&T study were so convincing that many
copied all or part of the original process. There was little
additional research done to test either the findings of the
original study or its validity in a new situation (Crooks,
1973). During this time, the selection of administrators in
public junior and community colleges has continued to follow
the traditional methods of reviewing a candidate's education,
experience, references, and conducting a personal interview.
The exercises completed in hiring college and university
administrators are frequently anything but a process. Too
often they are a set of actions which leave selection
committees dissatisfied with what they have done and
candidates dissatisfied with what they have undergone. No
universal process exists for improving what often is an inept
or discouraging attempt to fill an administrative vacancy with
the best available person.
The problem is magnified by the increasing number of
searches underway in institutions of higher education. In

7
many colleges and universities more searches are underway for
administrators than for faculty. In addition, affirmative
action and equal opportunity require that administrative
searches be more public, more extensive, and more expensive
than ever before. All this expenditure of time and resources
demands that more attention be paid to the selection process
(Kelly & Nelson, 1978) .
Much has been written about the problems of selecting
administrators in higher education, however, little research
has been done on the topic. A change in the procedures now
followed may be warranted. The results of many years of
research in personnel selection methods in other fields have
been usefully applied in this case. The conclusions of this
research helped to determine guidelines for improving the
selection of administrators for our public junior and
community colleges.
Research in administrative personnel selection methods has
been done in corporations, the military services, public
administrative agencies, and in education. The present study
was needed in order to synthesize this research and apply it
to the selection of community college administrators. The
conclusions of the research, when applied to this problem,
have indicated those selection methods which are useful in
predicting administrative achievement.

8
Statement of the Problem
The problem addressed in this study was to develop
guidelines for improving the selection of administrators of
public junior and community colleges.
The following questions were addressed:
1. What methods for the selection of administrative
personnel are commonly used by corporations, public
administration agencies, the military services, and
institutions of higher education?
2. What methods for the selection of administrative
personnel, derived from a meta-analysis of selection research,
are significantly related to achievement in administrative
positions?
3. What information, derived from a meta-analysis of the
methodologies of selection research, is useful in interpreting
the findings of this study?
4. What guidelines can be derived from this research for
improving personnel selection methods for administrators of
public community colleges?
Justification for the Study
Radical changes have occurred in the public workplace
in the last two decades. These changes have required
management to become more flexible in its thinking and to
reorient itself to new conditions. In higher education the
advancements in tenure, unionization, equal opportunity

9
legislation, and increased protection of individual rights
present community college administrators with more complex
situations than ever before. However, selection procedures
for these positions have rarely been reviewed to determine
their continued validity.
McIntyre (1966) stated that the process used in selecting
higher education administrators is less than optimal.
Of all the rituals encumbering the selection process,
interviewing is undoubtedly the hoariestand the
sorriest. Nothing in the research on selection
methodology is so completely established and
repeatedly verified as is the unreliability of short
interviews as they are usually conducted.
Unfortunately, the record of letters of recommendation
is as dismal as that of interviewing. Although the
subject has not been researched to any great extent,
all available evidence indicates that the reading of
letters of recommendation is approximately as
enlightening as the reading of tea leaves. Rating
scales vary considerably in usefulness, but the usual
scale is little if any better than the usual letter
recommendation. The traits to be rated are often of
limited relevance, the points on the scale are seldom
clearly defined, and leniency is so rampant that only
the upper end of the scale is ordinarily used. (pp.
7-8)
In seeking to improve selection methods, every phase of
the process should be investigated and reevaluated. Much can
be accomplished when those in control of an organization take
a broader view of the problem of selection of administrators
and allow new knowledge to influence their thinking.
Occasionally innovations in procedures occur serendipitously,
but more often they are the result of long hard thinking,
experimentation, and evaluation. Purposeful changes in

10
organizational methods usually grow out of an intensive
investigation of data. Sands (1963) stated that
further study is especially needed in the area of
predicting successful managerial functioning. The
forecast for success can be improved through
correlation studies using data already on hand. All
that is necessary is the willingness to search, the
ability to analyze, and the imagination to interpret
the relationships discovered. (p. 188)
The conclusions of research must be presented in a usable
form. Two decades of research in selection of administrative
personnel is not useful to those selecting administrators of
community colleges unless it is analyzed and aimed toward
solving that specific problem. Silvern (1971) examined 15
research studies by education systems from January 1965
through June 1967. He found that, although the published
material on education systems had been proliferate,
applications of this material were found in only 1% of all of
the literature. The present study attempts to fill a void by
presenting the results of 24 years of administrative selection
research in a usable format.
Delimitations
The delimitations set restrict the research used in this
study to a specific locale, time period, phase of the
selection process, and type of position considered.
1. The research reviewed in this study pertained to the
following areas: (a) corporations, (b) public administration,
(c) military services, and (d) education.

11
2. The study dealt with personnel selection methods rather
than with individual selection tests. When only one
particular test was administered, such as a Strong Vocational
Interest Blank, this test was recorded by the type of
selection method it represented.
3. The methods of personnel selection, the independent
variable, used in this study included (a) aptitude and
intelligence measures, (b) personal interviews, (c) job-
related skills indicators, (d) psychological attribute
indicators, (e) value systems assessments, (f) biographical
information, (g) peer ratings, (h) self-appraisals, and (i)
assessment center processes.
4. The outcome measures of administrative achievement, the
dependent variable, used in this study included (a)
administrative level achieved, (b) salary level attained, (c)
supervisor ratings, (d) number of years serving in an
administrative position within the same organization, (e)
achievement of tenure, (f) objective performance data, and (g)
promotion rate.
5. Research studies which defined administrative
achievement as measured by (a) admissions personnel ratings,
(b) success in being hired, or (c) performance success in
subseguent training programs, were not used in this study.
These criteria were judged to be too far removed from any
measurement of actual performance on the job.

12
6. The research studies analyzed were published in the
United States from January 1, 1962 through December 31, 1985.
Limitations
The following confinements were observed in the
investigation:
1. Performance ratings and promotions are based in part
upon subjective ratings given by an employee's supervisor.
As such, they may not be totally objective measures of job
performance.
2. A lack of independence between the predictor and the
criteria variables may occur when personnel selection
examination results are made available to an employee's
supervisor. The possible effects of this criterion
contamination, where it was identified, were addressed in this
study.
3. Restriction of range occurred when the original
population or sample limited itself, or was limited by,
factors which related to the selection and subsequent
employment process. Thus, if 50% of those interviewed were
hired, and the dependent variable, administrative achievement,
was determined only for those hired, the effective sample
variance for the study was cut in half. This affects the
subsequent correlation between that selection method and any
measure of administrative achievement. For this reason, the

13
possible effects of range restriction, where they were
identified, were addressed in this study.
Definition of Terms
Administrator. An administrator is any person who manages
or directs the affairs of an institution or any major part
thereof. As pertains to higher education, anyone above the
level of professor, such as department chairman or
professional central administrative personnel, is considered
an administrator.
Community college. A community college is a two-year
college, offering academic, general education, vocational
training, terminal, and transfer programs. It can also be
known as a junior college.
Corporation. A corporation is any company or related
group of companies which produces goods and/or services for
profit. Usually this type of organization has a quantifiable
measure of success by which to judge the effectiveness of the
administrators and employees, such as number of units produced
or amount of profit made. Alternative terms for "corporation"
in this study include those such as the corporate world,
business and industry, or companies.
Higher education. Higher education consists of public and
private community colleges, four-year colleges, and
universities in the United States.

14
Meta-analysis. The statistical analysis of a large
collection of analysis results from individual studies for the
purpose of integrating the findings.
Military services. Military services are national-level
military organizations in the United States.
Public administration. Any agency which provides public
services at the federal, state, county, city, district, or
regional level is considered a public administrative agency.
Supervisor. A supervisor is a person who oversees,
directs, or manages work or workers.
Assumptions
The following assumptions have been made in this study:
1. It has been assumed that a meta-analysis of the
research on administrative personnel selection was an
appropriate methodology for determining those methods which
may, when properly implemented, improve the selection of
administrators of community colleges.
2. It has been assumed that the research included in this
study was guarded against restriction of range and criterion
contamination wherever possible.
Procedures
The purpose of this study was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. The research method used was a

15
meta-analysis of administrative personnel selection research
done in the United States in the past 24 years. Procedures
used in this study follow those described by Glass (1977) and
in Hunter, Schmidt, and Jackson (1982) for the meta-analysis
of a sample of independent correlational studies. The steps
involved included the following:
1. Identification of the research studies.
2. Classification of each study dependent variable,
administrative achievement, according to the Delimitations
section of this chapter.
3. Classification of each study independent variable,
selection method, according to the Delimitations section of
this chapter.
4. Calculation of frequencies of selection methods used
to answer the questions: (a) What were the most commonly used
selection methods across all types of organizations, and (b)
what were the most commonly used selection methods in each
type of organization.
5. Calculations of the means and standard deviations from
the means in selected cross-sections of the data were used to
determine which selection methods were significantly related
to achievement in administrative positions.
6. Determination of what information, derived from the
meta-analysis of the methodologies of the research studies
used, can be useful in interpreting the findings of the above
analysis of selection methods.

16
7. Analyzation of the findings of the above questions to
determine how the frequency of use of each selection method
related to its ability to predict administrative achievement.
8. Derivation of guidelines useful in improving personnel
selection methods for administrators of public community
colleges.
9. Presention of the data in textual form supported by
frequency tables and figures, and supported by tables and
figures reporting means and standard deviations of the
correlational data.
Organization of the Dissertation
A review of the literature pertaining to administrative
personnel selection methods is provided in Chapter Two.
Chapter Three presents a detailed discussion of the meta-
analytic research procedures used in this study, such as
methodology, sources of data, data analysis techniques, and
presentation of the data. A presentation of the findings of
the meta-analysis of the research studies used is contained
in Chapter Four, thus answering the questions posed in the
Statement of the Problem. Chapter Five contains the
guidelines derived from the meta-analysis of the research
studies and the presentation of these guidelines for
application to the selection of community college
administrators.

CHAPTER II
BACKGROUND LITERATURE
Introduction
The responsibilities inherent in administrative positions
have undergone a complex evolution since the turn of the
century. In many organizations, however, the selection
process for administrative personnel has not seen appreciable
change. In this review, the focus is on the research
literature pertaining to the usefulness of various selection
procedures in the prediction of administrative success. The
review is organized into the following sections: the history
of personnel selection; a discussion of objectivity,
reliability, and validity in personnel selection research; a
review of the personnel selection methods used as independent
variables in personnel selection; introduction of the research
studies reviewed; and a summary and critique.
History of Personnel Selection
Traditionally, Munsterberg's 1911 experiment with motormen
has been viewed as the beginning of research in the use of
examinations for personnel selection (Ghiselli, 1973).
Nevertheless, anecdotal evidence leads one to suggest that,
even before 1910, other psychologists conducted similar
studies with tests, but these were small in scope and went
17

18
unpublished. Under the impetus of the scientific management
movement of the early 1900s, some efficiency experts of that
time were using simple exams for evaluating applicants for
jobs. They reported fragmentary evidence of validity in the
attempt to justify their activities (Ghiselli, 1973). The
earliest review of an industrial application was in 1915 when
Scott (in Arvey & Campion, 1982, p. 283) reported low
reliability for evaluations given by six personnel managers
who had interviewed the same 36 sales applicants.
During World War I, the large-scale testing both of
soldiers and industrial workers provided stimulation,
methodology, and respectability to the examination of the
utility of examinations in the assessment of occupational
aptitude. In 1923, Freyd (in Guin, 1987) published a 10-step
outline for personnel selection. This outline was so complete
that it differs little from the procedures used in personnel
selection research today. This all led to a post-war surge
of systematic research in personnel testing.
Selection Processes in Business and Industry
Prior to World War I, most businesses were owner-managed
by the individuals who had founded them. But the growth of
the economy since the 1920s brought changes which have
replaced these colorful characters with impersonal
corporations. The function of the administrator had become
such that it required many skills and much knowledge, so that
it often took a number of administrative personnel to assist

19
in decision making. By the 1960s, corporate management
required a knowledge of a variety of fields and operations
rarely needed in business a few years earlier. Corporations
were growing, combining, and expanding so fast that they
needed to hire competent executives at a pace which could not
be fulfilled by promotion from within the ranks (Sands, 1963,
p. 3 ) .
One particular study was more comprehensive than much of
the personnel selection research of the 1950s and 1960s. This
was the American Telephone and Telegraph (AT&T) (Bray & Grant,
1966) study from which came the assessment center method for
the selection of administrative personnel. Called the
Management Progress Study, initiated in 1956, it is the first
known corporate research and application of the assessment
center method. The 422 men brought into the study were
followed up annually and reassessed at 7-year intervals in a
effort to keep track of their professional development. The
assessment center results were not revealed to the higher AT&T
management personnel, so that the progress of the men would
not be affected by the assessment findings. The results of
this study have been well documented. Between 1961 and 1967,
only this AT&T study and two studies done by the armed forces
on assessment centers were reported in the literature
(Crooks, 1973, p. 1).
i

20
Selection Processes in Public Administration
When public administration systems were initiated, it was
assumed that only people of the highest caliber would apply
for service. Once reality dashed this assumption, public
administrative personnel selection systems were built upon the
cornerstone of the competitive examination, a method of
deselecting inferior job candidates (Stahl, 1976, p. 129).
From the beginning, this system has placed special emphasis
upon formal selection procedures such as testing for specific
knowledge and skills related to the particular job the person
was applying for. The starting points for the selection
process are basically two: a determination of the objective
for selection, which may be a given position, occupation,
program, or service-wide career, and a setting of basic
standards for selection, the skills and knowledge that are
necessary to meet the preceding objective. During the early-
to mid-1960s, some public administration organizations
realized a need to ensure at least an adequate intake of high
caliber people at all levels so that there would be no
shortage of talent when movement upward or outward took place.
Thus, some services began to give more attention to the
selection of persons who possessed a capability of growth and
development. To this end, public administrative organizations
began to implement the assessment center approach.
However, most public administrative organizations select
a person to fill a particular job vacancy, and thus their

21
testing is for a specific competency in that area, not an
overall assessment of the applicant's potential. Most of
these agencies expect personnel to have learned their skills
previous to application and do not intend to take in an
inexperienced person of promise and then train him or her for
an administrative position.
Most of the background studies cited as a basis for
personnel selection in public administration were actually
conducted by private corporations. Many of the selection
practices have been borrowed and adapted from business
research. AT&T's original industrial development of the
assessment center method was so well grounded in research, it
was often casually implied that therefore any assessment
center would pick the "right" person (Ross, 1979, p. 41).
Selection Processes in the Military Services
Although many fields acquire administrators already
trained for the positions they are hired to fill, the military
services often must select people of officer quality and then
pay the additional cost of training them. Thus, the basic
objective in the selection process is to identify measures
which would result in officers entering the force with a high
probability of success. The services must select personnel
not only to fill a particular position as it becomes
available, but also to identify people who have the ability
to be trained repeatedly, sometimes every three years, for a

22
variety of different positions (Akman & Nordhauser, 1974, p.
3) .
Often, the services must select officer-quality personnel
to fill positions which have no equivalent in the nonmilitary
world. In these cases, either very qlobal criteria must be
set to determine if the candidate has the intelligence or
aptitude to learn the job-required skills and then be
successful at it, or very specific abilities or natural
characteristics must be measured to ensure the candidate will
be well suited for an important aspect of the position.
Because a large number of people must be tested and screened
for placement, the military services have developed many
paper-and-pencil tests to measure both general and specific
aptitudes. They have relied heavily upon past achievement,
such as college degrees and the results of officer candidacy
tests. The importance of test scores and military class
standings is shown by the fact that often one's scores and
future attainment of rank are positively correlated.
Selection methods used in the military services and in
public administration are often classified as "successive
hurdles" techniques. A lower than acceptable score on any one
particular screening test, or part of a test, will either
disqualify the candidate from that service or from particular
jobs within the service (Shoop, 1974, p. 341).

23
Selection Processes in Higher Education
In higher education, search committees are often used for
recruiting administrative personnel. These committees provide
for maximum participation in the selection process by a
variety of constituencies within the institution. The
membership of the search committee depends upon the vacancy
to be filled. The committee may include faculty and staff
members, administrators, and, in some cases, students. A
member of the central personnel staff is often an ex-officio
committee member in order to orient the committee to the
proper and legal selection procedures (Sprunger & Bergquist,
1978, p. 116). Usually, the search committee recruits,
screens candidates, checks references, participates in
preliminary interviewing, and recommends a fixed number of
candidates to a designated administrator, who makes the final
selection (Fortunato & Waddell, 1981, p. 107).
The initial screening of candidates has usually been done
by a review of the vitae and job applications. Supplemental
information has usually been obtained by talking to references
and others who knew the individuals and their work. The on-
campus interview has been one of the most important steps in
the selection process. The manner in which this interview is
conducted has usually been critical to the success of the
recruitment-selection process (Sprunger & Berquist, 1978, p.
119). Webster (cited in Grove, 1981, p. 56) emphasized that
the interviewer must understand what behavior is required of

24
the person finally selected to fill the position. He stressed
such practices as reviewing the application form with the
applicant, giving the applicant an opportunity to talk about
himself or herself in a way that helps that individual to
present relevant evidence, making the interview more
systematic and probing, examining test scores and other
information, and weighting the various types of information
in order to reach an intelligent decision. Webster
recommended taking time following each interview to clarify
impressions and to formulate a judgment. He suggested making
use of multiple independent evaluations in which two or more
interviewers record decisions independently and subsequently
reconcile differences.
Objectivity, Reliability, and Validity in Personnel
Selection Research
These three primary considerations enter into the
effectiveness and propriety of examination processes for
employment selection: objectivity, reliability, and validity.
Objectivity is even more important in the 1980s than it was
previously due to the legal demand for testing to have a job-
specific relation. Reliability is important but is often not
reported in personnel selection research. Validity is the
major concern of this dissertation, and the research studies
reviewed here pertain to the validity of each selection method
in predicting administrative achievement.

25
Objectivity
According to Stahl (1976), one of the prime reasons for
professionalizing all steps in the selection process is to
ensure thoroughgoing objectivity (p. 131). Only those methods
which disregard extraneous factors such as race, religion,
politics, sex, residence, and age can be considered thoroughly
objective. An objective selection method should identify
those characteristics of mind and skill necessary, and only
those necessary, to the given purpose, whether the purpose is
to fill a particular position or to begin a career.
Objectivity is not only desirable but is mandated in
hiring for positions in the public sector. Throughout almost
all of American history, legislation has been enacted to
facilitate equal treatment in employment. An abundance of
case law has built up around civil rights legislation, and
rulings have occurred which have broadened the impact of such
legislation. This broadening has had the effect of placing
more and more constraints on what employers are and are not
allowed to do in terms of employee selection (Burrington,
1982, p. 55).
Two landmark cases have set the serious tone with which
violations of civil rights in hiring will be met. In the U.S.
Supreme Court case of Griggs vs. Duke Power (401 U.S. 424,
1971), an employer rejected black job applicants on the basis
of lack of completion of high school or on the results of a
general intelligence test. Evidence showed that employees who

26
had not completed high school or taken the tests had continued
to perform satisfactorily and made progress in departments for
which the high school and test criteria were not used
(Burrington, 1982). It was ruled that employment standards
and tests which are not significantly related to job
performance violate the Civil Rights Act of 1964, Title VII
(Alexander, 1980, p. 515).
The second case, Edward L. Kirkland et al.. Plaintiffs v.
New York State Department of Correctional Services et al..
Defendants (73 LIV. 1548, 1974), was brought under the Fifth
and Fourteenth Amendments to the U.S. Constitution and under
the Civil Rights Acts of 1866 and 1871. The case concerned
a Correction Sergeant (male) written examination which had the
impact of allowing only 1.9% of blacks, and no Hispanics, to
be eligible for promotion. No recourse was made to the Civil
Rights Act of 1964, so the Equal Employment Opportunity
Commission (EEOC) Guidelines, required under that act, were
not binding. However, the U.S. District Court, Southern
District, New York, considered these Guidelines in evaluating
the job relatedness of the examination. A number of
implications can be deduced from this case for personnel
selection in general and the examination process in particular
(Wisner, 1975, pp. 266-267).
1. In rendering a decision, the court is likely to
follow the EEOC Guidelines even though the case may not have
been brought under the Guidelines.

27
2. A job analysis should be performed prior to the
development of each examination.
3. Those critical elements, or more important
characteristics of the job, which distinguish between those
likely to be successful and those less likely to be
successful, must be included in the examination.
4. Subject matter experts, to be considered qualified,
must be of an appropriate rank and sufficiently experienced
in the current skills, knowledges, and abilities required in
the job for which the test is developed.
5. Discriminatory test results cannot be refuted by
demonstrating that the results occurred because of certain
environmental conditions or cultural characteristics related
to the protected group population unless it can be shown that
the examination is job-related.
6. The fact that specific parts of the test are not
found to be discriminatory does not serve to support the use
of an examination whose overall results are found to be
discriminatory.
7. All subtests included in the examination must be
weighted in accord with their importance to the job.
8. Data should be obtained concerning the examination
results immediately after administering the exam to determine
the examination's impact on protected groups.
9. Every effort should be made to set passing scores at
a point considered to separate the candidates most likely to

28
be successful from those considered to have the least chance
of success.
10. Be prepared to answer the question "why was this
method of measurement considered the best means to employ?"
11. In court, do not rely too heavily on your
professional witness to attest to the job relatedness of your
examination.
Thus, the public employer has but one choiceto validate
the employee selection procedures. Fortunately, the choice
is the best one to follow from the standpoint of good
personnel management.
Reliability
The consistency with which a method measures what it is
expected to measure is called reliability (Ary, Jacobs, &
Razavieh, 1979, p. 206). In order to achieve consistency, a
selection test or method must leave little room for chance in
the subject's final score or rating. If a test is reliable,
a person taking it at two different times should make
substantially the same score or be ranked in approximately the
same position each time. Test reliability depends also on the
nature of the variable being measured. For example, the trait
of academic achievement can be measured more consistently than
that of personality. Although all measurements of human
qualities contain some error, no test or method of personnel
selection is of value unless it has a high degree of
reliability (Stahl, 1976, p. 133).

29
Reliability can be affected by varied administration of
a selection procedure and by the scoring or judging of an
applicant's performance. Inexperienced persons or committees
may deviate from standardized or prescribed procedures in
testing an applicant. Vague observational techniques or
scoring of an applicant's behavior or responses leads to an
unreliability which has nothing to do with the competency of
the candidate. The difficulty of any selection method or
examination affects its reliability. When a method is
difficult, the subjects guess on most of the questions and a
low reliability results. Conversely, if the type of
examination is easy, all subjects have correct responses on
most of the items and only a few more difficult items
discriminate among subjects. This again results in a low
reliability. Thus, any selection committee must keep in mind
the fact that they must adjust the substance of questions
asked in order to discriminate subtle differences between
similar candidates on traits that are difficult to measure.
The achievement of a high degree of reliability in
selection procedures for administrative positions in a
community college may be difficult. In addition to the
problems inherent in assessing traits which elude measurement
by persons who are often inexperienced in the task,
reliability can be affected by the lack of heterogeneity of
the group of applicants. The greater the homogeneity of the
group of subjects being considered, the lower the reliability

30
of any selection method in differentiating among those
subjects. Candidates for these administrative positions are
likely to be quite similar in the traits being measured.
Therefore, it is more difficult to rank them than it would be
to rank the same number of subjects chosen randomly from the
entire population. This restriction in range is a problem in
most personnel selection research as very seldom is the sample
drawn from the population at large.
Validity
Validity is defined as the extent to which an instrument
or method measures what it is intended to measure. Unlike the
physical sciences, where there is a direct means of measuring
the outcome variable, in personnel selection indirect means
must be used to measure complex attributes.
The type of validity addressed in this study is criterion-
related validity. Criterion-related validity refers to the
relationship between the scores on a measuring instrument and
an independent external variable (criterion) believed to
measure directly the behavior or characteristic in question
(Ary, Jacobs, & Razavieh, 1979) In this type of validity the
emphasis is on the criterion rather than on the instrument
itself. One is primarily interested in what the instrument
can predict rather than in the test content.
There are several characteristics that a criterion measure
should possess. One must judge whether the criterion chosen
as the dependent variable really represents successful

31
performance of the behavior in question. If the criterion
does not reflect the attribute under study, it would be
meaningless to use it as a basis for validating another
instrument. In the case of this study, the outcome criteria
of supervisor's ratings, longevity in a position, and
salary/position or rank attained were deemed to be adequate
indicators of success in an administrative position.
A second characteristic is that a criterion must be
reliable. The criterion must be a consistent measure of the
attribute over time or from situation to situation. The
criteria chosen for this study, rank, position, and salary
achieved, have generally been considered to be related to
success. Length of time in an administrative position within
one organization may have been a mark of high achievement in
the past more than it is today. Now many administrators look
upon great longevity as an indicator of stagnation rather than
of achievement. So this indicator may not be as relevant to
later studies as to earlier ones.
Finally, a valid criterion should be free from bias. The
scoring should not be influenced by any factors other than
actual performance. The main problem in relation to personnel
selection research is a form of bias called criterion
contamination. This occurs when an individual's score on the
criterion, such as supervisor rating, is influenced by the
scorer's knowledge of the subject's predictor score.
Contamination of the criterion can be prevented by not

32
permitting the person who grades or rates the criterion to see
the predictor scores.
Although perfect criterion-related predictive validity is
practically impossible to achieve, a reasonable amount of
validity is possible. Selection upon the basis of tests or
methods which have no known validity may be little different
from selection upon the basis of a turn of a card. Yet some
personnel agencies have chosen to use methods which have not
been subjected to validity study.
The process of determining the validity of a selection
method involves statistical correlation between the various
examination results and some criteria of performance on the
job. Where the worker is engaged in production activities in
which it is easy to measure output, the question of the
criterion presents no problem. But the simplicity of the
problem disappears when one tries to find adequate criteria
of performance for an administrator. In these cases, the
researcher must rely upon appraisals by those closely
acquainted with the work of the particular administrator
(Stahl, 1976, p. 132).
Kirchner and Reisberg (1962) pointed out the problem of
determining an adequate criteria of managerial performance (p.
301). Supervisors' ratings of the performance of
administrative personnel were found to be subjective and
widely varied. Successful supervisors preferred subordinate
administrators who showed initiative toward organizational

33
goals, whereas supervisors considered to be less successful
rated their subordinate administrators more highly who
followed orders and got along with others.
There are two main procedures by which the validity of an
examination or method for selection is best determined. Each
is supplementary to the other. In the first instance,
concurrent validity, the exam may be given to administrators
of known ability already on the job. If those administrators
who have been predetermined to be most successful score
highest in the evaluation, while the least successful ones
score the lowest, the evaluation method is said to have
evidence of validity. The second procedure consists of long
term follow-up studies of the performance of those employees
who have been selected for administrative positions (Stahl,
1976, p. 132).
Review of the Research on Selection Methods
for Administrative Personnel
Methods for personnel evaluation have been described in
as many different ways as there have been researchers to
describe them. According to Northcott (1960, p. 289), the
procedures in standard use have fallen into three groups:
questionnaires, tests, and interviews. Stahl (1976, p. 136)
stated that the forms which examination can take may be
classified into five categories: (a) a systematic evaluation
of education and experience, (b) oral tests, (c) standardized
qualification inquiries, (d) written tests, and (e) tests of

34
personality traits. According to the "Uniform Guidelines on
Employee Selection Procedures," issued by the Equal Employment
Opportunity Commission, the United States Department of
Justice, the United States Civil Service Commission, and the
United States Department of Labor in 1978, selection
procedures are defined as including "the full range of
assessment techniques from traditional paper and pencil tests,
performance tests, training programs, or probationary periods
and physical, educational, and work experience requirements
through informal or casual interviews and unscored application
forms" (Quaintance, 1980, p. 126).
The selection methods reviewed in this study have been
delineated into the following types: (a) aptitude and
intelligence measures, (b) personal interviews, (c) job-
related skills indicators, (d) psychological attribute
indicators, (e) value systems assessments, (f) biographical
information, (g) peer ratings, (h) self appraisals, and (i)
assessment center processes. These methods are the
independent, or predictor variables of the administrative
personnel selection research studies used in this meta
analysis. Statistical data and methodological details of each
of the research studies reviewed for this dissertation are
found listed alphabetically by researcher's name(s) in the
Appendix.

35
Aptitude and Intelligence Measures as a Selection Method
In a large proportion of the research, at least one type
of intelligence or mental aptitude test has been used. Most
of the tests utilized measured verbal abilities, but some
measured critical thinking, or mathematical or mechanical
aptitudes.
In a review of 12 research studies aimed at relating a
measure of aptitude or I.Q. to later measures of
administrative achievement, Korman (1968) found that the
"typical managerial applicant population is already highly
preselected on abilities and is relatively homogeneous on
these variables" (p. 301). Thus, the tests of ability do not
tend to discriminate finely among such a population in which
preselection has already taken place. Korman states that
"verbal ability tests seem to show little usefulness for
predicting managerial performance above the first-line
supervisory level" (p. 297).
Dating back into the 1940s, cognitive ability research
showed mixed results. Williams and Leavitt (1947), using
Marine Corps officers as subjects, showed no significant
relationship with supervisor ratings. However, Knauft (1949)
found a significant relationship with later ratings. In the
1950's Handyside and Duncan (1954) and Meyer (1956), each
using supervisors as subjects, found significant relationships
with ratings, but Comrey and High (1955), Ricciuti (1955), and
Thorndike and Hagen (1959) did not.

36
Research used in this study. A total of 15 research
studies were found which used aptitude and intelligence
measures as a predictor variable and met the other
delimitations of this dissertation. These studies yielded 21
correlation coefficients which ranged in strength from r = .01
to r = .41. Samples ranged in size from 26 to 8,885 persons.
Hicks and Stone (1962) while working for Aerojet-General
Corporation and California State Polytechnic College,
respectively, evaluated the effectiveness of a test battery
in discriminating between successful and unsuccessful
managers. One part of this concurrent study related the
mental ability of managers, supervisors, shop foremen, and
engineering supervisors, to the outcome measure, supervisor
rating. Results were not significant. The researchers
concluded that mental abilities did not show a large
relationship to managerial success probably as the result of
sample attenuation through preselection on education and
career achievement.
Williams and Harrell (1964), while at San Francisco State
College and Stanford University, respectively, did a follow
up study of Stanford MBA graduates. The purpose was to find
predictors which were significantly correlated with on-the-
job achievement. While the grade point averages for
undergraduate courses and for required graduate courses fell
short of significant correlations with the success criterion,
there was a significant correlation between grades on elective

37
graduate courses and salary. The researchers suggested that
research attention might be directed at some figure which
would concentrate on the elective course area.
Dicken and Black (1965), while at Stanford University,
explored the validity of clinical interpretations of an
objective test battery in two different corporate settings.
Subjects were first-level supervisors in a manufacturing
organization and in an insurance company. Only the highest
of the derived correlations was significant. The researchers
acknowledged the restricted range of their samples and its
effect on predictability, stating that "measures of ability
and interest cannot be expected to make fine discriminations"
(p. 46).
In 1965, Tenopyr and Ruch (in Campbell, Dunnette, Lawler,
& Weick, 1970) studied the relationship between intelligence
test results and the salary level attained by corporate
production managers. The resulting correlation coefficient
in this concurrent study was significant.
Bray and Grant (1966), while with American Telephone and
Telegraph Company, sought to study the assessment center
process. This study has become the landmark study, to date,
in assessment center development. A sample of low-level
management employees of AT&T were tested in this carefully
designed assessment center. This was one of the few studies
in personnel selection research where no criterion
contamination was allowed. A follow-up of the subjects eight

38
years after testing showed a significant correlation between
three measures of mental aptitude and the outcome variable of
salary level achieved.
In favor of situational tests over mental ability tests,
the researchers stated that when mental ability as measured
by a paper-and-pencil test was partialed out of judged
ability, reliable variance still remained. They found that
overall and combined assessment ratings did tend to be higher
than the correlations for any individual technique (Bray &
Grant, 1966).
In 1967, Bentz (in Campbell et al., 1970) summarized the
results of the Sears, Roebuck Company psychological testing
program. In relating aptitude and intelligence measures to
the prediction of executive effectiveness, a significant
correlation was found. Bentz stated that the successful
administrators were superior in intellectual endowment.
Kraut (1969), while working for IBM Corporation, sought
to determine the relationship between high level managers'
promotional success and measures of their intellectual
ability. No significant relationships were found. The
researcher cited problems with the predictor tests and range
restriction of the sample as possible reasons for the results
obtained.
Wollowick and McNamara (1969), while with IBM Corporation,
sought to determine the validity of an assessment center
approach in predicting management potential and to determine

39
the relative value of the components of the program. Within
this study, they found that two measures of aptitude and
intelligence were not significantly related to the outcome
variable. The researchers suggested that "it may be possible
to consider eliminating the paper-and-pencil tests not
contributing to the predictive validity" (p. 352).
Campbell et al. (1970), in a study involving the Standard
Oil Company of New Jersey, attempted to discover how employees
who possess the potential to be successful in management could
be identified early in their careers. Although the
intelligence and aptitude measures used were not significantly
related to the outcome criterion, Laurent (in Campbell et al.,
1970) concluded that successful managers "have shown a total
life pattern of successful endeavors" (p. 169).
Moses (1972) studied the relationship between assessment
and subseguent progress in management for personnel in the
Bell Telephone System. This study was part of the AT&T
assessment center process, but not a part of the original
study (Bray & Grant, 1966). A significant relationship was
found between aptitude and intelligence measures. The study
involved a large sample of nonmanagement personnel.
Moses and Boehm (1975) replicated the study done by Moses
in 1972, but the sample consisted of nonmanagement females.
A significant relationship was found in this study. The
researchers stated that "the assessment process predicts the

40
future performance of women as accurately as it does that of
men" (p. 529) .
Grimsley and Jarrett (1975), while with the University of
Southern California and California State University,
respectively, studied the relationship between past
achievement test scores and criterion measures obtained in
administrative positions. The significant results found in
this concurrent validity study led the researchers to conclude
that "the differences in test scores of more or less
successful managers result from fundamental differences in
mental ability and personality rather than the influence of
on-the-job experience" (p. 226).
Huck and Bray (1976) studied a sample of non-management
females derived from the AT&T assessment center process.
Their purpose was to test the validity of an assessment center
process on a population different from the population of the
Management Progress Study (Bray & Grant, 1966). Although the
relationship between aptitude and intelligence measures and
the criterion variable did not prove to be significant, the
researchers found that "judgments made by assessment center
staffs are good predictors of later performance" (p. 27).
Gantz, Erickson, and Stephenson (1977) sought to determine
why some persons in a research and development population
achieve promotion, particularly into supervisory or managerial
positions, and other persons do not. Measures of aptitude and
intelligence were not significantly related to either

41
promotion rate or to supervisor ratings in this concurrent
study.
Hinrichs (1978) evaluated the predictive validity of the
AT&T assessment center process and compared the predictive
accuracy of the assessment center with the naturalistic
management evaluation. No significant findings were obtained
between aptitude and intelligence measures and administrative
level achieved. The researchers found that "prediction based
upon managerial review of the personnel files did as well as
the assessment center after 8 years" (p. 600).
Conclusions. Restriction of range was acknowledged as the
biggest drawback to the predictive ability of aptitude and
intelligence measures. Preselection of the samples by
management tended to reduce predictability (Dicken & Black,
1965). Intelligence, as measured typically by verbal ability
tests, is a fair predictor of first-line supervisory
performance but not of higher-level managerial performance
(Korman, 1968). The conclusions drawn by most researchers in
explaining the general lack of a significant relationship
between aptitude and intelligence measures and measures of
administrative achievement would support the statement that
"measures of ability and interest cannot be expected to make
fine discriminations" (Dicken & Black, 1965, p. 46).
Personal Interviews as a Selection Method
"The use of the interview as a device for appraising
applicants for a job is generally regarded with a good deal

42
of suspicion and distrust by industrial psychologists
(Ghiselli, 1966, p. 389) A factor that very likely has
produced disaffection with the employment interview is its
nebulous and intangible character. The interview involves a
social interaction between the interviewer and the applicant,
thus it varies substantially in form and content from one
applicant to another for one and the same interviewer.
In attempting to improve the selection interview,
industrial psychologists have sought to develop sets
of rules to follow in order to improve the reliability
and validity of the interview. To some extent these
rules are based upon scattered empirical evidence, but
because research on the interview is so very limited
they are more commonly based upon professional
judgment or sheer common sense. Mayfield's (1964)
attempt to integrate the pertinent research can only
be described as heroic. Yet the conclusions and
generalizations he arrives at are but weakly founded.
(Ghiselli, 1966, p. 390)
Early studies done on the personal interview method by
Binet and Scott (in Arvey & Campion, 1982) showed low
interrater reliabilities. Wagner (1949), in a summary of
interview research, noted low validities in most cases. In
1964, Mayfield reviewed all interview research done since
Wagner (1949) and found a median validity correlation
coefficient of r = .27. He stated that this value was not
"particularly high" (Arvey & Campion, 1982, p. 284).
Research used in this study. A total of 6 research
studies were found that used personal interviews as a
predictor variable and met the other delimitations of this
dissertation. These studies yielded 7 correlation

43
coefficients which ranged in strength from r = .03 to r = .35.
Samples ranged in size from 122 to 8,885 persons.
Ghiselli (1966), while at the University of California,
attempted to evaluate the validity of data gathered during
personal interviews in predicting administrative job
proficiency. The independent variable was the personnel
interview used in a corporation over a 17-year period in
hiring account executives. The outcome criterion was survival
with the company for a 3-year period. The interview was
relatively unstructured and included no questions that were
of a highly personal nature. However, some questions that
would not be legal today in a public employment interview were
used. A significant correlation was found between the two
variables in this highly range-restricted sample. Ghiselli
thus found the interview to have "at least a moderately
substantial validity" (p. 394).
Grant and Bray (1969) studied the contributions made by
the interview to the assessment center process, and also, the
relationships between interview variables and the progress
criterion of salary level attained after 9 years. These
relatively unstructured interviews covered job-related and
personal topics. The interviewers were professional
psychologists. For the purpose of the study, the researchers
divided the interview into 18 different personality variables
and related each variable to the outcome criterion. The
significant relationship found showed that "the interview did

44
produce reliable ratings of managerial qualities which related
significantly with ratings made on the basis of several other
techniques and with advancement" (p. 34).
Carleton (1970), while with the Standard Oil Company, Ohio
(SOHIO) assessment program, studied the relationships of
several personnel selection methods to the outcome criterion
of supervisor rating. A period of 2.5 to 5 years elapsed
between measurement of variables. The researcher concluded
that "results of the interview report appeared particularly
impressive in light of the usual criticism of the interview
as an assessment technique" (p. 566).
Campbell et al. (1970), while at the University of
Minnesota and Yale University (Lawler), in the Standard Oil
Company of New Jersey assessment center study, reviewed the
relationship between the personal interview and later measures
of administrative achievement. The outcome criterion
consisted of a combination of performance ratings to include
salary level attained, supervisor ratings, and administrative
level achieved.
Moses (1972) conducted a study of 8,885 nonmanagement
males. These men, while employed by the Bell System, were
interviewed as part of the assessment center process. A
significant correlation was found with the criterion of
administrative level achieved.
Turnage and Muchinsky (1984) conducted a study based upon
data from a one-day supervisory selection program (assessment

45
center) developed by a large manufacturing firm. The subjects
were 799 nonmanagement employees who were subsequently
promoted to supervisory positions. The individual interview
was based largely on the candidate's questionnaire responses,
in which the candidate was required to play the role of a
supervisor faced with discharging an employee. In this
respect, this interview method overlaps the method of job-
related skills indicators. The predictor variable, interview,
was correlated separately with three criteria: salary level
attained, supervisor ratings, and promotion rate. Only one
resulting coefficient was significant. Based on these, and
other findings in the study, the researchers concluded that
there appeared to be no appreciable relationship between how
one is evaluated in an assessment center and how one performs
on the j ob.
A number of reasons were surmised by the researchers for
the lack of correlation. Methodological factors such as low
criterion reliability, low predictor reliability, severe
restriction of range, marked skew in the data, procedural
inconsistencies, lack of comparability across assessment
groups, and errors in data collection, were cited as possible
problems within this concurrent study.
Conclusions. The structured personal interview, in which
the interviewer, or interviewers, follow a set schedule of
questions, has consistently shown higher reliabilities than
the unstructured interview (Wright, 1969). In addition,

46
interviews conducted by a board or panel appear to be
promising as a means of enhancing reliability and validity
(Arvey & Campion, 1982). Greater validities may be found if
researchers first decide the purpose the interview is intended
to serve. It has been suggested that the personal interview
is useful in determining interpersonal skills and motivation
(Schmitt, 1976), and in imparting job information from the
interviewer to the applicant (Arvey & Campion, 1982).
The employment interview continues to be used although
organizational psychologists are aware of the findings
concerning the method's limited reliability and validity.
Arvey and Campion stated that "interviewers ignore base rate
information, do not pay attention to disconfirming evidence,
and over-depend on case specific information in making their
judgments" (Arvey & Campion, 1982, p. 316).
Job-related Skills Indicators as a Selection Method
Situational methods offer the potential of adding to the
scope of human characteristics that can be evaluated. The
rationale behind using situational exercises is that they
simulate the type of work to which the candidate will be
exposed and allow his or her performance to be observed under
somewhat realistic conditions. Situational tests measure more
complex or dynamic behavior rather than aptitudes or traits
isolated by more traditional psychometric tests. The whole
personality is observed in interaction with simulations of the
future job environment (Howard, 1974).

47
The most commonly used situational tests of job-related
skills for administrative positions are examined here. The
in-basket exercise is one of those most frequently used in
assessment centers. The candidate is faced with an
accumulation of memos, reports, notes of incoming telephone
calls, letters, and other materials supposedly collected in
the in-basket of the job he or she is to take over. The
candidate is asked to dispose of these materials in the most
appropriate manner by writing letters, notes, self-reminders,
agenda for meetings, etc. Ratings of performance range from
subjective evaluations to highly standardized checklists.
In the leaderless group discussion, the participants are
given a discussion question and are instructed to arrive at
a group decision. Topics may include such things as promotion
decisions, disciplinary actions, or business expansion
problems. Sometimes participants are given a particular point
of view to defend. Personality dimensions such as
interpersonal skills, acceptance by the group, individual
influence, and leadership abilities can be evaluated in this
way.
Management games, such as stock market tasks,
manufacturing exercises, and merger negotiations are common
job-related selection exercises. Participants are asked to
solve problems, either cooperatively or competitively. These
games often bring out leadership, organizational abilities,
and interpersonal skills. Some games also permit observations

48
under stress, especially when conditions suddenly change or
when competition stiffens.
Research used in this study. A total of 10 research
studies were found that used job-related skills indicators as
a predictor variable and met the other delimitations of this
dissertation. These 10 studies yielded 12 correlation
coefficients which ranged in strength from r = .02 to r = .49.
Samples ranged in size from 31 to 8,885 persons.
In 1965, Tenopyr and Ruch (in Campbell et al., 1970)
studied the relationship between job-related skills and salary
level attained in a concurrent validity study. Subjects were
production managers at North American Aviation. They were
given a test designed to assess a supervisor's ability to
handle human relations problems. Although a significant
finding resulted, the researchers concluded that this
correlation was "useful" but not "overwhelming" (Campbell et
al., 1970, p. 193) .
Dicken and Black (1965) studied the validity of clinical
interpretations of an objective test battery in an industrial
setting. A sample of first-line supervisors in a
manufacturing company was given tests of job-related skills.
These tests were administered to the participants individually
and in groups. A significant correlation was found between
job-related skills indicators and salary level attained after
3.5 years.

49
Bray and Grant (1966) found that job-related, or
situational, selection methods had considerable influence on
the judgments of the assessor. Job-related skills indicators
of a sample of managers were related to salary level attained
8 years later. A significant correlation was found. The
situational exercises used consisted of an in-basket exercise,
a simulated manufacturing problem, and a leaderless group
discussion. The researchers concluded that the job-related
situational exercises contributed enough to predictivity to
warrant their relatively high costs in time and money.
Wollowick and McNamara (1969) studied the validity of an
assessment center approach in predicting management potential
and to determine the relative values of the components of the
program. One component of the assessment center consisted of
both written and performance tests of job-related skills.
These tests were administered, individually and in groups, to
a sample of low- and mid-management men. The results of these
tests were significantly related, approximately three years
later, to administrative level achieved.
Meyer (1970), while at General Electric Company, studied
the relationship between a job-related skills indicator, the
in-basket exercise, and an outcome criterion of supervisor
ratings. A demonstrated inability of a general aptitude test
to determine which employees would become successful unit
managers led to the use of this method. This exercise was
developed by John Hemphill, then of the Educational Testing

50
Service. A comparison of overall score on the skills
indicator test and the outcome variable yielded a significant
correlation in this concurrent study. Meyer concluded that
the "performance style one exhibits in handling carefully
selected, but true-to-life 'in-basket' items does correlate
with demonstrated on-the-job performance of a managerial
position, especially the ability to handle the planning and
administrative aspects of the job" (p. 307).
Campbell et al. (197 0) studied the relationship between
job-related skills indicators and a criterion measure of
administrative achievement. The outcome criterion was a
combination of performance ratings, to include salary level
attained, supervisor ratings, and administrative level
achieved. This concurrent validity study was a component of
the Early Identification of Management Potential (EIMP)
assessment program of the Standard Oil Company of New Jersey.
A subject group of mid- to high-level managers showed a
significant correlation between written measures of job-
related skills indicators and the outcome criterion.
The researchers found that job-related skills indicators,
in the form of a management judgment test, were among the four
best measures of achievement. However, they surmised that the
high correlation was, in part, attributable to possible
criterion contamination due to the managers' perceptions of
their own career success.

51
Moses (1972) studied the relationship between job-related
skills indicators and administrative level achieved seven
years later. The job-related skills indicators were
performance-type tests by which assessors determined a
subject's organizing, planning, decision-making, and
leadership skills. A large sample of nonmanagement men showed
a significant correlation coefficient between the two
variables.
Moses and Boehm (1975) studied the relationship between
several aspects of performance at an assessment center and
subsequent administrative level achieved. A large sample of
nonmanagement women was assessed between 1963 and 1971. A
significant correlation was found in this predictive study.
This study was not part of the landmark AT&T study by Bray and
Grant (1966) and some criterion contamination may have
occurred. The researchers concluded that the job-related
skills indicators of leadership, decisionmaking, organizing,
and planning related highly with the outcome criterion.
Huck and Bray (1976), while with the Wickes Corporation
and AT&T, respectively, studied the power of the AT&T
assessment center to predict future job performance of female
employees. Subjects were non-management personnel of Michigan
Bell Telephone. The independent variable of job-related
skills indicators consisted of performance tests of
leadership, decision-making, planning, and organizing. Some
criterion contamination was acknowledged. The researchers

52
concluded that the assessment center process was as valid a
selection method for women as for men.
Turnage and Muchinsky (1984), while at Iowa State
University, conducted a predictive study based upon data from
a one-day supervisory selection program (assessment center)
developed by a large manufacturing firm. The subjects were
employees who were subseguently promoted to supervisory
positions. An overall significant correlation was found
between job-related skills indicators and these methods of
measuring subsequent job performance combined. This was one
of the few significant results of this study.
Conclusions. Though much more expensive and time
consuming to administer than paper-and-pencil tests and
questionnaires, the need to find ways of evaluating
characteristics not covered by the latter is sufficient to
warrant extensive experimentation with relatively elaborate
techniques (Bray & Grant, 1966) There has been relatively
little research on the relationship between how a manager
behaves in a game and his behavior in an actual decision
situation. The behavior of a manager in a business game may
be quite different from his behavior on the job where the
rewards and punishments are much larger. Further, business
games tend to de-emphasize the interpersonal dimension in
managerial performance, whereas many management jobs appear
to emphasize it heavily (Lawler, 1967, p. 370).

53
Psychological Attribute Indicators as a Selection Method
McGregor (1960) reported that the greatest single factor
that apparently influences superior and inferior performance
by supervisory people was to be found in the area of
personality variables. According to Nash (1966),
it is reasonable to expect that the vocational
interests of a manager might be related to the
effectiveness of his job performance. His enthusiasm,
effort, and level of job satisfaction may be largely
determined by how interested he is in his work and
associates. (p. 250)
According to Edel (1968), for executive and managerial
positions, personality characteristics may well be more
important to success than skill or technical know-how (p.
231) To point up this importance, Micherson (in Edel, 1968)
surveyed 79 large and small business corporations and reported
that, of those executives who failed, over 70% did so because
of some flaw in their personality rather than from a lack of
ability.
A wide variety of research on personality tests of various
types has been conducted. This work has been successful in
defining problems and has contributed to the overall
understanding of personality adjustment. Three problems in
the use of personality tests in occupational prediction
deserve special mention: (a) the ease with which test scores
can be distorted by a test-wise applicant to portray the type
of personality desired, (b) the lack of reliability displayed
by many personality measures, and (c) the failure to design

54
the tests specifically for purposes of occupational prediction
(Edel, 1968).
Early studies gave mixed validity results. Knauft (1949)
studied a sample of 33 bakery shop managers and a correlation
of r = .39 was found, but the personality test used was later
withdrawn from the market (Korman, 1968). Comrey and High
(1955) tested the validity of some ability and interest
scores. Using a sample of over 200 production supervisors,
scores on several preference records and vocational scales
were compared to objective performance data. All correlations
except one were insignificant. La Gaipa (1960) studied the
validity of certain personality traits of over 400 naval
officer candidates compared to later supervisor ratings. Only
one of several correlations was significant. MacKinney and
Wolins (1960) studied the relationship between two interest
measures and several later measures of performance using three
overlapping samples of supervisors. Results were inconsistent
random patterns of significant and insignificant correlations.
Mixed results were also found by Robbins and King (1961) in
four samples of sales managers.
Research used in this study. A total of 22 research
studies were found which used psychological attribute
indicators as a predictor variable and met the other
delimitations of this dissertation. These studies yielded 37
correlation coefficients which ranged in strength from r = .01
to r = .47. Samples ranged in size from 20 to 1,375 persons.

55
Fleishman and Peters (1962), while at Yale University,
studied the relationships between psychological attribute
indicators and managerial achievement. The resulting
correlation was not significant. A small sample was used in
this concurrent study. The researchers found that (a)
managers who rated highly on "conformity" were less valued by
senior raters, (b) the leadership attitudes of consideration
and structure were not mutually exclusive, and (c) top
management tended to identify the effectiveness of subordinate
managers with the effectiveness of their superiors.
Hicks and Stone (1962) explored the relationship between
psychological attribute indicators and supervisors' ratings
in this concurrent study. A nonsignificant correlation was
found for the sample of shop foremen and engineering
supervisors. The researchers concluded that there may be
certain basic characteristics which the successful managers
possess regardless of their areas of specialization. They
described the personality of the successful manager as one of
emotional strength in a person who views things from a broad,
theoretical point of view, avoiding over involvement in
detail.
Goodstein and Schrader (1963), while with the University
of Iowa and the Civilian Personnel Field Agency, Ordnance
Field Activity, United States Army, respectively, studied the
validity of a personality inventory in identifying those
personality characteristics associated with managerial and

56
supervisory success in a large military industrial
organization. In their concurrent study of male civilian
managers working for the Army, the variable of psychological
attribute indicators was compared to the criterion of
administrative level achieved. A significant relationship was
found.
The researchers stated that the results suggested there
may be significant differences in personality characteristics
not only between managers and nonmanagers but also among
managers at different levels of responsibility. They
purported that success in first-line supervision may be
determined mainly by technical skill and knowledge which is
relatively independent of personality factors whereas, at the
upper levels of management, such personality-related variables
as organizing, directing, planning, and decision making become
important.
Williams & Harrell (1964) attempted to discover which
personality factors were related to business success. In
their predictive study they used the salary level attained
and administrative level achieved of Stanford MBA graduates
15 to 31 years after graduation. The overall comparison of
psychological attributes to later achievement was
insignificant. However, there was a significant positive
relationship between "success and the score on the
Masculinity-Femininity scale {of the Strong Vocational
Interest Blank}, indicating that those individuals with the

57
stronger masculine interests have a somewhat better chance of
success" (Williams & Harrell, 1964, p. 167).
Dicken and Black (1965) studied the predictive validity
of psychometric evaluation for the selection of supervisors.
Predictor variables were two well-known personality
inventories. Two criteria, salary level attained and
promotion rate, both measured 3.5 to 7 years after testing,
were used. Insignificant correlation coefficients were found
for each of the two samples. The researchers cited
restriction of range as a problem in studies of this type by
preselection of the samples by management. They stated that
"measures of ability and interest cannot be expected to make
fine discriminations" (p. 46).
Bray and Grant (1966) explored the relationship between
psychological attribute indicators and salary level attained
8 years later. Subjects were low-level managers. Although
an overall nonsignificant correlation was found between the
variables, several personality traits such as lack of
passivity and control of feelings were positively correlated
with success in management.
Nash (1966), while at the University of Maryland, sought
to discover the relationships between a manager's vocational
interests and the effectiveness of his job performance. This
concurrent validity study used corporation managers as
subjects. The correlation coefficient between psychological
attribute indicators and supervisor ratings was found to be

58
significant. Nash concluded that "effectiveness is
significantly related to the vocational interest patterns of
managers" (p. 254).
In 1967, Bentz (in Campbell et al., 1970) summarized a
series of studies which he had conducted within the
Psychological Research and Services Section of Sears, Roebuck
Company. In this study a significant relationship was found
between psychological attribute indicators and promotion rate
within Sears. Bentz's conclusion was that the Sears managers
accomplish their goals because they are superior "in
intellectual endowment, social competence, and emotional
stamina" (Campbell, et al. 1970, p. 187). Campbell et al.
(1970) stated that there was truth to Bentz's statement, but
that it was not strongly supported by the findings of his
concurrent validity study.
Cummin (1967), while at Harvard, sought to discover if the
"more successful" executives would display motivation toward
achievement, power, and autonomy, and the "less successful"
executives would show motivation toward affiliation,
aggression, and deference in a personality inventory. His
concurrent validity study of business persons compared
psychological attribute indicators to current salary level
attained. He found a significant correlation. Cummin
concluded that "the successful executive is one who is
determined to maintain a high standard of excellence in his
work, and to assume greater responsibilities and more control

59
over his environment as he advances within the organization"
(p. 81) .
Grant, Katkovsky, and Bray (1967) while at AT&T and
Fordham University (Katkovsky), studied the contributions of
projective techniques to the assessment of management
potential. The study used data collected in the Bell System
Management Progress Study (Bray, 1964). A comparison of
psychological attribute indicators to salary level attained
eight years later showed a nonsignificant correlation. The
researchers found that within this method several of the
projective variables were reliably related to salary progress.
Those variables were dependence and subordinate roles (with
negative relationships), achievement motivation, and
leadership role.
In a concurrent validity study, Ghiselli (1968) evaluated
motivational factors in the success of managers. He
administered a personality inventory to several samples of
corporation middle management personnel. Psychological
attribute indicators related significantly with supervisor
ratings.
As compared with the employed population as a whole,
Ghiselli found that persons in middle management positions
appeared to have a substantially lower desire for security and
for financial reward and a higher desire for self-
actualization. They did not differ from the employed
population in the desire for power over others.

60
Edel (1968), while working for the Department of Defense,
studied the relationship between a psychological "need for
success" and managerial performance. He found a significant
correlation in this concurrent study. Edel's study used one
test to measure need for success. Reliability data had not
yet been provided for this test.
In 1968, Miner (in Miner, 1977), while at Georgia State
University, studied the relationship between measures of
psychological attributes of school administrators and
separate, concurrent, outcome criteria of supervisor ratings
and salary level attained. He found no significant
correlations. He concluded that managerial motivation was not
rewarded in that school system.
Campbell et al., (1970) conducted the Early Identification
of Management Potential study for Standard Oil Company of New
Jersey. Within this study they compared measures of
psychological attributes to a combination of concurrent
outcome criteria to include salary level attained, supervisor
ratings, and administrative level achieved. Subjects were
mid- to high-level corporation managers. An insignificant
correlation was found. The researchers concluded that these
indicators of temperament showed no useful relationship with
any of the effectiveness measures.
Harrell and Harrell (1974) conducted a predictive study
in order to determine predictors of administrative
achievement. While working for the Office of Naval Research,

61
they found a significant correlation between psychological
attribute indicators and salary level attained 10 years later.
Subjects were former MBA graduates who had achieved management
positions in corporations.
Grimsley and Jarrett (1975) studied the relationship
between managerial achievement and test measures obtained in
the employment situation. This concurrent study used two
samples of mid- and top-level corporation managers. The
results of several measures of psychological attributes were
compared to the outcome criteria of administrative level
achieved. Insignificant correlation coefficients were found
with each of the two samples. The researchers concluded that
"the differences in test scores of more or less successful
managers result from fundamental differences in mental ability
and personality rather than the influence of on-the-job
experience" (p. 226).
Gantz, Erickson, and Stephenson (in Miner, 1977) sought
to determine if certain psychological attributes could be
causes of managerial success. Using a sample of research
scientists and engineers in a federally funded laboratory,
several measures of psychological attributes were
administered. These proved to be insignificantly related to
both supervisors' ratings and promotion rate. The researchers
concluded that perceived creativity was rewarded by promotion
into leadership roles, including formal managerial roles where

62
managerial skill was needed, and actual or potential
leadership ability was not rewarded.
Two consequent studies by Gantz, Erickson, and Stephenson,
were also reported in Miner (1977). In the first, a sample
of former research scientists was followed up 5.3 years after
testing and showed a significant correlation between measures
of psychological attributes related to motivation to manage
and promotion rate. As some of the subjects may have
experienced grade level changes based in part on research
competence, the success index, promotion rate, may not have
been based entirely on managerial competence. The researchers
concluded that the borderline level of the correlation
indicated that further analyses dealing with the predictive
power of the role-motivation theory ought to be conducted.
Using a second sample consisting of top sales persons and
marketing managers, the researchers conducted a similar study
with four years intervening between variables. The
correlation was significant. The researchers stated that,
within this department of the same company, "promotion into
the higher grade levels was based entirely on managerial
competence" (Miner, 1977, p. 31). The conclusion drawn was
that the motives measured by the particular psychological
attributes indicator used did serve as a cause of subsequent
managerial accomplishment (Miner, 1977).
Miner (1977) in a concurrent validity study, sought to
discover the relationships between different measures of

63
psychological attributes and two measures of administrative
achievement. Using a sample of business personnel managers,
he found insignificant relationships with both administrative
level achieved and with salary level attained. Although
neither correlation was significant, Miner stated that "the
overall measures of motivation to manage are quite
consistently related to the occupational success indexes" (p.
74) .
In a separate study by Miner (1977), faculty members and
administrators in three different business schools were given
an inventory of psychological attributes. Concurrently, each
subject's score was compared with his or her administrative
level attained. Miner reported no significant relationship,
and "if anything, the administrators have less motivation to
manage than the regular faculty, although the difference does
not approach significance" (p. 51).
McClelland and Boyatzis (1982), while at Harvard
University and McBer & Company, respectively, studied the
relationship between psychological attribute indicators and
long-term achievement in management. Subjects were entry-
level managers who were part of the AT&T assessment center
study. A significant correlation was found between the
predictor variable and administrative level achieved after 16
intervening years. The researchers concluded that a high need
for achievement was associated with managerial success at
lower levels of management, while at higher levels of

64
management persons with a need for influencing others
dominated.
Stahl (1983), while at Clemson University, used a measure
of psychological attributes to test the hypothesis that high
managerial motivation consisted of high personal needs for
achievement and power. He found a significant correlation
with each of two outcome criterion variables, supervisor
ratings and administrative level achieved. The researcher's
conclusions were drawn from this and other data collected from
other samples using the same methodology. He concluded that
"there was a higher proportion of subjects with high
managerial motivation among the managers than among the non
managers," and that "there was a higher proportion of managers
with high managerial motivation among the promoted managers
than among the non-promoted managers" (p. 786). Stahl
stressed the need for researchers to implement longitudinal
validation studies to confirm these results.
Turnage and Muchinsky (1984) compared the predictive
validity of assessment center evaluations versus traditional
measures in forecasting supervisory job performance. A sample
of employees who were subsequently promoted to supervisory
positions within a large manufacturing firm was used.
Correlation coefficients between findings of psychological
attribute indicators and three separate outcome variables
yielded no significant relationships. The researchers cited
many possible methodological shortcomings of the study that

65
could have contributed to the general lack of significant
findings.
Conclusions. Results obtained by the studies reviewed
herein suggest that there is evidence supporting the
proposition that effective managers have identifiable
interests which distinguish them from less effective managers.
However, most motivation studies have used concurrent validity
as the research method. Predictive validity studies that
assess the dynamic nature of personality may be needed.
According to Miner (1978), "the greatest research needs seem
to be for additional longitudinal studies of the relationships
between motivation to manage...and success" (p. 751).
Another limitation of interest measures is that they have
been demonstrated to be fakeable. Nash suggested that future
research should focus on the impact of fakability on the
actual use of measured interests in selection programs (1965,
P-34) .
Campbell et al. (1970) addressed achievement and power for
managers. Concerning job/task analysis of what effective
managers actually do, they listed the frequency of behavior
aimed at influencing others (power) and the frequency of
behavior concerned with setting and accomplishing goals
(achievement). They remarked that "better managers tend to
show a lifetime pattern of high achievement, power, and
economic motivation" (p. 361).

66
One of the most recent and extensive treatments of the
relationships of achievement to power is presented in Veroff's
book of readings in honor of McClelland.
Achievement motivation directs people to meeting
socialized standards of excellent performance and thus
to highly efficient task-centered strivings, whereas
power motivation directs people to doing whatever
draws most attention to their own effect on the world.
The two motives seem to be fused in instances where
the standard of excellence is to win in a social
competitive activity or to solve a problem that will
be given a great deal of recognition. (Veroff, 1982,
p. 100)
Value Systems Assessments as a Selection Method
The selection method of values systems assessments
measures the strength of a person's economic, aesthetic,
social, political, and religious concerns (Hinrichs, 1978).
According to England and Lee (1974),
a personal value system is viewed as a relatively
permanent perceptual framework which shapes and
influences the general nature of an individual's
behavior. Values are similar to attitudes but are
more ingrained, permanent, and stable in nature; they
are also more general and less tied to any specific
referent than is the case with many attitudes.
(p. 412)
Most personnel selection research studies which dealt with
the independent variable of psychological attribute indicators
verged on or overlapped the selection methods using values as
an indicator. The studies used in this section were those
that this researcher determined were predominantly concerned
with values as defined above.
Research used in this study. A total of 4 research
studies were found which used value systems assessments as a

67
predictor variable and met the other delimitations of this
dissertation. These studies yielded 5 correlation
coefficients which ranged in strength from r = 10 to r = .32.
Samples ranged in size from 30 to 1,375 persons.
In 1944, the Psychological Research and Services Section
of Sears, Roebuck Company established a psychological testing
program. Over a number of years researchers within this
program conducted a series of investigations into the
prediction of executive effectiveness in the Sears
organization. A predictive study was accomplished by Bentz
in 1967 (in Campbell et al., 1970). A span of 11 to 17 years
elapsed between the measurement of values and collection of
the criterion variable, promotion rate. A significant
correlation was found.
England and Lee (1974), while at the University of
Minnesota, studied the relationship between managerial values
and managerial success in several countries. The portion of
their study that dealt with managers in the United States was
used in this dissertation. Managerial success was defined as
a measure of salary level attained in this concurrent study.
The correlation of this measure with a written survey of
values was significant for a sample of corporation directors
and high level executives. The researchers concluded that the
value patterns were predictive of managerial success and could
be used in selection and placement decisions. The general
pattern that emerged from the study indicated that more

68
successful managers appeared to have pragmatic, dynamic,
achievement oriented values, whereas less successful managers
had more static and passive values, corresponding to a desire
for organizational stasis rather than an organization in flux
(England & Lee, 1974).
Grimsley and Jarrett (1975) studied the "effectiveness of
a particular methodological approach which can be used in
analyzing data gathered in the process of assessing managerial
applicants in the employment situation" (p. 215). Two sample
groups of mid- and top-level corporate managers were used in
this concurrent study. These managers came from more than 100
different companies and many industries in various parts of
the United States. All were administered a "study of values"
test and test scores were compared with administrative level
achieved. The results were not significant. The findings of
this study led the researchers to conclude that the
"differences in test scores of more or less successful
managers result from fundamental differences in mental ability
and personality rather than the influence of on-the-job
experience" (p. 226).
Hinrichs (1978), while with Management Decision Systems,
Inc., conducted an 8-year followup study of the IBM
Corporation assessment center. Within this study, the results
of a study of values were correlated to the outcome variable
of administrative level achieved. The resulting correlation
coefficient of r = .26 was not significant. High scores on

69
political and economic subsets of the study of values were
positively related to progression in formal business
hierarchy, while scores on the religious and social scales
related negatively.
Conclusions. The few research findings available on
values as a selection method consistently show that certain
economic and political beliefs correlate positively, and
strong social and religious beliefs correlate negatively, with
high administrative achievement (Grimsley & Jarrett, 1975).
England and Lee (1974) found that "more successful" managers
favored an achievement orientation and preferred an active
role in interaction with other individuals instrumental to
achievement of the manager's organizational goals, while "less
successful" managers had values associated with a static and
protected environment in which they took relatively passive
roles (pp. 418-419). According to Hinrichs (1978) those
traits or characteristics that seem to reflect a degree of
social awareness tend to be detrimental to success in
administration. "The flavor is one of the 'hard charging,'
perhaps somewhat socially insensitive and upward mobile
individual" (p. 600).
Biographical Information as a Selection Method
Biographical information, personal history data about an
employee, is most often furnished an employer in the form of
an application blank or a resume. According to Levine and
Flory (1975), the most widely used selection technique is the

70
evaluation of the job application blank or the resume. These
are used to determine whether an individual meets minimum
qualifications for a position. If these qualifications are
not met, the applicant is no longer considered. In some cases
this is the only information an employer receives about a
prospective employee, but in most cases the application is
followed by an interview, written test, or some additional
means of assessment.
To provide some idea of the frequency of use of
applications, Carlson (in Levine & Flory, 1975) posited that
if an arbitrary assumption were made that 50% of those filing
applications or resumes are ultimately interviewed, then over
1,000,000,000 applications and resumes per year are prepared
and screened in the United States. Yet, despite its
universality, the evaluation of the validity of biographical
information for personnel selection and placement has not been
researched in a systematic fashion to any great extent.
Research has been done on the empirically weighted
application, usually for predicting turnover. More recent
research on application evaluation has focused on the factors
affecting the rater's quality of evaluation of the information
rather than the quality of the method of evaluation per se.
According to Spencer and Worthington (1952) and Peck and
Parsons (1956), the few studies dealing with method of
evaluation other than empirical weighing have demonstrated

71
that projective evaluation of application blank information
could validly predict performance and tenure.
A trend in selection research in recent years has been the
increasing use of personal background variables in the
prediction of occupational success. Such variables are
considered to have several advantages for this purpose. The
most frequently cited advantages in utilizing these kinds of
variables are that (a) they are less threatening than the
items on typical personality inventories and thus are less
subject to "facade" effects, and (b) the behaviors described
by these items are often reflections of attitudinal and
personality variables.
Results of past studies of the relationship between
biographical information and administrative achievement have
been mixed. Vernon (1950), using civil service managers, and
La Gaipa (1960), using Naval officer candidates, found no
significant relationships. The findings of Riccuiti (1955),
with U.S. Naval officers, Meyer (1956), with firstline
supervisors, and MacKinney and Wolins (1960) with supervisors,
found no consistent relationships. Significant relationships
were found by Haggerty (1953), at the U.S. Military Academy,
Soar (1956), using service station managers, and Scollay
(1957), with promotion managers.
Childs and Klimoski (1986), in a study which involved
employees in both management and nonmanagement positions
investigated the validity of a biographical inventory in the

72
prediction of occupational success. They found that positive
social orientation, interpersonal confidence, and educational
achievement were positively related to their outcome measures
of "job and career success" (p. 7) .
Research used in this study. A total of 8 research
studies were found which used biographical information as a
predictor variable and met the other delimitations of this
dissertation. These studies yielded 11 correlation
coefficients which ranged in strength from r = .05 to r = .57.
Samples ranged in size from 30 to 799 persons.
In 1963, Haggerty (in Mayfield, 1970) studied the
relationships between certain predictor variables and
achievement of U.S. Army officers. In this predictive study
using a sample of officer cadets, the researchers found a
nonsignificant correlation between biographical information
and a combination of achievement measures.
Williams and Harrell (1964) sought to determine which, if
any, of a number of factors were related to business success.
The independent variable in this predictive study was that of
biographical information available to an employer at the time
of a student's graduation from business school. Salary level
attained, and administrative level achieved, measured many
years after graduation, were the criteria of success.
Insignificant correlations were found for this sample of
Stanford MBA graduates. A subset of this information that was
significantly and positively related to later achievement was

73
a person's previous participation as a leader in organizations
on campus.
Kotula and Haggerty in 1966 (in Korman, 1968) studied the
relationship between biographical information given on written
personal history blanks and supervisor ratings. Two groups
of Army officers were studied. Only one coefficient was
reported, r = .17, which was significant at the .05 level.
Campbell et al. (1970) attempted to discover how to
identify, early in their careers, those employees who possess
the potential to be successful in management. The assessment
procedure in this concurrent study contained a background
biographical survey the scores from which were correlated to
an "Overall Success Index", a combination of dependent
variables. A sample of mid- to high-level corporation
managers was used. A significant correlation was found
between biographical information and the outcome criterion of
achievement. Laurent, one of the researchers involved with
this study, concluded that "successful executives in the
Standard Oil of New Jersey organization have shown a total
life pattern of successful endeavors. They were good in
college, are active in taking advantage of leadership
opportunities, and see themselves as forceful, dominant,
assertive, and confident" (Campbell et al., 1970).
Harrell and Harrell (1974) conducted a predictive study
of MBA graduates in order to determine predictors of

74
management success. They found a significant relationship
between biographical information and salary level attained 10
years later.
Gantz, Erickson, and Stephenson (in Miner, 1977) conducted
a concurrent study of research scientists and engineers for
the purpose of gaining "more theoretical understanding of why
some persons in a research and development population achieve
promotion, particularly into formally designated supervisory
or managerial positions, and other persons do not" (p. 18).
The organization was a federally funded public laboratory.
The independent variable, the selection method of biographical
information, was found to be not related to either promotion
rate or supervisor ratings.
Hinrichs (1978) found a significant relationship between
biographical information and administrative level achieved
eight years later for a sample of 30 managers. The resulting
correlation coefficient was slightly higher than that for the
Overall Assessment Rating.
Turnage and Muchinsky (1984) sought to examine the
predictability of assessment center evaluation versus
traditional measures in forecasting job success. As part of
the study, they separately compared biographical information
with supervisor ratings and salary level attained up to
several years later. These variables were insignificantly
related for the mostly nonmanagement personnel tested. The

75
researchers concluded that certain background data predict
criteria as well as assessment center data.
Conclusions. Compared to the interview method, Levine and
Flory (1975) found the application blank information to be
superior in accuracy of information. However, they noted that
"sizable inaccuracies" had been found in each method (p. 384) .
Childs and Klimoski (1986) found that an outgoing and self-
confident personality preordains success in both job- and non
job-related situation. They also found that success in one's
educational history may aid the attainment of success in one's
career, regardless of whether that job is personally
satisfying. Most studies that showed significant positive
correlations between biographical information and
administrative achievement attributed the relationship to
personality characteristics which were also positive
attributes of leadership ability.
Peer Ratings as a Selection Method
Peer ratings of supervisory potential achievement consist
of impressions gained from interactions in an equal, non
supervisor-subordinate, nature. They are predictions of how
well a peer will do in a supervisory position, should he or
she be placed in one (Korman, 1968).
Among the more consistent findings concerning peer ratings
is the significant validity these afford in predicting later
performance. Many studies conducted in military settings
indicated that peer evaluations during officer training

76
successfully predicted later criteria of performance.
Williams and Leavitt (1947) found that peer ratings were
better predictors of long-term success in the Marine Corps
than were superiors' ratings. Much military research was
contributed by Hollander (1954, 1956, 1957, 1964, 1965) and
the evidence was consistently favorable. La Gaipa (1960), in
the Development of the Officer Candidate Biographical
Information Blank, sampled Naval officers in both shore duty
and fleet duty. He found a significant correlation between
peer ratings and later performance criterion.
There has also been evidence of the utility of peer
ratings from other spheres of activity, as Weitz found a
relationship of r = .40 between peer nominations and later
ratings of life insurance agents in a supervisory position
(1958). As Hollander has stated, "when employed with
discrimination, it [the peer rating method] can provide a
unique contribution to evaluation" (1965, p. 434).
Research used in this study. A total of 8 research
studies were found which used peer ratings as a predictor
variable and met the other delimitations of this dissertation.
These studies yielded 10 correlation coefficients which ranged
in strength from r = .04 to r = .53. Samples ranged in size
from 40 to 799 persons.
Haggerty (1963), in two separate predictive studies,
explored the predictive ability of peer ratings obtained on
Military Academy Cadets. The first group of men showed a

77
significant correlation with the outcome variable of
supervisors' ratings. The second group of cadets showed a
significant correlation with a later combination of overall
ratings.
Roadman's (1964) predictive study compared peer ratings,
taken in a management school setting at IBM, to promotion
rates of a sample of graduates who had received promotions at
least two years later. A significant correlation was found.
He concluded that a careful and comprehensive peer rating
administered in a middle manager training program can identify
those who later move into senior executive positions.
Hollander (1965), while at the State University of New
York at Buffalo, did a predictive study comparing the
relationship between peer assessments and supervisor ratings.
The study was begun in 1955 at the Naval Officer Candidate
School (OCS) in Newport, Rhode Island. An entire OCS class
was made available for this investigation. Four forms, each
setting out different qualities to be rated, were utilized;
these dealt with leadership, motivation for naval service,
probability of success in OCS training, and success as a
future officer. Results of part of the study were reported
by Hollander (1956) and showed that very early in the training
program, students were able to accurately determine which of
their peers would do well after training.
In the follow-up study (Hollander, 1965), supervisor
ratings of former trainees, who had become officers, were used

78
to test the validity of peer prediction of future success.
This follow-up of officers at least 3 years after graduation
showed a significant correlation. Hollander concluded that
peer nominations used early in training can make a distinctive
contribution to the prediction of a long-range criterion of
performance after training.
Lawler (1967), while working for the Department of
Administrative Sciences of Yale University, explored the
relationship between peer ratings and supervisors' ratings.
In this concurrent validity study, a sample of mid-and top-
level managers in a manufacturing company were rated by
several peer raters on three traits. Lawler found that
personnel selection "decisions (which include peer ratings)
will be of a higher quality than if just superiors' ratings
are relied upon" (p. 378).
Mayfield (1970), while employed with the Life Insurance
Agency Management Association, studied the relationship
between peer ratings and supervisors' ratings 2.5 years later.
The purpose of this predictive study was to investigate the
value of the buddy nomination procedure in the selection of
assistant managers for life insurance companies. The scores
were not used in promotion decisions to prevent criterion
contamination.
Two of the three samples showed significant correlations
between peer and supervisor ratings. Mayfield concluded that

79
peer ratings can be made in a "realistic" administrative
setting and retain their predictiveness.
Mitchel (1975), while at Bowling Green State University,
studied the predictive abilities of parts of the Standard Oil
Company of Ohio's assessment center program. The relationship
between peer ratings and a later measure of salary level
attained proved significant. He concluded that "peer and
assessor ratings, as well as combinations of variables, were
predictive of a salary criterion of managerial success"
(Mitchel, 1975, p. 578).
Gantz, Erickson, and Stephenson (in Miner, 1977) sought
to determine why some persons in a research and development
population achieve promotion into formally designated
supervisory or managerial positions and whether those who were
promoted were the ones who should have been promoted. A study
of research scientists and engineers in a large, federally-
funded laboratory showed a significant correlation between
peer ratings and promotion rate. These peer ratings of
overall ability and perceived creativity were statistically
significant at the .01 level.
Turnage and Muchinsky (1984), examined the ability of
assessment center evaluations to predict actual job
performance criteria, and to compare the predictability of
assessment center evaluations versus traditional measures in
forecasting job success. Their data came from a one-day
supervisory selection program developed by a large

80
manufacturing firm. The outcome data were gathered from 1
month to several years after assessment from a sample of
previous assessees. None of the peer ratings was significant.
Several possible explanations for the low validity
coefficients were given. These included poor experimental
control, insufficiently trained assessors, aggregation of data
over a 4-year period, and the effects of intervening
variables.
Conclusions. There appears to be a reasonable basis to
contend that peer nominations do provide distinctive
prediction of performance at a considerably later time. Peer
evaluations are relevant because peers are situated to
evaluate how a person performs in terms of the lateral
relationships in working toward organizational goals.
Further, peers often see the worker at times when his superior
is not viewing his behavior and, therefore, they may see
aspects of his behavior of which the superior is not aware
(Lawler, 1967) However, attention must be paid to the
question of the willingness of participants to rate each other
accurately if they know full use will be made of peer ratings
for administrative purposes (Roadman, 1964). A problem Lawler
(1967) acknowledged was that if the peer rater knew that his
or her opinion was "going to count" in an employment setting,
ratings may lose their validity, "particularly if a situation
exists where an individual's self-interests might be best
served by distortion of the peer ratings" (p. 379) According

81
to Korman (1968) there is a need for the initiation of
research on the selection method of peer ratings concerning
both its predictive validity and the general characteristics
that correlate with peer ratings.
Self-Appraisals as a Selection Method
Self-ratings of administrative ability are relevant in
selection because the individual has more information about
his own behavior than anyone else and because self-perceptions
are important determinants of an individual's future behavior
(Lawler, 1967, p. 371). A negative view of self-selection for
administrative positions has apparently been shared by those
behavioral scientists who study, develop, and validate
personnel selection procedures. The reason for this appears
to be rooted in a Theory X view of people as they are expected
to behave in a personnel selection setting. In these settings
people are assumed to lack objectivity in assessing their own
performance or personal attributes. They can be expected to
overestimate their performance, skills, knowledge, and
abilities to improve their chances for appointment (Levine,
1978) .
Published studies on self-assessment of skills, abilities,
knowledge, and other applicant attributes as predictors of
administrative job performance have been virtually nonexistent
until recently (Levine, 1978, p. 230). Studies on self
selection, done before 1962, have dealt mainly with
objectively measurable traits (Nickels & Renzaglia, 1958).

82
Research used in this study. A total of 8 research
studies were found which used self-appraisals as a predictor
variable and met the other delimitations of this dissertation.
These studies yielded 9 correlation coefficients which ranged
in strength from r = .01 to r = .26. Samples ranged in size
from 30 to 799 persons.
Prien and Liske (1962), while at Case-Western Reserve
University, explored the relationship between supervisor
ratings of job performance and incumbent self-ratings of job
performance on tasks which were intangible in nature. In
their concurrent validity study of employees of various
corporations, significant correlations were found between
self-ratings and first-level supervisor ratings, and
insignificant correlations were found between self-ratings and
second-level supervisor ratings. The preponderance of studies
showed, as this one did, that individuals rate themselves
higher than they are rated by comparison groups.
Lawler (1967) while with the Department of Administrative
Sciences of Yale University, studied mid- and top-level
managers in a manufacturing organization. Four coefficients
of the relationship between self-appraisal and supervisory
job performance ratings showed an average nonsignificant
correlation. This concurrent validity study also showed
evidence of self over-estimation of perceived administrative
abilities.

83
Thornton (1968), while a summer associate at the firm of
Rohrer, Hibler and Replogle in Chicago, investigated the
relationship between supervisory perceptions and incumbent
self-perceptions of the performance of executive personnel.
A sample of high-level managers in a large corporation were
included in this study. An average of coefficients of 27
traits, or behavioral characteristics, studied showed a
significant correlation between self-ratings and supervisor
perceptions of performance ratings. The tendency was for
self-evaluations of performance to be higher than supervisory
perception and, in this study, those executives who overrated
themselves were considered least promotable.
Campbell et al. (1970), in a long-term staff study,
reviewed the relationship between self-assessment and a
combination of performance ratings to include salary,
supervisory ratings, and administrative level attained. This
was part of the Early Identification of Management Potential
(EIMP) study carried out by the Standard Oil Company of New
Jersey (SONJ). A sample of managers of SONJ was used. The
resulting correlation coefficient was significant.
Contrary to previous evidence, Heneman (1974) found a
tendency for self-ratings to be less lenient than supervisory
ratings, with a significant correlation. His concurrent study
compared self- and supervisor ratings of job performance of
former MBA graduates of Indiana University, 7 to 10 years
after graduation. He suggested that future research on

84
managerial performance should include self-ratings where it
is made clear that these ratings will be used for research
purposes only.
Hinrichs (1978) completed one of the few predictive
validity studies of self-assessment of administrative
performance. The predictions of the assessment center were
not used administratively to prevent criterion contamination.
This study is included in an 8-year followup of a management
assessment center at International Business Machines (IBM).
Self-ratings of marketing personnel correlated significantly
with the outcome variable of administrative level achieved.
Steel and Ovalle (1984), while at the Air Force Institute
of Technology, sought to compare the relative validity of
self-ratings for predicting objective criteria of managerial
job performance. A concurrent validity study at a large
lending institution revealed an insignificant correlation
between self-rating and supervisor rating.
Turnage and Muchinsky (1984) conducted a predictive study
in a large manufacturing firm. The subjects were employees
who were subseguently promoted to supervisory positions.
Self-evaluations of job-related personal characteristics, such
as ability to withstand stress, intellectual abilities, and
interpersonal skill, made up the independent variable. Self-
evaluations related insignificantly and negatively with both
the outcome variables of salary level attained and supervisor
ratings. The problem of criterion contamination (i.e., the

85
availability of assessment scores to superiors) may have been
a factor in this study.
Conclusions. It has been suggested by Levine (1978) that,
as predictors of job performance, self-assessments should at
least supplement other information. Self-assessments may
replace more traditional selection methods, especially in
measuring psychological attributes that may be relatively
inaccessible by other means. He suggested that self-
assessments might have a motivational impact on those
applicants who are hired, as these people will strive to be
consistent with their self-perceived competencies. Levine
admits that research is needed to determine what applicant
attributes are most validly, or least validly, self-assessed.
Assessment Center Processes as a Selection Method
A highlighting feature of the assessment center is that
candidates are evaluated not on what they have done in present
or past jobs but on how they are likely to cope with a new
type of position. This involves using various situational
tests as well as incorporating some of the more classic
selection procedures, such as aptitude tests and interviews.
Assessments are conducted at least partially in groups, which
permits observing group interactions as well as obtaining peer
ratings (Howard, 1974).
The following dimensions are most often assessed: (a)
leadership, (b) organizing and planning, (c) decision making,
(d) oral and written communications skills, (e) initiative,

86
(f) energy, (g) analytical ability, (h) resistance to stress,
(i) use of delegation, (j) behavior flexibility, (k) human
relations competence, (1) originality, (m) controlling, (n)
self-direction, and (o) overall potential (Howard, 1974).
The origin of the use of multiple assessment procedures
on a large scale is credited to German military psychologists.
The British adapted the procedures to the screening of officer
candidates, and the United States Office of Strategic Services
(OSS) took over the approach from the British during World War
II (Bray & Grant, 1966). Howard (1974) stated that the first
industrial use of an assessment center has been generally
attributed to AT&T beginning in 1956.
Research used in this study. A total of 14 research
studies were found that used Overall Assessment Ratings (OAR)
of an assessment center process as a predictor variable and
met the other delimitations of this dissertation. These
studies yielded 16 correlation coefficients which ranged in
strength from r = .00 to r = .65. Samples ranged in size from
25 to 5,943 persons.
The first industrial use of an assessment center is
generally attributed to the American Telephone and Telegraph
Company. Other centers have been variations on AT&T's theme.
The AT&T experimentation was research-oriented and designed
to follow the development of managerial personnel for many
years after assessment (Howard, 1974).

87
Bray and Grant (1966) measured personal characteristics
hypothesized to be of importance either in developmental
change in early adulthood or success in business management.
The overall assessment rating was significantly correlated
with both salary level attained and administrative level
achieved 8 years after assessment. This was one of few
studies in which criterion contamination was eliminated. The
researchers concluded that "no single characteristic
determines progress in management" (Bray & Grant, 1966).
Moses (1972) studied the relationship between assessment
center processes and managerial achievement. Using a large
sample of assessees of an AT&T one-day assessment center,
Moses found a significant relationship between overall
assessment rating and administrative level achieved.
Moses and Boehm (1975) studied the relationship between
the assessment center process and subsequent progress in
management for women in the Bell System. Assessment center
data were obtained on 4,846 nonmanagement women from 1963
until 1971. The conclusion the researchers drew in this study
was that the assessment center process predicted the future
performance of women as accurately as it did that of men.
Huck and Bray (1976) tested the validity of an assessment
center process on a population different from the population
of the original Management Progress Study. Using a relatively
small sample of females who had been assessed previously, the

88
researchers found a significant correlation between overall
assessment rating and later administrative achievement.
The International Business Machines (IBM) Corporation
assessment center was patterned after the AT&T model and
included 2.5 days of assessment activity followed by 2 days
of developmental activity. Several studies have reported
results of the IBM assessment center (Hinrichs, 1978; Kraut
& Scott, 1972; Roadman, 1964; and Wollowick & McNamara, 1969).
Wollowick and McNamara (1969) studied the progress of low-
and mid-management males as subjects. The purpose of the
study was to determine the validity of an assessment center
approach in predicting management potential and to determine
the relative value of each of the assessment center
components. An overall assessment rating was determined for
each subject at the conclusion of the two-day assessment
session. The four observers who formed the assessment staff,
and who were operating management personnel at least two
levels above the participants, assigned each subject an
overall assessment rating taking into consideration all of the
variables in the program. According to the overall assessment
rating, each subject was ranked on a 5-point scale rating
potential for advancement within the company (Wollowick &
McNamara, 1969).
These overall assessment ratings were compared, 2.5 to 3.2
years later, to the subjects' administrative levels achieved.
This resulted in a significant correlation. It was evident

89
to the researchers that the subjectively derived overall
assessment rating utilized in this assessment program was a
valid predictor of management success.
Kraut and Scott (1972) studied the data provided by the
IBM assessment center on the progress of non-management males
into administrative positions. The purpose of the study was
to examine the validity of the assessment program in
predicting administrative achievement. Overall assessment
ratings were correlated with administrative level achieved
from 1 to 5 years later using the same 5-point rating system
that was used in the Wollowick and McNamara (1969) study. The
correlation between overall assessment rating and later
achievement was significant. This assessment program was one
of the few selection methods which showed validity in
predicting success beyond the first-level promotion after
assessment.
Hinrichs (1978) completed a predictive study using a small
sample of IBM managers. A significant relationship was found
between assessment center overall assessement ratings and
administrative level achieved eight years later. However,
Hinrichs also found that the relatively simple prediction
based upon managerial review of the personnel files did as
well in prediction as the assessment center.
Researchers in the IBM studies reviewed here concluded
that large-scale assessment programs appear useful in making
discriminations of management potential which are later

90
confirmed by the rate of promotions, as well as demotions
(Kraut & Scott, 1972). The studies contained criterion
contamination as the subjects' results were provided to their
supervisors. However, it was observed that the "relationship
of ratings to first promotions is moderate enough to reduce
fears of 'crown prince' or 'kiss of death' effects" (p. 124).
According to Kraut & Scott (1972),
compared to the normal promotional system in most
companies, the program typically has some obvious
advantages in reliability and validly measuring
management potential. Instead of judgments by one's
immediate manager which may be more or less
subjective, evaluations in the program are made by
several managers (raters) who are likely to be much
more objective. Further, they are not making
judgments about the individual's management potential
from his performance in a non-management job; instead
they evaluate all candidates against a common
yardstick comprised of standardized management-type
tasks. (p.124)
The Standard Oil Company, Ohio (SOHIO), created an
assessment center in 1963. This Formal Assessment of
Corporate Talents (FACT) Program was modeled after the AT&T
assessment process (Carleton, 1970) Most data from the SOHIO
program pointed to the superiority of the overall assessment
ratings but did not eliminate any single category of
assessment components (Howard, 1974).
Carleton (1970) sought to determine the relationships
between test and rating data and later measures of
administrative achievement. A significant correlation was
found between the overall assessment rating and subsequent
supervisor ratings. Carleton concluded that both test and

91
rating data generated in a multiple technique assessment
center held up well as predictors of behavioral ratings made
several years later.
Finley (1970) studied the predictive validity of
projective tests in the SOHIO management assessment center.
Using a sample derived from the FACT Program, he found a
significant relationship between overall assessment ratings
and subsequent supervisor ratings.
Mitchel (1975) studied a sample of managers from SOHIO's
FACT Program. An insignificant correlation was found between
overall assessment ratings, minus assessor's ratings of
potential, and salary level attained 5 years after assessment.
The overall rating was not the most valid predictor in this
study, nor did it appear in any of the stepwise equations done
by the researcher.
The SOHIO studies involved both trained and untrained
assessors. Evidence for discriminant validity was found for
the two trained assessor groups, but not for the group in
which assessors consisted of untrained supervisors. It was
concluded that the assessment center method was able to
predict multiple criteria fairly well, but that the poor
quality of criterion measures probably reduced the convergent
validity coefficients (Howard, 1974).
One of the most extensive validations of the assessment
center method against measures of management effectiveness was
the Early Identification of Management Potential (EIMP) study

92
carried out by the Standard Oil Company of New Jersey (SONJ).
The long-term staff study was extensively reported in Campbell
et al. (1970).
Through this study, the management of SONJ sought to
determine how employees who possess the potential to be
successful in management can be identified early in their
careers. Management personnel of SONJ were more hopeful of
identifying young men with potential for general management
positions and were less concerned with pinpointing those with
potential narrowly oriented toward specific functional areas.
A significant relationship was found between overall
assessment ratings and a combination of outcome variables in
the concurrent study.
Schmitt, Noe, Meritt, and Fitzgerald (1984) sought to
evaluate the use of the assessment center approach when
applied in an educational setting to select secondary and
elementary school administrators. Using an assessment center
designed by the National Association of Secondary School
Principals (NASSP) and the American Psychological Association,
they found a significant relationship between overall
assessment rating and subsequent supervisor ratings. However,
the number of years between the measurement of variables was
limited due to the fact that average job tenure of the
promoted participants was only 13 months. This study
represented the first validation of the use of the assessment
center process in the selection of school administrators.

93
Turnage and Muchinsky (1984) conducted a study based upon
data from a one-day assessment center developed by an
unidentified large manufacturing firm. The subjects were 799
nonmanagement employees who were subsequently promoted to
supervisory positions. Two overall assessment ratings were
provided using two separate outcome criteria, salary level
attained and supervisor ratings. Based on these, and other
findings in the study, the researchers concluded that there
appeared to be no appreciable relationship between how one is
evaluated in an assessment center and how one performs on the
job. They stated that "there is not a strong relationship
between job performance and promotion" (p. 600).
A number of reasons were surmised by the researchers for
the lack of correlation. Methodological factors such as low
criterion reliability, low predictor reliability, severe
restriction of range, marked skew in the data, procedural
inconsistencies, lack of comparability across assessment
groups, and errors in data collection were cited as problems
within this study (Turnage & Muchinsky, 1984).
Rankin (1981) conducted a predictive validity study in an
assessment center program in a military organization.
Subjects were low-level management civilian employees. No
relationship was found between overall assessment ratings and
a combination of outcome measures. In fact, Rankin (1981)
stated that,
the most significant correlation between overall
ratings was found within the group of assessees with

94
military supervisors (N = 19) In a comparison of
OVERAC (OAR), the overall 1981 rating, a correlation
of .40 was achieved, representing a significance of
.045. This could be considered to be a good result
were it not for the fact that it is a negative
coefficient. This implies that military supervisors
rate their personnel opposite from that of the
assessment center. For example, if the assessment
center rated an individual poorly, these results
indicate that chances are that this individual's
military supervisor would rate him highly. (p. 67)
Conclusions. In most cases, the ratings of candidates
based on the totality of assessment procedures seemed to have
validity superior to any of the specific components. Since
the situational tests represented a unique contribution to the
process, and were relied upon heavily, it was usually assumed
that these were what made the difference, although specific
data often were not reported. The in-basket's contribution
usually was considered critical, although it mainly tapped
administrative skills and was thus more narrow in scope than
some of the other exercises. Other situational tests were so
varied, and data were so seldom reported on them, that
conclusions were difficult to draw. The mental ability tests
seemed to work for some companies but not for others;
personality tests showed moderate to little success, but they
continued to be used. The projectives and interviews were
more successful than expected from their past reputations.
More research evidently is needed on the various components
of assessment centers and their integration (Howard, 1974).

95
Summary and Critique
The job of the administrator has changed considerably.
It is perceived as more complex than ever before, requiring
multiple skills and abilities, and is affected by many outside
forces. Some selection methods of administrative personnel
are used in sequence as screening devices, whereas other
methods consider many different personal qualities in concert
in the evaluation of a job candidate.
The use of one or several selection methods is a function
of the importance of the job, the money and the time available
for selection, and the accuracy of any one method compared
with a combination of methods. Research studies can be cited
that arrive at opposite conclusions. In some cases, one
selection method can be superior to several methods. In other
cases a combination of methods is superior to any one.
Theoretically, several methods can never be inferior to one
if they are combined on a statistically sound basis, but this
is rarely done. A valid method can be reduced in
effectiveness by the addition of an inferior one, except when
the less valid method measures attributes not otherwise
detected (Mandell, 1964, p. 22).
The quality of validity, addressed in this study, has been
studied in personnel selection research and literature for
decades. Often a method was "sold" based upon what it was
claimed to measure, without any evidence that the claims were
justified. In some cases, the endorsement of a user who had

96
conducted no research studies was used. More technical means
of distortion occurred when claims of overall validity were
made on the basis of the outcome scores in a training course,
correlation with another selection method, or with a criterion
that represented only a portion of the total job requirements
(Mandell, 1964, p. 2).
As Stahl (1976) stated:
The worst sins in examining personnel are not inherent
in any single selection instrument or method, but are
derived from using the wrong selection instrument or
method, using something that is inappropriate for
gauging the abilities really needed in a given
occupation or position. When the wrong measuring
instrument or method has been used, too often the
resulting criticism has been directed against the
method itself instead of against the bad judgment of
the selectors. (pp. 134-135)
Previous reviews of administrative personnel selection
research have been subjective in nature (Howard, 1974; Korman,
1968). In the present study, the statistical synthesis of the
research data has been done using the objective process of
meta-analysis. Specific procedures used in the conduct of the
meta-analysis of administrative personnel selection methods
are introduced in the following chapter.

CHAPTER III
PROCEDURES
Introduction
In this chapter, the purpose of the study, the rationale
for the use of the meta-analytic approach, the application to
selection research for administrative personnel, the
identification of the research studies used, the
classification of variables, and procedures for the meta
analysis are discussed. These procedures include: conversion
to a common metric; averaging correlations across studies and
variables; elimination of sampling error; and adjustments for
error of measurement.
The Purpose of the Study
The purpose of this study was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. Toward this end, the questions
posed in the Statement of the Problem in Chapter I have been
defined here in greater detail. In Chapter IV, the
presentation of the results of the meta-analysis, the
following questions are answered.
1. What methods for the selection of administrative
personnel are commonly used by corporations, public
97

98
administration agencies, the military services, and
institutions of higher education?
a. What are the most commonly used administrative
selection methods across all types of organizations?
b. What are the most commonly used administrative
selection methods in each type of organization?
c. How does the frequency of use of each selection
method relate to its ability to predict administrative
achievement?
2. What does research demonstrate are the methods which are
significantly related to achievement in administrative
positions?
a. To what extent do all selection methods, taken
inclusively, correlate with administrative achievement?
b. To what extent do different selection methods
correlate with administrative achievement?
c. To what extent do selection methods, taken both
inclusively and separately, correlate with different measures
of administrative achievement?
d. To what extent do selection methods predict
achievement in successively higher levels of administration?
e. To what extent do selection results of the assessment
center approach correlate with administrative achievement?
f. To what extent does the type of interaction (i.e.,
written, verbal, or performance) required by an applicant in

99
completing a selection test correlate with administrative
achievement?
3. What information, derived from a meta-analysis of the
methodologies of selection research, is useful in interpreting
the findings of this study?
a. To what extent does criterion contamination effect
the correlation between selection method results and
administrative achievement?
b. To what extent does restriction of range effect the
correlation between selection method results and
administrative achievement?
4. What guidelines can be derived from this research for
improving personnel selection methods for administrators of
public community colleges?
a. To what extent do the methods now used for the
selection of administrative personnel in public community
colleges correspond with those methods which show the highest
correlation to administrative achievement?
b. What selection methods, indicated by the results of
the meta-analysis, could be added to or dropped from the
methods currently in use to make the selection process more
accurate and efficient?
The Rationale for the Meta-analytic Approach
In his presidential address to the 1976 meeting of the
American Educational Research Association (AERA) Glass (1976)
introduced the concept of meta-analysis. In distinguishing

100
meta-analysis from primary analysis (the original analysis of
data from a research study) and secondary analysis (the
reanalysis of existing data in order to answer the original
research question), Glass remarked:
My major interest currently is what we have come to
callnot for want of a less pretentious namethe
meta-analysis of research. The term is a bit
grand, but it is precise, and apt.... Meta-analysis
refers to the analysis of analyses. I use it to
refer to the statistical analysis of a large
collection of analyses results from individual
studies for the purpose of integrating the
findings. It connotes a rigorous alternative to
the casual, narrative discussions of research
studies which typify our attempts to make sense of
the rapidly expanding research literature. (p. 3)
The major thrust of meta-analysis is the understandable
integration of existing research. In many areas of
sociological, psychological, and educational research, the
need to understand the research which has been done is much
greater than that of adding another study to the storehouse
of knowledge. This situation has led to conflicting results
which lead to no acceptable answers to the problems posed,
but instead yields unending calls for further research.
Scholars have access to thousands of research studies, but
progress in making sense out of them for any given topic is
painfully slow. Often we "know" more than we are able to
"understand." Furthermore, literature reviews of empirical
studies are notorious for depending on the subjective
judgments, preferences, and biases of the reviewers, and

101
conflicting interpretations of the evidence are not uncommon
(Wolf, 1986, p. 10).
The meta-analytic method permits quantitative reviews
and syntheses of the research literature. "What is needed
are methods that will integrate results from existing studies
to reveal patterns of relatively invariant underlying
relations and causalities, the establishment of which will
constitute general principles and cumulative knowledge"
(Hunter et al., 1982, p. 26). In the meta-analytic approach,
"the findings of multiple studies should be regarded as a
complex data set, no more comprehensible without statistical
analysis than would hundreds of data points in one study"
(Glass et al., 1981, p. 12).
The Application to Administrative Personnel Selection Research
The meta-analytic approach is warranted in the case of
administrative personnel selection research. Wolf (1986)
suggested that the method for synthesizing the results of
correlational studies was to take the average of the
correlations between the two variables that examine the same
research question across separate research studies. According
to Hunter, Schmidt, and Jackson (1982), "cumulation of results
can be used whenever there are at least two studies with data
bearing on the same relation" (p. 28) They specifically
addressed the use of meta-analysis in synthesizing personnel
selection research. They recognized that in personnel
selection research different tests were often used to measure

102
the same variable. Thus, "variation in reliability
contributes to the variation in correlations across studies
if the review covers all studies using a given predictor
rather than a fixed test" (p. 91).
Although hundreds of studies exist and many claims are
made, the results of existing administrative personnel
selection research are unsettlingly discrepant. Fifty-two
studies met the fairly strict delimitations set forth in this
research. Several studies yielded more than one correlation
coefficient. A total of 128 correlation coefficients were
derived from the 52 studies. Care was taken to maintain
independence of samples within and between studies. Where
subsamples were analyzed within a study, and the relationship
between the same variables was tested in each subsample, these
subsamples were combined to yield an average correlation
coefficient. The few negative correlations found were treated
as positive for the reason that this study deals with the
strengths, not the directions, of the methods to predict
achievement. The strengths of correlations between selection
methods and administrative achievement in the research studies
reviewed ranged from r = .00 to r = .65. Their mean
correlation was r = .22 which increased to r = .25 when the
correlations were weighted for sample size according to
formula (3) in this chapter.
The intent of meta-analysis is to help understand what
causes variance in previous studies and to provide a much more

103
stable framework upon which tentative theories can be based.
Meta-analysis gives the researcher the capability to study not
only the subject at hand but also the frailties of the
research designs. If design seems to be a problem in a number
of studies, or if specific design flaws are occurring
regularly, the proper approach is to code that characteristic
along with the studies' other characteristics and empirically
determine how much of the variance in results can be accounted
for by such design and analysis flaws. Such coding was done
in this research.
The rationale and need for meta-analysis of research
findings can be summarized by a final comment from Glass's
AERA speech:
We need more scholarly effort concentrated on the
problem of finding the knowledge that lies untapped
in completed research studies. (p. 5)
Identification of the Research Studies
Many research studies concerning administrative personnel
selection were present in the literature. The sources
included only published studies found in journals,
periodicals, information retrieval systems, and books. Glass
(1976) was aware that information was plentiful when he stated
that:
We are inundated with information. The ERIC system
fills over two million document requests yearly.
Some have termed our predicament "the misinformation
explosion." I assess it differently; we face an
abundance of information. Our problem is to find
the knowledge in the information. (p. 4)

104
Data sources concerning the selection of administrative
personnel for the period January 1, 1962 through December 31,
1985 were located in manual and computerized indexes. Books,
periodicals, and microfiche collections of The Library of
Congress and libraries of the University of Florida, the
University of South Florida, George Mason University, the
American Psychological Association, and various Department of
Defense installations were used. Computerized index searches
of the NEXIS, Educational Resources Information Center (ERIC),
PSYCINFO, the Defense Technical Information Center, and LC
MARC databases were obtained. Research studies were most
often found published in professional journals in the fields
of psychology, personnel, public administration, and
management.
Classification of Variables
Independent variables. This research dealt with
personnel selection methods rather than with individual
selection tests. Selection methods were used as the
independent, or predictor, variables in the present study.
When specific tests were administered in a research study,
each test was recorded by the type of selection method it
represented. In a 1968 review of personnel selection
research, Korman delineated selection methods in the following
manner:
I. Psychometric Prediction
A. Cognitive Ability Test

105
1. verbal ability
2. mechanical aptitude
3. adaptability
4. critical thinking
5. clerical ability
B.Objective Personality and Interest Inventory
1.
preferences
2.
vocational
interest
3 .
personality
inventory
4 .
temperament
inventory
5.
cooperation
test
6.
risk scale
C. Leadership Ability Test
1. leadership opinion
2. sentence completion
3. practical judgment
D. Personal History Data
1. personal history blank
2. age, education
3. academic grades
II. Judgmental Prediction
A. Executive Assessments (assessment centers)
B. Peer Rating
1. overall rating
2. critical incident
3.combat rating

106
C. Superior and Faculty Rating
A look at the research studies Korman (1968) reviewed
revealed 13 selection methods that were most commonly used.
Of these, nine appeared to be mutually exclusive and have been
used as classifications of the independent variables for the
purpose of this study:
1. aptitude and intelligence measures,
2. personal interviews,
3. job-related skills indicators,
4. psychological attribute indicators,
5. values systems assessments,
6. biographical information,
7. peer ratings,
8. self-appraisals, and
9. assessment center processes.
For the purpose of this study, the numerous correlations
contained in each report of an assessment center process have
been separated and used independently, by selection method,
where possible. In addition, the overall assessment ratings
(OAR) have been included in order to allow for a comparison
between this unique conglomeration of data and individual
selection methods.
Dependent variables. All dependent variables reflected
a measure of achievement at an administrative level. In a
review of selection research, Korman (1968) used at least
fourteen different measures of achievement: (a) supervisor

107
ratings, (b) turnover, (c) objective performance data, (d)
overall ratings, (e) promotion rate, (f) administrative level,
(g) tenure, (h) suggestion plan activity, (i) pay, (j)
mobility, (k) position level changes, (1) termination, (m)
critical incidents, and (n) combat ratings.
For the purpose of this study these have been condensed
to the following seven measures:
1. administrative level achieved,
2. salary level attained,
3. supervisor ratings,
4. number of years serving in an administrative
position within the same organization,
5. achievement of tenure,
6. objective performance data, and
7. promotion rate.
Research reviewed that defined administrative
achievement as measured by (a) admissions personnel ratings,
(b) success in being hired, or (c) performance success in
subsequent training programs were not used in this study.
These criterion were judged to be too far removed from any
measurement of actual on-the-job performance.
Additional variables. Additional variables were
classified to aid in the meta-analyses of both the content and
the methodology of the research studies used in this
dissertation. In order to understand better the validities

108
involved in administrative personnel research the following
aspects of each study were recorded and coded:
1. the year the study was published (where the year
was not given, the year of the report was used),
2. the type organization the sample was derived from
(corporations, educational institutions, military
services, or public administration),
3. the number of subjects in the sample,
4. the types of subjects used, their job levels and
sexes,
5. the number of years between measurement of the
independent and dependent variables,
6. an ordinal measure of range restriction,
7. the type validity, predictive or concurrent,
8. the presence or absence of criterion
contamination, and
9. whether the independent variable was derived from
an assessment center.
Procedures for the Meta-Analysis
The methods for synthesizing the results of
correlational studies that report on the relationship between
two variables are rather straightforward. Essentially, the
average of the correlations between the two variables that
examine the same research question across separate research
studies is obtained and analyzed. Reports of administrative
selection methods usually expressed findings in the form of

109
a correlation coefficient. Glass (1977) believed that studies
which were combined must have similar aspects. He was aware
that Light and Smith (1971) stated that the independent and
dependent variables analyzed had to be measured in the same
way, or in a manner which could be converted into the scales
used in the majority of studies. Procedures used in this
study followed those described by Glass (1977) for the meta
analysis of a sample of independent correlational studies.
Calculation of Data
Most studies contained more than one correlation
coefficient. Thus, each individual correlation within a study
was treated as a separate record as long as at least one of
the variables measured was different from all others. Each
record was recorded in a database management program which
contained fields for each of the variables listed above. The
answers to the questions posed in the Statement of the Problem
were derived from taking cross-sections of the data within the
variables by sorting studies based upon the fields listed
above. The following meta-analytic procedures were then
followed to better understand the data.
Conversion to a common metric. The correlations in each
study were first converted into a common metric in order to
determine their relative effect sizes. The effect size is
simply a measure of the strength of the relationship between
two variables. In this dissertation, the Pearson product-
moment correlation was used as the common measure of effect

110
size. Research studies reviewed here which were not
originally reported using this correlation were all converted
to their equivalent Pearson product-moment correlations using
the formulas given in Wolf (1986).
Averaging correlations across studies and variables.
From the correlation coefficients derived by each query, means
and standard deviations were calculated for the relationships
between selection methods and administrative achievement using
formulas (1) and (2). According to Wolf "the average of the
correlations between the two variables that examine the same
research question across separate research studies is
obtained" (1986, p. 28). Means are typically calculated by
averaging the raw Pearson product-moment correlation
coefficients (r) using the formula:
Zr
r = (1)
n
r = mean correlation
r = Pearson correlation
n = number of correlation coefficients combined
(Wolf, 1986, p.29)
The standard deviations of the means were then
calculated using the formula:
a
square root of
Zr£ -
(Sr)2
N
N
(2)
a = standard deviation

N = number of cases
r = correlation coefficient
111
However, with the widely varying sample sizes found in
the research studies reviewed, from 20 to 8,885 subjects, it
was deemed necessary to weight all mean correlations by sample
size. This was done according to the formula:
S[N,-r,]
r = (3)
zNj
r(. = correlation coefficient in study i
Nj = number of persons in study i
(Hunter, Schmidt, & Jackson, 1982, p. 41)
Elimination of sampling error. When an inference is
made from a sample to a population a certain amount of error
is involved because random samples can be expected to vary
from one to another. Sampling error is defined as the
difference between a population parameter and a sample
statistic. Researchers depend upon sample statistics to
estimate population parameters, thus, the knowledge of how
samples are expected to vary from populations is a basic
element in inferential statistics (Ary, Jacobs & Razavieh,
1979, p. 136).

112
Sampling error causes the variance across studies to
increase relative to the variance of the population
correlations. The effect of sampling error on the variance
is to add a known constant, which is called the sampling error
variance. To eliminate the effect of sampling error from a
meta-analysis the measure of sampling error variance is
subtracted from the observed variance. This is done using the
following formulas for finding the variance of correlations
across samples:
Z[N¡(r, ir)2]
S =
S2 = estimate of variance of r
r
Ni = number of persons in study i
r. = correlation coefficient in study i
r = mean correlation
(4)
e
(1 r2) 2K
EN
(5)
a2 = sampling error
e
r = mean correlation
K = number of studies
N = number of persons in study
Thus, from formulas (4) and (5) we see that the variance of
the population correlations is estimated by:

113
(1 r2) 2K
est a2 = S2 (6)
r N
J
a = estimated variance in population correlations
S2 = estimate of variance of r
r
r = mean correlation
K = number of studies
N = number of persons in study
(Hunter, Schmidt, & Jackson, 1982, p. 44)
Adjusting for error of measurement. To eliminate the
artifacts due to error of measurement, reliabilities of the
variables are computed according to reliability data given in
the research. The formulas for adjusting for error of
measurement are:
s7r
a = (7)
N
a = mean product variable
R = measure of reliability
N = number of persons in study
(Hunter, Schmidt, & Jackson, 1982, p. 82)
Then a correlation closer to the true score can be calculated
by:
P
TU
*y
a b
(8)

114
PTU = mean true score
= mean product variable
b = mean product variable
(Hunter, Schmidt, & Jackson, 1982, p. 83)
Reliability data were not found reported in all of the
research studies reviewed. Reliability data were presented
in several studies for the independent variables used (Bray
& Grant, 1966; Dicken & Black, 1965; Grant & Bray, 1969;
Grant, Katkovsky, & Bray, 1967; Hollander, 1965). The
reliability coefficient of .82, used in this study, was
derived by averaging the reliability findings of these five
research studies.
One reference was found for reliability data on the
outcome variable of supervisor ratings. This coefficient was
.60 (King, Hunter, & Schmidt, 1980). This coefficient has
been used for the reliability coefficient of the combination
of outcome variables in this study.
Conclusion
The results of the application of the above procedures
to the data derived from the 52 research studies reviewed are
presented in Chapter IV to answer the questions posed in the
Statement of the Problem. As Glass has stated:
Most of us were trained to analyze complex
relationships among variables in the primary
analysis of research data. But at the higher
level, where variance, nonuniformity and
uncertainty are no less evident, we too often

substitute literary exposition for quantitative
rigor. The proper integration of research
requires the same statistical methods that are
applied in primary data analysis. (1976, p. 6)

CHAPTER IV
PRESENTATION OF THE META-ANALYSIS OF THE DATA
Introduction
The Korman (1968) study combined with comments by Howard
(1974) provided a framework for this study of selection
methods for administrative personnel, their predictive
validities, and guidelines for administrative personnel
selection in public community colleges. Hunter, Schmitt, and
Jackson (1982) supplied the statistical technique, meta
analysis, by which one could quantify the answers to the
questions.
In the present study answers to the following questions
were sought:
1. What methods for the selection of administrative
personnel are commonly used by corporations, public
administration agencies, the military services, and
institutions of higher education?
2. What methods for the selection of administrative
personnel, derived from a meta-analysis of selection research,
are significantly related to achievement in administrative
positions?
116

117
3. What information, derived from a meta-analysis of the
methodologies of selection research, are useful in
interpreting the findings of this study?
4. What guidelines can be derived from this research for
improving personnel selection methods for administrators of
public community colleges?
In the study, the findings of research of administrative
personnel selection methods for the years 1962 through 1985
inclusive were investigated. This study built upon a
classification system used by Korman (1968) and was carried
further by the use of meta-analysis. Since summary statistics
could be standardized into standard effect size measures, it
was possible to combine the quantitative findings of different
studies and to identify those selection methods which showed
validity in predicting administrative achievement.
Initially, approximately 300 research studies on
administrative personnel selection methods were surveyed.
Many reports contained outcome criterion measures which were
not usable considering the delimitations of this study.
Finally, 52 research studies which contained a total of 128
correlation coefficients were identified as meeting the
criterion for this study.
In this chapter, the results of the meta-analysis of the
data derived from the 52 research studies reviewed are
presented. Further reporting of the studies is based on the

118
128 correlation coefficients rather than discussing the
studies individually.
Selection methods are abbreviated in the tables as
follows: (a) aptitude and intelligence measures (APT), (b)
personal interviews (INTER), (c) job-related skills indicators
(JOB), (d) psychological attribute indicators (PSYCH), (e)
value systems assessments (VAL), (f) biographical information
(BIO), (g) peer ratings (PEER), (h) self-appraisals (SELF),
and (i) overall assessment ratings (OAR). Outcome criteria
are abbreviated in the tables as follows: (a) supervisor
ratings (SUPRAT), (b) administrative level achieved (ADLEV),
(c) promotion rate (PRO), (d) salary level attained (SAL), (e)
number of years serving in an administrative position within
the same organization (NUMYR), and (f) a combination of any
of the above criteria (COMB).
A General View of the Research
In Tables 1 and 2, descriptive analyses of the research
are displayed. The frequency of reporting of correlations by
year and selection method is indicated in Table 1. The nine
independent variable classifications are those used throughout
this dissertation. The greatest number of correlations (16)
was reported in 1984, while 14 were gathered in 1977, 13 in
1965, and 12 in 1970. Other years yielded an average of four

119
Table 1
Frequency of Independent Variable Classifications of
Correlations by Reporting Year
Selection method
Year
APT
INTER
JOB
PSYCH VAL
BIO
PEER
SELF
OAR
1962
1
2
1
1963
1
1
2
1964
2
2
2
1
1965
5
3
4
1
1966
1
1
1
2
1
2
1967
1
3 1
1
1
1968
6
1
1969
2
1
1
1
1970
1
2
2
1
1
1
1
3
1971
1972
1
1
1
2
1973
1974
1 1
1
1
1975
3
1
2 2
1
2
1976
1
1
1
1977
2
9
2
1
1978
1
1
1
1
1
1979
1980
1981
1
1982
1
1983
1
1984
2
2
2
2
2
3
3
1985

120
studies each with five years yielding none. Research on
administrative personnel selection methods may have been done
during these years, however, any such research did not meet
the fairly strict delimitations of this study.
The study of the validity of psychological attribute
indicators, such as motivation, has received by far the most
attention in research. Of the total of 128 correlation
coefficients found, 37 were derived from research of
psychological attributes. Clearly there was an interest in
finding the motivations and attitudes held by successful
administrators.
The second most frequently researched administrative
selection method was aptitude and intelligence measures.
Researchers have persisted in studying the validity of
intelligence measures despite the fact that administrative
position applicants were already highly preselected on this
trait. Possible reasons for this emphasis in the research
are simply the availability of the data and the relative
reliability of intelligence measures.
The selection methods of job-related skills indicators
and assessment center overall assessment ratings are almost
equally represented in the research. This equality in study
is most likely due to the fact that most assessment centers
rely heavily on job-related skills indicators, so for each

121
overall assessment rating at least one measure of job-related
skills was also recorded.
The remaining administrative personnel selection methods
were represented relatively egually in the research with
between 5 and 11 correlation coefficients each recorded in
this study. Interestingly, these "least studied" methods
contain all three of the methods which are used most
freguently in actual administrative personnel selection.
A review of the data derived from the research studies
revealed that 91% of the administrative personnel selection
research was done within corporations, none within public
administration agencies, 6% within the military services, and
3% within educational institutions.
In Table 2, descriptive data are presented for the
distributions of the validity coefficients before weighting
for sample sizes or correcting for sampling error and
reliability. The mean correlation, median, standard
deviation, and range are shown for each selection method. The
mean and standard deviation were calculated using formulas (1)
and (2) respectively from Chapter III.
It is clear that the correlations which make up each mean
correlation have a wide range. Typically the standard
deviations from the mean egual approximately 50% of the mean.
For most selection methods, the distributions were fairly

122
Table 2
Mean Correlations. Medians. Standard Deviations, and Ranges
of Uncorrected Data
Selection Mean correlations, medians, standard
method deviations, and ranges
OAR [ ( X + ) ]
JOB [ ( X+ ) ]
VAL [ ( X + ) ]
INTER [ ( X+ )-]
APT [ ( +X ) ]
PEER [ ( X + ) ]
BIO [ ( + X ) ]
PSYCH [-( +X ) ]
SELF [-( X +])
-.10 .00 .10 .20 .30 .40 .50 .60 .70
Note. [ ] = range, X = mean, + = median, ( ) = SD.
symmetric. For self-appraisals, however, the distribution is
severely skewed to the left.
Administrative Personnel Selection Methods
In this section, administrative personnel selection
methods are discussed in terms of which methods have been most

123
commonly used in personnel selection overall and in each
specific areas of (a) corporations, (b) public administration,
(c) military services, and (d) education. The frequency of
use of each selection method was compared to its ability to
predict administrative success. The statistical data in this
section were derived by calculating mean correlations across
selection methods and by then weighting these correlations by
sample size using formula (3) presented in Chapter III. The
complete meta-analysis of each method, in which corrections
were made for sampling error and reliability, is presented in
a later section.
Most Commonly Used Administrative Personnel Selection Methods
The most commonly used selection methods for
administrative personnel remain essentially the same
throughout all types of organizations considered.
Biographical information was used to weed out those who did
not meet minimum
qualifications
for
the
position.
Self
appraisals were
often provided
for
administrative
level
positions in the
form of cover
letters
to resumes.
The
military services and public administrative agencies were
likely to follow up with tests of aptitude and intelligence.
Almost all organizations included a personal interview. Peer
ratings were often provided in the form of references when
required. Measures of psychological attributes, values

124
systems, job-related skills, and overall assessment center
ratings were less often used.
One of the most commonly used administrative personnel
selection methods was the biographical information method.
This information is presented in the form of application
blanks and resumes (Levine & Flory, 1975). Its popularity
stemmed from the accepted notion that the best predictor of
future behavior is past behavior (Childs & Klimoski, 1986) .
In most cases, biographical information has been used to help
determine if an individual meets the minimum qualifications
for a particular job. Most often this has been the way in
which a large group of job applicants was reduced to a
manageable number for further consideration.
Another often used method has been the personal
interview. According to Carlson (in Levine & Flory, 1975)
approximately 50% of those persons filing applications or
resumes are ultimately interviewed. Spriegel and James (in
Ulrich & Trumbo, 1965) revealed that 99% of 852 firms surveyed
had interviewed applicants before hiring. This method may be
combined with tests of aptitude or job-related skills, but at
the administrative level often it is not. According to Jablin
(1975) ,
the interview should serve as the key predictor of
the prospective employee's ability to fit into the
organizational environment. In other words, the
interview would not evaluate the candidate's
occupational skills and aptitudes but rather his/her

125
potential to fit into the social-functional
environment of the job and organization. (p. 5)
Until recently, evidence for the validity of the interview
was particularly low (Arvey, Miller, Gould, & Burch, 1986).
Peer ratings have been frequently collected in the form
of references and recommendations. Research on peer ratings
may be generalized to include references given by an
applicant's peers or co-workers. According to Korman, (1968)
peer ratings are "impressions gathered from interactions of
an equal, non-supervisor-subordinate nature" (p. 313). No
research was found which addressed references per se.
The remaining selection methods were used less frequently
than the ones mentioned above. At progressively higher levels
of management, fewer inquiries have been made into an
applicant's values, motivations, and intelligence other than
what can be gathered in the most commonly used methods. Job-
skills indicators and assessment center processes have been
used only in organizations which process a fairly large amount
of applicants.
The selection methods, and the order in which they are
employed, vary slightly from one type organization to another.
Research done in corporations pervades the literature. Most
of this research has been conducted by university-based
researchers who either have been hired to do research for the
company or have been doing personnel selection research for

126
a doctoral dissertation. Little research has been done with
public administrative agencies. None of the research which
fits the delimitations of this study was conducted in public
administration.
A moderate amount of personnel selection research had
been completed within the military services; however, most of
this research did not meet the parameters of this study. Very
little research has been done within educational institutions.
Although most administrative personnel selection research has
been conducted by university-based researchers, the subjects
of their studies were almost always drawn from employees of
a corporation and not from educational institutions. This
research was often funded by the individual corporation.
Validities of Selection Methods and Their Frequency of Use
All personnel selection methods, combined, correlated
significantly at the .05 level with administrative achievement
with a mean correlation coefficient of r = .22 (a = .14),
calculated according to formulas (1) and (2) in Chapter III,
and r = .25 when weighted for sample size according to formula
(3). Thus, with combined samples the correlations were
significant meaning that personnel selection methods, taken
as a whole, were predictive of administrative achievement.
For sample sizes of at least 100 subjects, the .05
significance level is set at r = .195 (Ary, Jacobs, &
Razavieh, 1979). However, the frequency of use of

127
administrative selection methods appeared to bear little
relationship to each method's ability to predict achievement.
The correlation coefficients used here were mean correlations
taken across each selection method and weighted for sample
size (see Table 3).
The assessment center overall assessment rating was
ranked highest in validity of the selection procedures. A
total of 14,224 subjects were included in 14 studies which
yielded 16 correlation coefficients involving assessment
center overall assessment ratings. The mean correlation of r
= .32 (a = .14) was raised to r = .37 when weighted according
to sample size. Although this method is highly valid it has
been rarely used outside large corporations because of the
costs in time and money.
Peer ratings correlated with administrative achievement
showing a mean coefficient of r = .28 (a = .14). When
weighted for sample size, this correlation dropped to r = .20.
This selection method dropped from its second place ranking,
according to strength, to sixth place when weighted.
Essentially related to references and recommendations, the
predictive ability of this method indicated that references
and recommendations may have some validity in the selection
process. Unfortunately, considering the legal problems that

128
Table 3
Rankinas
of Validities of Selection
Methods With
and Without
Weiahtincr for Sample
Size
Rank
Method
Unweighted r
Method
Weighted r
1
OAR
. 32
OAR
. 37
2
PEER
.28
JOB
.30
3
JOB
.25
VAL
.25
4
BIO
.23
INTER
.24
5
INTER
.20
APT
.21
6
VAL
.20
PEER
.20
7
PSYCH
. 19
BIO
. 18
8
APT
. 19
PSYCH
. 15
9
SELF
. 17
SELF
. 10
truthfulness in references might bring, there may be a large
gap between the validity of this method when used in research
as opposed to its use in actual hiring.
Job-related skills indicators were ranked third in
strength of validity. The correlation coefficient of this
method was strengthened from i = .25 (o = .13) to r = .30 by
weighting for sample size. Thus, it was ultimately ranked as
the second most valid method after weighting. This method has

129
been rarely used outside an assessment center program. The
measurement of job-related skills indicators usually involves
rating a performance on tests of job skills, and this involves
costs in both time and money in setting up reliable
performance tests of administrative skills.
Biographical information correlation coefficients showed
a decrease in predictive strength from r = .23 (a = .16) tor
= .18 after weighting for sample size. Thus, it slipped from
fourth place to seventh place in predictive validity. This
is a mediocre ranking for such a frequently used selection
method.
The selection method of personal interviews, was
strengthened when weighted for sample size. Originally ranked
fifth with a correlation of r = .20 (a = .10), it was raised
to r = .24, fourth place, when weighted. Thus, at least one
of the three most frequently used selection methods ranked in
the top half in predictive validity.
The selection method of values systems assessments was
strengthened when weighted for sample size. The mean
correlation coefficient was raised from r = .20 (a = .09) to
i = .25. Thus, this method rose from sixth to third in
ranking by predictive strength. According to England and Lee
(1974), "it is evident that value patterns are predictive of
managerial success and could be used in selection and
placement decisions. A number of legal and ethical issues

130
must be considered, however, before such use is justified" (p.
418) .
Ranked seventh was the method of psychological attribute
indicators with a mean correlation of r = .19 (a = .11) This
correlation was lowered to r = .15 when weighted for sample
size. Thus, this method dropped to eighth place out of the
nine methods. Measures of interests have not been used
extensively in managerial selection (Nash, 1965). Often it
was not used past the first-level of supervisory selection.
The intelligence and aptitude measures selection method
originally displayed a mean correlation coefficient of r = .19
(a = .11) which was strengthened to r = .21 by weighting for
sample size. Its ranking was raised from eighth to fifth
place by this increase in strength. The use of this method
has become increasingly infreguent as the higher levels of
management are reached.
Ninth in original and final rankings was the method of
self-appraisals with a mean correlation coefficient of r = .17
(a = .10). Predictive validity was dropped to r = .10 by
weighting for sample size. Self-appraisal remained the lowest
ranked administrative personnel selection method. This method
has been freguently seen in the form of some resumes and in
the cover letters to resumes.
In Table 4 the frequency of use of each selection method
compared to the relative strength of the mean effect size of

131
each method is presented. The mean correlation coefficient
is referred to as an estimate of "effect size" (Hunter et al.,
1982, p. 145). Effect size ranges were based upon definitions
for small, medium, and large effect sizes as stated for point
biserial correlations in Cohen (1977). Thus, as shown in
Table 4, the two selection methods which showed medium effect
sizes, overall assessment ratings and job-related skills
indicators, were also the least used.
Table 4
Selection Method Frequency of Use Compared With Mean Effect
Size
Frequency
Mean effect size
Low
Medium
High
Frequent
PEER
INTER
BIO
Moderate
SELF
APT
PSYCH
Seldom
VAL
OAR
JOB
Two of the three most frequently used methods,
biographical information and peer ratings, showed low effect
sizes. Only one frequently used method, personal interviews,

132
showed a medium effect size. However, the validities of these
selection methods have not been corrected for sampling error
or reliability, and these corrections, done in the meta
analysis, were found to increase the effect sizes of the mean
correlations.
Selection Methods Significantly Related to Administrative
Achievement: A Meta-analvsis
The results presented in this section were derived by a
meta-analysis of the 128 correlation coefficients sorted by
independent variable, selection method. Formulas (4) through
(8), presented in Chapter III, were used.
Eight of the nine personnel selection methods were
significantly related to achievement in administrative
positions. The one method which was not significantly related
was self-appraisals. The rankings of the validities of the
selection methods remained the same after these corrections
for sampling error and reliability were completed in the meta
analysis (see Table 5). The fact that the relative rankings
of the selection methods remained the same after these
corrections were made is understandable. Correction for
sampling error had little effect due to the facts that (a) the
amount of sampling error is inversely proportional to the
sample sizes, and (b) the sample sizes, when combined across
studies, were large. Adjustments for reliability of
independent and dependent variables effected the strengths of

133
Table 5
Selection Method Frequency of Use Compared with Mean Effect
Size Derived Through Meta-analysis
Freguency
Mean effect size
Low
Medium
High
Freguent
INTER
BIO
PEER
Moderate
SELF
PSYCH
APT
Seldom
VAL
OAR
JOB
the correlation coefficients but not the ranking. Since
reliability data were scantily reported in the research, the
same reliability data were used for each meta-analysis done
on each independent variable.
In the following discussion of the results of the meta
analysis, the selection methods were ranked in the order of
their predictive validity, from highest to lowest. These
data, along with the corrected means, are presented in
Table 6. Corrected means were derived through calculations
done using formulas (4) through (8) presented in Chapter III.

134
Table 6
Mean Correlations. Sample Size Weighted Correlations, and
Corrected Correlations of Selection Methods
Method Mean r Weighted r Corrected r
OAR
.32
.37
.52
JOB
.25
.30
.43
VAL
.20
.25
.35
INTER
. 20
.24
.34
APT
. 19
.21
.30
PEER
.28
.30
.28
BIO
.23
. 18
.26
PSYCH
. 19
. 15
.21
SELF
. 17
. 10
. 14
Overall assessment ratinas
The
assessment center
overall assessment rating was by
far the
"most
valid" of the
selection
procedures. A total
of 14
, 224
subjects were
included in 14 studies which yielded 16 correlation
coefficients involving assessment center overall assessment
ratings. The mean correlation of r = .32 (a = .14) was raised
to r = .37 when weighted according to sample size. These

135
figures were obtained using formulas (1) through (3) in
Chapter III. The correction for unreliability caused the
weighted mean correlation to be raised from r = .37 to r
= .52, which was calculated according to formula (8) in
Chapter III.
Job-related skills indicators. A total of 16,517
subjects were included in 10 studies which yielded 12
correlation coefficients involving job-related skills
indicators. The mean correlation of r = .25 (a = .13) was
raised to r = .30 when weighted according to sample size.
Correction for unreliability caused the weighted mean
correlation to be raised from r = .30 to r = .43.
Value systems assessments. A total of 2,483 subjects
were included in the 4 studies which yielded 5 correlation
coefficients involving value systems assessments. The mean
correlation was r = .25 (o = .09) when weighted for sample
size. Correction for unreliability caused the mean
correlation to be raised from r = .25 to .35.
Personal interviews. A total of 11,526 subjects were
included in 6 studies which yielded 7 correlation coefficients
involving personal interviews. The original mean correlation,
.20 (a = .10), weighted for sample size, was r = .24. The
correction for the error of measurement caused the mean
correlation to be raised from r = .24 to .34.

136
Aptitude and intelligence measures. A total of 17,482
subjects were included in 15 studies which yielded 21
correlation coefficients involving aptitude and intelligence
measures. The mean correlation of r = .19 (a = .11) was
raised to r = .21 when weighted according to sample size.
Correction for unreliability caused the weighted mean
correlation to be raised from r = .21 to r = .30.
Peer ratings. A total of 3,413 subjects were included
in 8 studies which yielded 10 correlation coefficients
involving peer ratings. The original mean correlation of r
= .28 (a = .14), weighted for sample size, was r = .20. The
adjustments for error of measurement caused the mean
correlation to be raised from r = .20 to .28.
Biographical information. A total of 3,326 subjects were
included in 8 studies which yielded 11 correlation
coefficients involving biographical information. The mean
correlation of r = .23 (a = .16) was depressed to r = .18 when
weighted for sample size. Correction for unreliability caused
the mean correlation to be raised from r = .18 to r = .26.
Psychological attribute indicators. A total of 7,503
subjects were included in 22 studies which yielded 37
correlation coefficients involving psychological attribute
indicators. The mean correlation of r = .19 (o = .11) was
depressed to r = .15 when weighted for sample size.

137
Correction for unreliability caused the weighted mean
correlation to be raised from r = .15 to r = .21.
Self-appraisals. A total of 2,847 subjects were included
in 8 studies which yielded 9 correlation coefficients
involving self-appraisals. The mean correlation of r = .17
(a = .10) was depressed to r = .10 when weighted for sample
size. Correction for unreliability caused the mean
correlation to be raised from r = .10 to r = .14.
The corrected correlations of the nine selection methods
showed a mean correlation of r = .31 and a median of .30.
The validity of each selection method, with the exception of
self-appraisals, was significant at the .05 level. The
corrected mean correlations presented here, using formulas
(4) through (8) in Chapter III, are closer to the true mean
correlations than are the unweighted or sample size weighted
means. However, because of the varied methods of measuring
both the independent and dependent variables, the corrections
done here could only partially rectify the mean correlation
coefficients in the direction of their true scores. This is
evidenced by the large amount of variance that remained
unaccounted for after correction.
Relationships Between Selection Methods and Separate
Measures of Administrative Achievement
A review of the literature revealed general
dissatisfaction with all measures of administrative

138
achievement (Huck & Bray, 1976; Korman, 1968; Steel & Ovalle,
1984) What is called for is an outcome measure that
accurately and objectively assesses an administrator's level
of achievement on the job, rather than someone's subjective
opinion of such achievement. No plausible solutions to this
dilemma were suggested in the literature. In fact,
discrepancies in inter-dependent variable validities were
found in the literature. The relationships between selection
methods, both inclusively and separately, with the six
different measures of dependent variables used in this study
are shown in Table 7.
The outcome criterion which showed the highest mean
correlation was derived from an average of only two widely
disparate correlation coefficients and, thus, may not have
been a reliable measure of achievement. However, the measure
of administrative achievement which was a combination of two
or more of the other outcome criteria had a relatively high
correlation, r = .26, and was based upon a larger sample of
correlations. This finding substantiates the Campbell et. al
(1970) view that such an "overall success index" (p. 166),
while not explicitly identifying or quantifying actual
managerial behavioral dimensions, did identify managers
differing substantially in their ability to utilize
organizational resources effectively.

139
Table 7
Selection Method Correlation to Outcome Criteria
Method
SUPRAT
ADLEV
PRO
SAL
NUMYR
COMB
OAR
. 33
.38
-
.24
-
. 15
JOB
. 19
.28
. 11
. 26
-
.49
VAL
-
. 16
.22
. 32
-
-
INTER
. 13
.27
-
. 17
.35
.20
APT
. 13
.21
. 14
.25
-
-
PEER
. 32
-
.29
. 11
-
.33
BIO
. 14
. 11
. 17
. 17
-
.38
PSYCH
.24
. 17
.20
. 13
. 19
.08
SELF
. 16
.26

.01

.24
Mean r
.22
.24
. 19
. 18
.27
.26
Prediction
to Successivelv
Hicrher
Levels
of Administration
Several
attempts
have
been made (Korman, 1968;
Kraut,
1969; Roadman, 1964)
to determine
those
selection
methods
which can predict administrative achievement at succeedingly
higher levels of management. Roadman (1964) reported that
psychological tests had been found useful in predicting which

140
employees would be successful junior or middle managers. In
such tests, the mean correlation with mental ability was
definitely higher for top level executives than for middle
managers, but the large amount of overlap between the
distribution of scores reduced any ability to make predictions
from the tests.
Korman (1968) also found that verbal ability tests showed
little usefulness for predicting managerial performance above
the first-line supervisory level, because the typical
managerial applicant population was already highly preselected
on abilities. Biographical information, likewise, showed
"some predictive value" for first-line supervisors only
(Korman, 1968, p. 308).
There were not sufficient correlation coefficients
present in this study to do a complete meta-analysis in which
each selection method was averaged by each level of management
achieved. However, by combining all selection methods, and
comparing them to the different administrative levels
achieved, the following conclusions were found.
Selection methods correlated r = .36 with attainment of
the first-level of supervisory achievement. These same
methods correlated r = .27 and r = .28, respectively, with
the mid- and high-levels of administrative achievement. Thus,
the results confirmed the literature findings that selection
methods can more accurately predict achievement of the

141
first-level of administration, but have less validity in the
prediction of those applicants who will succeed at higher
administrative levels.
Validity of the Assessment Center Approach
Howard (1974) stated that, because of the expense
involved, psychologists tried to develop assessment centers
in more than a haphazard way. She surmised that projective
tests, interviews, and job-related exercises were designed
with care toward reliability and validity for each particular
organization. The results of this study, however, showed
little overall difference between the validities of each and
all selection methods when used within an assessment center
versus when used independent of any assessment center.
Table 8 shows that, of the eight selection methods (OAR
was not included), half displayed higher correlations within
an assessment center and half without. The mean correlations
of each group, r = .20 for the methods within an assessment
center, and r = .21 for the methods taken independent of any
assessment center, were almost identical.
Thus, it does not seem that the assessment center
approach added any value to each separate selection method.
Still the assessment center overall assessment rating was, in
11 out of 14 assessment center studies, significantly
correlated with administrative achievement. Several reasons
have been suggested for this phenomenon. Turnage and

142
Muchinsky (1984) recognized that the "Pygmalion effect" may
play a part. The self esteem of a person selected for
assessment may have been raised substantially and "they then
seek to perform both in assessment and later on the job at a
Table 8
Correlations of Selection Methods Measured Within and Without
Assessment
Center Processes
Selection
Assessment
Non-assessment
method
center
center
JOB
.26
.25
PSYCH
. 14
.20
INTER
. 18
.35
APT
.20
. 18
VAL
.24
. 18
BIO
. 32
. 18
PEER
. 12
.35
SELF
. 13
. 19
Mean r
.20
.21
level high enough to substantiate the lofty esteem in which
they are perceived to be held" (p. 602).

143
The validity of the job-related skills indicators used
in every assessment center process has been suggested as a
cause of the high validities of overall assessment ratings.
Howard (1974) stated that "since the situational tests
represented a unigue contribution to the process and were
relied upon heavily, it was usually assumed that these were
what made the difference, although specific data often were
not reported" (p. 126). The results of this study bear this
out. Thus, the job-related skills indicators contributed a
great deal to the overall assessment rating validity.
A final reason for the high validity of the assessment
center process concerns the statistical combination of the
data derived from each selection method within the assessment
center process. A stepwise multiple regression, used to form
a composite of the results of all selection methods within the
assessment center, has been used to almost double the validity
between the selection process and the outcome criterion. For
example, Wollowick and McNamara (1969) found an overall
assessment rating of r = .37 in the IBM assessment center
study. Using all the assessment data in an actuarial fashion
produced a multiple correlation of R = .62, accounting for 38%
of the variance (Howard, 1974). Thus, using the most common
selection methods, and combining them statistically, is an
efficient way of increasing validity.

144
Correlation of Types of Interactions Within Selection
Methods and Measures of Administrative Achievement
The predictive strength of each type of interaction,
written, verbal, or performance, required from an applicant
was found to be dependent upon the type of selection method
used. Selection tests which required primarily written
responses showed the lowest amount of correlation, r = .19,
when weighted for sample size. Verbal interactions showed a
weighted mean correlation of r = .24. Interactions involving
a performance of a task by the applicant showed the highest
validity, r = .31. The predictive strengths of these
interactions were not independent from the nine selection
methods considered.
The verbal interaction correlation coefficients were
derived solely from the studies involving the selection method
of personal interviews. The performance interaction
correlation coefficients were derived predominantly from the
selection method of job-related skills indicators. Thus,
these mean coefficients were almost identical to the selection
methods from which they were drawn.
Analysis of the Research Methods Used in Administrative
Personnel Selection Research
Information derived from a meta-analysis of the
methodologies used in personnel selection research were found
to be useful in interpreting the findings of the research.
The problems of criterion contamination, restriction of range,

145
and concurrent validity studies are discussed. The results
of an analysis of the merits of predictive versus concurrent
validity are shown. Results are given for an analysis of the
relative advantages of various time intervals between the
measurement of the independent and dependent variables in the
predictive studies used.
Criterion Contamination
The extent to which criterion contamination effects
hiring and promotion of administrative personnel has been a
constant concern of researchers. Howard (1974) stated that
problems of criterion contamination have confounded predictive
validity studies when personnel decisions based upon selection
ratings were used in determining future promotions. However,
she thought that even where these data were used to aid in
initial hiring or promotion decisions, the effect on later
promotions was not large.
The results of the present meta-analysis showed that
criterion contamination had no effect on the combined
validities of the independent variables. When weighted by
sample size, predictive validity studies containing criterion
contamination showed a mean correlation of r = .25.
Predictive validity studies for which the researchers reported
that no criterion contamination was allowed showed a mean
correlation of r = .26. Thus, it appears that the fears of
researchers, that subsequent measures of administrative

146
achievement may be biased based upon an employee's past rating
on a selection procedure, may be unwarranted.
Range Restriction
According to the results of the present meta-analysis,
range restriction had little effect on the combined validities
of the independent variables. One would expect that, as
amount of range restriction increased, the ability of the
correlation statistics to determine differences between
variables would decrease. That is, as range restriction
increases it is expected that the strengths of the correlation
coefficients will decrease. This was not proven to be the
case in this study. In Table 9, the effects of range
restriction on outcome measures of administrative achievement
are presented. All correlation coefficients reported were
weighted by sample size.
Studies in which there was no appreciable range
restriction, in which subjects were in nonmanagement
positions, showed a mean correlation coefficient of r = .25.
Where range restriction was considered low, where the original
subjects were low-level managers, the mean correlation
coefficient was r = 27. Where the subjects were mid-level
managers at the time of measurement of the predictor variable,
the mean correlation coefficient was r = .23. Where range

147
Table 9
Effects of Range Restriction on Outcome Measures of
Administrative Achievement
Effect size measurement
Range
restriction
Small
Medium
Large
None
. 25
Low
. 27
Medium
. 23
High
. 24
restriction was considered to be high, meaning the subjects
were either high-level administrators or the study was one of
concurrent validity, the mean correlation coefficient was
r = .24.
Predictive Versus Concurrent Validity
According to Stahl (1983), concurrent validity studies
might be expected to show higher correlation coefficients than
predictive validity studies. It has been suggested that, in
a concurrent study on administrative achievement, a supervisor
completing an inventory such as a psychological attributes

148
inventory, might reveal that he or she has many of the traits
needed to be a "successful" manager. He or she may not have
had these traits originally in his or her career, but may have
developed and reinforced them over time. However, the results
of this meta-analysis showed significance of, and little
difference between, the mean correlations of the predictive
validity and the concurrent validity studies, r = .25 and r
= .22, respectively, when weighted for sample size.
Years Between Variable Measurement
According to Turnage and Muchinsky (1984), if
organizations collect their criterion data immediately after
assessment, the candidates may not have had ample time to
demonstrate their performance on the job. But, if too much
time has elapsed between the assessment and the performance
evaluations, many extraneous factors may account for the
candidate's level of performance.
Data from this study showed that this may be the case.
The optimal time between measurement of variables may be from
3 through 10 years. Eighty-three correlation coefficients
were derived from studies in which the dependent variable was
measured less than 3 years after the measurement of the
independent variable. These included all coefficients derived
from concurrent validity studies. The sample size weighted
mean correlation was r = .21. Predictive versus concurrent
validity studies showed no appreciable difference in validity.

149
When intervening time between independent and dependent
variable measurement was increased to 3 through 10 years, the
mean correlation was raised to r = .30 when weighted for
sample size. By definition, these 34 coefficients were drawn
only from predictive studies. However, when years between the
measurement of variables was greater than 10 years, the 10
correlation coefficients concerned yielded a weighted mean
correlation of r = .19.
The weighted mean correlation of r = .30, for the studies
where time between variables was from 3 through 10 years, was
of interest to this researcher. Explanations for this
relatively high validity were sought. The most logical
explanation would be for an included study to contain a large
sample size and high validity coefficients. Studies by Moses
(1972) met these requirements. However, when the weighted
mean coefficients were recalculated omitting the studies by
Moses, the corresponding mean correlation coefficient was only
reduced to r = .26. Therefore, it is suggested that further
investigation be made into determining the optimal length of
time needed between measurement of variables in administrative
personnel selection research.
Conclusions of the Meta-analvsis of the Content and
Methodologies of Personnel Selection Research
The following is a discussion of the findings of the
content of administrative personnel selection research as

150
pertains to the predictive validity of each method. Although
a sufficient number of studies and correlation coefficients
were found to gain an accurate assessment of the predictive
validities of the nine selection methods, insufficient
research/reporting was found on details within each study
needed to develop moderator variables for in-depth research.
Sufficient data were provided to complete the review of
several methodological concerns. A summary of these results
is presented here.
Content
The overall assessment rating displayed a decidedly
higher correlation than all other methods. This correlation
was derived from a combination of both clinical judgments by
trained assessors or psychologists, and statistical synthesis
of these judgments (Korman, 1968). Thus, this method has
validity-enhancing aspects rarely found in other methods.
Job-related skills indicators, although lower in mean
correlation than the overall assessment rating, have been
attributed with contributing a great deal to the overall
assessment rating in terms of validity (Howard, 1974). This
study supported this view.
Values systems assessments showed a relatively high
correlation with administrative success. The results of this
study supported the findings of England and Lee (1974) that
"values patterns were significantly predictive of managerial

151
success" (p. 411) This was the "least researched" of the
nine selection methods in this study. In the light of the
relatively high validity shown, this method may warrant
further research. Care should be taken to define "values"
precisely and not consider them an extension of psychological
attributes indicators.
Ghiselli (1966) found the interview to have at least
moderately substantial validity. This study supported his
finding. However, there has been concern over just what
factors, or moderator variables, contribute to the validity
of the personal interview. It has been suggested that both
structure, such as with an interview schedule (Ghiselli), and
multiple assessors contribute to this validity (Carleton,
1970). A paucity of data concerning these specific factors
in the seven studies reviewed precluded any further analysis
within the selection method of personal interviews. Further
research is needed, with better reporting of the details
involved, to allow for a meta-analysis for the effects of
moderator variables within this method.
According to Korman (1968), ability tests have shown
little usefulness in predicting managerial performance above
the first-line supervisory level. In this study, aptitude
and intelligence measures fell on the median and near the mean
of the nine selection methods.

152
Korman (1968) found that peer ratings could be highly
predictive of later promotion rate. The present study showed
that peer ratings were significantly related to later
administrative achievement. According to Korman (1968), "when
one sees how consistently this is an effective predictor of
officer success, it is surprising that the method has not been
studied more thoroughly in the industrial context" (p. 313) .
Biographical information ranked third from the lowest,
but was still significant, as a selection method. Hinrichs
(1969) concluded that using biographical information was as
effective as using an assessment center process in identifying
management potential. This study does not bear this
conclusion out, however, in light of the possibility of
obtaining higher correlation, further research for moderator
variables within this method may be warranted.
The selection method of psychological attribute
indicators was significant in predicting administrative
success. However, it was the lowest ranked significant
method. This was a poor showing considering that 29% of the
research effort has been put toward understanding this method.
Thus, this study substantiated Korman's (1968) view, 20 years
later, that "there seems little reason for thinking that we
have learned much about the psychological variables indicative
of managerial behavior insofar as these variables are
determinable by objective personality inventories" (p. 302).

153
Self-appraisals were not found to be significantly
associated with administrative achievement. With the
exception of Heneman (1974), most researchers found that
"individuals rate themselves higher than they are rated by
comparison groups" as such they may lead to "inflated
statements of qualification" (Thornton, 1980, p. 265, p. 269).
This study substantiated this view.
Methodologies
The findings of the meta-analysis of the research
methodologies used in personnel selection research revealed
that the predictive validities of the predictor variables were
affected very little by the research methodologies. The
presence of criterion contamination and range restriction
produced minimal variation across studies. The results of
concurrent validity studies and predictive validity studies
similarly showed little deviation from one another.
These findings prompt this researcher to suggest that
personnel selection researchers might review the need for the
tightly controlled, long-term studies which are often called
for at the conclusion of concurrent validity studies. Is
there empirical evidence that such studies result in "more
valid" conclusions than concurrent studies, or is this an
assumption that has been perpetuated?
In Chapter V the processes currently used for the
selection of administrative personnel will be discussed. The

154
extent to which those methods now in use for the selection of
administrative personnel in public community colleges
correspond with those methods which show the highest
correlation to administrative achievement will be explored.
Guidelines, derived from the present study, for improving
personnel selection methods for administrators of public
community colleges will be suggested.

CHAPTER V
GUIDELINES FOR SELECTION OF ADMINISTRATIVE PERSONNEL
IN PUBLIC COMMUNITY COLLEGES
Introduction
The purpose of this study was to develop a set of
guidelines for improving the selection of administrators of
public community colleges. The findings of research on
administrative personnel selection methods were surveyed and
a meta-analysis was conducted on the data. In this chapter
the results of this meta-analysis are applied to the selection
of administrative personnel in public community colleges. The
results are discussed and guidelines for improving personnel
selection methods for administrators of public community
colleges are presented.
The Need for Valid Personnel Selection Methods
The history of the two-year college movement was
punctuated by a tremendous growth spurt in the 1960s and
1970s, which brought about an increased demand for
administrators to staff the new and growing colleges. During
this period, the two-year colleges did not always have time
to "grow their own" administrators. This historical fact
probably helped to contribute to a relatively open labor
market for two-year college administrators (Twombly, 1987, p.
14) .
155

156
During the 1980s, following this period of unprecedented
growth, community college administrators faced a new set of
challenges. Enrollments plateaued; state governments replaced
local communities as the colleges' principal source of
funding; steady state enrollments and greater fiscal
accountability meant that, for the first time, community
colleges had to look at educational outcomes (Bernstein,
1988) .
The diversity of the student body of the 1980s added to
the complexity the community college administrator faced
(Wattenbarger, Haynes, & Smith, 1982). Wolf (1985) stated
that a community college is an organization in which "goals
are unclear, technology is ambiguous, and participation is
fluid among faculty and administrators" (p. 55) Thus, he
suggested that the best way to achieve organizational outcomes
was to place emphasis on hiring intelligent people as
administrators. As Etzioni pointed out, the more selective
the organization, the less control it needs, and the fewer
problems its administrators face (in Webb, 1983).
Present Administrative Personnel Selection Procedures
In the past, the process of hiring college administrators
has usually been conducted in one of the following ways: (a)
the implementation of a search committee (Kelly & Nelson,
1978), (b) by placing a blind ad in the Chronicle of Higher
Education, or by (c) using an employment agency or executive
search firm (Mottram, 1983). The committee or person

157
responsible for the search described the position to be filled
and wrote an appropriate position description. The position
opening was often advertised in the Chronicle of Higher
Education. Whatever the method used, the rule has been for
each applicant to complete an application form, supply
transcripts, and send three letters of recommendation
(Scigliano, 1979). Thus, the "first cut" made by the search
committee has been based upon the selection methods of
biographical information and peer ratings in the form of the
completed application blank and recommendations. Those
applicants who passed this stage were asked to supply previous
employer experience affidavits, copies of all licenses or
certificates, and transcripts from all institutions attended
(Scigliano, 1979).
According to Huegli and Eich (1979), the most crucial
decisions reached in the search process were those which
rested on the results of on-site interviews with candidates.
Tremendous weight has traditionally been placed on the results
of the interview. The candidate pool has usually been pared
down to only a handful of applicants and great pressure was
on to select one of them or begin the whole process over
again. After this final screening, the recommendation was
made by the committee to the chief administrative officer and
board (Huegli & Eich, 1979).
Thus, according to the findings of the present study, the
search process mainly involves the selection methods of

158
biographical information, peer ratings, and personal
interviews. These three methods showed medium effect sizes
after meta-analysis, meaning that they can be valid predictors
of administrative achievement.
Two selection methods, overall assessment rating and job-
related skills indicators, were ranked high in effect size
ratings, but are seldom used in selection for administration
in higher education. Only one use of an assessment center
process for higher education was identified by this researcher
(Brubaker, 1983) It was not used for selection, but for
career development.
Three additional selection methods, psychological
attributes indicators, aptitude and intelligence measures, and
value systems assessements, also showed medium effect sizes,
but the use of these methods in a public institution may lead
to legal problems if the hiring institution cannot prove that
such tests are specifically job related.
One method, self-appraisal, showed a low effect size.
This was the only one of the nine selection methods that was
proven to be invalid in the prediction of administrative
achievement.
Legal Constraints Upon Personnel Selection
In selecting administrators in public community colleges,
the persons responsible for the search must comply with legal
constraints which affect their ability to apply conclusions
drawn directly from the research. Although personal

159
interviews probably will continue to be the most important
part of the process (Webb, 1983), testing has often been the
most controversial part. From Griggs v. Duke Power Co., 1971
(in Wisner, 1975) several rules for testing in personnel
selection were derived. Tests must be used only as
supplements to other selection methods and all tests must be
validated within the organization in order to be considered
job-related (Webb, 1983).
The case of Edward L. Kirkland et al.. Plaintiffs V. New
York State Department of Correctional Services et al..
Defendants (73 LIV. 1548, 1974) strengthened the requirement
for job-relatedness of selection tests. The court mandated
that every measure available be used to assure that tests
would not discriminate against any group of persons. Through
its institution, in 1972, of the Higher Education Guidelines,
the federal government called for an extension to recruitment
and hiring activities beyond the generally perceived network
approach. Dingerson, Rodman, and Burns (1985) sought to
assess what changes have occurred in hiring procedures since
implementation of the Higher Education Guidelines. They found
that there has been no substantial change in the hiring of
candidates from underrepresented populations for nationally
advertised positions. There is simply no evidence that change
is underway (Dingerson, Rodman, & Burns, 1985).
Sunshine laws are another source of constraint in hiring
in public community colleges. Some search committees are

160
required by state law to conduct all search business in
public. The names, resumes, and letters of recommendation of
all candidates must be available to the press and general
public and all committee votes are a matter of public record
(McLaughlin, 1985). Thus, the problem of the reliability of
the data provided within some selection methods may be
suspect, even if that method itself has been proven a valid
predictor of administrative achievement.
Guidelines for Selecting Community College Administrators
In deriving guidelines from the research, it must first
be determined whether such guidelines can be generalized to
aid in the selection of community college administrators.
Hammons and Ivery (1987) found statistically significant
differences between the tasks of five presidents of community
colleges and five chief executive officers of corporations.
Bare (1986) identified two unlike "cores" in college and
university administration. He found that work in the
administrative core was organized in a mechanistic manner with
an emphasis on systematic approaches to problem solving.
Bare found that a more organic form of work organization
prevailed within the technical core. Departmental
chairpersons employed the more consultative leadership style
appropriate to academic decision-making. He stated that
researchers must be aware that the core, rather than the
institution, may be the most fruitful level for theoretical
analysis. Thus, it may be the case that the personnel

161
selection guidelines from the research can be applied with
differing validity within different levels and cores of
community college administration.
From the conclusions of the research conducted in the
present study, it was found that eight of the personnel
selection methods considered are valid predictors of
administrative achievement. These methods are the overall
assessment rating derived from assessment center processes,
job-related skills indicators, value systems assessments,
personal interviews, aptitude and intelligence measures, peer
ratings, biographical information, and psychological attribute
indicators.
This meta-analysis has provided an objective view of the
accumulated findings of the 52 research studies involved. The
following are specific guidelines derived from the meta
analysis of the research. Taken into consideration were the
legal challenges presented.
1. Set up an assessment center for the identification of
persons who might be selected as community college
administrators. The assessment center overall assessment
rating showed the highest validity of all the selection
methods. To offset the costs involved, use the center also
for management training. Expected outcomes of the University
of California assessment center method are the improved
performance by participants in their present jobs and improved

162
success in competing for entry-level and mid-level management
positions in the university system (Brubaker, 1983) .
2. Implement a statistical combination of the data derived
from other selection processes. Multiple correlations of
scores derived from several selection methods increased the
validity of the methods to predict administrative achievement
(Howard, 1974). The results of this research have shown that,
where selection methods were used independently of any
assessment center, they were as valid as the same methods used
within an assessment center. Thus, the assessment center
process does not enhance the validity of the specific
selection methods, but the computational synthesis of the
combined data does enhance validity.
3. Write a detailed, task-oriented job description for any
administrative position to be filled. This process is
necessary (a) to provide selectors with specific skills,
traits, and capabilities to be sought within all selection
processes used, and (b) to satisfy the legal constraints on
job-relatedness of personnel selection procedures. Job-
related skills indicators ranked second in overall validity,
also having a confidence range above the significance level.
4. Use job-related skills indicators wherever possible
within the selection process. As indicated by the results of
this study, a job-related selection method can only add to the
validity of the overall selection process. The process of
developing job-related skill indicators assures that the legal

163
aspects of that portion of the job search have been met
(Huegli & Eich, 1979) .
5. Do not use value systems assessments in the selection
process for administrators in public community colleges. This
selection method may not be viewed by the legal system as
conforming to the Higher Education Guidelines for job-related
validity (Wisner, 1975). However, value system assessments
ranked third in validity, showing a small confidence interval
well above the significance level. Within the course of the
research for this study, the researcher noticed specific value
systems tests which showed high validity in predicting
administrative achievement.
6. Continue to use personal interviews in the selection
process. The personal interview almost always enhances the
validity of the personnel selection process. However, more
detailed reporting of research is needed to determine which
specific aspects of the interview method enhance its ability
to validly predict future administrative achievement.
Personal interviews ranked fourth in overall predictive
validity. This finding is encouraging since the personal
interview is the second most used method in the selection of
administrative personnel.
It has been suggested that structured interviews and
interviews involving more than one rater show higher
predictive validities. The paucity of detailed information
concerning these aspects of the research reviewed for this

164
study precluded a meta-analysis for the effects of structured
versus nonstructured, and singly-rated versus multiply-rated,
moderator variables. Thus, research within this selection
method is needed to show selectors ways to boost the validity
of this often-used method.
7. Use aptitude and intelligence measures to raise the
caliber of people being admitted to educational administration
programs in colleges and universities, and thus, eventually
improve the labor pool from which administrators are drawn
(Farquhar & Piele, 1972). In this study it was found that
aptitude and intelligence measures ranked fifth from the
highest in predictive validity of selection methods. The
confidence interval was almost wholly above the significance
level. Thus, in this study, it was shown that aptitude and
intelligence measures are valid predictors of future
administrative achievement. The problem in selection of
educational administrators is more a result of admitting weak
applicants into the selection process than of lacking strong
candidates. Legality may be an issue in the requirement of
an aptitude test as a selection process in a public community
college.
8. Suspect the validity of letters of recommendation. The
actual validity of peer ratings used in real life situations
may well be less than is shown by this meta-analysis. Peer
ratings ranked below the median, but still showed significant
validity in predicting administrative achievement. The

165
subjective reviews of the research on peer assessments
(Haggerty, 1963; Hollander, 1965; Roadman, 1964) lead one to
believe that the validity of this method might be higher than
it actually is.
However, attention must be given to the willingness of
participants to rate each other accurately when they know full
use will be made of such ratings for administrative purposes
(Roadman, 1964) In the 1980s, many persons may not give
truthful opinions of another person's abilities, or lack
thereof, for legal reasons. This tendency may not be evident
in studies conducted for research purposes only.
9. Conduct detailed research within the selection method
of biographical information. This most used method ranked
seventh in predictive validity. The findings from detailed
study of this method, with a search for those moderator
variables which add to predictive validity, may aid in
improving the use of this method.
10. Avoid using psychological attribute indicators as a
selection method for the reasons that (a) although
significant, it has marginal value as a predictor of
administrative achievement, and (b) there may be legal
problems with testing for motivations as opposed to job-
related skills.
The measurement of psychological attributes as a
predictor of administrative achievement has received by far
the most attention in the research. Little progress has been

166
made in applying knowledge toward improving the predictive
validity of this method. The mean correlation of this method
fell near the significance level. Appleby and Nunnery (1980)
found that various psychological attributes did not account
for differences in managerial effectiveness among community
college department heads. Although several flaws were
acknowledged in the study, it was representative of the
lackluster contribution of this selection method in predicting
achievement. Perhaps reconsideration is called for in
researchers' avid pursuit of a selection method that has
persistently failed to enhance selection validity.
Recommendations for Research
More research should be conducted on personnel selection
within education in general and in community colleges in
particular. An institutional research department or
organization dealing with community colleges may be warranted.
Only 3% of the research reviewed for this study was conducted
using educational administrators as subjects. However, 46%
of the researchers involved with the 52 research studies
worked for universities at the time of their research. As
Richardson (1987) stated, "university professors are sometimes
criticized for studying everything but their own university
settings" (p. 39).
The understaffed and inexperienced colleges of the
sixties have been replaced by mature institutions
with their own research and support staffs. The
cutting-edge community college of this decade
possesses more resources and more expertise in
virtually every line or administrative activity than

167
all of the university programs combined.
(Richardson, 1987, p. 39)
Hammons and Ivery (1987) found significant differences
between the tasks of corporation executives and community
college leaders. Thus, it is possible that the results of
administrative personnel selection research, which has been
done predominately within corporations, cannot be generalized
to the field of education. If this is the case, such research
is needed within the field of higher education.
The results of this study show that concurrent validity
studies may serve as well as predictive studies in determining
the validity of administrative personnel selections methods.
Thus, the information which could be obtained from concurrent
validity studies conducted within public community colleges
may add to our knowledge of the validities of administrative
personnel selection in the near term.
Three methodological practices are suggested as a result
of this study: (a) use a combination of two or more outcome
measures to add to validity in measuring administrative
achievement, (b) plan for the length of time allowed to elapse
between collection of the independent and dependent variables
to be at least three years but not greater than ten years, and
(c) report details of research conducted so that researchers
can determine what moderator variables within each method
contribute to validity.

168
Perhaps most important of all, community colleges
and university graduate programs can work together
to promote high-quality research. Such research,
to be worthy of doctoral recognition, should conform
to rigorous standards, contribute to our knowledge
base on community colleges, and have an impact on
professional practice. (Richardson, 1987, p. 41)
Summary and Conclusion
The two administrative personnel selection methods which
showed high predictive validity were the overall assessment
ratings of the assessment center process and job-related
skills indicators. The first of these methods is global in
aiding in the selection of personnel who can be expected to
achieve in various environments. The second method acquires
its validity from its specificity to a particular set of job
skills. Guin (1987) stated that
if your purpose is to hire generally good people,
a global criterion is useful. If you aren't very
sure what distinguishes good from better employees,
mix up a little of every kind of job behavior in
developing a truly global criterion. However, if
you need to solve a very specific problem, then a
more specific criterion is needed. If there is more
than one specific problem, then more than one
specific criterion is called for. But in most
situations, a global measure will serve quite well.
(p. 205)
The personnel selection methods which are currently used
for the selection of administrators in public community
colleges showed moderate predictive validity in this study.
Since these methods likely will continue to be used,
suggestions for improvement in their predictive validities
have been suggested.

169
This objective view of the research, applied to the
selection of administrative personnel in public community
colleges, has led to the following conclusions. Use
assessment centers in higher education for the purposes of
both personnel selection and career development. Use job-
related skills indicators to aid in defining the position to
be filled, to enhance predictive validity in selection, and
to avoid legal problems. Avoid any testing that is not job-
related, such as value system assessments and psychological
attribute indicators.
Use aptitude and intelligence measures to improve the
labor pool feeding educational administrative positions. Look
within the selection methods of personal interviews, peer
ratings, and biographical information for the moderator
variables which increase the predictive validities of these
methods. Use computational methods to combine selection
information of a job candidate in order to simulate the
validity of an assessment center overall assessment rating.

APPENDIX
ABSTRACTS OF SELECTED STUDIES
1. Bentz, V. J. (1967). Sears, Roebuck studies. In J. P.
Campbell, M. D. Dunnette, E. E. Lawler, & K. E. Weick, Jr.,
Measuring executive effectiveness (pp. 184-187). New York:
Appleton-Century-Crofts.
Purpose: To develop a psychological testing program for the
prediction of executive effectiveness in the Sears
organization.
Type organization: Corporation.
Subjects: 1,375 managers.
Type validity: Predictive.
Years between variables: 11 through 17.
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: Assumed yes.
Results:
a. Predictor: Aptitude and intelligence measures.
Criterion:
Promotion rate.
Correlation
coefficient (r):
0.21
Predictor:
Value systems assessments.
Criterion:
Promotion rate.
Correlation
coefficient (r):
0.22
Predictor:
Psychological attribute indicators.
Criterion:
Promotion rate.
Correlation
coefficient (r):
0.20
170

171
2. Bray, D. W., & Grant, D. L. (1966). The assessment
center in the measurement of potential for business
management. Psychological Monographs. 80(17, Whole No.
625) .
Purpose; To measure personal characteristics hypothesized
to be of importance either in developmental change in early
adulthood or success in business management.
Type organization; Corporation.
Subjects; 422 male, low-level managers.
Type validity; Predictive.
Years between variables; 8
Assessment center?; Yes.
Range restriction; Medium.
Criterion contamination?; No.
Results;
a. Predictor: Job-related skills indicators.
Criterion: Salary level attained.
Correlation coefficient (r^: 0.26 (N=269)
b. Predictor: Aptitude and intelligence measures.
Criterion: Salary level attained.
Correlation coefficient (r): 0.27 (N=269)
c. Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.16 (N=269)
d. Predictor: OAR.
Criterion: Salary level attained.
Correlation coefficient frl: 0.47 (N=203)

172
e. Predictor: OAR.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.57 (N= 422)
3. Campbell, J. P., Dunnette, M. D., Lawler, E. E., &
Weick, K. E. (1970). Managerial behavior, performance, and
effectiveness. New York: McGraw Hill.
Purpose: To determine how employees who possess the
potential to be successful in management can be identified
early in their careers.
Type organization: Corporation.
Subjects: 443 mid- to high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A.
Results:
a* Predictor: Aptitude and intelligence measures.
Criterion: Combination.
Correlation coefficient (r): 0.16
b. Predictor: Psychological attribute indicators.
Criterion: Combination.
Correlation coefficient (r) : 0.08
c. Predictor: Self-appraisals.
Criterion: Combination.
Correlation coefficient (r^:
0.24

173
d.
Predictor:
Job-related skills
indicators
Criterion:
Combination.
Correlation
coefficient (r):
0.49
e.
Predictor:
Personal interviews.
Criterion:
Combination.
Correlation
coefficient (r):
0.20
f.
Predictor:
Biographical information.
Criterion:
Combination.
Correlation
coefficient (r):
0.57
g-
Predictor:
OAR
Criterion:
Combination.
Correlation
coefficient (r):
0.30
4. Carleton, F. 0. (1970). Relationships between follow-up
evaluations and information developed in a management
assessment center. Proceedings of the 78th Annual
Convention of the American Psychological Association, pp.
565-568.
Purpose: To determine if test and rating data generated in
a multiple-technique assessment center hold up well as
predictors of behavioral ratings made several years later.
Type organization: Corporation.
Subjects: 122 managers.
Type validity: Predictive.
Years between variables: 2.5 to 5
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: Yes.
Results:
a* Predictor: Personal interviews.

Criterion: Supervisor ratings.
Correlation coefficient (r): 0.23
174
b. Predictor (selection method): OAR
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.65
5. Cummin, P. C. (1967). TAT correlates of executive
performance. Journal of Applied Psychology. 51. 78-81.
Purpose: To determine if successful executives rate high in
the psychological attributes of achievement, power, and
autonomy, and less successful executives rate high in
affiliation, aggression, and deference.
Type organization: Corporation.
Subjects: 52 mid- and high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A.
Results:
Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.21
6. Dicken, C., & Black, J. (1965). Predictive validity of
psychometric evaluation of supervisors. Journal of Applied
Psychology. 49. 34-47.
Purpose: To study the validity of clinical interpretations
of an objective test battery in two different industrial
settings.
Type organization: Corporation.
Subjects: 2 samples: (a) 31 male, low-level managers, (b)
26 male, low-level managers.

Type validity: Predictive.
Years between variables; (a) 3.5, (b) 7
Assessment center?; No.
Range restriction: Medium.
Criterion contamination?; Yes.
Results:
a. Predictor: Aptitude and intelligence measures
Criterion: Salary level attained.
Correlation coefficient (r): 0.41 (N=31)
b. Predictor: Psychological attribute indicators
Criterion: Salary level attained.
Correlation coefficient (r): 0.15 (N=31)
c. Predictor: Job-related skills indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.35 (N=31)
d. Predictor: Aptitude and intelligence measures
Criterion: Promotion rate.
Correlation coefficient fr): 0.21 (N=31)
e. Predictor: Psychological attribute indicators
Criterion: Promotion rate.
Correlation coefficient m: 0.11 (N=31)
f- Predictor: Job-related skills indicators.
Criterion: Promotion rate.
Correlation coefficient (r): 0.11 (N=31)
<3 Predictor: Aptitude and intelligence measures
Criterion: Salary level attained.

176
Correlation coefficient (r): 0.29 (N=26)
h. Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.15 (N=26)
i. Predictor: Aptitude and intelligence measures.
Criterion: Promotion rate.
Correlation coefficient (r): 0.18 (N=26)
j. Predictor: Psychological attribute indicators.
Criterion: Promotion rate.
Correlation coefficient (r): 0.20 (N=26)
7. Edel, E. G. (1968). "Need for success" as a predictor
of managerial performance. Personnel Psychology. 21. 231-
240.
Purpose: To determine the concurrent validity of a conative
measure which purports to assess "need for success" by
measuring perceptual distortion to "loaded" stimuli in an
indirect manner (Edel, 1968, p. 232).
Type organization: Corporation.
Subjects: 232 first-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.43

177
8. England, G. W., & Lee, R. (1974). The relationship
between managerial values and managerial success in the
United States, Japan, India, and Australia. Journal of
Applied Psychology. 39., 411-419.
Purpose: To investigate the relationship between values of
managers and their success as managers in four countries.
Type organization: Corporation.
Subjects: 878 high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Value systems assessments.
Criterion: Salary level attained.
Correlation coefficient (r): 0.32
9. Finley, R. M., Jr. (1970). Predictions from projective
tests given in a management assessment center. In A. Howard
(1974). An assessment of assessment centers, Academy of
Management Journal. 17, 123.
Purpose: To study the predictive validity of projective
tests in a management assessment center.
Type organization: Corporation.
Subjects: 109 subjects.
Type validity: Predictive.
Years between variables: .75 through 5
Assessment center?: Yes.
Range restriction: Assumed medium.
Criterion contamination?:
Yes.

178
Results:
Predictor: OAR
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.33
10. Fleishman, E. A., & Peters, D. R. (1962).
Interpersonal values, leadership attitudes, and managerial
"success". Personnel Psychology. 15, 127-143.
Purpose: To study the interrelationships among criteria of
leadership effectiveness, value dimensions, and leader
behavior and attitudes.
Type organization: Corporation.
Subjects: 25 Managers
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.19
11. Gantz, B. S., Erickson, C. 0., & Stephenson, R. W.
Some determinants of promotion in a research and development
population. In J. B. Miner, & M. G. Miner (1977).
Motivation to manage: A ten year update on the "studies in
management education" research (pp. 18-22). Atlanta:
Organizational Measurement Systems Press.
Purpose: To gain more theoretical understanding of why some
persons in a research and development population achieve
promotion into managerial positions and others do not.
Type organization: Corporation.
Subjects: 117 engineers and research scientists.

Type validity: Concurrent.
Years between variables: 0
Assessment center?; No.
Range restriction: High.
Criterion contamination?: N/A
Results;
a.
Predictor:
Aptitude and intelligence measures
Criterion:
Promotion rate.
Correlation
coefficient ir): 0.06
b.
Predictor:
Aptitude and intelligence measures
Criterion:
Supervisor ratings.
Correlation
coefficient (r): 0.07
c.
Predictor:
Psychological attribute indicators
Criterion:
Promotion rate.
Correlation
coefficient (r): 0.07
d.
Predictor:
Psychological attribute indicators
Criterion:
Supervisor ratings.
Correlation
coefficient fri: 0.05
e.
Predictor:
Biographical information.
Criterion:
Promotion rate.
Correlation
coefficient (r): 0.17
f.
Predictor:
Biographical information.
Criterion:
Supervisor ratings.
Correlation
coefficient (r): 0.15
g-
Predictor:
Peer ratings.
Criterion:
Promotion rate.

180
Correlation coefficient (r): 0.26.
12. Gantz, B. S., Erickson, C. O., & Stephenson, R. W. The
prediction of managerial and research success. In J. B.
Miner, & M. G. Miner (1977). Motivation to manage: A ten
year update on the "studies in management education"
research (pp. 28-34). Atlanta: Organizational Measurement
Systems Press.
Purpose: To determine whether motives measured by a
psychological attribute indicators are the cause of
managerial success.
Type organization: Corporation.
Subjects: 49 male research scientists.
Type validity: Predictive.
Years between variables: 5.3
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: Assumed yes.
Results:
Predictor: Psychological attribute indicators.
Criterion: Promotion rate.
Correlation coefficient (r): 0.25
13. Gantz, B. S., Erickson, C. 0., & Stephenson, R. W. The
marketing department follow-up study. In J. B. Miner, & M.
G. Miner (1977). Motivation to manage: A ten year update
on the "studies in management education11 research (pp. 28-
34). Atlanta: Organizational Measurement Systems Press.
Purpose: To determine whether motives measured by a
psychological attribute indicators are the cause of
managerial success.
Type organization: Corporation.
Subjects: 81 top salesmen and marketing managers.
Type validity: Predictive.

181
Years between variables: 4
Assessment center?: No.
Range restriction: Medium.
Criterion contamination?; No.
Results:
Predictor; Psychological attribute indicators.
Criterion; Promotion rate.
Correlation coefficient (r); 0.39
14. Ghiselli, E. E. (1966). The validity of a personnel
interview. Personnel Psychology. 19. 389-394.
Purpose; To examine the proposition that superficial data
pertaining to what an individual has done has little power
in predicting job "success".
Type organization: Corporation.
Subjects: 275 applicants for low-level executive positions.
Type validity; Predictive.
Years between variables: 3
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
Predictor: Personal interviews.
Criterion: Number of years serving in an
administrative position within the same
organization.
Correlation coefficient (r): 0.35
15. Ghiselli, E. E. (1968). Some motivational factors in
the success of managers. Personnel Psychology. 21. 431-440.

182
Purpose: To study the relationship between the strength of
desire for self-actualization and performance among
managers.
Type organization: Corporation.
Subjects: 233 mid-level male managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient fr): 0.22 (N=89)
b. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient fr): 0.30 (N=21)
c. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r^: 0.47 (N=20)
d. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.22 (N=22)
e. Predictor: Psychological attribute indicators.
Criterion: Number of years serving in an
administrative position within the same
organization.
Correlation coefficient (r): 0.19 (N=81)

183
16. Goodstein, L. D., & Schrader, W. J. (1963). An
empirically-derived managerial key for the California
Psychological Inventory. Journal of Applied Psychology. 47
42-45.
Purpose: To determine the usefulness of a specific
indicator of psychological attributes in identifying those
personality characteristics associated with managerial and
supervisory success in a large industrial organization.
Type organization: Military.
Subjects: 603 mid- and top-level male, civilian managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.23
17. Grant, D. L., & Bray, D. W. (1969). Contributions of
the interview to assessment of managerial potential.
Journal of Applied Psychology. 53, 24-34.
Purpose: To present information on the contributions made
by the interview to the assessment center process and to
study the relationships of interview variables to the
progress criterion.
Type organization: Corporation.
Subjects: 203 low-level male managers.
Type validity: Predictive.
Years between variables: 9
Assessment center?: Yes.
Range restriction: Medium.

184
Criterion contamination?; No.
Results:
Predictor: Personal interviews.
Criterion: Salary level attained.
Correlation coefficient (r): 0.23
18. Grant, D. L., Katkovsky, W., & Bray, D. W. (1967).
Contributions of projective techniques to assessment of
management potential. Journal of Applied Psychology. 51,
226-232.
Purpose: To present data from the Bell System Management
Progress Study (Bray, 1964) on the relationships between
projective techniques and managerial achievement.
Type organization: Corporation.
Subjects: 201 male low-level managers.
Type validity: Predictive.
Years between variables: 8
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: No.
Results:
Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.18
19. Grimsley, G., & Jarrett, G. F. (1975). The relation
of past managerial achievement to test measures obtained in
the employment situation: Methodology and results-II.
Personnel Psychology. 28., 215-231.
Purpose: To test the effectiveness of a particular
methodological approach which can be used in analyzing data
gathered in the process of assessing managerial applicants
in the employment situation.

185
Type organization: Corporation.
Subjects: 2 samples of 100 mid- and high-level managers
each.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a. Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.36
b. Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.17
c. Predictor: Value systems assessments.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.11
d. Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient fr): 0.35
e* Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r) : 0.17
f. Predictor: Value systems assessments.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.10

186
20. Haggerty, H. R. (1963). Status report on research for
the U.S. Military Academy (Cadet Leaders Task). Washington,
DC: U.S. Army Personnel Research Office.
Purpose: To study the relationships between certain
predictor variables and achievement of Army officers.
Type organization: Military.
Subjects: 2 samples of 78 and 420 U.S. Military Academy
Cadets.
Type validity: Predictive.
Years between variables: Unknown.
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a. Predictor: Peer ratings.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.53 (N=78)
b. Predictor: Peer ratings.
Criterion: Combination.
Correlation coefficient (r): 0.33 (N=420)
c. Predictor: Biographical information.
Criterion: Combination.
Correlation coefficient (r): 0.18 (N=78)
21. Harrell, T. W., & Harrell, M. S. (1974). Predictors
of management success (Technical Report No. 3). Arlington,
VA: Office of Naval Research.
Purpose: To determine predictors of management success.
Type organization: Corporation.
Subjects: 205 MBA graduates.

Type validity: Predictive.
Years between variables: 10
187
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a. Predictor; Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.19
b. Predictor: Biographical information.
Criterion: Salary level attained.
Correlation coefficient (r^: 0.28
22. Heneman, H. G. Ill (1974). Comparisons of self- and
superior ratings of managerial performance. Journal of
Applied Psychology. 59, 638-642.
Purpose: To identify broad dimensions of managerial
performance, obtain self- and superior performance ratings,
and assess these ratings in terms of leniency, restriction
of range, halo, and convergent-discriminant validity.
Type organization: Combination.
Subjects: 102 MBA graduates.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Self-appraisals.

Criterion: Supervisor ratings.
Correlation coefficient (r): 0.22
188
23. Hicks, J. A., & Stone, J. B. (1962). The
identification of traits related to managerial success.
Journal of Applied Psychology. 46. 428-432.
Purpose: To determine if a broad battery of tests covering
aptitudes, temperament, and creativity could be used to
identify certain basic characteristics for selection,
promotion, and training purposes.
Type organization: Corporation.
Subjects: 76 low-level supervisors.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a. Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.12
b. Predictor: Aptitude and intelligence measures.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.17
24. Hinrichs, J. R. (1978). An eight-year follow-up of a
management assessment center. Journal of Applied
Psychology. 63., 596-601.
Purpose: To evaluate the predictive validity of the
assessment center process and to compare the predictive
accuracy of the assessment center with the naturalistic
management evaluation.
Type organization: Corporation.

189
Subjects; 30 male nonmanagement and low-level management
personnel.
Type validity: Predictive.
Years between variables: 8
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: No.
Results:
a.
Predictor:
Value systems assessments.
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.26
b.
Predictor:
Self-appraisals.
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.26
c.
Predictor:
Aptitude and intelligence measures.
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.21
d.
Predictor:
OAR
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.46
e.
Predictor:
Biographical information.
Criterion:
Administrative level achieved.
Correlation
coefficient (r): 0.55
25. Hollander, E. P. (1965). Validity of peer nominations
in predicting a distant performance criterion. Journal of
Applied Psvcholoav. 49, 434-438.
Purpose: To determine the validity of peer ratings in
predicting future administrative achievement.

190
Type organization: Military.
Subjects: 639 Naval Officer Candidate School students.
Type validity: Predictive.
Years between variables: 3
Assessment center?: No.
Range restriction: Low
Criterion contamination?: Assumed yes.
Results:
Predictor: Peer ratings.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.35
26. Huck, J. R., & Bray, D. W. (1976). Management
assessment center evaluations and subsequent job performance
of white and black females. Personnel Psychology. 29., 13-
30.
Purpose: To test the validity of an assessment center
process on a population different from the population of the
Management Progress Study.
Type organization: Corporation.
Subjects: 126 nonmanagement women.
Type validity: Predictive.
Years between variables: 1.2 through 5.8
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
s* Predictor: Aptitude and intelligence measures.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.15

191
b. Predictor; Job-related skills indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.27
c. Predictor: OAR
Criterion; Supervisor ratings.
Correlation coefficient (r): 0.38
27. Kotula, L. J., & Haggerty, H. R. (1966). Research in
the selection of officer candidates and cadets. Washington,
DC: U.S. Army Personnel Research Office.
Purpose: To determine the predictive validity of the
personal history blank.
Type organization: Military.
Subjects: 426 Army officers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Biographical information.
Criterion: Supervisor ratings.
Correlation coefficient fr): 0.17
28. Kraut, A. I. (1969). Intellectual ability and
promotional success among high level managers. Personnel
Psychology. 22., 281-290.
Purpose: To study the relationship of high level managers'
promotional success to two tests of intellectual
functioning.
Type organization: Corporation.

192
Subjects: 365 mid- and high-level managers.
Type validity: Predictive.
Years between variables: 4 through 7
Assessment center?: No.
Range restriction: High.
Criterion contamination?: Yes.
Results:
Predictor: Aptitude and intelligence measures.
Criterion: Promotion rate.
Correlation coefficient fr): .02
29. Kraut, A. I., & Scott, G. J. (1972). Validity of an
operational management assessment program. Journal of
Applied Psychology. 56, 124-129.
Purpose: To study the predictive validity of an assessment
center.
Type organization: Corporation.
Subjects: 432 nonmanagement employees.
Type validity: Predictive.
Years between variables: 1 through 5
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
Predictor: OAR
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.25
30. Lawler, E. E. Ill (1967). The multitrait-multirater
approach to measuring managerial job performance. Journal
of Applied Psychology. 51, 369-381.

193
Purpose: To study the advantages of the multitrait-
multirater approach to measuring managerial job performance.
Type organization: Corporation.
Subjects: 113 mid- to high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a. Predictor: Self-appraisals.
Criterion: Supervisor ratings.
Correlation coefficient fr): Approximately 0.07
b. Predictor: Peer ratings.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.43
31. Mayfield, E. C. (1970). Management selection: Buddy
nominations revisited. Personnel Psychology. 23. 377-391.
Purpose: To investigate the value of the buddy nomination
procedure in the selection of assistant managers.
Type organization: Corporation.
Subjects: 154 real estate agents.
Type validity: Predictive.
Years between variables: 2.5
Assessment center?: No.
Range restriction: Low.
Criterion contamination?: No.

194
Results:
Predictor: Peer ratings.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.23
32. McClelland, D. C., & Boyatzis, R. E. (1982).
Leadership motive pattern and long-term success in
management. Journal of Applied Psychology. 67, 737-743.
Purpose: To determine whether specific personality
characteristics relate to success in management.
Type organization: Corporation.
Subjects: 237 male, entry-level managers.
Type validity: Predictive.
Years between variables: 16
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: No.
Results:
Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.33
33. Meyer, H. H. (1970). The validity of the in-basket
test as a measure of managerial performance. Personnel
Psychology. 23. 297-307.
Purpose: To determine to what degree performance on an in-
basket test actually does correspond to observed performance
of an actual managerial job.
Type organization: Corporation.
Subjects: 81 male, low-level managers.
Type validity: Concurrent.
Years between variables: 0

195
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Job-related skills indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.29
34. Miner, J. B., & Miner, M. G. (1977). Motivation to
manage: A ten year update on the "Studies in Management
Education" research. Atlanta: Organizational Measurement
Systems Press.
Purpose: To study the relationship between two
psychological attribute measures and indexes of management
success.
Type organization: Corporation.
Subjects: 101 high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a- Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.10
b. Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.13

196
35. Miner, J. B., & Miner, M. G. (1977). Motivation to
manage: A ten year update on the "Studies in Management
Education" research. Atlanta: Organizational Measurement
Systems Press.
Purpose: To study the relationship between psychological
attribute indicators and professional and administrative
success in educational administration.
Type organization: Educational.
Subjects: 49 university administrators and faculty members.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.10
36. Miner, J. B., & Miner, M. G. (1977). Motivation to
manage: A ten year update on the "Studies in Management
Education" research. Atlanta: Organizational Measurement
Systems Press.
Purpose: To determine the relationships between motivation
to manage and managerial success in educational
organizations.
Type organization: Educational.
Subjects: 219 mid-to high-level educational administrators.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.

Criterion contamination?; N/A
Results:
197
Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.05
37. Miner, J. B., & Miner, M. G. (1977). Motivation to
manage: A ten year update on the "Studies in Management
Education" research. Atlanta: Organizational Measurement
Systems Press.
Purpose: To test the hypothesis that sex differences in
motivation to manage do exist among practicing managers.
Type organization: Educational.
Subjects: 219 male and female mid- to high-level
educational administrators.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.18
38. Mitchel, J. O. (1975). Assessment center validity: A
longitudinal study. Journal of Applied Psychology. 60, 573-
579.
Purpose: To examine the predictive validity of an
assessmment center in a developmental paradigm, and to
observe changes in the validity coefficients resulting from
measuring a criterion of salary growth at different points
in time.

198
Type organization: Corporation.
Subjects: 254 low-level managers.
Type validity: Predictive.
Years between variables: 5
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: Yes.
Results:
a. Predictor: Peer ratings.
Criterion: Salary level attained.
Correlation coefficient (r): 0.23
b. Predictor: OAR
Criterion: Salary level attained.
Correlation coefficient (r): 0.14
39. Moses, J. L. (1972). Assessment center performance
and management progress. Studies in Personnel Psychology.
4(1), 7-12.
Purpose: To study the relationship between assessment
center processes and managerial achievement.
Type organization: Corporation.
Subjects: 8,885 nonmanagement males.
Type validity: Predictive.
Years between variables: 7
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a. Predictor: Personal interviews.

199
Criterion; Administrative level achieved.
Correlation coefficient (r): 0.27
b. Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.21
c. Predictor: Job-related skills indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.35
d. Predictor: OAR
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.44 (N=5f943)
40. Moses, J. L., & Boehm, V. R. (1975). Relationship of
assessment-center performance to management progress of
women. Journal of Applied Psychology. 60, 527-529.
Purpose: To study the relationship between assessment and
subsequent progress in management for women in the Bell
System.
Type organization: Corporation.
Subjects: 4,846 nonmanagement women.
Type validity: Predictive.
Years between variables: 2 through 10
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a* Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.25

200
b. Predictor: Job-related skills indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.28
c. Predictor: OAR
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.37
41. Nash, A. N. (1966). Development of an SVIB key for
selecting managers. Journal of Applied Psychology. 50, 250-
254 .
Purpose: To determine if the measured interests of a large
sample of managers were related to an internal criterion of
managerial effectiveness.
Type organization: Corporation.
Subjects: 230 managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.33
42. Prien, E. P., & Liske, R. E. (1962). Assessments of
higher-level personnel: III. Rating criteria: A
comparative analysis of supervisor ratings and incumbent
self-ratings of job performance. Personnel Psychology. 15.
187-194.
Purpose: To explore the relationship between supervisor
ratings of job performance and incumbent self-ratings of job
performance on tasks which are intangible in nature.

201
Type organization: Corporation.
Subjects: 96 supervisory position incumbents.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Self-appraisals.
Criterion: Supervisor ratings.
Correlation coefficient fr): 0.25
43. Rankin, K. K. (1981). A predictive validity study of
an assessment center for research and development superiors.
Wright-Patterson AFB, OH: Air Force Institute of Technology.
Purpose: To study the relationships between assessment
center ratings and several measures of administrative
achievement.
Type organization: Military.
Subjects: 67 civilian nonmanagement personnel.
Type validity; Predictive.
Years between variables: 6
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
Predictor: OAR
Criterion: Combination.

202
Correlation coefficient (r): 0.00
44. Roadman, H. E. (1964). An industrial use of peer
ratings. Journal of Applied Psychology. 48. 211-214.
Purpose: To determine if peer ratings can identify managers
who later are promoted rapidly in a large corporation.
Type organization: Corporation.
Subjects; 40 mid-level managers.
Type validity; Predictive.
Years between variables: 2
Assessment center?: No.
Range restriction: High.
Criterion contamination?: No.
Results:
Predictor: Peer ratings.
Criterion: Promotion rate.
Correlation coefficient fr): 0.31.
45. Schmitt, N., Noe, R. A., Meritt, R., & Fitzgerald, M.
P. (1984). Validity of assessment center ratings for the
prediction of performance ratings and school climate of
school administrators, Journal of Applied Psychology. 69,
pp. 207-213.
Purpose: Evaluates the use of the assessment center
approach applied in an educational setting to select
secondary and elementary school administrators.
Type organization: Education.
Subjects: 153 low-level administrators.
Type validity; Predictive.
Years between variables: 1
Assessment center?: Yes.
Range restriction: Low.

203
Criterion contamination?: Yes.
Results:
Predictor: OAR.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.24.
46. Stahl, M. J. (1983). Achievement, power, and
managerial motivation: Selecting managerial talent with the
job choice exercise. Personnel Psychology. 36, 775-789.
Purpose: To determine if high managerial motivation is
related to high needs for achievement and power.
Type organization: Corporation.
Subjects: 45 mid-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.36
47. Steel, R. P., & Ovalle, N. K. 2d. (1984). Self
appraisal based upon supervisory feedback. Personnel
Psychology. 37, 667-685.
Purpose: To examine the relationship between self
appraisals and objective criteria of job performance.
Type organization: Corporation.
Subjects: 401 mid-level managers.
Type validity: Concurrent.

204
Years between variables; 0
Assessment center?: Yes.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Self-appraisals.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.19
48. Tenopyr, M. L., & Ruch, W. W. (1965). In J. P.
Campbell, M. D. Dunnette, E. E. Lawler, & K. E. Weick
(1970). Managerial behavior, performance, and effectiveness
(p. 193). New York: McGraw Hill.
Purpose: To study the relationship between personnel
selection measures and measures of job achievement.
Type organization: Corporation.
Subjects: 113 mid-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a* Predictor: Aptitude and intelligence measures.
Criterion: Salary level attained.
Correlation coefficient (r): 0.29
b. Predictor: Job-related skills indicators.
Criterion: Salary level attained.
Correlation coefficient m: 0.36

205
49. Thornton, G. C. (1968). The relationship between
supervisory- and self-appraisals of executive performanace.
Personnel Psychology. 21, 441-455.
Purpose: To investigate the relationship between
supervisory-perceptions and incumbent self-perceptions of
the performance of executive personnel.
Type organization: Corporation.
Subjects: 64 high-level administrators.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Self-appraisals.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.24
50. Turnage, J. J., & Muchinsky, P. M. (1984). A
comparison of the predictive validity of assessment center
evaluations versus traditional measures in forecasting
supervisory job performance: Interpretive implications of
criterion distortion for the assessment paradigm. Journal
of Applied Psychology. 69. 595-602.
Purpose: To examine the ability of assessment center
evaluations to predict actual job perfomance criteria and to
compare the predictability of assessment center evaluations
versus traditional measures in forecasting job success.
Type organization: Corporation.
Subjects: 799 mostly nonmanagement employees.
Type validity: Predictive.
Years between variables: 1 month to several years.
Assessment center?: Yes.

206
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a.
Predictor:
Self-appraisals.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.01
b.
Predictor:
Psychological attribute indicators.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.01
c.
Predictor:
Biographical information.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.09
d.
Predictor:
Personal interviews.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.03
e.
Predictor:
Job-related skills indicators.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.02
f.
Predictor:
Peer ratings.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.08
g.
Predictor:
OAR
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.03
h. Predictor: Self-appraisals.
Criterion: Salary level attained.

207
Correlation coefficient (r): -0.01
i.
Predictor:
Psychological attribute indicators.
Criterion:
Salary level attained.
Correlation
coefficient (r): -0.02
j
Predictor:
Biographical information.
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.05
k.
Predictor:
Personal interviews.
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.10
1.
Predictor:
Job-related skills indicators.
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.07
m.
Predictor:
Peer ratings.
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.04
n.
Predictor:
OAR
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.12
51. Williams, F. J., & Harrell, T. W. (1964). Predicting
success in business. Journal of Applied Psychology. 48.
164-167.
Purpose: To identify those predictors which are
significantly correlated with on-the-job achievement.
Type organization: Corporation.
Subjects: 196 MBA graduates.
Type validity: Predictive.
Years between variables: 15
Assessment center?: No.

Range restriction: Low.
Criterion contamination?:
208
Yes.
Results:
a. Predictor: Aptitude and intelligence measures.
Criterion: Salary level attained.
Correlation coefficient (r): -0.01
b. Predictor: Aptitude and intelligence measures.
Criterion: Adminstrative level achieved.
Correlation coefficient fr): -0.01 (N=116)
c. Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.08
d. Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.09 (N=116)
e. Predictor: Biographical information.
Criterion: Salary level attained.
Correlation coefficient (r): 0.17
f. Predictor: Biographical information.
Criterion: Administrative level achieved.
Correlation coefficient fr): 0.11 (N=116)
52. Wollowick, H. B., & McNamara, W. J. (1969).
Relationship of the components of an assessment center to
management success. Journal of Applied Psychology. 53. 348-
352.
Purpose: To determine the validity of an assessment center
approach in predicting management potential and to determine
the relative value of the components of the program.

209
Type organization: Corporation.
Subjects: 94 low- and mid-level management males.
Type validity: Predictive.
Years between variables: 2.5 through 3.2
Assessment center?: Yes.
Range restriction: High.
Criterion contamination?: Yes.
Results:
a. Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.09
b. Predictor: Job-related skills indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.20
c. Predictor: OAR
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.37

References
Akman, A. & Nordhauser, F. (1974). A conceptual view of the
officer procurement model (TOPOPS) (Report No.
AFHRL-TR-73-73). Silver Spring, MD: System Automation.
(ERIC Document Reproduction Service No. ED 105 174)
Alexander, S. K. (1980). School law. St. Paul: West.
Appleby, S. A. & Nunnery, M. Y. (1980). Power motivation,
leadership style, and managerial effectiveness of
community college department heads. Community/Junior
College Research Quarterly. 5, 1-10.
Arvey, R. D., & Campion, J. E. (1982). The employment
interview: A summary and review of recent research.
Personnel Psychology. 35, 281-322.
Arvey, R. D., Miller, H. E., Gould, R., & Burch, P. (1987).
Interview validity for selecting sales clerks. Personnel
Psychology. 40. 1-12.
Ary, D., Jacobs, L. C., & Razavieh, A. (1979). Introduction
to research in education. New York: Holt, Rinehart and
Winston.
Bare, A. C. (1986). Managerial behavior of college
chairpersons and administrators. Research in Higher
Education. 24.(2), 128-138.
Bentz, V. J. (1967). Sears, Roebuck studies. In J. P.
Campbell, B. D. Dunnette, E. E. Lawler, & K. E. Weick,
Jr., Measuring executive effectiveness (pp. 184-187).
New York: Appleton-Century-Crofts.
Bernstein, A. (1988). Community colleges: Coming of age.
Change. 2^(1) 4.
Bray, D. W. (1964). The management progress study, American
Psychologist. 19. 419-420.
Bray, D. W., & Grant, D. L. (1966). The assessment center
in the measurement of potential for business management.
Psychological Monographs. 80(17), (Whole No. 625).
210

211
Bray, D. W. & Moses, J. L. (1972). Personnel selection. In
P. H. Mussen & M. R. Rosenzweig (Eds.), Annual Review of
Psychology. 23. Palo Alto, CA: Annual Reviews, 545-576.
Brubaker, L. (1983). The assessment center method:
Developing employees for mid-management positions. The
Journal of College and University Personnel. 34.(3) ,
32-36.
Burrington, D. D. (1982). A review of state government
employment application forms for suspect inquiries.
Public Personnel Management Journal. 11. 55-60.
Campbell, J. P., Dunnette, M.D., Lawler, E. E., Ill, &
Weick, K. E., Jr. (1970). Managerial behavior,
performance, and effectiveness. New York: McGraw-Hill.
Carleton, F. 0. (1970). Relationships between follow-up
evaluations and information developed in a management
assessment center. Proceedings of the 78th Annual
Convention of the American Psychological Association. 2,
565-566.
Childs, A., & Klimoski, R. J. (1986). Successfully
predicting career success: An application of the
biographical inventory. Journal of Applied Psychology.
71, 3-8.
Comrey, A., & High, W. (1955). Validity of some ability and
interest scores. Journal of Applied Psychology. 39.
247-248.
Cohen, J. (1977). Statistical power analysis for the
behavioral sciences. New York: Academic Press.
Crooks, L. A. (1973). The selection and development of
performance measures for assessment center programs.
Paper presented at the First Annual Industrial and
Organizational Psychology Conference, Columbus, OH.
Cummin, P. C. (1967). TAT correlates of executive
performance. Journal of Applied Psychology. 51, 78-81.
Dicken, C., & Black, J. (1965). Predictive validity of
psychometric evaluation of supervisors. Journal of
Applied Psychology. 49. 34-47.
Dingerson, V. R., Rodman, J. A., & Burns, D. (1985). The
hiring of underrepresented individuals in academic
administrative positions: 1972-1979. Research in Higher
Education. 23.(2), 115-133.

212
Dunnette, M. D. (1963). A modified model for test
validation and selection research. Journal of Applied
Psychology. 47., 317-323.
Edel, E. G. (1968). "Need for success" as a predictor of
managerial performance. Personnel Psychology. 21.
231-240.
England, G. W., & Lee, R. (1974). The relationship between
managerial values and managerial success in the United
States, Japan, India, and Australia. Journal of Applied
Psychology. 39. 411-419.
Farquahar, R. H., & Piele, P. K. (1972). Preparing
educational leaders: A review of recent literature,
ERIC/CEM/State-of-the Knowledge Series, No. 14. Fourteen.
UCEA Monograph Series. No 1. Columbus, Ohio: University
Council for Educational Administration.
Finley, R. M., Jr. (1970). Evaluation of behavior
predictions from projective tests given in a management
assessment center. Proceedings of the 78th Annual
Convention of the American Psychological Association. 2,
567-568.
Fleishman, E. A., & Peters, D. R. (1962). Interpersonal
values, leadership attitudes, and managerial "success".
Personnel Psychology. 15, 127-143.
Fortunato, R. T., & Waddell, D. G. (1981). Personnel
administration in higher education. San Francisco:
Jossey-Bass.
Gantz, B. S., Erickson, C. O., & Stephenson, R. W. (1977).
Some determinants of promotion in a research and
development population. In J. B. Miner & M. G. Miner,
Motivation to manage: A ten year update on the "studies
in management education" research (pp. 18-22). Atlanta:
Organizational Measurement Systems Press.
Ghiselli, E. E. (1955). The measurement of occupational
aptitude. Berkley: University of California.
Ghiselli, E. E. (1966). The validity of a personnel
interview. Personnel Psychology. 19. 389-394.
Ghiselli, E. E. (1968). Some motivational factors in the
success of managers. Personnel Psychology. 21, 431-440.

213
Ghiselli, E. E. (1973). The validity of aptitude tests in
personnel selection, Personnel Psychology. 26. 461-477.
Glass, G. (1976). Primary, secondary, and meta-analysis of
research. Educational Researcher. 5, 3-8.
Glass, G. (1977). Integrating findings: The meta-analysis
of research. Review of Research in Education. 5,
351-379.
Glass, G. McGaw, B., & Smith, M. L. (1981). Meta-analvsis
in social research. Beverly Hills: Sage.
Goodstein, L. D., & Schrader, W. J. (1963). An empirically-
derived managerial key for the California Psychological
Inventory. Journal of Applied Psychology. 47. 42-45.
Grant, D. L., & Bray, D. W. (1969). Contributions of the
interview to assessment of management potential. Journal
of Applied Psychology. 53. 24-34.
Grant, D. L., Katkovsky, W., & Bray, D. W. (1967).
Contributions of projective techniques to assessment of
management potential. Journal of Applied Psychology. 51,
226-232.
Grimsley, G., & Jarrett, H. F. (1975). The relation of past
managerial achievement to test measures obtained in the
employment situation: Methodology and Results II.
Personnel Psychology. 28., 215-231.
Grove, D. A. (1981). A behavioral consistency approach to
decision making in employment selection. Personnel
Psychology. 34. 55-64.
Guin, R. M. (1987). Changing views for personnel selection
research. Personnel Psychology, 40, 199-213.
Haggerty, H. R. (1953). Personnel research for the U.S.
Military Academy 1942-1953. U.S. Army Personnel Research
Board Technical Research Report 1077. Wash., D.C.: U.S.
Army Personnel Research Office.
Haggerty, H. R. (1963). Status report on research for the
U.S. Military Academy (Cadet Leaders Task). Wash., D.C.:
U.S. Army Personnel Research Office
Halpin, A. W. (1954). The leadership behavior and combat
performance of airplane commanders. Journal of Abnormal
and Social Psychology. 49. 19-22.

214
Hammons, J., & Ivery, C. (1987). Does a difference make a
difference?: A comparison of the activities of community
college presidents with those of chief executive officers
in other settings. Community College Review. 15(4),
18-27.
Handyside, J., & Duncan, D. C. (1954). Four years later: A
follow-up of an experiment in selecting supervisors.
Occupational Psychology, 28. 9-23.
Harrell, T. W., & Harrell, M. S. (1974). Predictors of
management success. Arlington, VA: Office of Naval
Research. (Technical Report No. 3)
Hemphill, J. K. (1960). Dimensions of executive positions.
Columbus, OH: Ohio State University.
Heneman, H. H., III. (1974). Comparisons of self and
superior ratings of managerial performance. Journal of
Applied Psychology. 59, 638-642.
Hicks, J. A., & Stone, J. B. (1962). The identification of
traits related to managerial success. Journal of Applied
Psychology. 46. 428-432.
Hinrichs, J.R. (1969). Comparison of "real life"
assessments of management potential with situational
exercises, paper-and-pencil ability tests, and
personality inventories. Journal of Applied Psychology.
53, 425-432.
Hinrichs, J. R. (1978). An eight-year follow-up of a
management assessment center. Journal of Applied
Psychology. 63., 596-601.
Hollander, E. P. (1954). Buddy ratings: Military research
and industrial implications. Personnel Psychology. 1_,
385-393.
Hollander, E. P. (1956). The friendship factor in peer
nominations. Personnel Psychology. 9, 425-444.
Hollander, E. P. (1957). The reliability of peer
nominations under various conditions of administration.
Journal of Applied Psychology. 41, 85-90.
Hollander, E. P. (1964). Leaders, groups, and influence.
New York: Oxford University Press.

215
Hollander, E. P. (1965). Validity of peer nominations in
predicting a distant performance criterion. Journal of
Applied Psychology. 49. 434-438.
Howard, A. (1974). An assessment of assessment centers,
Academy of Management Journal. 17, 115-134.
Huck, J. R., & Bray, D. W. (1976). Management assessment
center evaluations and subsequent job performance of
white and black females. Personnel Psychology. 29.
13-30.
Huegli, J. M., & Eich, K. E. (1979). The administrator
search process: Managing the interview for desired
results. Journal of the College & University Personnel
Association. 3.0(2), 1-12.
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982).
Meta-analysis: Cumulating research findings across
studies. Beverly Hills: Sage.
Jablin, F. (1975). The selection interview: Contingency
theory and beyond. Human Resource Management. 14(1) .
2-9.
Kelly, S. P., & Nelson, D. F. (1978). Other quarters: The
administrative hiring process. Journal of the College
and University Personnel Association. 2.9(3), 51-58.
King, L. M., Hunter, J. E., & Schmitt, F. L. (1980). Halo
in a multidimensional forced-choice performance
evaluation scale. Journal of Applied Psychology. 65,
507-516.
Kirchner, W. K., & Reisberg, D. J. (1962). Differences
between better and less-effective supervisors in
appraisal of subordinates. Personnel Psychology. 15.
301.
Korman, A. K. (1968). The prediction of managerial
performance: A review. Personnel Psychology. 21. 295-
322 .
Kotula, L. J., & Haggerty, H. R. (1966). Research in the
selection of officer candidates and cadets. Wash., D.C.:
U.S. Army Personnel Research Office.
Knauft, E. C. (1949). A selection battery for bake shop
managers. Journal of Applied Psychology. 32. 434-438.

216
Kraut, A. I. (1969). Intellectual ability and promotional
success among high level manager. Personnel Psychology.
22, 281-290.
Kraut, A. I., & Scott, G. J. (1972). Validity of an
operational management assessment program. Journal of
Applied Psychology. 56, 124-129.
La Gaipa, J. J. (1960). Development of Officer Candidate
Biographical Information Blank: I. Validation of some
experimental inventories. Bureau of Naval Personnel
Technical Bulletin 60-9. June.
Lawler, E. E., III. (1967). The multitrait-multirater
approach to measuring managerial job performance, Journal
of Applied Psychology. 51, 369-381.
Levine, E. L. (1978). Self-assessment for personnel
selection: Bane or boon? Public Personnel Management.
7(4), 230-235.
Levine, E. L., & Flory, III, A. (1975). Evaluation of job
applications A conceptual framework, Public Personnel
Management. 4(6), 378-385.
Light, R. J., & Smith, P. (1971). Accumulating evidence:
Procedures for resolving contradictions among different
research studies, Harvard Educational Review. 41. 429-
471.
MacKinney, A., & Wolins, L. (1960). Validity information
exchange. Personnel Psychology. 13., 443-447.
Mandell, M. M. (1964). The selection process: Choosing the
right man for the job. New York: American Management
Association.
Mayfield, E. C. (1970). Management selection: Buddy
nominations revisited. Personnel Psychology. 23. 377-
391.
McClelland, D.C., & Boyatis, R.E. (1982). Leadership motive
pattern and long-term success in management. Journal of
Applied Psychology. 67. 737-743.
McGregor, D. (1960). The human side of enterprise. New
York: McGraw-Hill.

217
McIntyre, K. E. (1966). Selection of Administrators.
Columbus, OH: University Council for Educational
Administration.
McLaughlin, J. B. (1985). From secrecy to sunshine: An
overview of presidential search practice. Research in
Higher Education. 22.(2), 195-209.
Meyer, H. H. (1956). An evaluation of a supervisory
selection program. Personnel Psychology. 9, 499-513.
Meyer, H. H. (1970). The validity of the in-basket test as
a measure of managerial performance. Personnel
Psychology. 23. 297-307.
Miner, J. B. (1977). Motivation to manage: A ten year
update on the "studies in management education" research.
Atlanta: Organizational Measurements Systems Press.
Miner, J. B. (1978). Twenty years of research on role-
motivation theory of managerial effectiveness. Personnel
Psychology. 31. 739-760.
Mitchel, J. O. (1975). Assessment center validity: A
longitudinal study. Journal of Applied Psychology. 60,
573-579.
Moses, J. L. (1972). Assessment center performance and
management progress. Studies in Personnel Psychology.
4(1), 7-12.
Moses, J. L., & Boehm, V.R. (1975). Relationship of
assessment center performance to management progress of
women. Journal of Applied Psychology. 60, 527-529.
Nash, A. N. (1965). Vocational interests of effective
managers: A review of the literature. Personnel
Psychology. 18. 21-37.
Nash, A. N. (1966). Development of an SVIB key for
selecting managers. Journal of Applied Psychology. 50,
250-254.
Nickels, J. B., & Renzaglia, G. A. (1958). Additional data
on the relationships between expressed and measured
values. Journal of Applied Psychology. 42, 99-104.
Northcott, C. H. (1960). Personnel management: Principles
and practices. London: Sir Isaac Pitman.

218
Peck, R. F., & Parsons, J. W. (1956). Personality factors
in work output: Four studies of factory workers.
Personnel Psychology. 9, 49-79.
Prien, E. P., & Liske, R. E. (1962). Assessments of
higher-level personnel: III. Rating criteria: A
comparative analysis of supervisor ratings and incumbent
self-ratings of job performance. Personnel Psychology.
15, 187-194.
Quaintance, M. K. (1980). The impact of the uniform
selection guidelines on public merit systems. Public
Personnel Management Journal. 9, 125-132.
Rankin, K. K. (1981). A predictive validity study of an
assessment center for research and development superiors.
Wright-Patterson AFB, OH: Air Force Institute of
Technology.
Rawls, J. R., & Rawls, D. J. (1974). Recent trends in
management selection. Personnel Journal. 53. 104-109.
Ricciuti, H. (1955). Ratings of leadership potential at the
U.S. Naval Academy and subsequent officer performance.
Journal of Applied Psychology. 39. 194- 199.
Richardson, R. Jr. (1987). A question of quality:
University programs for community college leaders.
Community, Technical, and Junior College Journal. 57(4),
39-41.
Roadman, H. E. (1964). An industrial use of peer ratings.
Journal of Applied Psychology. 48. 211-214.
Robbins, J. & King, D. (1961). Validity information
exchange. Personnel Psychology. 13. 217-219.
Ross, J. D. (1979). A current review of public sector
assessment centers: Cause for concern. Public Personnel
Management. 8, 41-46.
Sands, E. (1963). How to select executive personnel. New
York: Reinhold.
Schmitt, N. (1976). Social and situational determinants of
interview decision: Implications for the employment
interview. Personnel Psychology. 29, 79-101.

219
Schmitt, N., Noe, R. A., Meritt, R., & Fitzgerald, M. P.
(1984). Validity of assessment center ratings for the
prediction of performance ratings and school climate of
school administrators, Journal of Applied Psychology. 69,
207-213.
Scigliano, V. S. (1979). The search committee: Don't call
us, we'll call you. Journal of the College & University
Personnel Association. 30(3), 36-40.
Scollay, R. W. (1957). Personal history data as a predictor
of success. Personnel Psychology. 10. 23-26.
Sharp P. F. (1984). American college presidents since World
War II. Educational Record. 65(3), 11-16.
Shoop, R. (1974). Public personnel selection: A matter of
consistency. Public Personnel Management. 3, 341- 343.
Silvern, L. C. (1971). Systems engineering of education:
The evolution of systems thinking in education. Los
Angeles: Education and Training Consultants.
Soar, R. S. (1956). Personal history data as a predictor of
success in service station management. Journal of
Applied Psychology. 40, 383-385.
Spencer, G. J., & Worthington, (1952). Validity of a
projective technique in predicting sales effectiveness,
Personnel Psychology. 5, 125-144.
Sprunger, B. E., & Bergquist, W. H. (1978). Handbook for
college administration. Washington, D.C.: Council for
the Advancement of Small Colleges.
Stahl, M. J. (1983). Achievement, power, and managerial
motivation: Selecting managerial talent with the job
choice exercise. Personnel Psychology. 36. 775-789.
Stahl, O. G. (1976). Public personnel administration. (7th
ed.). New York: Harper & Row.
Steel, R. P., & Ovalle, N. K. 2d. (1984). Self-appraisal
based upon supervisory feedback. Personnel Psychology.
37. 667-685.
Tenopyr, M. L., & Ruch, W. W. (1970). In J. P. Campbell, M.
D. Dunnette, E. E. Lawler, & K. E. Weick, Managerial
behavior, performance and effectiveness (p. 193). New
York: McGraw Hill.

220
Thorndike, R., & Hagen, E. (1959). Ten thousand careers.
New York: John Wiley & Sons.
Thornton, G. C. (1968). The relationship between
supervisory- and self-appraisals of executive
performance. Personnel Psychology. 21. 441-455.
Thornton, G. C. (1980). Psychometric properties of self
appraisals of job performance. Personnel Psychology. 33,
263-271.
Turnage, J. J., & Muchinsky, P. M. (1984). A comparison of
the predictive validity of assessment center evaluations
versus traditional measures in forecasting supervisory
job performance: Interpretive implications of criterion
distortion for the assessment paradigm. Journal of
Applied Psychology. 69., 595-602.
Twombly, S. B. (1987). The importance of beginnings: The
relationship of entry positions to career outcomes for
two-year college presidents, Community College
Review.15(2). 14-21.
Ulrich, L., & Trumbo, D. (1965). The selection interview
since 1949. Psychological Bulletin. 63., 100-116.
Vernon, P. (1950). The validation of civil service
selection board procedures. Occupational Psychology. 24.,
75-95.
Veroff, J. (1982). Assertive motivations: Achievement
versus power. In A. J. Stewart (Ed.), Motivation and
society: A volume in honor of David C. McClelland. San
Francisco: Jossey-Bass, 99-132.
Wagner, R. (1949). The employment interview: A critical
summary. Personnel Psychology. 2, 17-46.
Wattenbarger, J. L., & Haynes, F. T., Jr. (1982). Coping
with complexity: Another viewpoint for community
colleges. Community College Review. .10(2) 3-11.
Webb, M. (1983) Employment administration: Elements and
effects. The Journal of College and University
Personnel. 12.(1), 17-22.
Weitz, J. (1958). Selecting supervisors with peer ratings.
Personnel Psychology. 11, 25-35.
Williams, F.J., & Harrell, T.W. (1964). Predicting success
in business. Journal of Applied Psychology. 48, 164-167.

221
Williams, S. B., & Leavitt, H. J. (1947). Group opinion as
a predictor of military leadership. Journal of
Consulting Psychology. 11. 283-291.
Wisner, R. W. (1975) The Kirkland Case: Its implications
for personnel selection, Public Personnel Management. 4.,
263-267.
Wolf, D. B. (1985). A contrary look at community college
management: An ERIC review. Community College Review.
13(2), 51-59.
Wolf, F. M. (1986). Meta-analvsis: Quantitative methods for
research synthesis. Beverly Hills: Sage.
t
Wollowick, H. B., & McNamara, W. J. (1969). Relationship of
the components of an assessment center to management
success. Journal of Applied Psychology. 53. 348-352.
Wright, O. R., Jr. (1969). Summary of research on the
selection interviews since 1964. Personnel Psychology.
22, 391-413.

BIOGRAPHICAL SKETCH
Betty-June Eldridge was born in Maryland on June 17,
1947. She attended public schools in Montgomery County,
Maryland, from elementary school through junior college. She
completed her undergraduate studies at West Chester State
College, Pennsylvania, in 1970. She received an M.Ed. degree
from the University of Florida in 1983, and has pursued her
doctoral program there since that date.
Most of her professional career has been spent as an
intelligence analyst with the Central Intelligence Agency,
serving at the National Photographic Interpretation Center in
Washington, D.C. She is currently employed by the Defense
Intelligence Agency as an intelligence operations specialist
in the United States Special Operations Command, Tampa,
Florida. She and her family reside in Brandon, Florida.
222

I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality,
as a dissertation for the deqree of Doctor of Philosophy.
Phili/A. Clark, Chair
Professor of Educational
Leadership
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality,
as a dissertation for the degree of Doctor of Philosophy.
James L.
professor of Educational
/Leadership
I certify that I have read this study and that in my
opinion it conforms to acceptable standards of scholarly
presentation and is fully adequate, in scope and quality,
as a dissertation for the degree of Doctor of Philosophy.
, 1 J 7 ^
J iJames W. Hensl, Professor
of Educational Leadership
This dissertation was submitted to the Graduate
Faculty of the College of Education and to the Graduate
School and was accepted as partial fulfillment of the
requirements for the degree of Doctor of Philosophy.
December 1988
yrr~u
Dean, College of Education
Dean, Graduate School



184
Criterion contamination?; No.
Results:
Predictor: Personal interviews.
Criterion: Salary level attained.
Correlation coefficient (r): 0.23
18. Grant, D. L., Katkovsky, W., & Bray, D. W. (1967).
Contributions of projective techniques to assessment of
management potential. Journal of Applied Psychology. 51,
226-232.
Purpose: To present data from the Bell System Management
Progress Study (Bray, 1964) on the relationships between
projective techniques and managerial achievement.
Type organization: Corporation.
Subjects: 201 male low-level managers.
Type validity: Predictive.
Years between variables: 8
Assessment center?: Yes.
Range restriction: Medium.
Criterion contamination?: No.
Results:
Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.18
19. Grimsley, G., & Jarrett, G. F. (1975). The relation
of past managerial achievement to test measures obtained in
the employment situation: Methodology and results-II.
Personnel Psychology. 28., 215-231.
Purpose: To test the effectiveness of a particular
methodological approach which can be used in analyzing data
gathered in the process of assessing managerial applicants
in the employment situation.


168
Perhaps most important of all, community colleges
and university graduate programs can work together
to promote high-quality research. Such research,
to be worthy of doctoral recognition, should conform
to rigorous standards, contribute to our knowledge
base on community colleges, and have an impact on
professional practice. (Richardson, 1987, p. 41)
Summary and Conclusion
The two administrative personnel selection methods which
showed high predictive validity were the overall assessment
ratings of the assessment center process and job-related
skills indicators. The first of these methods is global in
aiding in the selection of personnel who can be expected to
achieve in various environments. The second method acquires
its validity from its specificity to a particular set of job
skills. Guin (1987) stated that
if your purpose is to hire generally good people,
a global criterion is useful. If you aren't very
sure what distinguishes good from better employees,
mix up a little of every kind of job behavior in
developing a truly global criterion. However, if
you need to solve a very specific problem, then a
more specific criterion is needed. If there is more
than one specific problem, then more than one
specific criterion is called for. But in most
situations, a global measure will serve quite well.
(p. 205)
The personnel selection methods which are currently used
for the selection of administrators in public community
colleges showed moderate predictive validity in this study.
Since these methods likely will continue to be used,
suggestions for improvement in their predictive validities
have been suggested.


165
subjective reviews of the research on peer assessments
(Haggerty, 1963; Hollander, 1965; Roadman, 1964) lead one to
believe that the validity of this method might be higher than
it actually is.
However, attention must be given to the willingness of
participants to rate each other accurately when they know full
use will be made of such ratings for administrative purposes
(Roadman, 1964) In the 1980s, many persons may not give
truthful opinions of another person's abilities, or lack
thereof, for legal reasons. This tendency may not be evident
in studies conducted for research purposes only.
9. Conduct detailed research within the selection method
of biographical information. This most used method ranked
seventh in predictive validity. The findings from detailed
study of this method, with a search for those moderator
variables which add to predictive validity, may aid in
improving the use of this method.
10. Avoid using psychological attribute indicators as a
selection method for the reasons that (a) although
significant, it has marginal value as a predictor of
administrative achievement, and (b) there may be legal
problems with testing for motivations as opposed to job-
related skills.
The measurement of psychological attributes as a
predictor of administrative achievement has received by far
the most attention in the research. Little progress has been


120
studies each with five years yielding none. Research on
administrative personnel selection methods may have been done
during these years, however, any such research did not meet
the fairly strict delimitations of this study.
The study of the validity of psychological attribute
indicators, such as motivation, has received by far the most
attention in research. Of the total of 128 correlation
coefficients found, 37 were derived from research of
psychological attributes. Clearly there was an interest in
finding the motivations and attitudes held by successful
administrators.
The second most frequently researched administrative
selection method was aptitude and intelligence measures.
Researchers have persisted in studying the validity of
intelligence measures despite the fact that administrative
position applicants were already highly preselected on this
trait. Possible reasons for this emphasis in the research
are simply the availability of the data and the relative
reliability of intelligence measures.
The selection methods of job-related skills indicators
and assessment center overall assessment ratings are almost
equally represented in the research. This equality in study
is most likely due to the fact that most assessment centers
rely heavily on job-related skills indicators, so for each


146
achievement may be biased based upon an employee's past rating
on a selection procedure, may be unwarranted.
Range Restriction
According to the results of the present meta-analysis,
range restriction had little effect on the combined validities
of the independent variables. One would expect that, as
amount of range restriction increased, the ability of the
correlation statistics to determine differences between
variables would decrease. That is, as range restriction
increases it is expected that the strengths of the correlation
coefficients will decrease. This was not proven to be the
case in this study. In Table 9, the effects of range
restriction on outcome measures of administrative achievement
are presented. All correlation coefficients reported were
weighted by sample size.
Studies in which there was no appreciable range
restriction, in which subjects were in nonmanagement
positions, showed a mean correlation coefficient of r = .25.
Where range restriction was considered low, where the original
subjects were low-level managers, the mean correlation
coefficient was r = 27. Where the subjects were mid-level
managers at the time of measurement of the predictor variable,
the mean correlation coefficient was r = .23. Where range


89
to the researchers that the subjectively derived overall
assessment rating utilized in this assessment program was a
valid predictor of management success.
Kraut and Scott (1972) studied the data provided by the
IBM assessment center on the progress of non-management males
into administrative positions. The purpose of the study was
to examine the validity of the assessment program in
predicting administrative achievement. Overall assessment
ratings were correlated with administrative level achieved
from 1 to 5 years later using the same 5-point rating system
that was used in the Wollowick and McNamara (1969) study. The
correlation between overall assessment rating and later
achievement was significant. This assessment program was one
of the few selection methods which showed validity in
predicting success beyond the first-level promotion after
assessment.
Hinrichs (1978) completed a predictive study using a small
sample of IBM managers. A significant relationship was found
between assessment center overall assessement ratings and
administrative level achieved eight years later. However,
Hinrichs also found that the relatively simple prediction
based upon managerial review of the personnel files did as
well in prediction as the assessment center.
Researchers in the IBM studies reviewed here concluded
that large-scale assessment programs appear useful in making
discriminations of management potential which are later


183
16. Goodstein, L. D., & Schrader, W. J. (1963). An
empirically-derived managerial key for the California
Psychological Inventory. Journal of Applied Psychology. 47
42-45.
Purpose: To determine the usefulness of a specific
indicator of psychological attributes in identifying those
personality characteristics associated with managerial and
supervisory success in a large industrial organization.
Type organization: Military.
Subjects: 603 mid- and top-level male, civilian managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.23
17. Grant, D. L., & Bray, D. W. (1969). Contributions of
the interview to assessment of managerial potential.
Journal of Applied Psychology. 53, 24-34.
Purpose: To present information on the contributions made
by the interview to the assessment center process and to
study the relationships of interview variables to the
progress criterion.
Type organization: Corporation.
Subjects: 203 low-level male managers.
Type validity: Predictive.
Years between variables: 9
Assessment center?: Yes.
Range restriction: Medium.


6
collectively. A low score on one part of the examination
could be offset or disregarded due to adequate performance on
other parts. This approach replaced the previously used
"successive hurdles" method in which a failing score on any
one part of an selection procedure would disqualify a
candidate from further consideration (Shoop, 1974).
By the 1960s, many companies and public administration
organizations had implemented the assessment center approach.
The results of the AT&T study were so convincing that many
copied all or part of the original process. There was little
additional research done to test either the findings of the
original study or its validity in a new situation (Crooks,
1973). During this time, the selection of administrators in
public junior and community colleges has continued to follow
the traditional methods of reviewing a candidate's education,
experience, references, and conducting a personal interview.
The exercises completed in hiring college and university
administrators are frequently anything but a process. Too
often they are a set of actions which leave selection
committees dissatisfied with what they have done and
candidates dissatisfied with what they have undergone. No
universal process exists for improving what often is an inept
or discouraging attempt to fill an administrative vacancy with
the best available person.
The problem is magnified by the increasing number of
searches underway in institutions of higher education. In


207
Correlation coefficient (r): -0.01
i.
Predictor:
Psychological attribute indicators.
Criterion:
Salary level attained.
Correlation
coefficient (r): -0.02
j
Predictor:
Biographical information.
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.05
k.
Predictor:
Personal interviews.
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.10
1.
Predictor:
Job-related skills indicators.
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.07
m.
Predictor:
Peer ratings.
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.04
n.
Predictor:
OAR
Criterion:
Salary level attained.
Correlation
coefficient (r): 0.12
51. Williams, F. J., & Harrell, T. W. (1964). Predicting
success in business. Journal of Applied Psychology. 48.
164-167.
Purpose: To identify those predictors which are
significantly correlated with on-the-job achievement.
Type organization: Corporation.
Subjects: 196 MBA graduates.
Type validity: Predictive.
Years between variables: 15
Assessment center?: No.


176
Correlation coefficient (r): 0.29 (N=26)
h. Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.15 (N=26)
i. Predictor: Aptitude and intelligence measures.
Criterion: Promotion rate.
Correlation coefficient (r): 0.18 (N=26)
j. Predictor: Psychological attribute indicators.
Criterion: Promotion rate.
Correlation coefficient (r): 0.20 (N=26)
7. Edel, E. G. (1968). "Need for success" as a predictor
of managerial performance. Personnel Psychology. 21. 231-
240.
Purpose: To determine the concurrent validity of a conative
measure which purports to assess "need for success" by
measuring perceptual distortion to "loaded" stimuli in an
indirect manner (Edel, 1968, p. 232).
Type organization: Corporation.
Subjects: 232 first-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.43


190
Type organization: Military.
Subjects: 639 Naval Officer Candidate School students.
Type validity: Predictive.
Years between variables: 3
Assessment center?: No.
Range restriction: Low
Criterion contamination?: Assumed yes.
Results:
Predictor: Peer ratings.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.35
26. Huck, J. R., & Bray, D. W. (1976). Management
assessment center evaluations and subsequent job performance
of white and black females. Personnel Psychology. 29., 13-
30.
Purpose: To test the validity of an assessment center
process on a population different from the population of the
Management Progress Study.
Type organization: Corporation.
Subjects: 126 nonmanagement women.
Type validity: Predictive.
Years between variables: 1.2 through 5.8
Assessment center?: Yes.
Range restriction: Low.
Criterion contamination?: Yes.
Results:
s* Predictor: Aptitude and intelligence measures.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.15


154
extent to which those methods now in use for the selection of
administrative personnel in public community colleges
correspond with those methods which show the highest
correlation to administrative achievement will be explored.
Guidelines, derived from the present study, for improving
personnel selection methods for administrators of public
community colleges will be suggested.


163
aspects of that portion of the job search have been met
(Huegli & Eich, 1979) .
5. Do not use value systems assessments in the selection
process for administrators in public community colleges. This
selection method may not be viewed by the legal system as
conforming to the Higher Education Guidelines for job-related
validity (Wisner, 1975). However, value system assessments
ranked third in validity, showing a small confidence interval
well above the significance level. Within the course of the
research for this study, the researcher noticed specific value
systems tests which showed high validity in predicting
administrative achievement.
6. Continue to use personal interviews in the selection
process. The personal interview almost always enhances the
validity of the personnel selection process. However, more
detailed reporting of research is needed to determine which
specific aspects of the interview method enhance its ability
to validly predict future administrative achievement.
Personal interviews ranked fourth in overall predictive
validity. This finding is encouraging since the personal
interview is the second most used method in the selection of
administrative personnel.
It has been suggested that structured interviews and
interviews involving more than one rater show higher
predictive validities. The paucity of detailed information
concerning these aspects of the research reviewed for this


117
3. What information, derived from a meta-analysis of the
methodologies of selection research, are useful in
interpreting the findings of this study?
4. What guidelines can be derived from this research for
improving personnel selection methods for administrators of
public community colleges?
In the study, the findings of research of administrative
personnel selection methods for the years 1962 through 1985
inclusive were investigated. This study built upon a
classification system used by Korman (1968) and was carried
further by the use of meta-analysis. Since summary statistics
could be standardized into standard effect size measures, it
was possible to combine the quantitative findings of different
studies and to identify those selection methods which showed
validity in predicting administrative achievement.
Initially, approximately 300 research studies on
administrative personnel selection methods were surveyed.
Many reports contained outcome criterion measures which were
not usable considering the delimitations of this study.
Finally, 52 research studies which contained a total of 128
correlation coefficients were identified as meeting the
criterion for this study.
In this chapter, the results of the meta-analysis of the
data derived from the 52 research studies reviewed are
presented. Further reporting of the studies is based on the


96
conducted no research studies was used. More technical means
of distortion occurred when claims of overall validity were
made on the basis of the outcome scores in a training course,
correlation with another selection method, or with a criterion
that represented only a portion of the total job requirements
(Mandell, 1964, p. 2).
As Stahl (1976) stated:
The worst sins in examining personnel are not inherent
in any single selection instrument or method, but are
derived from using the wrong selection instrument or
method, using something that is inappropriate for
gauging the abilities really needed in a given
occupation or position. When the wrong measuring
instrument or method has been used, too often the
resulting criticism has been directed against the
method itself instead of against the bad judgment of
the selectors. (pp. 134-135)
Previous reviews of administrative personnel selection
research have been subjective in nature (Howard, 1974; Korman,
1968). In the present study, the statistical synthesis of the
research data has been done using the objective process of
meta-analysis. Specific procedures used in the conduct of the
meta-analysis of administrative personnel selection methods
are introduced in the following chapter.


103
stable framework upon which tentative theories can be based.
Meta-analysis gives the researcher the capability to study not
only the subject at hand but also the frailties of the
research designs. If design seems to be a problem in a number
of studies, or if specific design flaws are occurring
regularly, the proper approach is to code that characteristic
along with the studies' other characteristics and empirically
determine how much of the variance in results can be accounted
for by such design and analysis flaws. Such coding was done
in this research.
The rationale and need for meta-analysis of research
findings can be summarized by a final comment from Glass's
AERA speech:
We need more scholarly effort concentrated on the
problem of finding the knowledge that lies untapped
in completed research studies. (p. 5)
Identification of the Research Studies
Many research studies concerning administrative personnel
selection were present in the literature. The sources
included only published studies found in journals,
periodicals, information retrieval systems, and books. Glass
(1976) was aware that information was plentiful when he stated
that:
We are inundated with information. The ERIC system
fills over two million document requests yearly.
Some have termed our predicament "the misinformation
explosion." I assess it differently; we face an
abundance of information. Our problem is to find
the knowledge in the information. (p. 4)


Criterion contamination?; N/A
Results:
197
Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.05
37. Miner, J. B., & Miner, M. G. (1977). Motivation to
manage: A ten year update on the "Studies in Management
Education" research. Atlanta: Organizational Measurement
Systems Press.
Purpose: To test the hypothesis that sex differences in
motivation to manage do exist among practicing managers.
Type organization: Educational.
Subjects: 219 male and female mid- to high-level
educational administrators.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.18
38. Mitchel, J. O. (1975). Assessment center validity: A
longitudinal study. Journal of Applied Psychology. 60, 573-
579.
Purpose: To examine the predictive validity of an
assessmment center in a developmental paradigm, and to
observe changes in the validity coefficients resulting from
measuring a criterion of salary growth at different points
in time.


209
Type organization: Corporation.
Subjects: 94 low- and mid-level management males.
Type validity: Predictive.
Years between variables: 2.5 through 3.2
Assessment center?: Yes.
Range restriction: High.
Criterion contamination?: Yes.
Results:
a. Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.09
b. Predictor: Job-related skills indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.20
c. Predictor: OAR
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.37


35
Aptitude and Intelligence Measures as a Selection Method
In a large proportion of the research, at least one type
of intelligence or mental aptitude test has been used. Most
of the tests utilized measured verbal abilities, but some
measured critical thinking, or mathematical or mechanical
aptitudes.
In a review of 12 research studies aimed at relating a
measure of aptitude or I.Q. to later measures of
administrative achievement, Korman (1968) found that the
"typical managerial applicant population is already highly
preselected on abilities and is relatively homogeneous on
these variables" (p. 301). Thus, the tests of ability do not
tend to discriminate finely among such a population in which
preselection has already taken place. Korman states that
"verbal ability tests seem to show little usefulness for
predicting managerial performance above the first-line
supervisory level" (p. 297).
Dating back into the 1940s, cognitive ability research
showed mixed results. Williams and Leavitt (1947), using
Marine Corps officers as subjects, showed no significant
relationship with supervisor ratings. However, Knauft (1949)
found a significant relationship with later ratings. In the
1950's Handyside and Duncan (1954) and Meyer (1956), each
using supervisors as subjects, found significant relationships
with ratings, but Comrey and High (1955), Ricciuti (1955), and
Thorndike and Hagen (1959) did not.


49
Bray and Grant (1966) found that job-related, or
situational, selection methods had considerable influence on
the judgments of the assessor. Job-related skills indicators
of a sample of managers were related to salary level attained
8 years later. A significant correlation was found. The
situational exercises used consisted of an in-basket exercise,
a simulated manufacturing problem, and a leaderless group
discussion. The researchers concluded that the job-related
situational exercises contributed enough to predictivity to
warrant their relatively high costs in time and money.
Wollowick and McNamara (1969) studied the validity of an
assessment center approach in predicting management potential
and to determine the relative values of the components of the
program. One component of the assessment center consisted of
both written and performance tests of job-related skills.
These tests were administered, individually and in groups, to
a sample of low- and mid-management men. The results of these
tests were significantly related, approximately three years
later, to administrative level achieved.
Meyer (1970), while at General Electric Company, studied
the relationship between a job-related skills indicator, the
in-basket exercise, and an outcome criterion of supervisor
ratings. A demonstrated inability of a general aptitude test
to determine which employees would become successful unit
managers led to the use of this method. This exercise was
developed by John Hemphill, then of the Educational Testing


Criterion: Supervisor ratings.
Correlation coefficient (r): 0.23
174
b. Predictor (selection method): OAR
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.65
5. Cummin, P. C. (1967). TAT correlates of executive
performance. Journal of Applied Psychology. 51. 78-81.
Purpose: To determine if successful executives rate high in
the psychological attributes of achievement, power, and
autonomy, and less successful executives rate high in
affiliation, aggression, and deference.
Type organization: Corporation.
Subjects: 52 mid- and high-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A.
Results:
Predictor: Psychological attribute indicators.
Criterion: Salary level attained.
Correlation coefficient (r): 0.21
6. Dicken, C., & Black, J. (1965). Predictive validity of
psychometric evaluation of supervisors. Journal of Applied
Psychology. 49. 34-47.
Purpose: To study the validity of clinical interpretations
of an objective test battery in two different industrial
settings.
Type organization: Corporation.
Subjects: 2 samples: (a) 31 male, low-level managers, (b)
26 male, low-level managers.


68
successful managers appeared to have pragmatic, dynamic,
achievement oriented values, whereas less successful managers
had more static and passive values, corresponding to a desire
for organizational stasis rather than an organization in flux
(England & Lee, 1974).
Grimsley and Jarrett (1975) studied the "effectiveness of
a particular methodological approach which can be used in
analyzing data gathered in the process of assessing managerial
applicants in the employment situation" (p. 215). Two sample
groups of mid- and top-level corporate managers were used in
this concurrent study. These managers came from more than 100
different companies and many industries in various parts of
the United States. All were administered a "study of values"
test and test scores were compared with administrative level
achieved. The results were not significant. The findings of
this study led the researchers to conclude that the
"differences in test scores of more or less successful
managers result from fundamental differences in mental ability
and personality rather than the influence of on-the-job
experience" (p. 226).
Hinrichs (1978), while with Management Decision Systems,
Inc., conducted an 8-year followup study of the IBM
Corporation assessment center. Within this study, the results
of a study of values were correlated to the outcome variable
of administrative level achieved. The resulting correlation
coefficient of r = .26 was not significant. High scores on


137
Correction for unreliability caused the weighted mean
correlation to be raised from r = .15 to r = .21.
Self-appraisals. A total of 2,847 subjects were included
in 8 studies which yielded 9 correlation coefficients
involving self-appraisals. The mean correlation of r = .17
(a = .10) was depressed to r = .10 when weighted for sample
size. Correction for unreliability caused the mean
correlation to be raised from r = .10 to r = .14.
The corrected correlations of the nine selection methods
showed a mean correlation of r = .31 and a median of .30.
The validity of each selection method, with the exception of
self-appraisals, was significant at the .05 level. The
corrected mean correlations presented here, using formulas
(4) through (8) in Chapter III, are closer to the true mean
correlations than are the unweighted or sample size weighted
means. However, because of the varied methods of measuring
both the independent and dependent variables, the corrections
done here could only partially rectify the mean correlation
coefficients in the direction of their true scores. This is
evidenced by the large amount of variance that remained
unaccounted for after correction.
Relationships Between Selection Methods and Separate
Measures of Administrative Achievement
A review of the literature revealed general
dissatisfaction with all measures of administrative


166
made in applying knowledge toward improving the predictive
validity of this method. The mean correlation of this method
fell near the significance level. Appleby and Nunnery (1980)
found that various psychological attributes did not account
for differences in managerial effectiveness among community
college department heads. Although several flaws were
acknowledged in the study, it was representative of the
lackluster contribution of this selection method in predicting
achievement. Perhaps reconsideration is called for in
researchers' avid pursuit of a selection method that has
persistently failed to enhance selection validity.
Recommendations for Research
More research should be conducted on personnel selection
within education in general and in community colleges in
particular. An institutional research department or
organization dealing with community colleges may be warranted.
Only 3% of the research reviewed for this study was conducted
using educational administrators as subjects. However, 46%
of the researchers involved with the 52 research studies
worked for universities at the time of their research. As
Richardson (1987) stated, "university professors are sometimes
criticized for studying everything but their own university
settings" (p. 39).
The understaffed and inexperienced colleges of the
sixties have been replaced by mature institutions
with their own research and support staffs. The
cutting-edge community college of this decade
possesses more resources and more expertise in
virtually every line or administrative activity than


196
35. Miner, J. B., & Miner, M. G. (1977). Motivation to
manage: A ten year update on the "Studies in Management
Education" research. Atlanta: Organizational Measurement
Systems Press.
Purpose: To study the relationship between psychological
attribute indicators and professional and administrative
success in educational administration.
Type organization: Educational.
Subjects: 49 university administrators and faculty members.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.10
36. Miner, J. B., & Miner, M. G. (1977). Motivation to
manage: A ten year update on the "Studies in Management
Education" research. Atlanta: Organizational Measurement
Systems Press.
Purpose: To determine the relationships between motivation
to manage and managerial success in educational
organizations.
Type organization: Educational.
Subjects: 219 mid-to high-level educational administrators.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.


36
Research used in this study. A total of 15 research
studies were found which used aptitude and intelligence
measures as a predictor variable and met the other
delimitations of this dissertation. These studies yielded 21
correlation coefficients which ranged in strength from r = .01
to r = .41. Samples ranged in size from 26 to 8,885 persons.
Hicks and Stone (1962) while working for Aerojet-General
Corporation and California State Polytechnic College,
respectively, evaluated the effectiveness of a test battery
in discriminating between successful and unsuccessful
managers. One part of this concurrent study related the
mental ability of managers, supervisors, shop foremen, and
engineering supervisors, to the outcome measure, supervisor
rating. Results were not significant. The researchers
concluded that mental abilities did not show a large
relationship to managerial success probably as the result of
sample attenuation through preselection on education and
career achievement.
Williams and Harrell (1964), while at San Francisco State
College and Stanford University, respectively, did a follow
up study of Stanford MBA graduates. The purpose was to find
predictors which were significantly correlated with on-the-
job achievement. While the grade point averages for
undergraduate courses and for required graduate courses fell
short of significant correlations with the success criterion,
there was a significant correlation between grades on elective


211
Bray, D. W. & Moses, J. L. (1972). Personnel selection. In
P. H. Mussen & M. R. Rosenzweig (Eds.), Annual Review of
Psychology. 23. Palo Alto, CA: Annual Reviews, 545-576.
Brubaker, L. (1983). The assessment center method:
Developing employees for mid-management positions. The
Journal of College and University Personnel. 34.(3) ,
32-36.
Burrington, D. D. (1982). A review of state government
employment application forms for suspect inquiries.
Public Personnel Management Journal. 11. 55-60.
Campbell, J. P., Dunnette, M.D., Lawler, E. E., Ill, &
Weick, K. E., Jr. (1970). Managerial behavior,
performance, and effectiveness. New York: McGraw-Hill.
Carleton, F. 0. (1970). Relationships between follow-up
evaluations and information developed in a management
assessment center. Proceedings of the 78th Annual
Convention of the American Psychological Association. 2,
565-566.
Childs, A., & Klimoski, R. J. (1986). Successfully
predicting career success: An application of the
biographical inventory. Journal of Applied Psychology.
71, 3-8.
Comrey, A., & High, W. (1955). Validity of some ability and
interest scores. Journal of Applied Psychology. 39.
247-248.
Cohen, J. (1977). Statistical power analysis for the
behavioral sciences. New York: Academic Press.
Crooks, L. A. (1973). The selection and development of
performance measures for assessment center programs.
Paper presented at the First Annual Industrial and
Organizational Psychology Conference, Columbus, OH.
Cummin, P. C. (1967). TAT correlates of executive
performance. Journal of Applied Psychology. 51, 78-81.
Dicken, C., & Black, J. (1965). Predictive validity of
psychometric evaluation of supervisors. Journal of
Applied Psychology. 49. 34-47.
Dingerson, V. R., Rodman, J. A., & Burns, D. (1985). The
hiring of underrepresented individuals in academic
administrative positions: 1972-1979. Research in Higher
Education. 23.(2), 115-133.


185
Type organization: Corporation.
Subjects: 2 samples of 100 mid- and high-level managers
each.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a. Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.36
b. Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.17
c. Predictor: Value systems assessments.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.11
d. Predictor: Aptitude and intelligence measures.
Criterion: Administrative level achieved.
Correlation coefficient fr): 0.35
e* Predictor: Psychological attribute indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r) : 0.17
f. Predictor: Value systems assessments.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.10


164
study precluded a meta-analysis for the effects of structured
versus nonstructured, and singly-rated versus multiply-rated,
moderator variables. Thus, research within this selection
method is needed to show selectors ways to boost the validity
of this often-used method.
7. Use aptitude and intelligence measures to raise the
caliber of people being admitted to educational administration
programs in colleges and universities, and thus, eventually
improve the labor pool from which administrators are drawn
(Farquhar & Piele, 1972). In this study it was found that
aptitude and intelligence measures ranked fifth from the
highest in predictive validity of selection methods. The
confidence interval was almost wholly above the significance
level. Thus, in this study, it was shown that aptitude and
intelligence measures are valid predictors of future
administrative achievement. The problem in selection of
educational administrators is more a result of admitting weak
applicants into the selection process than of lacking strong
candidates. Legality may be an issue in the requirement of
an aptitude test as a selection process in a public community
college.
8. Suspect the validity of letters of recommendation. The
actual validity of peer ratings used in real life situations
may well be less than is shown by this meta-analysis. Peer
ratings ranked below the median, but still showed significant
validity in predicting administrative achievement. The


127
administrative selection methods appeared to bear little
relationship to each method's ability to predict achievement.
The correlation coefficients used here were mean correlations
taken across each selection method and weighted for sample
size (see Table 3).
The assessment center overall assessment rating was
ranked highest in validity of the selection procedures. A
total of 14,224 subjects were included in 14 studies which
yielded 16 correlation coefficients involving assessment
center overall assessment ratings. The mean correlation of r
= .32 (a = .14) was raised to r = .37 when weighted according
to sample size. Although this method is highly valid it has
been rarely used outside large corporations because of the
costs in time and money.
Peer ratings correlated with administrative achievement
showing a mean coefficient of r = .28 (a = .14). When
weighted for sample size, this correlation dropped to r = .20.
This selection method dropped from its second place ranking,
according to strength, to sixth place when weighted.
Essentially related to references and recommendations, the
predictive ability of this method indicated that references
and recommendations may have some validity in the selection
process. Unfortunately, considering the legal problems that


133
Table 5
Selection Method Frequency of Use Compared with Mean Effect
Size Derived Through Meta-analysis
Freguency
Mean effect size
Low
Medium
High
Freguent
INTER
BIO
PEER
Moderate
SELF
PSYCH
APT
Seldom
VAL
OAR
JOB
the correlation coefficients but not the ranking. Since
reliability data were scantily reported in the research, the
same reliability data were used for each meta-analysis done
on each independent variable.
In the following discussion of the results of the meta
analysis, the selection methods were ranked in the order of
their predictive validity, from highest to lowest. These
data, along with the corrected means, are presented in
Table 6. Corrected means were derived through calculations
done using formulas (4) through (8) presented in Chapter III.


83
Thornton (1968), while a summer associate at the firm of
Rohrer, Hibler and Replogle in Chicago, investigated the
relationship between supervisory perceptions and incumbent
self-perceptions of the performance of executive personnel.
A sample of high-level managers in a large corporation were
included in this study. An average of coefficients of 27
traits, or behavioral characteristics, studied showed a
significant correlation between self-ratings and supervisor
perceptions of performance ratings. The tendency was for
self-evaluations of performance to be higher than supervisory
perception and, in this study, those executives who overrated
themselves were considered least promotable.
Campbell et al. (1970), in a long-term staff study,
reviewed the relationship between self-assessment and a
combination of performance ratings to include salary,
supervisory ratings, and administrative level attained. This
was part of the Early Identification of Management Potential
(EIMP) study carried out by the Standard Oil Company of New
Jersey (SONJ). A sample of managers of SONJ was used. The
resulting correlation coefficient was significant.
Contrary to previous evidence, Heneman (1974) found a
tendency for self-ratings to be less lenient than supervisory
ratings, with a significant correlation. His concurrent study
compared self- and supervisor ratings of job performance of
former MBA graduates of Indiana University, 7 to 10 years
after graduation. He suggested that future research on


42
of suspicion and distrust by industrial psychologists
(Ghiselli, 1966, p. 389) A factor that very likely has
produced disaffection with the employment interview is its
nebulous and intangible character. The interview involves a
social interaction between the interviewer and the applicant,
thus it varies substantially in form and content from one
applicant to another for one and the same interviewer.
In attempting to improve the selection interview,
industrial psychologists have sought to develop sets
of rules to follow in order to improve the reliability
and validity of the interview. To some extent these
rules are based upon scattered empirical evidence, but
because research on the interview is so very limited
they are more commonly based upon professional
judgment or sheer common sense. Mayfield's (1964)
attempt to integrate the pertinent research can only
be described as heroic. Yet the conclusions and
generalizations he arrives at are but weakly founded.
(Ghiselli, 1966, p. 390)
Early studies done on the personal interview method by
Binet and Scott (in Arvey & Campion, 1982) showed low
interrater reliabilities. Wagner (1949), in a summary of
interview research, noted low validities in most cases. In
1964, Mayfield reviewed all interview research done since
Wagner (1949) and found a median validity correlation
coefficient of r = .27. He stated that this value was not
"particularly high" (Arvey & Campion, 1982, p. 284).
Research used in this study. A total of 6 research
studies were found that used personal interviews as a
predictor variable and met the other delimitations of this
dissertation. These studies yielded 7 correlation


149
When intervening time between independent and dependent
variable measurement was increased to 3 through 10 years, the
mean correlation was raised to r = .30 when weighted for
sample size. By definition, these 34 coefficients were drawn
only from predictive studies. However, when years between the
measurement of variables was greater than 10 years, the 10
correlation coefficients concerned yielded a weighted mean
correlation of r = .19.
The weighted mean correlation of r = .30, for the studies
where time between variables was from 3 through 10 years, was
of interest to this researcher. Explanations for this
relatively high validity were sought. The most logical
explanation would be for an included study to contain a large
sample size and high validity coefficients. Studies by Moses
(1972) met these requirements. However, when the weighted
mean coefficients were recalculated omitting the studies by
Moses, the corresponding mean correlation coefficient was only
reduced to r = .26. Therefore, it is suggested that further
investigation be made into determining the optimal length of
time needed between measurement of variables in administrative
personnel selection research.
Conclusions of the Meta-analvsis of the Content and
Methodologies of Personnel Selection Research
The following is a discussion of the findings of the
content of administrative personnel selection research as


32
permitting the person who grades or rates the criterion to see
the predictor scores.
Although perfect criterion-related predictive validity is
practically impossible to achieve, a reasonable amount of
validity is possible. Selection upon the basis of tests or
methods which have no known validity may be little different
from selection upon the basis of a turn of a card. Yet some
personnel agencies have chosen to use methods which have not
been subjected to validity study.
The process of determining the validity of a selection
method involves statistical correlation between the various
examination results and some criteria of performance on the
job. Where the worker is engaged in production activities in
which it is easy to measure output, the question of the
criterion presents no problem. But the simplicity of the
problem disappears when one tries to find adequate criteria
of performance for an administrator. In these cases, the
researcher must rely upon appraisals by those closely
acquainted with the work of the particular administrator
(Stahl, 1976, p. 132).
Kirchner and Reisberg (1962) pointed out the problem of
determining an adequate criteria of managerial performance (p.
301). Supervisors' ratings of the performance of
administrative personnel were found to be subjective and
widely varied. Successful supervisors preferred subordinate
administrators who showed initiative toward organizational


12
6. The research studies analyzed were published in the
United States from January 1, 1962 through December 31, 1985.
Limitations
The following confinements were observed in the
investigation:
1. Performance ratings and promotions are based in part
upon subjective ratings given by an employee's supervisor.
As such, they may not be totally objective measures of job
performance.
2. A lack of independence between the predictor and the
criteria variables may occur when personnel selection
examination results are made available to an employee's
supervisor. The possible effects of this criterion
contamination, where it was identified, were addressed in this
study.
3. Restriction of range occurred when the original
population or sample limited itself, or was limited by,
factors which related to the selection and subsequent
employment process. Thus, if 50% of those interviewed were
hired, and the dependent variable, administrative achievement,
was determined only for those hired, the effective sample
variance for the study was cut in half. This affects the
subsequent correlation between that selection method and any
measure of administrative achievement. For this reason, the


134
Table 6
Mean Correlations. Sample Size Weighted Correlations, and
Corrected Correlations of Selection Methods
Method Mean r Weighted r Corrected r
OAR
.32
.37
.52
JOB
.25
.30
.43
VAL
.20
.25
.35
INTER
. 20
.24
.34
APT
. 19
.21
.30
PEER
.28
.30
.28
BIO
.23
. 18
.26
PSYCH
. 19
. 15
.21
SELF
. 17
. 10
. 14
Overall assessment ratinas
The
assessment center
overall assessment rating was by
far the
"most
valid" of the
selection
procedures. A total
of 14
, 224
subjects were
included in 14 studies which yielded 16 correlation
coefficients involving assessment center overall assessment
ratings. The mean correlation of r = .32 (a = .14) was raised
to r = .37 when weighted according to sample size. These


109
a correlation coefficient. Glass (1977) believed that studies
which were combined must have similar aspects. He was aware
that Light and Smith (1971) stated that the independent and
dependent variables analyzed had to be measured in the same
way, or in a manner which could be converted into the scales
used in the majority of studies. Procedures used in this
study followed those described by Glass (1977) for the meta
analysis of a sample of independent correlational studies.
Calculation of Data
Most studies contained more than one correlation
coefficient. Thus, each individual correlation within a study
was treated as a separate record as long as at least one of
the variables measured was different from all others. Each
record was recorded in a database management program which
contained fields for each of the variables listed above. The
answers to the questions posed in the Statement of the Problem
were derived from taking cross-sections of the data within the
variables by sorting studies based upon the fields listed
above. The following meta-analytic procedures were then
followed to better understand the data.
Conversion to a common metric. The correlations in each
study were first converted into a common metric in order to
determine their relative effect sizes. The effect size is
simply a measure of the strength of the relationship between
two variables. In this dissertation, the Pearson product-
moment correlation was used as the common measure of effect


84
managerial performance should include self-ratings where it
is made clear that these ratings will be used for research
purposes only.
Hinrichs (1978) completed one of the few predictive
validity studies of self-assessment of administrative
performance. The predictions of the assessment center were
not used administratively to prevent criterion contamination.
This study is included in an 8-year followup of a management
assessment center at International Business Machines (IBM).
Self-ratings of marketing personnel correlated significantly
with the outcome variable of administrative level achieved.
Steel and Ovalle (1984), while at the Air Force Institute
of Technology, sought to compare the relative validity of
self-ratings for predicting objective criteria of managerial
job performance. A concurrent validity study at a large
lending institution revealed an insignificant correlation
between self-rating and supervisor rating.
Turnage and Muchinsky (1984) conducted a predictive study
in a large manufacturing firm. The subjects were employees
who were subseguently promoted to supervisory positions.
Self-evaluations of job-related personal characteristics, such
as ability to withstand stress, intellectual abilities, and
interpersonal skill, made up the independent variable. Self-
evaluations related insignificantly and negatively with both
the outcome variables of salary level attained and supervisor
ratings. The problem of criterion contamination (i.e., the


139
Table 7
Selection Method Correlation to Outcome Criteria
Method
SUPRAT
ADLEV
PRO
SAL
NUMYR
COMB
OAR
. 33
.38
-
.24
-
. 15
JOB
. 19
.28
. 11
. 26
-
.49
VAL
-
. 16
.22
. 32
-
-
INTER
. 13
.27
-
. 17
.35
.20
APT
. 13
.21
. 14
.25
-
-
PEER
. 32
-
.29
. 11
-
.33
BIO
. 14
. 11
. 17
. 17
-
.38
PSYCH
.24
. 17
.20
. 13
. 19
.08
SELF
. 16
.26

.01

.24
Mean r
.22
.24
. 19
. 18
.27
.26
Prediction
to Successivelv
Hicrher
Levels
of Administration
Several
attempts
have
been made (Korman, 1968;
Kraut,
1969; Roadman, 1964)
to determine
those
selection
methods
which can predict administrative achievement at succeedingly
higher levels of management. Roadman (1964) reported that
psychological tests had been found useful in predicting which


104
Data sources concerning the selection of administrative
personnel for the period January 1, 1962 through December 31,
1985 were located in manual and computerized indexes. Books,
periodicals, and microfiche collections of The Library of
Congress and libraries of the University of Florida, the
University of South Florida, George Mason University, the
American Psychological Association, and various Department of
Defense installations were used. Computerized index searches
of the NEXIS, Educational Resources Information Center (ERIC),
PSYCINFO, the Defense Technical Information Center, and LC
MARC databases were obtained. Research studies were most
often found published in professional journals in the fields
of psychology, personnel, public administration, and
management.
Classification of Variables
Independent variables. This research dealt with
personnel selection methods rather than with individual
selection tests. Selection methods were used as the
independent, or predictor, variables in the present study.
When specific tests were administered in a research study,
each test was recorded by the type of selection method it
represented. In a 1968 review of personnel selection
research, Korman delineated selection methods in the following
manner:
I. Psychometric Prediction
A. Cognitive Ability Test


126
a doctoral dissertation. Little research has been done with
public administrative agencies. None of the research which
fits the delimitations of this study was conducted in public
administration.
A moderate amount of personnel selection research had
been completed within the military services; however, most of
this research did not meet the parameters of this study. Very
little research has been done within educational institutions.
Although most administrative personnel selection research has
been conducted by university-based researchers, the subjects
of their studies were almost always drawn from employees of
a corporation and not from educational institutions. This
research was often funded by the individual corporation.
Validities of Selection Methods and Their Frequency of Use
All personnel selection methods, combined, correlated
significantly at the .05 level with administrative achievement
with a mean correlation coefficient of r = .22 (a = .14),
calculated according to formulas (1) and (2) in Chapter III,
and r = .25 when weighted for sample size according to formula
(3). Thus, with combined samples the correlations were
significant meaning that personnel selection methods, taken
as a whole, were predictive of administrative achievement.
For sample sizes of at least 100 subjects, the .05
significance level is set at r = .195 (Ary, Jacobs, &
Razavieh, 1979). However, the frequency of use of


221
Williams, S. B., & Leavitt, H. J. (1947). Group opinion as
a predictor of military leadership. Journal of
Consulting Psychology. 11. 283-291.
Wisner, R. W. (1975) The Kirkland Case: Its implications
for personnel selection, Public Personnel Management. 4.,
263-267.
Wolf, D. B. (1985). A contrary look at community college
management: An ERIC review. Community College Review.
13(2), 51-59.
Wolf, F. M. (1986). Meta-analvsis: Quantitative methods for
research synthesis. Beverly Hills: Sage.
t
Wollowick, H. B., & McNamara, W. J. (1969). Relationship of
the components of an assessment center to management
success. Journal of Applied Psychology. 53. 348-352.
Wright, O. R., Jr. (1969). Summary of research on the
selection interviews since 1964. Personnel Psychology.
22, 391-413.


81
to Korman (1968) there is a need for the initiation of
research on the selection method of peer ratings concerning
both its predictive validity and the general characteristics
that correlate with peer ratings.
Self-Appraisals as a Selection Method
Self-ratings of administrative ability are relevant in
selection because the individual has more information about
his own behavior than anyone else and because self-perceptions
are important determinants of an individual's future behavior
(Lawler, 1967, p. 371). A negative view of self-selection for
administrative positions has apparently been shared by those
behavioral scientists who study, develop, and validate
personnel selection procedures. The reason for this appears
to be rooted in a Theory X view of people as they are expected
to behave in a personnel selection setting. In these settings
people are assumed to lack objectivity in assessing their own
performance or personal attributes. They can be expected to
overestimate their performance, skills, knowledge, and
abilities to improve their chances for appointment (Levine,
1978) .
Published studies on self-assessment of skills, abilities,
knowledge, and other applicant attributes as predictors of
administrative job performance have been virtually nonexistent
until recently (Levine, 1978, p. 230). Studies on self
selection, done before 1962, have dealt mainly with
objectively measurable traits (Nickels & Renzaglia, 1958).


63
psychological attributes and two measures of administrative
achievement. Using a sample of business personnel managers,
he found insignificant relationships with both administrative
level achieved and with salary level attained. Although
neither correlation was significant, Miner stated that "the
overall measures of motivation to manage are quite
consistently related to the occupational success indexes" (p.
74) .
In a separate study by Miner (1977), faculty members and
administrators in three different business schools were given
an inventory of psychological attributes. Concurrently, each
subject's score was compared with his or her administrative
level attained. Miner reported no significant relationship,
and "if anything, the administrators have less motivation to
manage than the regular faculty, although the difference does
not approach significance" (p. 51).
McClelland and Boyatzis (1982), while at Harvard
University and McBer & Company, respectively, studied the
relationship between psychological attribute indicators and
long-term achievement in management. Subjects were entry-
level managers who were part of the AT&T assessment center
study. A significant correlation was found between the
predictor variable and administrative level achieved after 16
intervening years. The researchers concluded that a high need
for achievement was associated with managerial success at
lower levels of management, while at higher levels of


144
Correlation of Types of Interactions Within Selection
Methods and Measures of Administrative Achievement
The predictive strength of each type of interaction,
written, verbal, or performance, required from an applicant
was found to be dependent upon the type of selection method
used. Selection tests which required primarily written
responses showed the lowest amount of correlation, r = .19,
when weighted for sample size. Verbal interactions showed a
weighted mean correlation of r = .24. Interactions involving
a performance of a task by the applicant showed the highest
validity, r = .31. The predictive strengths of these
interactions were not independent from the nine selection
methods considered.
The verbal interaction correlation coefficients were
derived solely from the studies involving the selection method
of personal interviews. The performance interaction
correlation coefficients were derived predominantly from the
selection method of job-related skills indicators. Thus,
these mean coefficients were almost identical to the selection
methods from which they were drawn.
Analysis of the Research Methods Used in Administrative
Personnel Selection Research
Information derived from a meta-analysis of the
methodologies used in personnel selection research were found
to be useful in interpreting the findings of the research.
The problems of criterion contamination, restriction of range,


71
that projective evaluation of application blank information
could validly predict performance and tenure.
A trend in selection research in recent years has been the
increasing use of personal background variables in the
prediction of occupational success. Such variables are
considered to have several advantages for this purpose. The
most frequently cited advantages in utilizing these kinds of
variables are that (a) they are less threatening than the
items on typical personality inventories and thus are less
subject to "facade" effects, and (b) the behaviors described
by these items are often reflections of attitudinal and
personality variables.
Results of past studies of the relationship between
biographical information and administrative achievement have
been mixed. Vernon (1950), using civil service managers, and
La Gaipa (1960), using Naval officer candidates, found no
significant relationships. The findings of Riccuiti (1955),
with U.S. Naval officers, Meyer (1956), with firstline
supervisors, and MacKinney and Wolins (1960) with supervisors,
found no consistent relationships. Significant relationships
were found by Haggerty (1953), at the U.S. Military Academy,
Soar (1956), using service station managers, and Scollay
(1957), with promotion managers.
Childs and Klimoski (1986), in a study which involved
employees in both management and nonmanagement positions
investigated the validity of a biographical inventory in the


82
Research used in this study. A total of 8 research
studies were found which used self-appraisals as a predictor
variable and met the other delimitations of this dissertation.
These studies yielded 9 correlation coefficients which ranged
in strength from r = .01 to r = .26. Samples ranged in size
from 30 to 799 persons.
Prien and Liske (1962), while at Case-Western Reserve
University, explored the relationship between supervisor
ratings of job performance and incumbent self-ratings of job
performance on tasks which were intangible in nature. In
their concurrent validity study of employees of various
corporations, significant correlations were found between
self-ratings and first-level supervisor ratings, and
insignificant correlations were found between self-ratings and
second-level supervisor ratings. The preponderance of studies
showed, as this one did, that individuals rate themselves
higher than they are rated by comparison groups.
Lawler (1967) while with the Department of Administrative
Sciences of Yale University, studied mid- and top-level
managers in a manufacturing organization. Four coefficients
of the relationship between self-appraisal and supervisory
job performance ratings showed an average nonsignificant
correlation. This concurrent validity study also showed
evidence of self over-estimation of perceived administrative
abilities.


206
Range restriction: Low.
Criterion contamination?: Yes.
Results:
a.
Predictor:
Self-appraisals.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.01
b.
Predictor:
Psychological attribute indicators.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.01
c.
Predictor:
Biographical information.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.09
d.
Predictor:
Personal interviews.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.03
e.
Predictor:
Job-related skills indicators.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.02
f.
Predictor:
Peer ratings.
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.08
g.
Predictor:
OAR
Criterion:
Supervisor ratings.
Correlation
coefficient fr): 0.03
h. Predictor: Self-appraisals.
Criterion: Salary level attained.


13
possible effects of range restriction, where they were
identified, were addressed in this study.
Definition of Terms
Administrator. An administrator is any person who manages
or directs the affairs of an institution or any major part
thereof. As pertains to higher education, anyone above the
level of professor, such as department chairman or
professional central administrative personnel, is considered
an administrator.
Community college. A community college is a two-year
college, offering academic, general education, vocational
training, terminal, and transfer programs. It can also be
known as a junior college.
Corporation. A corporation is any company or related
group of companies which produces goods and/or services for
profit. Usually this type of organization has a quantifiable
measure of success by which to judge the effectiveness of the
administrators and employees, such as number of units produced
or amount of profit made. Alternative terms for "corporation"
in this study include those such as the corporate world,
business and industry, or companies.
Higher education. Higher education consists of public and
private community colleges, four-year colleges, and
universities in the United States.


204
Years between variables; 0
Assessment center?: Yes.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Self-appraisals.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.19
48. Tenopyr, M. L., & Ruch, W. W. (1965). In J. P.
Campbell, M. D. Dunnette, E. E. Lawler, & K. E. Weick
(1970). Managerial behavior, performance, and effectiveness
(p. 193). New York: McGraw Hill.
Purpose: To study the relationship between personnel
selection measures and measures of job achievement.
Type organization: Corporation.
Subjects: 113 mid-level managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
a* Predictor: Aptitude and intelligence measures.
Criterion: Salary level attained.
Correlation coefficient (r): 0.29
b. Predictor: Job-related skills indicators.
Criterion: Salary level attained.
Correlation coefficient m: 0.36


11
2. The study dealt with personnel selection methods rather
than with individual selection tests. When only one
particular test was administered, such as a Strong Vocational
Interest Blank, this test was recorded by the type of
selection method it represented.
3. The methods of personnel selection, the independent
variable, used in this study included (a) aptitude and
intelligence measures, (b) personal interviews, (c) job-
related skills indicators, (d) psychological attribute
indicators, (e) value systems assessments, (f) biographical
information, (g) peer ratings, (h) self-appraisals, and (i)
assessment center processes.
4. The outcome measures of administrative achievement, the
dependent variable, used in this study included (a)
administrative level achieved, (b) salary level attained, (c)
supervisor ratings, (d) number of years serving in an
administrative position within the same organization, (e)
achievement of tenure, (f) objective performance data, and (g)
promotion rate.
5. Research studies which defined administrative
achievement as measured by (a) admissions personnel ratings,
(b) success in being hired, or (c) performance success in
subseguent training programs, were not used in this study.
These criteria were judged to be too far removed from any
measurement of actual performance on the job.


58
significant. Nash concluded that "effectiveness is
significantly related to the vocational interest patterns of
managers" (p. 254).
In 1967, Bentz (in Campbell et al., 1970) summarized a
series of studies which he had conducted within the
Psychological Research and Services Section of Sears, Roebuck
Company. In this study a significant relationship was found
between psychological attribute indicators and promotion rate
within Sears. Bentz's conclusion was that the Sears managers
accomplish their goals because they are superior "in
intellectual endowment, social competence, and emotional
stamina" (Campbell, et al. 1970, p. 187). Campbell et al.
(1970) stated that there was truth to Bentz's statement, but
that it was not strongly supported by the findings of his
concurrent validity study.
Cummin (1967), while at Harvard, sought to discover if the
"more successful" executives would display motivation toward
achievement, power, and autonomy, and the "less successful"
executives would show motivation toward affiliation,
aggression, and deference in a personality inventory. His
concurrent validity study of business persons compared
psychological attribute indicators to current salary level
attained. He found a significant correlation. Cummin
concluded that "the successful executive is one who is
determined to maintain a high standard of excellence in his
work, and to assume greater responsibilities and more control


147
Table 9
Effects of Range Restriction on Outcome Measures of
Administrative Achievement
Effect size measurement
Range
restriction
Small
Medium
Large
None
. 25
Low
. 27
Medium
. 23
High
. 24
restriction was considered to be high, meaning the subjects
were either high-level administrators or the study was one of
concurrent validity, the mean correlation coefficient was
r = .24.
Predictive Versus Concurrent Validity
According to Stahl (1983), concurrent validity studies
might be expected to show higher correlation coefficients than
predictive validity studies. It has been suggested that, in
a concurrent study on administrative achievement, a supervisor
completing an inventory such as a psychological attributes


151
success" (p. 411) This was the "least researched" of the
nine selection methods in this study. In the light of the
relatively high validity shown, this method may warrant
further research. Care should be taken to define "values"
precisely and not consider them an extension of psychological
attributes indicators.
Ghiselli (1966) found the interview to have at least
moderately substantial validity. This study supported his
finding. However, there has been concern over just what
factors, or moderator variables, contribute to the validity
of the personal interview. It has been suggested that both
structure, such as with an interview schedule (Ghiselli), and
multiple assessors contribute to this validity (Carleton,
1970). A paucity of data concerning these specific factors
in the seven studies reviewed precluded any further analysis
within the selection method of personal interviews. Further
research is needed, with better reporting of the details
involved, to allow for a meta-analysis for the effects of
moderator variables within this method.
According to Korman (1968), ability tests have shown
little usefulness in predicting managerial performance above
the first-line supervisory level. In this study, aptitude
and intelligence measures fell on the median and near the mean
of the nine selection methods.


56
supervisory success in a large military industrial
organization. In their concurrent study of male civilian
managers working for the Army, the variable of psychological
attribute indicators was compared to the criterion of
administrative level achieved. A significant relationship was
found.
The researchers stated that the results suggested there
may be significant differences in personality characteristics
not only between managers and nonmanagers but also among
managers at different levels of responsibility. They
purported that success in first-line supervision may be
determined mainly by technical skill and knowledge which is
relatively independent of personality factors whereas, at the
upper levels of management, such personality-related variables
as organizing, directing, planning, and decision making become
important.
Williams & Harrell (1964) attempted to discover which
personality factors were related to business success. In
their predictive study they used the salary level attained
and administrative level achieved of Stanford MBA graduates
15 to 31 years after graduation. The overall comparison of
psychological attributes to later achievement was
insignificant. However, there was a significant positive
relationship between "success and the score on the
Masculinity-Femininity scale {of the Strong Vocational
Interest Blank}, indicating that those individuals with the


100
meta-analysis from primary analysis (the original analysis of
data from a research study) and secondary analysis (the
reanalysis of existing data in order to answer the original
research question), Glass remarked:
My major interest currently is what we have come to
callnot for want of a less pretentious namethe
meta-analysis of research. The term is a bit
grand, but it is precise, and apt.... Meta-analysis
refers to the analysis of analyses. I use it to
refer to the statistical analysis of a large
collection of analyses results from individual
studies for the purpose of integrating the
findings. It connotes a rigorous alternative to
the casual, narrative discussions of research
studies which typify our attempts to make sense of
the rapidly expanding research literature. (p. 3)
The major thrust of meta-analysis is the understandable
integration of existing research. In many areas of
sociological, psychological, and educational research, the
need to understand the research which has been done is much
greater than that of adding another study to the storehouse
of knowledge. This situation has led to conflicting results
which lead to no acceptable answers to the problems posed,
but instead yields unending calls for further research.
Scholars have access to thousands of research studies, but
progress in making sense out of them for any given topic is
painfully slow. Often we "know" more than we are able to
"understand." Furthermore, literature reviews of empirical
studies are notorious for depending on the subjective
judgments, preferences, and biases of the reviewers, and


39
the relative value of the components of the program. Within
this study, they found that two measures of aptitude and
intelligence were not significantly related to the outcome
variable. The researchers suggested that "it may be possible
to consider eliminating the paper-and-pencil tests not
contributing to the predictive validity" (p. 352).
Campbell et al. (1970), in a study involving the Standard
Oil Company of New Jersey, attempted to discover how employees
who possess the potential to be successful in management could
be identified early in their careers. Although the
intelligence and aptitude measures used were not significantly
related to the outcome criterion, Laurent (in Campbell et al.,
1970) concluded that successful managers "have shown a total
life pattern of successful endeavors" (p. 169).
Moses (1972) studied the relationship between assessment
and subseguent progress in management for personnel in the
Bell Telephone System. This study was part of the AT&T
assessment center process, but not a part of the original
study (Bray & Grant, 1966). A significant relationship was
found between aptitude and intelligence measures. The study
involved a large sample of nonmanagement personnel.
Moses and Boehm (1975) replicated the study done by Moses
in 1972, but the sample consisted of nonmanagement females.
A significant relationship was found in this study. The
researchers stated that "the assessment process predicts the


148
inventory, might reveal that he or she has many of the traits
needed to be a "successful" manager. He or she may not have
had these traits originally in his or her career, but may have
developed and reinforced them over time. However, the results
of this meta-analysis showed significance of, and little
difference between, the mean correlations of the predictive
validity and the concurrent validity studies, r = .25 and r
= .22, respectively, when weighted for sample size.
Years Between Variable Measurement
According to Turnage and Muchinsky (1984), if
organizations collect their criterion data immediately after
assessment, the candidates may not have had ample time to
demonstrate their performance on the job. But, if too much
time has elapsed between the assessment and the performance
evaluations, many extraneous factors may account for the
candidate's level of performance.
Data from this study showed that this may be the case.
The optimal time between measurement of variables may be from
3 through 10 years. Eighty-three correlation coefficients
were derived from studies in which the dependent variable was
measured less than 3 years after the measurement of the
independent variable. These included all coefficients derived
from concurrent validity studies. The sample size weighted
mean correlation was r = .21. Predictive versus concurrent
validity studies showed no appreciable difference in validity.


200
b. Predictor: Job-related skills indicators.
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.28
c. Predictor: OAR
Criterion: Administrative level achieved.
Correlation coefficient (r): 0.37
41. Nash, A. N. (1966). Development of an SVIB key for
selecting managers. Journal of Applied Psychology. 50, 250-
254 .
Purpose: To determine if the measured interests of a large
sample of managers were related to an internal criterion of
managerial effectiveness.
Type organization: Corporation.
Subjects: 230 managers.
Type validity: Concurrent.
Years between variables: 0
Assessment center?: No.
Range restriction: High.
Criterion contamination?: N/A
Results:
Predictor: Psychological attribute indicators.
Criterion: Supervisor ratings.
Correlation coefficient (r): 0.33
42. Prien, E. P., & Liske, R. E. (1962). Assessments of
higher-level personnel: III. Rating criteria: A
comparative analysis of supervisor ratings and incumbent
self-ratings of job performance. Personnel Psychology. 15.
187-194.
Purpose: To explore the relationship between supervisor
ratings of job performance and incumbent self-ratings of job
performance on tasks which are intangible in nature.


62
managerial skill was needed, and actual or potential
leadership ability was not rewarded.
Two consequent studies by Gantz, Erickson, and Stephenson,
were also reported in Miner (1977). In the first, a sample
of former research scientists was followed up 5.3 years after
testing and showed a significant correlation between measures
of psychological attributes related to motivation to manage
and promotion rate. As some of the subjects may have
experienced grade level changes based in part on research
competence, the success index, promotion rate, may not have
been based entirely on managerial competence. The researchers
concluded that the borderline level of the correlation
indicated that further analyses dealing with the predictive
power of the role-motivation theory ought to be conducted.
Using a second sample consisting of top sales persons and
marketing managers, the researchers conducted a similar study
with four years intervening between variables. The
correlation was significant. The researchers stated that,
within this department of the same company, "promotion into
the higher grade levels was based entirely on managerial
competence" (Miner, 1977, p. 31). The conclusion drawn was
that the motives measured by the particular psychological
attributes indicator used did serve as a cause of subsequent
managerial accomplishment (Miner, 1977).
Miner (1977) in a concurrent validity study, sought to
discover the relationships between different measures of


142
Muchinsky (1984) recognized that the "Pygmalion effect" may
play a part. The self esteem of a person selected for
assessment may have been raised substantially and "they then
seek to perform both in assessment and later on the job at a
Table 8
Correlations of Selection Methods Measured Within and Without
Assessment
Center Processes
Selection
Assessment
Non-assessment
method
center
center
JOB
.26
.25
PSYCH
. 14
.20
INTER
. 18
.35
APT
.20
. 18
VAL
.24
. 18
BIO
. 32
. 18
PEER
. 12
.35
SELF
. 13
. 19
Mean r
.20
.21
level high enough to substantiate the lofty esteem in which
they are perceived to be held" (p. 602).


214
Hammons, J., & Ivery, C. (1987). Does a difference make a
difference?: A comparison of the activities of community
college presidents with those of chief executive officers
in other settings. Community College Review. 15(4),
18-27.
Handyside, J., & Duncan, D. C. (1954). Four years later: A
follow-up of an experiment in selecting supervisors.
Occupational Psychology, 28. 9-23.
Harrell, T. W., & Harrell, M. S. (1974). Predictors of
management success. Arlington, VA: Office of Naval
Research. (Technical Report No. 3)
Hemphill, J. K. (1960). Dimensions of executive positions.
Columbus, OH: Ohio State University.
Heneman, H. H., III. (1974). Comparisons of self and
superior ratings of managerial performance. Journal of
Applied Psychology. 59, 638-642.
Hicks, J. A., & Stone, J. B. (1962). The identification of
traits related to managerial success. Journal of Applied
Psychology. 46. 428-432.
Hinrichs, J.R. (1969). Comparison of "real life"
assessments of management potential with situational
exercises, paper-and-pencil ability tests, and
personality inventories. Journal of Applied Psychology.
53, 425-432.
Hinrichs, J. R. (1978). An eight-year follow-up of a
management assessment center. Journal of Applied
Psychology. 63., 596-601.
Hollander, E. P. (1954). Buddy ratings: Military research
and industrial implications. Personnel Psychology. 1_,
385-393.
Hollander, E. P. (1956). The friendship factor in peer
nominations. Personnel Psychology. 9, 425-444.
Hollander, E. P. (1957). The reliability of peer
nominations under various conditions of administration.
Journal of Applied Psychology. 41, 85-90.
Hollander, E. P. (1964). Leaders, groups, and influence.
New York: Oxford University Press.