This item is only available as the following downloads:
United States Civil Service Commission
FPM Lelter 335-13
SFederal Personnel Manual System
Guidelines for Evaluation of Employees for
Promotion and Internal Placement
Published in advance
of incorporallion in FPM
RETAIN UNTIL SUPERSEDE
Washington, D. C. 20415
December 31, 1979
Heads of Departments and Independent Establishments:
1. The purpose of this letter is to announce the issuance of Appendix B to FPM Supplement
335-1. (Supplement 335-1, "Evaluation of Employees for Promotion and Internal Placement"
was issued through FPM Letter 335-12 dated December 29, 1978.)
2. A draft of Appendix B, "Guidelines for Evaluating Employees for Promotion and Internal
placement," was transmitted to agencies and unions for comment. Only ten comments
were received, indicating basic concurrence with the document. Many of the comments,
however, did contain suggestions for specific changes. The final version reflects
revisions made as a result of these suggestions.
3. Appendix B contains useful advisory material in this area but is not binding on
4. This Appendix reviews certain concepts and problems in evaluating employees for promo-
tion and internal placement and describes methods and necessary resources for
agencies to carry out the principles set forth in the supplement. Topics discussed
in the appendix include, for example, needed technical competence, description of
the evaluative process, types of evaluation methods, determining effectiveness of
instruments, and information on widely used OPM evaluation procedures.
Jule M. Sugarman
Personnel Research and Development Center, Staffing Services
335, Promotion and Internal Placement
Attachment to FPM Ltr. 335-13 (2)
(e) Ability to plan objectives and staffing and resource requirements to implement
the personnel measurement program throughout the agency.
(f) Ability to determine training needs in personnel measurement areas and to carry
out training activities designed to meet objectives aof the personnel measurement program.
c. Personnel measurement capability needed by personnel specialists and technicians.
(1) Many recurring staffing activities which involve personnel measurement (e.g.
determination and documentation of work behaviors, skills, knowledge, and abilities!
development of vacancy announcements; development or modification of instruments for
performance appraisal; evaluation of training, education and experience administration
of tests and other procedures; monitoring of rating panels) are carried out by personnel
specialists. Program effectiveness is largely dependent upon the extent to which the
personnel specialists are able to apply appropriate job analysis procedures and psycho-
metric principles in these activities. Special attention to selection for these posi-
tions and to training programs for current staff may be warranted.
(2) Major attention should be given to structuring the work activities in personnel
measurement areas to best utilize staff resources. Some recurring data operations and
statistical procedures, for example, may best be carried out by personnel technicians
or psychology technicians under the guidance of staffing specialists of personnel
(3) Employee development specialists and EEO specialists in many instances need
specialized abilities in personnel measurement. A special awareness is also needed
in these areas of the relationship and distinctions among objectives and activities
of counseling and employee development, equal employment opportunity and affirmative
action, and merit promotion and placement.
d. Personnel measurement capabilities needed by supervisors and selecting officials.
(1) Supervisors, as well as other employees, are frequently primary information
sources for information concerning jobs to be filled and the important requirements
for successful performance on the job. They need to understand and support the job
analysis and measurement activities of the personnel office. Orientation sessions and
materials for supervisory personnel on objectives and planned procedures for improving
personnel measurement can facilitate their support. As improved personnel measurement
procedures yield results in terms of placing the best qualified persons on the job,
the support of managers and supervisors will be further strengthened.
(2) Supervisory appraisals of employee performance are usually of major importance
in personnel measurement programs, both as selection procedures themselves and as criteria
against which the predictive validity of other procedures is evaluated. Substantial
research indicates that the relevance, objectivity, reliability, and validity of
supervisory appraisals is enhanced when supervisors are provided special training on
making appraisals and how to avoid common types of rater error.
(3) Selecting officials need information concerning the rating process to assist
them in making decisions from the list of best-qualified candidates. They need to
know Whether the differences in final rating of those referred are likely to be
important in terms of job performance differences. Usually this can be best accomplished
through orientation of selecting officials concerning the selection instruments and
procedures used for jobs under their cognizance and, additionally, through providing
interpretive information in conjunction with each certificate which discusses the
degree of importance of differences in candidate ranking on that particular certificate.
B-4. OVERVIEW OF THE EVALUATION PROCESS
a. General. Except when screening criteria have been used to retrieve information
from a skills file, an initial review of all candidates in the area of consideration
or all who respond to a vacancy announcement will be necessary to screen out
unqualified candidates. The following are two approaches which may be used for
identifying the candidates for competitive evaluation:
(1) Separate screening and ranking. Screen all candidates on the basis of elements
that are essential to adequate performance in the specific position to be filled. A
careful and thorough job analysis is needed to identify only those elements actually
Attachment to FPM Ltr. 335- 13 (3)
essential to minimum acceptable performance. It is important that screenout criteria
distinguish only between those who could perform adequately in the job and those who
could not. It is also important to avoid use of search or screenout criteria which
unduly restrict competition or impose artificial barriers to advancement, for instance
by including as an element a skill which can be easily learned. Unless a job element
qualification standard is authorized by the Office of Personnel Management (OPM) for
internal placement in the position being filled, care should be taken that persons
screened by this method meet the OPM's minimum qualification requirements or the
requirements under such agency standards as may be authorized.
(2) Combining screening and ranking. When the number of candidates is small, it
may be possible to eliminate the separate screening stage and perform the basic qualifica-
tion determination and competitive evaluation at the same time. Those found unquali-
fied for the job could be so notified and eliminated from further consideration. If
this procedure is used by an evaluation panel, it may be necessary for advisory partici-
pation by a personnel office representative to assure that the qualification standards
are understood and that all OPM requirements are met.
b. Evaluation procedures. The overall objectives of evaluation are to narrow the
number of eligible candidates to a reasonable number from which a selection may be
made and to assure that selection is made from among the best qualified candidates.
It is necessary that candidates are evaluated on the basis of predetermined criteria
to establish their relative merit for promotion to the position being filled. To
achieve these objectives, evaluation procedures should:
(1) Be based on a thorough analysis of the job to determine the work behaviors
and knowledge, skills, abilities and other characteristics to be used to measure
the probability or expectancy of successful performance among candidates;
(2) Provide for an evaluation of each candidate's qualifications against the work
behaviors of KSAds required for successful performance of the target position;
(3) Result in a distinction among candidates on the basis of relative qualification
(4) Provide for a consideration of those qualifications which indicate a candidate's
potential for future promotion, when the job being filled leads to further advancement.
(See Section 51 of Appendix A).
c. Criteria. Criteria, as used in the evaluation process, are the standards against
which the qualifications of candidates are assessed. In order for each criterion to
be useful for rating purposes, it must be important to successful performance of the
duties of the position to be filled; not be subject to being learned or acquired
on the job within a short period of time after promotion or placement; and distin-
quish among candidates according to expected levels of job performance (e.g., successful
versus superior). Failure to meet these requirements tends to result in placing
artificial limits on competition, to foster inbreeding,and to restrict the effective-
ness of manpower planning and career development by the agency.
Establishment of specialized evaluation factors which far exceed actual job demands
may suggest an attempt to tailor the job to a preselected candidate.
(1) Minimum qualification standards, such as those in OPM Handbooks X-118 and
X-118C, are used to eliminate those persons who are not eligible for further
consideration because of the low expectancy that those who fail to meet the standard
will perform the duties of the job to be filled at at least a minimally acceptable
level. These standards are set at a level which maximize the probability of correctly
identifying and distinguishing between potentially satisfactory and potentially
poor job performers.
(2) Rating criteria can also be used to identify well qualified candidates when
the standards are set at a level which maximizes the probability of correctly
identifying persons who have the ability to perform on the job in a superior, rather
than merely in a satisfactory, manner. Usually, these criteria are different from
those associated with satisfactory performance
d. Methods of evaluating candidates. To the maximum extent possible, the same
evaluation methods should be applied to all persons being considered for appointment to
Attachment to FFP Ltr. 335-L3 (4)
higher grade positions. When that is not possible, for example, such as with per-
formances tests where candidates may not be in the same geographical area, con-
sideration should be given to either not applying that type of evaluation method
or using two or instruments with similar relevance and validity to obtain the same
type of information. ......
e. Grouping candidates. Evaluation procedures may provide for the ranking of
candidates in 1-2-3 order on the basis of scores obtained by application of the
evaluation instruments or may provide for assigning candidates to quality groups
udch as qualified" with the best designated for referral to the selecting officials i
(See section B-11).
B-5. STEPS IN DEVELOPMENT OF THE EVALUATION PROCESS
The steps in the screening and evaluation of employees should be designed to determine
both basic eligibility and relative qualifications of promotion candidates. In
developing the process, careful consideration should be given to each of the follow-
a. Determining the important work behaviors, knowledge, skills, and abilities, and
the level of each, that are needed (1) for satisfactory and (2) for highly success-
ful job performance.
b. Identifying selective factors. (1) are they appropriate; (2) are there any other
job requirements not subject to evaluation but which must be met. (For example willing-
ness to travel by air.)
c. Determining which evaluation instruments should be used by finding out:
(1) What information about employees' qualifications is already available
(2) What additional information is needed to determine employees' basic qualifications;
(3) What additional information is needed to determine the best qualified employees;
(4) Which potential instruments (a) are relevant to the job, (b) are sound and
dependable measures of the qualifications needed, (c) make meaningful distinctions among
the employees, and (d) are practical and feasible. (See B-B, Types of Evaluation Methods).
d. Determining specifically for each instrument:
(1) At what stage in the total evaluation process it will be applied
(2) To whom it will be applied:
(3) Who will apply it; and
(4) How the data yielded by this instrument will be handled, by whom, and under what
instructions for interpretation.
e. Determining how the data or results of each instrument are to be combined with other
information in ranking employees, and how they are to be used in the final selection
B-6. DETERMINING MINIMUM REQUIREMENTS
a. General. Minimum job requirements are in the qualification standard for the occupa-
tion in OPM Handbook X-118 and X-118C. The standard is developed on the basis of a
comprehensive occupational analysis, a careful study to identify the basic and (where
required) specialized work behaviors, and the knowledge, skills, and abilities (job
elements or factors) needed for acceptable performance on the job. Candidates have
basic eligibility for the position when they satisfy the minimum requirements of the
standard for the occupation and in addition any specialized requirements (selective
factors) necessary for the position.
b. Exceptions. Employees may be eligible for promotion under specific provisions
of training agreements approved by the OPM, which maes it possible to substitute
intensive, accelerated training for a portion of the normal qualification requirements
(see FPM chapter 271, subchapter 7).
Attachment to FPM Ltr. 335-13 (5)
B-7 NEED FOR ANALYSIS OF JOBS
a. Importance of job analysis. The foundation of a sound evaluation process is the
careful analysis of the job. This critical step is necessary to determine minimum
qualification requirements, to identify the KSAds to be used to identify those
candidates who can be expected to perform in a superior manner, and to determine the
measurement instruments that will be used to assess qualifications of candidates,
resulting in the ranking of candidates in order of relative ability. The design of the
analysis process requires personnel measurement skills. Once selected and designed,
the job analysis procedures can be accomplished by subject matter experts under the
guidance of carefully trained personnel specialists. A complete record should be
retained to show the sources of job data, the data obtained, and a statement of any
conclusions supported by the data. In this way, a logical explanation can be given
to support and justify the use of a particular evaluation instrument for a specific
job or groups of jobs.
b. Scope of job analysis. (1) Job analysis should identify not only the work
behaviors and knowledge, skills, abilities, and other characteristics needed
for successful job performance, but also the level of amount of KSAOD needed. On
the basis of this information, appropriate measurement methods can be selected or
developed in terms of these jobs needs.
(2) One way to utilize the information about job requirements gathered from a job
analysis is to group jobs by job families, i.e., jobs having similar requirements.
For instance a variety of clerical jobs such as correspondence clerk, information and
editorial clerk, etc., might be considered together because the same basic requirements
are cannon to all the jobs. Another group of clerical jobs which may have a common
core of qualifications is made up of jobs such as: retirement clerk; accounting clerk;
voucher examining clerk; and payroll clerk. These jobs may have in common accuracy
and speed in working with numbers, attention to detail, and memory for a variety
of items. Thus, the employee qualifications needed are similar and one evaluation
program might suffice for promoting employees into these jobs at the entrance levels.
(3) Jobs above the entry level will usually need to be analyzed separately and will
probably necessitate separate measurement procedures, since the work behaviors and
KSAOs which differentiate the potentially superior employees may likely be different
from those at the entry level.
(4) The job analysis should also include information about the total environment
of the job, its position in the career ladder, and its training features. For such a
job analysis, a number of sources of information can be utilized: statements of
employees and their supervisors; official position descriptions review of work materials
and work products; observation of work processes; job performance standards; and materials
relating to other personnel and operating programs and policies which affect this
c. Selective factors. If an element is found to be essential to successful perform-
ance in the job to be filled and is not covered by the qualification standard, it
may be added as a selective factor. For example, a position in a particular location
may require knowledge of a language other than English. Although the qualification
standard does not require this knowledge, it may be added as a selective factor.
B-8. TYPES OF EVALUATION METHODS
a. A variety of methods are available for evaluating qualifications. Among these
are evaluation of training and experience, written, oral, and performance tests, inter-
views, performance appraisals, and assessment centers. Within each of these broad
types of evaluation methods, there are various specific measuring instruments and
procedures that may be applied. These include:
(1) Measures of demonstrations of specific job skills such as typing;
(2) Measures of work behaviors or KSAO's demonstrated in structured individual
or group interviews and exercises which are standardized and scored;
(3) Measures of abilities related to learning in various situations;
Attachment to FPM Ltr. 335-13 (6)
(4) Measures of specific classes of abilities, verbal, numerical, mechanical,
clerical, and others;
(5) Measures of dexterity and coordination;
(6) Measures of knowledge and proficiency;
(7) Occupational and other interest measures; and
(8) All other formal, scored, quantified, or standardized techniques for assessing job
b. Evaluation of training, education and experience.
(1) Purpose. An evaluation of training and experience is used to predict the prob- -
able effectiveness of future performance of the basis of the quality of relevant
past performance. This evaluation should cover work behaviors or knowledge, skills,
and abilities (the outcomes of training, education, and experience) rather than
training, education, and experience for their own sake. Furthermore, the objective
is to determine how well the candidate is prepared for higher level work. Therefore,
the employee's training, education, and experience should be evaluated in terms of
the work behaviors or knowledge, skills, and abilities needed for success in the job
to be filled. (2) Credit for training, education, and experience. Training, education
and experience are evaluated in terms of the degree to which the employee possesses
the work behaviors or the knowledge, skills and abilities (job elements or factors)
identified in the job analysis. For each factor, various degrees of ability are
described in terms of the kinds and quality, rather than length, of training, education,
and experience. These descriptions should be as specific as possible care should be
taken not to rely on job titles since different employees with similar job titles may
be performing different job tasks requiring different knowledge, skills, and abilities.
Normally, it is sufficient to distinguish four to six levels of ability. Greater
refinement is not usually necessary and makes the descriptions of levels as well as
the appraisal of employees more difficult and prone to error. See the Job Qualification.
System for Trades and Labor Occupations (Handbook X-118C) for examples of procedures
for crediting training, education, and experience in terms of job elements. Usually,
supplemental forms are needed which provide candidates the opportunity to describe
their KSAOs specifically in terms of their relationship to job requirements.
(3) Selection of evaluators. Evaluation of training, education, and experience I
requires considerable judgment; the employees who do this should be selected carefully
and trained well in the use of the rating plan, so that the evaluations by different "
raters will be comparable and dependable. One effective approach is to use represen-
tatives both of the personnel office and of operating offices (subject-matter experts)
(SME's). (See section B-13 for further discussion of the use of panels.) Personnel
specialists can contribute a knowledge of rating techniques as well as of career staffing
concepts and methods, while the subject-matter experts can assess the relative value
of certain types of experience for their particular jobs.
c. Employment tests. (1) Purpose. Professionally prepared written, performance,
or oral tests can provide a reliable and valid means for measuring knowledge, skills,
and abilities (KSAO's). Tests are particularly helpful:
(a) If the possession of specific KSAO's must be established; for example, in selecting
people for jobs which are new because of technological developments, or when information
about the KSAO's of employees is not available;
(b) If the ability to learn specified new job behavior or concepts is an important
factor, for example, in evaluating persons for trainee positions or for training to
qualify for higher level position;
(c) If the present job duties of the employees are not closely related to those of the
position being filled; for example, for evaluating potential of nonsupervisory employees
for supervisory positions;
(d) If knowledge of important subject matter is to be evaluated,
(e) If the outcome of training designed to prepare employees for higher level positions
is to be evaluated; or
Attachment to FPM Ltr. 335-13 (7)
(f) If relevant knowledge, skills, and abilities among the employee group would not
otherwise be measured.
(2) Written tests. (a) Written tests are valuable in identifying KSAOD's that employ-
ees may have had little opportunity to demonstrate through past performance. They also
are appropriate measures of information or knowledge about given subjects.
(b) Written tests are sometimes in the form of questionnaires designed to identify
employee occupational interest patterns. These questionnaires are usable if the items
are relevant to the specific work situations or tasks and thereby enable the employee
to indicate his or her interest in the particular job. Questionnaires measuring general
traits, such as social interest, usually are not acceptable for selection purposes.
(c) Written questionnaires intended to measure personality traits are, at their
present stage of development, generally of questionable value in selection of personnel
for promotion. FPM chapter 337, section 1-5i, restricts the use of personality tests in
personnel actions affecting employees in the competitive service.
(3) Performance tests. Tests in which the employee uses physical equipment or simul-
ates or replicates a work process or product are commonly labeled performance tests.
Well-known examples are the typing test and road test of driving skills. Performance
tests are particularly useful in the skilled trades, such as machinist or electrician.
If they require much material and individual examining, administration on a large scale
is expensive. On the other hand, properly constructed performance tests can often
provide more direct measures of work behaviors and KSAO's than written tests. Addi-
tionally, they are more acceptable to employees.
(4) Oral tests. Oral tests are standard questions which are used to measure the
employee's ability to respond to problems and to demonstrate job knowledge by oral
responses which are individually scored. Oral tests are more difficult to standardize
and more costly to give and to score than are similar written tests.
d. Assessment Centers.
(1) Purpose. Frequently, positions are filled by individuals whose present or previous
job requirements are somewhat different from those of the new position. The assess-
ment center technique is a valuable tool in identifying job-related behaviors and
KSAO's of candidates before they are placed on the job.
(2) Format. The assessment center consists of a multiple evaluation procedure in
which assessment reports are developed from the combined judgments of a team of trained
assessors. These assessors have observed the candidates in various standardized and job-
related simulated situations. An advantage of assessment programs is that the exercises
can be tailored to the organization and job requirements. Because of this flexibility,
there are a number of variations as to method.
(3) Content. The job simulations may include exercises such as group discussions,
simulations of interviews with subordinates, in-basket simulations, oral presentation
exercises, written communication exercises, and factfinding exercises. The exercises
are constructed to provide the candidate an opportunity to demonstrate skills that have
been identified as important for success on the job.
(4) Design and operation. (a) All assessment center exercises should be selected
and designed to measure specific job requirements as determined from job analysis. As
previously referred to, the simulations must be job-related and standard for all candi-
dates who are competing for the same job(s). In addition, those work behaviors which
are to be observed in each exercise should be documented and specified for assessors.
(b) An important part of the assessment process is assessor training. The assessors
should be adequately prepared for their evaluative task. This preparation includes
familiarization with the assessment exercises and the relevant skills, training on
the use of behavior recording forms and procedures, and instruction as to the final
rating process. The assessor training usually requires the assessors to evaluate
a group of practice candidates which provides an excellent indication of rater reli-
ability. Where large numbers of candidates necessitates multiple teams of assessors,
the teams should be equated in terms of rating characteristics.
Attachment to FPM Ltr. 335-13 (8)
(c) The intended use of assessment reports is stipulated in advance, and the candi-
dates are informed of the assessment center procedures and objectives. The information
obtained during the assessment process may be appropriate for decisions such as selection,
placement, or career development. Candidate feedback is, therefore, an additional con-
sideration. This feedback should focus on the specific skills evaluated during the
assessment center. Strengths and weaknesses of the candidate can be discussed, and ....... ...
developmental efforts can be suggested.
(1) Purpose. Some important elements of the job may be identified from the job
analysis which involve personal characteristics of the employee. In some cases, observ-
able information pertinent to these elements cannot be obtained adequately by means
such as appraisals by supervisors. In these instances, an interview with the employee
may be useful. Interviews should of course be job related and be used consistently with
other methods. In addition, an interview may be useful as a means for giving information .
about the job to the employee.
(2) Uses in ranking. The interview may be used in the ranking process in two ways:
(a) The interviewers evaluate the employee on specific behaviors and responses and
this information is later combined with other available measures to arrive at a final
total score. In this way, the interview information is simply recorded by the inter-
viewers and turned over to those making the final evaluation. The interviewers do not
combine the interview results with results obtained on other measures.
(b) The interviewers evaluate the employee on specific behaviors and responses
considering simultaneously the interview data as well as other available information.
In this second way, the panel of interviews must combine the interview results with the i"
results obtained on other measures to arrive at the final evaluation.
(3) Planning and conducting the interview.
(a) It is critical that the interview be well planned in terms of the behaviors and
responses to be observed, the evaluation standards to be applied, and the procedures for
conducting the interview session. It is particularly important to pay careful attention
to objectivity. The interview should be structured so that the information to be
obtained is well defined and recorded in a standard manner. The guide for determining
employee's score on the factors to be observed should specifically state how the informa-
tion from the interview is combined with information from other sources.
(b) In addition to the careful design of the interview, there should be at least two
interviewers who are well trained in the procedures and the purpose of the interview.
(c) Since the interview at best is only a small sample of the employee's behavior
and is conducted in a setting which may not be representative of the work situation,
care should be taken not to attribute more precision to the interview than it warrants.
Small differences among employees are unlikely to be meaningful.
(d) Interview records should be retained with other merit promotion plan records.
(e) If no work behaviors or KSAO's are identified that must be measured by an inter-
view, it is not necessary to interview the candidates. However, if personal contact
is desired, a selection interview may be held in accordance with FPM chapter 332, Appendix
(f) Interviewers should take special care not to ask questions which violate equal
f. Appraisals of work behaviors, knowledge, skills, abilities and other characteristics.
(1) Purpose. (a) Appraisals of employee performance are used for various purposes,
e.g., promotion, employee development, and performance ratings required by statute. A
principal difference between appraisals for promotion purposes and appraisals for other
purposes is that appraisals for promotion focus on the job to be filled, and appraisals
for other purposes focus on the employee's present job. Both the orientation of the
rater and the content of the appraisal instrument frequently differ according to the
purpose of the appraisal. It is important that appraisals used to evaluate employees
for promotion be relevant to the job for which they are being considered.
Attachment to FPM Ltr. 335-13 (9)
(b) Frequently, employees have not had an opportunity to demonstrate some of the
important work behaviors or KSAob needed for the job to be filled. For work behaviors
and KSAds which differ markedly from those which are needed to perform the candidates'
present jobs, performance appraisals may not offer useful information for evaluation
of those work behaviors and KSAds. However, care should be taken in the design of
appraisal procedures to ensure that candidates get due credit for related work beha-
viors and KSAds; for work behaviors and KSAds which have been demonstrated at slightly
different levels than those needed on the new job; and for work behaviors and KSAds
which have been demonstrated in other ways, such as training programs or temporary
assignments. It is usually advisable to supplement appraisals by supervisors with
information about work behaviors and KSAds from other sources, including the candi-
(2) Periodic appraisals of performance. Some agencies confronted with the need
to evaluate large numbers of employees have developed systems in which appraisals
are obtained at regular intervals. Same of these systems are computer based and
cover a wide range of work behaviors or factors designed to record information for
use in appraising the promotability of employees for a large number of different
positions. To develop an appraisal score, a subset of factors that are relevant
to the job to be filled is selected from the large number of factors rated. If
relevant factors for a particular job are not included in the periodic appraisal,
the periodic appraisal should be supplemented with special appraisals. Similarly,
if the periodic appraisal includes factors which are not relevant to the job to be
filled, the ratings on these nonrelevant factors should not be used in determining
the appraisal score.
(3) Format. Often one of two forms--narrative or check-off--is used. Each form
has its advantages and disadvantages. The check-off may provide the supervisor less
freedom in reporting his or her observations of the employee's behavior. However,
since the check-off is more structured, it is more likely to have satisfactory
objectivity and reliability. Frequently, a combination of the two is used in which
the supervisor first completes the check-off portion and then supports his or her
judgments with narrative statements. The forms should encourage a spread of scores.
This can be facilitated by identifying the various points on the scale for each factor
in terms of specific behaviors, and requiring the supervisor to report his or her
observations rather than make a judgment of whether the employee is high or low.
(4) Improving consistency of appraisal and assessment. When the employees are
scattered throughout a large organization, they will be evaluated by different people.
The various people may differ in their understanding of the job requirements and the
factors to be covered. Use of the following procedures may help to provide consistency
of judgments when different people evaluate the employee.
(a) The factors used are based on careful analysis of job requirements in order to
reach a common understanding.
(b) Evaluators, when possible, discuss the factors and job requirements (but not
individual employees) in order to reach a common understanding.
(c) Supervisory judgments are obtained in a systematic way so that work in different
situations under different people can be evaluated in a standard manner.
(d) Training is given to ensure that, before they start to make the evaluations, the
evaluators are given careful instructions. Also, the effects of their evaluations
on quality staffing as well as on the careers and the attitudes of employees are made
(e) Results are obtained from several independent evaluations, if appropriate to
furnish a more balanced picture than a single appraisal. In addition, knowledge of
the fact that future opportunities will not be determined by just one supervisor's
judgment is likely to increase the employee's acceptance of a promotion plan.
B-9. DETERMINING EFFECTIVENESS OF INSTRUMENTS
a. General. The major consideration in selecting and using an evaluation instrument
is its effectiveness in measuring the degree to which the employees possess the work
behaviors and knowledge, skills, and abilities needed for successful job performance.
The effectiveness of any instrument depends on itq job relatedness, objectivity,
Attachment to FPM Ltr. 335-43 (10)
reliability, and validity. These qualities, individually and in interaction, are critical
to a sound program of identifying persons best able to perform or learn a job. Each
of these qualities is defined in general terms below and should be considered by
persons responsible for selecting and applying evaluation instruments.
b. Job relatedness. An evaluation instrument is job related if the work behaviors or
knowledge, skills, abilities, and other characteristics (KSAO's) measured by it have .. |I
been determined to be necessary for successful job performance through a job analysis. -
For example, a supervisory appraisal of performance is relevant only to the extent I
that the work behaviors or KSAO's being rated are important for success on the job
to be filled. If accuracy in working a calculator is not important in a supervisory
statistician's job, appraisals of that skill are not relevant in evaluating employees i
for the supervisory job. The need for job relatedness again points up the need for
careful job analysis. Instruments are selected, modified, or developed on the basis
of the findings of the job analysis to assure they measure important job characteristics.
This helps assure that all qualified employees receive proper consideration.
c. Objectivity. An evaluation instrument should be as objective as possible;
that is, it should be structured to elicit responses which are factual and observable,
and which are reported in a precise, specified way. In addition, specific rules should
be available for determining the value of each response. For example, in evaluating
employees for computer operator positions, it would be more objective to ask them to
list the computers they had operated, and the kinds of operations they performed on
each rather than to try to make judgments based on general descriptions of duties.
Evaluation of this experience can then be systematized. The machines and tasks
indicative of the highest level of ability can be specified and each lower level of
ability can be defined in similar terms. The less objective a measurement procedure
is, the greater the possibility of unreliability.
d. Reliability. Reliability refers to the consistency and stability of the scores
obtained with the instrument. Any measurement is subject to error and the extent to
which repeated measurements differ limits the reliability of the measuring instrument.
Reliability can be estimated in a number of ways: On tests, for example, by comparison
of results of repeated administrations of the same test, on parallel forms, or on
representative subsets of items; on appraisals, by comparison of the ratings assigned
by different appraisers. The more reliable an instrument, the less likely it is
that a person's score is affected by chance errors. It should be noted that an
instrument can be reliable, but invalid.
e. Validity. An instrument is valid if it measures the work behaviors or knowledge,
skills, abilities and other characteristics (KSAO's) it is intended to measure. Vali-
dity is a critical requirement for the selection and use of any measuring instrument.
However, an instrument may provide a highly valid measure of a KSAO and still not be
appropriate for a particular job because the KSAO itself is not related to successful
job performance. Thus, both validity and job relatedness should be considered. There
are three generally accepted ways of demonstrating validity: content, construct, and
criterion related. At least one form of validity evidence is normally required to sup-
port the use of an instrument. Principal concepts of these approaches are provided
below in a highly simplified manner. Further guidance on validation and documentation
is covered in Appendix A. (1) Content validity depends on well-informed judgments that
the content of the measuring instrument is representative of important aspects of perfor-
mance on the job (see paragraph 14C of Appendix A). Job information tests and other
achievement tests are typically evaluated on the basis of content validity. Supervisory
appraisals, ratings of training and experience, and assessment centers are also frequently
evaluated on the basis of content validity. In the case of evaluation of training and
experience where information provided by candidates is evaluated in terms of behavioral
examples of demonstrated KSAOh, it is important to content validity that the candidates
be provided a description of the important work behaviors or KSAds needed on the job so
that they can select appropriate examples from their experience. In such cases, the
candidates sample from their experience those accomplishments and behavioral examples
which most closely match the KSACOs required on the job.
Attachment to FPM Ltr. 335- 13 (11)
(2) Construct validity is based on the inference that a certain measurable ability
or personal characteristic (construct) is required in the job and that the selection
procedure in fact measures this construct. For example, a job analysis may indicate
that quantitative reasoning ability is important for successful job performance. A
test is then devised or selected to measure the quantitative reasoning ability. If
the test can be shown to be related to important job duties it may be used opera-
tionally. Identification and measurement of constructs is an involved process.
Again, job analysis is an essential part of the process, and usually, evidence of
the relationship of the measure of the construct with other measures of the same
construct is necessary for demonstration of construct validity.
(3) Criterion-related validity is determined by statistically comparing performance
on the measuring instrument with some other indicator (the criterion) which is accepted
as a measure of job performance. Criterion-related validity can be approached in two
ways: predictive validity and concurrent validity. Predictive validity involves
relating scores on a measuring instrument to later measures of job performance. For
example, scores on an ability test might be used to predict later success in job per-
formance. In concurrent validity, the criterion score is obtained at approximately
the same time as the score on the evaluation instrument. For example, a test of
verbal ability might be administered to a group of employees and compared with
their present job performance.
f. General standards for validation. An evaluation instrument is acceptable if either
of two sets of conditions is met:
(1) There is evidence from a careful job analysis that the work behaviors or KSAOh
being measured are in fact an important part of the job (class of jobs, or occupation)
and logically related to successful performance on that job (class of jobs or occupa-
tion); and there is competent evidence of content or construct validity in the evalua-
tion instrument or (2) There is evidence of a useful degree of criterion-related
validity arrived at by comparing scores on the instrument with a criterion which is
legitimately based on the needs of the Federal Government.
B-10. OTHER CONSIDERATIONS IN SELECTING INSTRUMENTS
a. General. Many factors enter into the decision on what instruments should be used
in the evaluation process. There are many administrative considerations. The primary
consideration, however, is obviously the effectiveness with which the instrument can
provide adequate measure of the work behaviors, knowledge, skills, abilities or other
characteristics (KSAO's) required.
b. Differences between selection for entry and for inservice placement. (1) In
selection from among applicants outside the agency, the selecting official chooses from
among many persons not directly known to him or her. In inservice actions, however,
the agency also must recognize the existing employment relationship and the nature of
the commitments between the employees and the agency. The methods used help determine
whether the employee will consider the organization fair in its treatment of employees.
(2) In making promotion decisions, the agency is likely to have available more com-
plete and dependable information about its employees than it is likely to have about
outside applicants. Therefore, it can and should use evaluation instruments and proce-
dures which make full use of available information relevant to evaluation of specific
work behaviors or KSAO's required for particular positions.
c. Effect of new .r modified instruments. (1) The introduction of new or modified
measuring instruments calls for careful study of the agency's staffing function and of
selection factors already being used. It is important to find out what is already known
about the work behaviors and knowledge, skills, and abilities of employees in various
occupations to see whether new or modified instruments are needed. What selection
devices have already been used, and at what level of difficulty? How narrow or how
broad is the range of individual differences among present employees as a result of
high selection standards for initial appointment, of turnover, and of training? Employ-
ment standard- and procedures in the Federal Government have a very marked selective
effect. The persons from the total employment population who are hired represent a
selected yroup both in terms of general and special KSAO's.
This selectivity means that a more restricted range of individual differences exists among
employees than among applicants. Thus, it becomes more difficult to identify large differ-
ences among qualified employees than between qualified and unqualified applicants in
open, competitive examinations.
(2) The instruments and procedures under consideration, if otherwise technically sound,
Should meet the test of acceptability and reasonableness to the employees affected and
the supervisors and other officials who will be making use of the evaluations for reach-
ing personnel decisions.
d. Administrative considerations. Many problems of an administrative nature will affect
the choice and use of specific instruments. Although these administrative problems must
be considered, the main goal--to provide a sound promotion program--should not be over-
looked. In some cases, the decision to select one instrument over another may result in
the need to put more time and effort into the evaluation of employees, but this additional
workload may be worthwhile in terms of increased effectiveness of the method, employee
confidence in the program, etc. The administrative considerations include such matters
as the number of employees to be considered in relation to cost of program, number of
vacancies, etcg whether the program is designed for large groups in populous occupational
areas and grade levels, or for small groups in specialized positions, or both.
B-11 GENERAL PRINCIPLES ON ARRANGING EMPLOYEES IN ORDER OF MERIT
a. Introduction. To identify the best qualified employees, the results of evaluation
instruments should be used to rank or group eligible candidates in order of relative
merit. Such an arrangement can range from broad categories, each including a sizable
group of employees, to specific rankings. Two interrelated problems are involved: First
how to combine the results of several instruments; second, whether the score produced
by the combination is suitable for fine ranking or for broad categorization.
b. Principles. (1) The degree of refinement expected should not exceed the sensiti-
vity of the measurement instrument itself. For example, interviewers or supervisors may
be able reasonably to appraise various dimensions of employee performance on a broad
5-point scale. If, however, the procedure requires them to use a highly refined scale
with a large number of different divisions, small differences in the appraisals of
employees will likely reflect only apparent rather than real differences in ability.
(2) It is important that instruments and procedures intended to measure work
behaviors or KSAds which differentiate superior employees from less successful
employees actually differentiate among employees when applied. If almost all candi-
dates get the maximum available points of evaluation of performance, for example,
other possibly less important factors may then account for the rankings of candidates.
When work behaviors or XKSds are identified in job analysis which distinguish superior
performers, and attempts at measurement of these particular work behavior or KSAdS
fail to demonstrate differences, the job analysis should be reviewed with respect
to the appropriateness of the work behaviors or KSAOs identified. If their appro-
priateness is confirmed, it then becomes appropriate to determine ways to improve
the measurement methods and procedures to obtain a more accurate spread of measures
on employees and valid ranking.
(3) The person who makes the final selection normally should not be confronted with
so many employees that he or she cannot reasonably review every case and make compari-
sons among all the employees in the best qualified category. At the same time, it
is improper to reduce the number in the best qualified group on the basis of minute
score differences, or arbitrary or artificial factors, which are unlikely to have
real meaning in terms of likelihood of success on the job. Where too large a number
of employees in the best qualified group continues to be a problem, it is probably
indicative of a need to review the job information and selection procedure to focus
more clearly on work behaviors or KSAds which distinguish highly successful performance.
Where distinctions simply cannot be made tie-breaking factors such as length of
experience or seniority may be used.
Attachment to FPM Ltr. 335-13 (13)
(4) In all cases, even when only a few employees are under considerations, the results
obtained by the employees on the evaluation instruments are of direct value in describing
the relative abilities of the employees for the selecting official.
B-12 COMBINING RESULTS OF VARIOUS INSTRUMENTS
a. Successive hurdles. Combining measurement results is one of the most important
functions of the professional personnel measurement specialist. Among the considera-
tion faced by a measurement specialist is a procedure by which the various evaluation
instruments are used as a sequence of hurdles. Under such a system, the employee
must attain a certain score on each instrument as that instrument is applied. Only
those who attain the specified score on each instrument are given final consideration.
This procedure may be appropriate in situations in which a particular work behavior
or KSAO, or a specific level of work behavior or KSAO, is necessary, in and of itself,
for successful performance on the job; and absence of it cannot reasonably be compen-
sated for by other XSAO's. Hurdles or screening factors, are usually appropriate
only in determining basic eligibility. In distinguishing among the qualified and
the best qualified candidates, the interests of the employee and the agency are best
served by basing the employee's evaluation on the total picture drawn from a variety
of evaluation instruments. In this way, employees who are unable to show their ability
on one instrument might be able to show it on another.
b. Methods of combining scores.
(1) The combining process is difficult when the results on each of two or more
evaluation instruments are to affect the rank ordering of the employees. The various
scores must be combined to yield a single final score. There are various ways in which
evaluation data can be combined. The particular way selected determines which
evaluation instrument has the greatest effect on the final ranking.
(2) The weighting of scores and ratings is not a mechanical process. Sometimes when
weights have been set and scores have been multiplied by the weights, the real influence
of the weighted scores may be vastly different from the apparent or intended one because
of the statistical properties of the measuring instruments. When information from several
different measuring instruments is to be considered, the information is often expressed
in different terms for the different measures. For example, supervisory appraisals may
be in terms of descriptions of levels of performances, whereas test results may be in
terms of numerical scores. Morever, the terms of evaluation may be of dissimilar magni-
tude, with supervisory appraisals ranging from one to five levels and test scores ranging
from 50 to 100. These differences affect the final ranking. Raw scores on a written
tests and on an evaluation of training and experience, for example, cannot simply be
added together with the thought that equal weight is being given to the two parts. Since
the written test scores are likely to be spread out over a much wider range than were
the ratings of experience, the written test will therefore, have greater weight in the
composite. Before deciding upon how to combine scores from measuring instruments, it is,
therefore, essential to know their real effects.
(3) One of the best ways of controlling the effects of differences between the terms
used for different evaluation instruments and thereby making interpretation, comparison,
and combination of data from various sources easier is to convert the raw scores on each
instrument to standard scores. The procedure for computing standard scores is explained
in tests on psychological statistics. This results in scores on one instrument that can
be directly compared or combined with scores on another. For the OPM tests covered in
this appendix, standard scores have been computed and are included in the discussion
of the application of scores. Agencies may find this information useful as an aid in
interpreting scores on these tests.
c. Relating information to elements rather than total job. (1) In some cases, the score
obtained on each evaluation instrument is considered as a separate, overall measure of
the employee's potential for success on the job and the results of the various measures
are then combined into a single total score. It may be more appropriate, however, to
view the results of the various instruments as alternate indications of ability on each of
the job elements (KSAO's) to which the information is related. A scoring plan can be set
Attachment to FPM Ltr. 335-13 (14)
up for each element to consider all the pieces of information that are indications of
ability in that element regardless of the evaluation instrument that produced the infor-
mation. In this way, all the available information is used to arrive at the separate
element score. (The key is that the plan must specifically state how the various pieces
of information are used to determine the score on the element.) Then, the scares on
the elements are combined to form an overall indication of how good the employee should
be on the job.
(2) This technique is used in the job element method described in the Job Qualificatios
System for Trades and Labor Occupations (Handbook X-118C). In this method a logical
general scoring system is provided to assure that the information relating to each element
is looked at in a consistent way. Since the method uses only those few elements which atre
most effective in identifying the employees who are most likely to be successful on the
job, the snores on the elements are given equal weight unless evidence is available to
support assigning a higher weight to specific elements.
B-13. USING PROMOTION PANELS
A promotion panel is a permanent or ad hoc committee established to evaluate, compare,
and rank employees for promotion. The membership of the panel normally consists of three
to seven agency program or operating officials and personnel office representatives. At
least one personnel office representative should be in attendance at the panel meetings,
so that control and documentation of panel actions meet validity requirements. Usually*
some of the line officials on the panel are representatives of the organizations in
which the vacancy exists. To assure objectivity the selecting official should not be
a panel member. In exceptional situations his or her expertise and knowledge of
the position being filled may be essential for the effective evaluation of the candidates.
In such cases it would be advisable for the personnel office representative to document
the reasons for his or her inclusion on the panel. This situation calls for a very
clear understanding of the functions of the panel members in the evaluation process
as distinct from the selection decision to be made by the selecting official. If
other officials are available who have a sufficient knowledge of the job to be filled,
such individuals should be included on the panel rather than the selecting official.
The procedures to be followed by the panel, the way results of evaluation instruments
are interpreted and used, and the method for combining results to arrive at the final
ranking should be precisely stated and clearly understood by the panel members.
B-14. DETERMINING EFFECTIVENESS OF THE EVALUATION PROCESS
a. Need for review. After the evaluation process has been initially developed or after
it has been used to place employees on the job, the question arises, "Are there addi-
tional ways in which the effectiveness of the process may be approved?" Any information
that indicates deficiencies or suggests improvements is of value in refining the process
so that it results in identification of the best qualified employees.
b. Tryout of evaluation process. One procedure is to try out the process with employees
of known competence, so that instruments and procedures can be improved before they are
actually used in the evaluation process. In many cases, however, the conditions neces-
sary to make a sound pretest of this kind do not exist. Frequently there are too few
employees who are working in similar positions and for whom meaningful evidence of job
performance can be obtained. Another limitation is that, since employees already on the
job represent a highly selected group, the narrowed range of ability within the employed
group makes it difficult to relate differences in job performance to measurable diffe-
rences in knowledge, skill, and ability.
c. Placement fbllowup. A practicable method of determining how to improve the evalua-
tion process is to see how well people who were promoted are doing on the job after 6
months or a year. Feedback of this type can be useful for identifying possible weaknesses
in evaluation procedures.
Attachment to FPM Ltr. 335-13(15)
(1) If the employees promoted are successful on the job, it can generally be
concluded that the methods of selection has been sound and that they have contributed
to the selection process.
(2) One rough guide is the number of prcmnotees who had to be reassigned or given
special training (other than training normally planned or expected) because of inability
to perform the duties of the job. Analysis of the particular deficiencies provides a
basis for improvement of the evaluation process. Did the evaluation process include
some instrument which was supposed to guard against this particular deficiency and
apparently did not function as intended? Should techniques be added to cover this
qualification requirement? Frequently, careful analysis of the reasons for success or
failure of particular persons may provide insights into the nature of the entire promo-
tion plan and directions of improvements.
d. Other benefits. Observation and employee feedback provide a means to check up on
other aspects of the promotion program, e.g., the usefulness and effect of orientation
or similar training efforts; the adjustment (as distinguished from performance) of
employees in their work and its environment; and the reactions of employees and super-
visors to the promotion program. From this type of information it should be possible,
over a period of time, to develop an appreciation of the real effect and value of the
promotion plans in the agency's overall personnel management system, and the problems
which need attention.
B-15. INFORMATION ON WIDELY USED OFFICE OF PERSONNEL MANAGEMENT EVALUATION PROCEDURES
a. General. The procedures listed in this section may be used by any agency. They
must be used in accordance with the occupational and grade level coverage and any other
restrictions prescribed in the considerations for use of each test or other procedure.
(When a test is authorized for use at more than one grade, however, it is not the intent
of the OPM that it be applied more than once to the same employees. If the test is used
for selecting employees at one level within an occupation, it may not then be used for
evaluating these employees for promotion to a higher grade in the same or related occupa-
tion.) The test must be properly administered and scored. Information to assist
agencies in interpreting and applying scores is provided for use in combining the
test results with results of other procedures for which standard scores are available.
(See Appendix B-12).
Attachment to FPM Ltr. 335-13 (16)
b. Clerical abilities test. (1) Introduction. The clerical abilities test includes
measures of (a) clerical abilities (Test 800) and (b) verbal abilities (Test 801A).
(2) Considerations for use.
(a) OPM Tests 800 and 801A may be used by agencies for those GS-2 and GS-3 clerical
positions for which the tests may be used competitively. A determination must be made
that the abilities measured by the tests are relevant to the job. --
(b) Test 800 and 801A may not be used above the GS-3 level.
(c) If the agency desires to use the clerical ability test for other positions or
levels prior approval of the OPM is required.
(3) Interpretation of scores.
(a) The following information for Tests 800 and 801 is based on a sample of applicaift ,
in hiring examinations. The mean was 76.4 and the standard deviation was 25.3
(b) The table below includes suggested interpretations of total test scores.
Total Raw Score
(Test 800 & 801A)
Percent of Sample at or
above the Given Score
99 20 High
90 30 Good
82 40 Average
69 60 Low
Note: Scores of 45 or below should be interpreted with caution since these
are near or below a chance level. It is possible that such scores are not
accurate indicators of ability. Other factors, such as faulty test admin-
istration or scoring, may have affected the score.
(c) The table below includes suggested interpretations of part scores for use when
consideration of part scores is relevant to the ranking process.
52 and above
31 and below
52 and above
31 and below
Average to High
Low to Average
Attachment to FPM Ltr. 335- 13 (17)
(d) The table below can be used for converting raw scores to standard scores.
Conversion Table, Test 800 + 801A
(4) Scoring Formula
Attachment to FPM Ltr. 335-13 (18)
c. Trades abilities test.
The trades abilities test is designed to measure abilities required in a wide variety
of occupations, at levels which require the performance of a series of simple trades
and labor tasks. This test includes measures of (a) gross dexterity, (b) alignment ..........
dexterity, (c) shop arithmetic, and (d) following directions.
(2) Consideration for use.
(a) Test 100A, 100B, 100D, and 100E are used for filling positions in which the
ability to learn and advance in the occupation is essential such as helper-trainee
positions. Tests 100A and 100E are also applicable for filling positions in which
the ability to learn and advance is not critical. (See Job Qualification System for
Trades and Labor Occupations Handbook X-118C). A determination must be made that
the abilities measured by the test are relevant to the job.
(b) Only series of Test 100 which are numbered 110 or above are approved for use.
(c) If the agency desires to use the test other than as described above prior
approval of the OPM is required.
(3) Interpretation scores.
(a) The following tables show how the scores on the trades aptitude test may be
used for assigning credit on individual job elements.
(b) Table 1 provides suggested crediting on elements for positions in which the
ability to learn and advance in the occupation is not essential.
(c) Table 2 provides suggested crediting on elements for positions in which the
ability to learn and advance in the occupation is essential.
Dexterity Following Directions Ability Level
Test 100A Test 100E
70 and above 141 and above Superior (4 points)
60-69 131-140 Satisfactory (3 points)
45-59 111-130 Potential (2 points)
35-44 81-110 Weak (1 point)
Attachment to FPM Ltr. 335-13 (19)
Tests 100 A, B, D,
258 and above
141 and above
100A and B
93 and above
Superior (4 points)
Satisfactory (3 points)
Potential (2 points)
Weak (1 point)
3 1262 08729 3444
xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E9MKL56GT_D8JOGA INGEST_TIME 2013-01-18T14:23:09Z PACKAGE AA00012996_00045
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC