• TABLE OF CONTENTS
HIDE
 Front Cover
 Executive summary
 University mission, quality, and...
 Appendix
 Notes






Title: Florida quality evaluation project
CITATION PAGE IMAGE ZOOMABLE
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/UF00055260/00001
 Material Information
Title: Florida quality evaluation project
Alternate Title: College summaries
Physical Description: v. : ; 29 cm.
Language: English
Creator: University of Florida -- Office of Institutional Research
University of Florida -- Office of the Provost
Publisher: <University of Florida, Office of the Provost>
Place of Publication: Gainesville Fla
Frequency: annual
regular
 Subjects
Subject: Universities and colleges -- Evaluation -- Statistics -- Periodicals -- Florida   ( lcsh )
Genre: government publication (state, provincial, terriorial, dependent)   ( marcgt )
 Notes
Statement of Responsibility: University of Florida.
Issuing Body: Issued by: Office of the Provost, <1991/92>-1994/95; Office of Institutional Research, 1995/96- .
General Note: Description based on: 1991-92; Title from cover.
 Record Information
Bibliographic ID: UF00055260
Volume ID: VID00001
Source Institution: University of Florida
Holding Location: George A. Smathers Libraries, University of Florida
Rights Management: All rights reserved, Board of Trustees of the University of Florida
Resource Identifier: aleph - 002422673
oclc - 45090497
notis - AMC7742
lccn - 00229099

Table of Contents
    Front Cover
        Front Cover 1
        Front Cover 2
    Executive summary
        Unnumbered ( 3 )
        Unnumbered ( 4 )
    University mission, quality, and performance
        Page 1
        Page 2
        Page 3
        Page 4
        Page 5
        Page 6
        Page 7
    Appendix
        Page 8
        Appendix A: Resource balance categories
            Page 9
            Sample data tables for resource balance
                Page 10
                Page 10a
                Page 10b
                Page 10c
                Page 10d
                Page 10e
                Page 10f
                Page 10g
                Page 10h
                Page 10i
                Page 10j
                Page 10k
                Page 10l
                Page 10m
                Page 10n
                Page 10o
        Appendix B: Examples of teaching, research, and service criteria
            Page 11
            Page 12
        Appendix C: Nine measures in state university system accountability plan
            Page 13
    Notes
        Page 14
        Page 15
        Page 16
        Page 17
        Page 18
Full Text







University Mission, Quality, and Performance


The Florida Quality Evaluation Project














John V. Lombardi and Elizabeth D. Capaldi








UNIVERSITY OF
FLORIDA







Mission Quality, and Performance


The Florida Quality Evaluation Project


University of Florida 1992


t

























Universities have no immunity from competition for resources, and in that intense competition,
quality is of the utmost importance. This paper describes the University of Florida's mission,
defines its community and constituencies, explains how we evaluate quality, and reviews our
planned process of continual improvement.

Mission, Community, and Constituencies: The University of Florida belongs to an ancient
tradition of great universities. The undergraduate experience, based in the arts and sciences,
remains at the core of higher education in America. Universities extend the range of this
undergraduate education to include advanced or graduate study leading to the Ph.D. and a
variety of professional programs. As part of this tradition, the University of Florida is a major,
public, comprehensive, land-grant, research university. Our university community of faculty,
staff, and students serves a complex constituency of government, research agencies,
foundations, corporations, regents, legislators, citizens, parents, alumni, and the general public.
The university community commits its energy and talent to the programs that deliver teaching,
research, and service--our three-part mission. As our faculty, students, and staff maintain their
commitment to quality, we will serve our many constituencies well, and we will compete
successfully for the resources our work requires.

Quality Evaluation: Within the university, it is the coordinated and structured work of faculty,
staff, and students that produces quality programs. We identify and measure quality and
productivity in comparison to the standard of the nation's best universities of our type to
improve our effectiveness. For the University of Florida, our reference group includes the
public, and especially land-grant, universities in the American Association of Universities. In
the colleges and their departments, we evaluate quality performance in comparison to
comparable institutions, and the individuals who produce the quality must design the
assessment.

A cear understanding of mission, community, and constituencies helps define the
program's approach to research, teaching, and service. Does a program focus primarily
on teaching, research, or service, or does it balance the three parts of our mission? To
assess our quality we need to recognizewhich parts of the university community or which
of the university's many constituencies we serve.

The identification of indicators of quality makes explicit the criteria we use to judge
quality, for unless we can define quality, we cannot improve it. We identify research and
creative activity using different criteria than we do teaching excellence or service quality.
In defining and assessing quality we also pay attention to how much quality our programs
can produce. If we have superior quality but we produce very little of it, we need to
understand why. Everyone wants us to do more and better, but we cannot forever do
more and better for less. We use the national reference to place our quality and resources


Executive Summary


University Mission, Quality, and Performance

The Florida Quality Evaluation Project


University of Florida
1992






Mission, Qualty, and Performance The Florida Quality Evaluation Project

into perspective. Teaching, research, and service never start or stop, have no beginning or
end, but instead operate on continuous, never-ending cycles. Each program and each part
of the mission involves processes that have cycles of productivity and achievement that
may extend over many years. We accommodate these cycles in this project, and we
evaluate interactions among programs and units to identify where we can improve their
quality.

The resource balance includes all the resources a program produces and consumes.
None of these quantitative data speak directly to quality. Instead, they help us put the
relative success of each of our units in context. If the resources appear inadequate when
compared with our reference group, then we need to reevaluate how we distribute our
resources. We evaluate our university's auxiliary operations for their quality and positive
resource balance. Activities such as an intercollegiate athletic program or a bookstore
must break even and should earn a surplus.

Constraints on the achievement of quality can come from inadequate space,
equipment, or salary support; unnecessary rules or restrictions; foolish bureaucratic
complexity; or poor coordination among related programs. Identification of constraints
allows us to minimize their effect on quality.

Constant Commitment: The Florida Quality Evaluation Project needs the continuing
involvement of the university community. We must identify quality indicators, analyze our
resource balance, assess our quality, and take the actions required to improve quality. To
guarantee our ability to capture the resources required to meet the best competitors in America,
we must pursue quality and productivity with a relentless enthusiasm.






















In this era of constant competition for scarce resources, quality is of the utmost importance to
universities. We have no immunity from competition for resources, but we cannot rely on
profit-and-loss mechanisms to understand ourselves. Instead, we focus on the quality of our
programs and evaluate them within the context of competing institutions. Evaluation is
necessary to identify and enhance quality within our mission. Without mission and purpose
and without a means of assessing quality, a university stagnates. We describe in this paper the
University of Florida's mission, its community and constituencies, how we can evaluate quality,
and our planned process of continual improvement.1

Mission
The University of Florida belongs to an ancient tradition of great universities. We participate
in an elaborate conversation among scholars and students that extends over space and time,
linking the experiences of Western Europe with the traditions and histories of all cultures, that
explores the limits of the physical and biological universes, and that nurtures and prepares
generations of educated people to address the problems of our societies. While this university
recognizes no limits on its intellectual boundaries, and our faculty and students remain free to
explore wherever the mind and imagination lead, we live in a real world whose constraints limit
what we can do. Out of the conflict between our universal intellectual aspirations and the
limitations of our environment comes the definition of the university's goals.
American colleges and universities share the fundamental educational mission of teaching
students. The undergraduate experience, based in the arts and sciences, remains at the core of
higher education in America. The formation of educated people, the transformation of mind
through learning, and the launching of a lifetime of intellectual growth: these goals remain
central to every university. This undergraduate foundation of American higher education has
grown more complex as the knowledge we teach has grown more complex. Where once we had
a single track through the arts and sciences leading to a degree, we now have multiple tracks
leading to many degrees in arts and sciences as well as in a variety of professional schools. Yet
even with many degrees, American university undergraduate education still rests on the
fundamental knowledge of the liberal arts and sciences.
In our academic world we recognize two rather imprecisely defined categories of higher
education: colleges and universities. The traditional American college specializes in a carefully
crafted four-year undergraduate program, generally focused almost entirely on the arts and
sciences. Universities extend the range of this undergraduate education to include advanced or
graduate study leading to the-Ph.D. Most American universities also include a variety of
undergraduate and graduate professional programs, master's degree programs, and the like.
The University of Florida shares these traditions. As an American university, we have a major
commitment to undergraduate education as the foundation of our academic organization, and
we pursue graduate education for the Ph.D. as well as many other advanced degrees in
professional fields.
We are, in addition, a major, public, comprehensive, land-grant, research university. Each of
these adjectives defines one of our characteristics, and, through frequent repetition, this


University Mission, Quality, and Performance

The Florida Quality Evaluation Project

John V. Lombardi and Elizabeth D. Capaldi
University of Florida
1992






Mision Quliy an Fefouac Th_ Fird nlt vatn v


description takes on the style of a ritual incantation: rhythmic, reverent, and infrequently
examined. What, then, does each of these key words mean?
Major. Here, at the head of the list, we find one of our most important aspirations. We will
be, we must be, and we are a major university. We define ourselves in comparison to the best
universities we can find. We need not be the absolutely unambiguously best, but we must be
among the best universities in the world. Exact ranking of the best universities is a meaningless
exercise, but most of us can name 60 great universities. By whatever indicator of quality we
choose, our university should fall into this group. If we define a group of universities who share
our adjectives (major, public, comprehensive, land-grant, research), then we fall into a group of
perhaps the best 15 in this country. This definition of major need not be very precise. What
matters is not the precision of the measuring scale but the inclusion of our university in the
group.
Public. We exist thanks to the commitment and investment of the people of the state of
Florida. Generations of tax dollars constructed the facilities we enjoy and have paid the major
portion of our operating budget. The graduates of this institution, educated with tax dollars,
provide the majority of our private funtling. Our state legislators created the conditions that
permit our faculty to educate our students, pursue their research, conduct their clinical practice,
and serve their statewide constituencies. We exist, then, within the public sector, responsible
and responsive to the needs of the citizens of our state. The obligations we assume as a public
university determine many of our characteristics.
We have many more undergraduates than graduates, we respond quickly to the needs of the
state's economy, we accommodate complex linkages with other state universities and
community colleges, and we operate in cooperative symbiosis with our state's media. We also
experience an often too-close interaction with the political process. Private universities, that
have a difference profile, do not respond in the same ways to these issues. We, as a public
university, must maintain close, continuous, and effective communication with our many
publics.
Comprehensive. This adjective recognizes the universal reach of our pursuit of knowledge.
As a matter of principle, we exclude no field from our purview. We believe that our approach
to knowledge and learning, to understanding and wisdom, requires us to be ready to examine
any field, cultivate any discipline, and explore any topic that offers insight or intellectual tools.
Resource limits, human or financial, may constrain us from cultivating one or another academic
subspecialty, but we accept, in principle, no limit on our field of view. Even when we struggle
with budget problems and must reduce a program or miss an intellectual opportunity, we do so
only to meet the practical constraints of our current environment. We never relinquish
commitment to the holistic pursuit of knowledge.
Land-grant. Florida belongs to the set of American universities whose mandate includes a
commitment to the development and transmission of practical knowledge. As one of the land-
grant universities identified by the Morrill Act of 1862, Florida has a special focus on agriculture
and engineering and a mandate to deliver the practical benefits of university knowledge to
every county in the state. In our university, the Institute of Food and Agricultural Sciences and
the College of Engineering respond to this definition most obviously; but over time, the entire
university has come to recognize its commitment to translating the benefits of abstract and
theoretical knowledge into the marketplace to sustain the economic growth that supports us all.
This commitment permeates the institutional culture and defines us as one of some 72 such
institutions in America. The land-grant university is, of course, a peculiarly American invention
and captures one of the powerful cultural beliefs of our country: that knowledge passes the test
of utility by remaining vitally connected to industry and commerce.
Research. Research defines this university. Our faculty dedicate themselves not only to the
bedrock function of education, not only to the land-grant function of service, but equally to the
essential activity of research.
By research we mean the effort to expand our understanding of the natural world, the world
of the mind, and the world of the senses. We define research to include the theoretical
abstractions of the mathematician, the experimental discoveries of the geneticist, the insights of


Mbs"lon, Quality, d Performance


The Flnrld OnnHtv F~Ysm~HRn Pmirrt









the semiotician, the re-creations of the historian, or the analysis of the anthropologist. We define
research to capture the business professor's analysis of economic organization, the architect's
design, and the musician's interpretation or the artist's special vision. Research by agronomists
improves crops, and research by engineers enhances materials. Medical and clinical research
cure and prevent disease. The list of research fields continues as endlessly as the intellectual
concerns of our faculty and the academic vision of our colleges.
We must publish university research, whatever the field. The musician who never performs,
the scientist whose work never appears for review by colleagues, the historian whose note
cards never become a book may have accomplished much, but their accomplishments remain
incomplete. When we say research, we mean research and creative activity that contributes to
the international public conversation about the advancement of knowledge.2

University Community
The university community serves a complex constituency of individuals, groups, and
organizations. From national, state, and local governments to research agencies, foundations,
and corporations; from regents, legislators, and citizens to parents, alumni, and the general
public; universities have many constituents. Our university community of faculty, staff, and
students serves these constituents in complex ways. If we pursue our mission with quality and
effectiveness, our constituencies will support us.
The university community commits its energy and talent to the programs that deliver
teaching, research, and service-our three-part mission. Many members of the university
community participate in all three parts of the mission. Graduate students do research, teach
students, and provide public service at the same time they learn. Faculty, who offer the highest
level of instruction, often learn from their colleagues in other fields or from staff with technical
expertise. Staff who support the teaching program also participate in the university's research
activities.
Many of the university's constituents, of course, also participate directly in the university's
life as they attend concerts or receive medical care, as they contribute the time and treasure to
support our work or serve on academic advisory committees. Others benefit indirectly when
they buy quality food made possible by university agricultural research or enjoy technology
derived from university science.4
If we in the university community of faculty, students, and staff maintain our commitment to
quality, we will serve our many constituencies well, and we will compete successfully for the
resources our work requires.

Quality Evaluation
Universities organize themselves and evaluate their accomplishments by focusing on
programs. We bring quality people together in programs and evaluate our progress and our
quality as programs rather than as individuals. Of course, we want to keep and support first-
rate individuals, and we have many methods of identifying individual achievement through
grades and degrees for students, and promotions and other recognition for faculty and staff.
Nonetheless, our success as an institution depends on our programs. We want a first-rate
engineering college, an outstanding chemistry department, a nationally ranked medical college,
a first-class agricultural college, a nationally competitive history department, and a superior
journalism college-just to mention a few programs. Although we cannot have these exemplary
programs without quality people, it is the coordinated and structured work of faculty, students,
and staff that produces quality programs. To improve and enhance quality, we must focus our
attention on programs.
Some of our constituents, who see themselves as consumers of educational products, try to
evaluate universities as if education were a consumer good. This one-dimensional perspective
fails to capture the complex accomplishments of universities and cannot serve as a basis for
understanding the quality of our multi-dimensional institutions.5
Some within our university community see the institution as being so complex with so many
overlapping goals and constituencies that the evaluation of quality programs becomes


Mission. Quality, and Performnce


The Florida Quality Evaluation Project









impossible. This view holds that universities most resemble living organisms that grow,
mature, and prosper, adapting to their changing environment as needed in a reactive fashion.
The enhancement of quality, in this view, becomes the result of long-term, almost Darwinian,
selection. The best will survive and prosper; the mediocre will stagnate and decline. Although
recognizing the complexity of universities and emphasizing the long-term development of many
university programs, this perspective fails to understand how the evaluation and improvement
of quality give the university a critical advantage in the competition for the scarce resources it
needs.6
Neither of these approaches serves the university well as we evaluate quality to improve our
effectiveness. Instead, we identify and measure quality and productivity by comparing
ourselves to the standard of the nation's best universities of our type. For the University of
Florida, our most immediate reference group includes the public universities in the American
Association of Universities and, for some aspects of our institution, the public land-grant
universities of the AAU. We refine these measures of university quality to assess our
performance at the college and division level with reference to comparable institutions. So, for
example, the College of Medicine would, within its mission, identify national counterparts for
comparison. Those should begin with the colleges of medicine in our institutional reference
group, the AAU public institutions, but may well include other competitive colleges of
medicine, whether in public or private universities. This recognizes that our colleges must
compete nationally and internationally for faculty, students, staff, and resources, and that the
relevant comparisons may involve colleges of medicine from private universities such as
Washington University (St. Louis) or Johns Hopkins or from non-land-grant public universities
such as the University of California at San Francisco or UCLA. 7
Within colleges, quality comes from departments or programs. The department or program
serves as the fundamental unit for the improvement of quality. Each department or program
can assess its quality and evaluate its performance in comparison to counterparts at comparable
institutions. Here, too, we compare first with programs in our institutional reference group: the
AAU public universities. Departments may, however, recognize other important reference
programs in private universities or other public institutions, and we can add these into the
comparative evaluation. 8
Within each level-university, college, or department--the individuals who produce the
quality and have the expertise must design the criteria for assessment. Faculty, students, and
staff design the evaluation to reflect their expertise. Each unit designs its own assessment of
quality within the general structure of this project. If the general structure fails to accommodate
important perspectives on our programmatic quality, we will change the structure.
While these examples address the academic colleges, the same comparative methodology
applies to the academic and administrative support programs of the university. Whether we
address financial aid, the registrar, student services, accounting, or purchasing; whether we look
at the president's office or the foundation; the athletic program or the bookstore; every one of
our programs will find that comparison and assessment enhance the quality of performance.9

The Project
For this project to succeed, we each need to understand our mission, community, and
constituencies; identify the indicators of our quality and reference them to national standards;
understand the balance of resources provided to support quality; and recognize the constraints
on improving quality.
A dear understanding of mission, community, and constituencies helps define the program's
approach to teaching, research, and service. Does a program focus primarily on teaching,
research, or service, or does it balance the three parts of our mission? We do not need highly
detailed mission statements, only a dear expression of the contribution a college or department
expects to make to the university's mission. In outlining a department or program mission, we
also help define its community and its external constituents. To assess our quality we need to
recognize which parts of the university community or which of the university's many
constituencies we serve.10


Mission quality. and Performance


The Florida Ouality Evaluation Prolect









The identification of indicators of quality and the collection of information about the quality
produced by the program make explicit the criteria we use to judge quality. Fundamental to
this effort is the conviction that unless we can define quality, we cannot improve it.11 We may
identify research quality through peer-reviewed sources such as publications in specialized
journals. Some departments look for books published by highly regarded presses and use
journal reviews to assess quality.. Other programs may use prestigious awards, scholarship,
prizes, fellowships, academic elections, and other marks of distinction. In some high-
technology departments, we might recognize research quality through the award of patents and
the achievement of licensing agreements. In the fine arts we might assess major refereed
exhibits, reviewed performances, prizes, and other artistic recognition. In financial aid or
purchasing we might recognize quality through the speed, accuracy, and courtesy of service.
Teaching requires much different measures of achievement and quality than research.
Teaching core courses within the general distribution requirements and teaching advanced
courses within a major or a graduate program require different assessment techniques.
Traditional student evaluation, alone, cannot adequately capture a program's success in
education. We need, in addition, a form of peer review and other quality-improvement
techniques that can approximate the measures we use to evaluate research. Laboratory sections,
foreign language study, studio art classes, and music studios call for different quality
assessments than lectures or seminars in history, for example. We need to know how well those
who employ our graduates think we have taught them; and we need to know how well those
who were our students some years ago assess the quality of their education. Those who
participate in the teaching and learning need to design the system that lets us understand and
improve the quality of the education we provide.
Equally challenging, we must understand the quality of the service we do. Some parts of our
university understand the service part of their mission nearly, and they have carefully
developed techniques for evaluating its quality. Agricultural research stations and extension
programs directly serve a wide constituency throughout the state and have established methods
for addressing quality. The colleges that serve patients in clinical settings have recognized
systems of quality control. Programs that provide faculty and staff to serve on national
commissions, edit journals, serve as association officers, or participate in review panels have less
nearly defined measures of quality. We need to understand the importance of service on major
university committees, participation in departmental searches, and work with student life. The
university and each department or program needs to recognize service quality.
In defining and assessing quality we also pay attention to how much quality our programs
can produce. If we have superior quality but we produce very little of it, we need to understand
why: perhaps we have invested too few resources, perhaps we need to improve our
organization or our management. If we have a fine instructional program but few students, we
may need to review our curriculum. If we have a superior research program producing few
results, we may need better facilities or support. Everyone wants us to do more and better, but
we cannot forever do more and better for less. For our university as a whole and for each
program, we use the national reference to place our quality and resources into perspective. The
university, its departments, and its programs have already demonstrated that they can produce
nationally competitive programs when provided reasonable support. We must carefully
calibrate our quality and our success at the national level because we compete nationally for
faculty, students, and resources. When our students graduate, they too must compete
nationally, even internationally, in whatever career they pursue. We cannot know how well we
do unless we can compare our quality and our resources with those of our national reference
group.1
The resource balance establishes the limits and provides the opportunities to excel. No
university or its programs, whether in pursuit of teaching, research, or service, succeed without
resources. A resource balance for the university or a program consists of all the resources it
produces and consumes. We earn funds from the state, grants and contracts, tuition and fees,
gifts and endowments, and auxiliary enterprises. To understand our resources, we need to
identify these sources. Not every program enjoys support from every possible source of funds.


Misdon OalHtv. and Performance


The Florida Onalit Evaluation Project






Miuuaoii. Quality. and Performance The Florid. fln.fltv F~vnlnntInn PmI.~


Some programs earn contract and grant funds; some do not. Some attract endowment and gift
support; others do not. Some have many students and earn substantial tuition and fee revenue;
others do not. The resource balance begins with a table that shows all the resources earned by
the program.
Once we know exactly what resources the program earns, we then determine the resources
consumed by the program. We pay salaries, buy supplies, pay for equipment, purchase books,
pay fellowship stipends, and incur other expenses. The resource balance tells us how we spend
our money.
The raw numbers of the resource balance, the funds earned and consumed, help us
understand only the simplest of relationships. We also need to know something about the
structure of the colleges or units involved. As part of the resource balance, we calculate some
simple ratios. We derive these from established accounting and budget categories rather than
invent new data. Even with the limitations of these categories, we can identify colleges that
provide strong support for each faculty member, from which we can expect to achieve
substantial quality and productivity. We can find colleges with low support who have,
nonetheless, produced substantial quality and quantity. We can see how we have distributed
the task of teaching undergraduates. And, we can make comparisons with other universities
and their programs.
None of these quantitative data speak directly to quality. Instead, they help us put the
relative success of each of our units in context. If we have a college with good quality and we
want it to improve, then we need to look at the resources spent by that college. If the resources
appear inadequate when compared with our reference group, then we need to reevaluate how
we distribute our resources. We may find that a college is producing an unusually high level of
quality, given the low level of resources it earns. Thus, the resource balance serves to facilitate
the conversation about the improvement of quality and about the expectations for productivity.
Out of this conversation, we can identify where we can best use new resources to enhance our
quality and our productivity.13
Constraints on improving quality come in many forms. Sometimes we suffer from something
as simple as inadequate space, equipment, or salary support. Often, however, we contend with
unnecessary rules or restrictions, foolish bureaucratic complexity, poor communication and
coordination among related programs, poor articulation between support groups and academic
units, and unsound statewide policies or regulations. Even if we cannot remove all constraints,
precise identification allows us to minimize their effect on quality.14
In managing these constraints we refer to the State University System's planning exercises.
Like the activities of various accrediting agencies, these plans set limits and constrain the
university's operations on five-year or other chronological cycles. Constructed and developed
primarily for state-wide programmatic planning purposes, the state's plans establish the limits
of competition among the state universities in Florida. Required by law and regents rule, we
pay dose attention to these plans and work to expand the limits established by them.15
Our project sees improvement in quality as a continuous and incremental activity. Our
success depends on our own determination and ability as well as the resources we can bring to
our programs. The university's emphasis on quality, evaluation, and improvement will make us
as successful as possible.

Process
This quality evaluation project may appear as a static exercise, a snapshot of institutional life
taken at intervals. This view distorts the process of university life. Teaching, research, and
service never start or stop, have no beginning or end, but instead operate on continuous, never-
ending cycles.
Each program and each part of the mission involves processes that have cycles of
productivity and achievement. Many university activities have multiple time cycles. The
teaching process has at least three or four easily identifiable cycles. One involves the individual
course experience, a semester in length. Another involves the bachelor of arts or science life
cycle of four or five years. A third includes the professional degree cycle. And a fourth might


Mission. Quaity. Pmd Performnce


The Flarldr CkuWv Evr~ndbn PmM









be the long-term impact of instruction evaluated five or ten years after graduation. We must
accommodate each of these cycles in any evaluation system, and programs need to approximate
these life cydes in their evaluation, wherever the differences in time sequence make a difference
in quality and productivity.
Other processes cross program and mission boundaries. An undergraduate degree involves
a wide range of program and operational units from the academic departments through the
registrar to the academic advising system. Accounting and budgeting and other bureaucratic
systems may improve or discourage the effectiveness of many research programs. We need to
evaluate these processes to identify where we can improve their quality.
Every university encourages the development of auxiliary operations that provide
supporting services but do not participate directly in meeting the institution's three-part
mission. An intercollegiate athletic program serves many useful purposes but does not deliver
teaching, research, or service in the traditional sense. Nonetheless, many people in the
university community and among our constituencies find this auxiliary program of great value.
The university operates a bookstore that provides essential materials to the university
community but does not itself serve a teaching, research, or service function.
We evaluate each of these units using the same techniques developed for other programs.
However, auxiliary enterprises will have a positive resource balance. The intercollegiate athletic
program and the bookstore must at least break even and should earn a surplus. To do
otherwise diverts resources from the university's primary mission.17

Constant Commitment
Quality and effectiveness require constant and permanent attention from everyone in the
university. The price of a high-quality university is constant commitment to evaluation and
improvement. Academic life exists within a framework of testing, reviewing, evaluation, and
challenge. We approach this process with much greater expertise and experience than many
other complex organizations. We will approach the evaluation and improvement of our
programs and our institution with the same commitment we now apply to the evaluation of
colleagues, students, and research results.18
The Florida Quality Evaluation Project needs the continuing involvement of the university
community. We must identify quality indicators, analyze our resource balance, assess our
quality, and take the actions required to improve quality. We expect to improve this process as
we gain experience, we will report our results annually, and we will constantly improve our
quality.19
Program quality and productivity, the critical elements of a nationally competitive
university, require a permanent commitment. Quality happens every day in the work that each
of us does and in the decisions we make. The university already has people and programs of
high quality. To guarantee our ability to capture the resources required to meet the best
competitors in America, we must pursue quality and productivity with a relentless enthusiasm.
Universities, like other major American institutions, find themselves under suspicious
scrutiny by many of our most loyal constituencies. We can argue about that critique, or we can
demonstrate quality and productivity so dearly that our success speaks for itself. We labor
under many constraints, but we have many opportunities. As we succeed in overcoming or
removing the constraints that inhibit us and as we capture the opportunities that quality and
productivity bring, we will do ourselves and our university proud.


Mbrio Onality. mnd Performance


The Florid OQuality Evaluatlon Project






Mison, Quality, and Performance


The Florida Quality Evaluation Project


APPENDICES


Appendix A:


Appendix B:.

Appendix C


Resource Balance Categories
Sample Data Tables for Resource Balance

Examples of Teaching, Research, and Service Criteria

Nine Measures in State University System Accountability Plan









Appendix A: Resource Balance Categories


The resource balance is computed at the college level. The data categories are as follows for
colleges in the Educational and General budget:

Sources: State money (generated student fees, approximate formula funding using legislative
formula and actual incremental funding values, fundable student credit hours, formula
generations not funded by fees, Quality Improvement and Centers of Excellence, Undergraduate
Improvement, Scientific and Technical equipment, other special appropriations, and other state
funding); research funds (generated indirect cost recoveries, indirect cost allocated or awarded
through DSR, direct costs); private support; other (grants and donations trust fund, auxiliary
trust funds, auxiliary overhead, etc.).

Expenditures: State funds, research (EIES and sponsored research), private, auxiliaries and
other.

Available Faculty: Total FTE; FTE supported by state funds, by contracts and grants, and by
other sources; academic effort as reported for the college on Academic Activity Reports

Faculty Effort: Student credit hours produced; contact hours produced; course sections
taught by faculty rank, by course level, and by section size for each college.

For IFAS sources also include research incidental, extension incidental, Federal-formula
generated research money, Federal-formula generated extension money, and USDA Federal
Extension Service funded special projects.

For the Health Center sources also include clinical fee revenue, VA contributions, Shands
contributions, and VMTH fees.


Mibsion. Outy. and Performance


The Florida Oality Evaluation Project







Mission Quality. md Performance


Sample Data Tables for Resource Balance


The Florda Quality Evaluation Project





Educational & General
1990-91









ACADEMIC
ARCHITECTURE
Percent of Total

BUSINESS ADMINISTRATION
Percent of Total

CENTERS AND INSTITUTES
Percent of Total

CENTERS AND INSTITUTES TYPE I
Percent of Total

DIVISION OF CONTINUING EDUCATION
Percent of Total

EDUCATION
S Percent of Total

ENGINEERING
Percent of Total

FINE ARTS
Percent of Total


HEALTH & HUMAN PERFORMANCE
Percent of Total

JOURNALISM & COMMUNICATIONS
Percent of Total


Percent ol Total


LIBERAL ARTS & SCIENCES
Percent of Total

Total Academic
Percent of Total


Resources
Generated Gross Approximate Formula Balance Indirect IC I/C Gen. Research Auxiliaries,
Student Formula Formula Not Funded QIPI S&T Other d State Cost (VIC) Allocations > (<) Direct Private MGG & Total
Fees Funding Funding by Fees COE UIP OCO Special Funding Generations & Awards AllocJAwrds. Costs Support Other Resources
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
[Non-Add] [Non-Add] [Non-Add]

























*i


Page 1







Educational & General
1990-91


ACADEMIC
ARCHITECTURE
Percent of Total

BUSINESS ADMINISTRATION
Percent ol Total

CENTERS AND INSTITUTES
Percent of Total

CENTERS AND INSTITUTES -TYPE I
Percent of Total

DMSION OF CONTINUING EDUCATION
Percent of Total

EDUCATION
Percent of Total

ENGINEERING
Percent of Total

FINE ARTS
Percent ol Total

HEALTH & HUMAN PERFORMANCE
Percent of Total

JOURNALISM & COMMUNICATIONS
Percent ol Total

LAW
Percent of Total

LIBERAL ARTS a SCIENCES
Percent of Total

Total Academic
Percent of Total


Expenditures
EIES & Auxiliaries.
State Sponsored MGG & Total
Funds Research Private Other Expenditures
17 18 19 20 21


Net
Ratio of
StateS to
Non-State
22















































Page 2


Academic Effort
Manyeare Expended/Percentages
Public Acad.
Instruction Advising Research Service Admin. Gov. Total
23 24 25 26 27 28 29


Available Faculty
FTE Positions from Final APF
State Research Other Total
Funded Funded Faculty Faculty
30 31 32 33




Educational & General
1990-91

Credit Hours Produced
Fees Total State Research Private Total Fundable Student Credit Houre (SC
Generated Support Funding Funding Funding Thesis
per Fac FTE per Fac FTE per Fac FTE per Fac FTE per Fac FTE Lower Upper Graduate & Dis. Total
34 35 36 37 38 39 40 41 42 43


ACADEMIC
ARCHITECTURE
Percent of Total

BUSINESS ADMINISTRATION
Percent of Total

CENTERS AND INSTITUTES
Percent of Total

CENTERS AND INSTITUTES TYPE I
Percent of Total

DMSION OF CONTINUING EDUCATION
Percent of Total

EDUCATION
Percent ol Total

ENGINEERING
Percent of Total

FINE ARTS
Percent of Total

HEALTH & HUMAN PERFORMANCE
Percent o Total

JOURNALISM & COMMUNICATIONS
Percent of Total

LAW
Percent of Total

LIBERAL ARTS A SCIENCES
Percent of Total

Total Academic
Percent of Total


Page 3







Educational & General
1990-91

Instructional Contact Hours (ICH)
Lower Upper UIP Graduate Thes/Dissertation Total
199X-8X 199X-OX 199X-9X 199X-9X 199X-9X 199X-9X 199X-9X 199X-X 19GX-OX 199X-9X l10X-9X Over
Required Actual Required Actual Required Required Actual Required Actual Required Actual (Short)
46 47 48 49 50 51 52 53 54 55 56 57


ACADEMIC
ARCHITECTURE
Percent of Total

BUSINESS ADMINISTRATION
Percent of Total

CENTERS AND INSTITUTES
Percent ol Total

CENTERS AND INSTITUTES TYPE I
Percent of Tolal

DMSION OF CONTINUING EDUCATION
Percent of Total

EDUCATION
Percent of Total

ENGINEERING
Percent of Total

FINE ARTS
Percent of Total

HEALTH & HUMAN PERFORMANCE
Percent of Total

JOURNALISM A COMMUNICATIONS
Percent ol Total

LAW
Percent of Total

LIBERAL ARTS & SCIENCES
Percent of Total

Total Academic
Percent of Total


Page 4




Educational & General
1990-91

COURSE SECTIONS TAUGHT (Fixed Credit)
by Faculty Rank by Course Level
Associate Assistant Instructor Graduate Adjunct
Professor Professor Professor Other Faculty Assistant Faculty Other Total Lower Upper Graduate Total
58 59 60 61 62 63 64 65 66 67 68 69


ACADEMIC
ARCHITECTURE
Percent of Total

BUSINESS ADMINISTRATION
Percent of Total

CENTERS AND INSTITUTES
Percent of Total

CENTERS AND INSTITUTES -TYPE I
Percent of Total

DIVISION OF CONTINUING EDUCATION
Percent of Total

EDUCATION
Percent of Total

ENGINEERING
Percent of Total

FINE ARTS
Percent of Total

HEALTH & HUMAN PERFORMANCE
Percent of Total

JOURNALISM a COMMUNICATIONS
Percent of Total

LAW
Percent of Total

LIBERAL ARTS & SCIENCES
Percent of Total

Total Academic
Percent of Total


Page 5







Educational & General
1990-91









SUPPORT

Academic Support
ACADEMIC AFFAIRS
Percent of Total
ADVANCEMENT
Percent of Total
FLA. MUSEUM OF NATURAL HISTORY
Percent of Total
GRADUATE SCHOOL
Percent of Total
LBRARIES
Percent of Total
MILITARY SCIENCE
Percent of Total

NORTHEAST REGIONAL DATA CENTER
Percent of Total

OFFICE OF INSTRUCTIONAL RESOURCES
Percent of Total

REGISTRARS OFFICE
Percent of Total

PRESIDENTS OFFICE
Percent of Total
S.U.S. PRESS
Percent of Total

UNIVERSITY RELATIONS
Percent of Total

RESEARCH & DEVELOPMENT
Percent of Total


Resources
Generated Gross Approximate Formula Balance Indrect C C Gen. Research Auxiliaries
Student Formula Formula Not Funded QIPI S&T Other of State Cost (I/C) Allocations > (<) Direct Private MGG & Total
Fees Funding Funding by Fees COE UIP OCO Special Funding Generations & Awards AllocJAwrds. Costs Support Other Resources
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
[Non-Add] [Non-Add] [Non-Add]


Page 6





Educational & General
1990-91


SUPPORT

Academic Support
ACADEMIC AFFAIRS
Percent of Total
ADVANCEMENT
Percent of Total
FLA. MUSEUM OF NATURAL HISTORY
Percent of Total
GRADUATE SCHOOL
Percent of Total
LIBRARIES
Percent of Total
MILITARY SCIENCE
Percent of Total
NORTHEAST REGIONAL DATA CENTER
Percent of Total

OFFICE OF INSTRUCTIONAL RESOURCES
Percent ol Total

REGISTRAR'S OFFICE
Percent of Total

PRESIDENTS OFFICE
Percent of Total
S.U.S. PRESS
Percent of Total
UNIVERSITY RELATIONS
Percent ol Total
RESEARCH & DEVELOPMENT
Percent of Total


Expenditures
EIES & Auxiliaries,
State Sponsored MGG & Total
Funds Research Private Other Expenditures
17 18 19 20 21


Net
Ratio of
State S to
Non-State
22


Academic Effort
Manyears Expended/Percentages
Public Acad.
Instruction Advising Research Service Admin. Gov. Total
23 24 25 26 27 28 29


Available Faculty
FTE Positions from Final APF
State Research Other Total
Funded Funded Faculty Faculty
30 31 32 33


Page 7







Educational & General
1990-91


SUPPORT

Academic Support
ACADEMIC AFFAIRS
Percent of Total

ADVANCEMENT
Percent of Total

FLA. MUSEUM OF NATURAL HISTORY
Percent of Total
GRADUATE SCHOOL
Percent of Tolal

LIBRARIES
Percent of Total

MILITARY SCIENCE
Percent of Total

NORTHEAST REGIONAL DATA CENTER
Percent of Total

OFFICE OF INSTRUCTIONAL RESOURCES
Percent of Total

REGISTRAR'S OFFICE
Percent of Total

PRESIDENT'S OFFICE
Percent of Total

S.U.S. PRESS
Percent of Total

UNIVERSITY RELATIONS
Percent of Total

RESEARCH & DEVELOPMENT
Percent of Total


Fees Total State Research Private Total
Generated Support Funding Funding Funding
per FacFTE perFacFTE perFacFTE perFacFTE per FacFTE
34 35 36 37 38


Credit Hours Produced
Fundable Student Credit Hours (SCH)
Thesis
Lower Upper Graduate & Dis. Total
39 40 41 42 43


SCH State
per Support
Fac FTE per SCH
44 45


Page 8





Educational & General
1990-91

Instructional Contact Hours (ICH)
Lower Upper UIP Graduate Thesis/Dissertation Total
199X-OX 199X-8X 199X-9X 199X-9X 199X-9X 1B9X-9X 199X-9X 199X-9X 1IGX-9X 199X-9X 199X-gX Over
Required Actual Required Actual Required Required Actual Required Actual Required Actual (Short)
46 47 48 49 50 51 52 53 54 55 56 57


SUPPORT

Academic Support
ACADEMIC AFFAIRS
Percent of Total
ADVANCEMENT
Percent of Total

FLA. MUSEUM OF NATURAL HISTORY
Percent of Total
GRADUATE SCHOOL
Percent ol Tolal
LIBRARIES
Percent of Total
MILITARY SCIENCE
Percent ol Total
NORTHEAST REGIONAL DATA CENTER
Percent of Total

OFFICE OF INSTRUCTIONAL RESOURCES
Percent ol Total

REGISTRAR'S OFFICE
Percent ol Total
PRESIDENTS OFFICE
Percent of Total
S.U.S. PRESS
Percent of Total
UNIVERSITY RELATIONS
Percent of Total
RESEARCH & DEVELOPMENT
Percent of Total


Page 9







Educational & General
1990-91

COURSE SECTIONS TAUGHT (Fixed Credit)
by Faculty Rank by Course Level
Associate Assistant Instructor Graduate Adjunct
Professor Professor Professor Other Faculty Assistant Faculty Other Total Lower Upper Graduate Total
58 59 60 61 62 63 64 65 66 67 68 69


SUPPORT

Academic Support
ACADEMIC AFFAIRS
Percent of Total
ADVANCEMENT
Percent of Total

FLA. MUSEUM OF NATURAL HISTORY
Percent of Total
GRADUATE SCHOOL
Percent of Total
LIBRARIES
Percent of Total
MILITARY SCIENCE
Percent of Total
NORTHEAST REGIONAL DATA CENTER
Percent of Total

OFFICE OF INSTRUCTIONAL RESOURCES
Percent of Total

REGISTRAR'S OFFICE
Percent of Total

PRESIDENTS OFFICE
Percent of Total
S.U.S. PRESS
Percent of Total
UNIVERSITY RELATIONS
Percent of Total

RESEARCH & DEVELOPMENT
Percent of Total


Page 10





Educational & General
1990-91








Student Support
STUDENT AFFAIRS
Percent of Total
CAREER RESOURCE CENTER
Percent of Total

STUDENT HOUSING
Percent of Total

University Support
ADMINISTRATION. SECURITY,
PHYSICAL PLANT, HEALTH &
SAFETY, WORKING CAPITAL
Percent of Tolal

TOTAL SUPPORT
Percent of Total

GRAND TOTAL
Percent of Total


Resources
Generated Gross Approximate Formula Balance Indirect IC I/C Gen. Research Auxiliaries,
Student Formula Formula Not Funded QIPI S&T Other of State Cost (IC) Allocations (<) Direct Private MGG & Total
Fees Funding Funding by Fees COE UIP OCO Special Funding Generations & Awards AlbcJAwrds. Costs Support Other Resources
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
[Non-Add] (Non-Add] [Non-Add]
























NOTES:
BfL Notes and Comments:
1 GENERATED STUDENT FEES Matriculation and Out-of-State fee "assessments" were generated based on actual student credit hours (SCH) from the
Student Data Course File (SDCF) and actual per-credit-hour fee charges for the appropriate year and term.
2 GROSS FORMULA FUNDING Calculations applied to the legislative funding formula and funding values to actual, fundable student credit hours.
3 APPROXIMATE FORMULA FUNDING Calculated as 76.5% multiplied by Gross Formula Funding (col. 2).
4 FORMULA NOT FUNDED BY FEES (col. 3 less cl. 1).
S QIP & COE Quality Improvement Program and Centers of Excellence.
6 UIP Undergraduate Improvement Program special appropriations.
7 S&T Scientific & Technical maintenance funds and special OCO.
OTHER SPECIAL Lbrary Resources, Plant Operations & Maintenance, Fee Waivers, etc.
9 BALANCE OF STATE FUNDING Total Expenditures from State Funds (col. 17) less cols. (1+4++6++7+8).
10 INDIRECT COST GENERATIONS Total actual indirect cost assessments (object code 810010) from the UF General Ledger.
11 INDIRECT COST ALLOCATIONS & AWARDS -Indirect cost recoveries allocated or awarded per Division of Sponsored Research.
12 VC GENERATIONS greater than (less than) ALLOCATIONS/AWARDS (col. 10 less col. 11).
13 RESEARCH DIRECT COSTS Total research expenditures (Sponsored Research Trust Fund & EIES) less Allocations/Awards (cal. 18 less col.1 1).
14 PRIVATE SUPPORT Total Private Transfers to UF and 'Other Expense' per the University of Florida Foundation (col. 19).
1 OTHER Grants & Donations Trust Fund, Auxiliary Trust Funds, auxiliary overhead, etc. (col. 20).
18 TOTAL RESOURCES Total of cola. (1. 4 thru 9,11, 13thru 15).

Page 11







Educational & General
1990-91








Student Support
STUDENT AFFAIRS
Percent of Total
CAREER RESOURCE CENTER
Percent of Total

STUDENT HOUSING
Percent of Total

University Support
ADMINISTRATION, SECURITY,
PHYSICAL PLANT, HEALTH &
SAFETY, WORKING CAPITAL
Percent of Total

TOTAL SUPPORT
Percent of Total

GRAND TOTAL
Percent of Total


Page 12


Expenditures
EIES & Auxiliaries.
State Sponsored MGG & Total
Funds Research Private Other Expenditures
17 18 19 20 21


Academic Effort
Manyears Expended/Percentages
Public Acad.
Instruction Advising Research Service Admin. Gov. Total
23 24 25 26 27 28 29


Available Faculty
FTE Positions from Final APF
State Research Other Total
Funded Funded Faculty Faculty
30 31 32 33


NOTES:
Bef Notes and Comments:
17-21 EXPENDITURES Total fiscal year expenditures as recorded in the UF General Ledger. Private funds expenditures as reported by
the UF Foundation.
17 STATE FUNDS Total Expenditures from General Revenue. Incidental Trust, and Educational Enhancement (Lottery) Trust Funds.
22 RATIO OF STATE TO NON-STATE FUNDS Total state expenditures (col 17) divided by the total of research (col. 18). private (col. 19).
and other (col. 20) expenditures.
23-29 ACADEMIC EFFORT Academic manyears expended by activity as calculated from the Academic Activity Reports.
30-33 AVAILABLE FACULTY Authorized and allocated faculty FTE as reported on the end-of-year Authorized Position Fie (APF).





Educational & General
1990-91


Student Support
STUDENT AFFAIRS
Percent of Total

CAREER RESOURCE CENTER
Percent of Total

STUDENT HOUSING
Percent of Total

University Support
ADMINISTRATION, SECURITY,
PHYSICAL PLANT, HEALTH A
SAFETY. WORKING CAPITAL
Percent of Total

TOTAL SUPPORT
Percent of Total

GRAND TOTAL
Percent of Total


SCH State
per Support
Fac FTE per SCH
44 45


Page 13


Fees Total State Research Private Total
Generated Support Funding Funding Funding
perFacFTE perFacFTE perFacFTE perFacFTE perFacFTE
34 35 36 37 38


Credit Hours Produced
Fundable Student Credit Hours (SCH)
Thesis
Lower Upper Graduate & Diss. Total
39 40 41 42 43


NOTES:
BLf. Notes and Comments:
34 FEES GENERATED PER FAC FTE Student fees generated (col. 1) divided by Total Available Faculty FTE (cd. 33).
35 TOTAL STATE SUPPORT PER FAC FTE Total expenditures from State funds (col. 17) divided by Total Available Faculty FTE (col. 33).
36 RESEARCH FUNDING PER FAC FTE Total research expenditures (col. 18) divided by Total Available Faculty FTE (col. 33).
37 PRIVATE FUNDING PER FAC FTE Total expenditures from private sources (col. 19) divided by Total Available Faculty FTE (col. 33).
38 TOTAL FUNDING PER FAC FTE Total expenditures from all sources (col. 20) divided by Total Available Faculty FTE (col. 33).
39-43 CREDIT HOURS PRODUCED Total Fundable Student Credit Hours as reported on the Student Data Course File (SDCF).
44 SCH PER FAC FTE Total Student Credit Hours (col. 43) divided by Total Available Faculty FTE (col. 33).
45 STATE SUPPORT PER SCH Total State Funds (col.17) divided by Total Student Credit Hours (cal. 43).







Educational & General
1990-91








Student Support
STUDENT AFFAIRS
Percent of Total
CAREER RESOURCE CENTER
Percent ol Tolal
STUDENT HOUSING
Percent of Total

University Support
ADMINISTRATION. SECURITY,
PHYSICAL PLANT, HEALTH &
SAFETY. WORKING CAPITAL
Percent of Tolal
TOTAL SUPPORT
Percent of Total

GRAND TOTAL
Percent of Total


Page 14


Instructional Contact Hours (ICH)
Lower Upper UIP Graduate Thesis/Dissertation Total
199X-QX 199X-QX 199X-9X 199X-9X 199X-1X 199X-9X 1999X-9X IX-9X 19oX-9X 199X-9X i9X-oX Over
Required Actual Required Actual Required Required Actual Required Actual Required Actual (Short)
46 47 48 49 50 51 52 53 54 55 56 57




Educational & General
1990-91








Student Support
STUDENT AFFAIRS
Percent of Total
CAREER RESOURCE CENTER
Percent ol Total
STUDENT HOUSING
Percent of Total

University Support
ADMINISTRATION, SECURITY,
PHYSICAL PLANT. HEALTH &
SAFETY, WORKING CAPITAL
Percent ol Tota
TOTAL SUPPORT
Percent of Total

GRAND TOTAL
Percent of Total


COURSE SECTIONS TAUGHT (Fixed Credit)
by Faculty Rank by Course Level
Associate Assistant Instructor Graduate Adjunct
Professor Proessor Professor Other Faculty Assistant Faculty Other Total Lower Upper Graduate Total
58 59 60 61 62 63 64 65 66 67 68 69
























NOTES:
BeL Notes and Comments:
58-69 COURSE SECTIONS TAUGHT (FIXED CREDIT)- Actual fixed credit (excludes variable credit and Thesis/Dissertation) course sections taught as reported on
the Instructional Activity File.
58-65 BY FACULTY RANK Course sections tallied by the rank of the predominant Instructor of record.
66-69 BY COURSE LEVEL- Course sections tallied by the level of the course.














Page 15









Appendix B: Examples of Teaching, Research, and Service Criteria
An asterisk indicates the measure was used by the National Research Council in An
Assessment ofResearch-Doctorate Programs in the United States (Washington D.C.: National
Academy Press, 1982).

Sample Measures of Teaching Quality (relevance of measures depends on field)
* Surveys of alumni, employers, parents, and clients
* Peer evaluation of teaching quality
* Student evaluation of teaching quality
* For entering undergraduates, proportion accepted, SAT scores and high school class
standing of freshmen
* Percentage of undergraduates going on to postgraduate education
* Pass rates on professional licensure examinations
* Performance of our undergraduates on GRE, GMAT, LSAT, and other entrance exams
* Post-graduation employment of undergraduates
* Total degrees awarded, undergraduate and graduate
* Median number of years from first enrollment in graduate school to receipt of the
doctorate*
* Fraction of graduate students with fellowship or training grant support during graduate
education*
* For entering graduate students: median undergraduate GPA, median scores on entrance
exams (GRE, GMAT, LSAT etc.), percentage of applicants accepted, percentage of those
accepted who attend
* Fraction of Ph.D. graduates employed at time of graduation*
* Fraction of graduate students employed in Ph.D.-granting institutions at time of
graduation*
* Any other measure deemed relevant by a unit

Sample Measures of Research (relevance of measures depends on field)
* Number of publications in refereed journals (can be divided into top tier and other
journals)*
Number of books divided into sole authored, co-authored, edited, chapters (quality
measured by press, reviews received)
Creative endeavor: recognition in the competitive arena, e.g., regional or national
exhibitions, special recognition via respected awards, etc.
Number of grants
Total research expenditures*
* Research expenditures per faculty FTE
* Fraction of program faculty members holding grants from NIH, NSF or ADAMHA*
* Citations in science citation index*
* Peer evaluation of program*
* Invited presentations at national and international meetings
* Honors and awards received by faculty from national and international societies, election
to membership in National Academy of Science, National Academy of Engineering,
Institute of Medicine
* Appointment of faculty members to editorial review boards of national and international
peer-reviewed journals, and to grant review panels of national granting agencies
* Fellowships (Fulbright, Guggenheim, NEH, other), scholarly achievements (e.g, honorary
degrees, patents/ inventions, book/article awards)
* Any other measure deemed relevant by unit


Mission Quality, and Performance


The Florida Quality Evaluation Project






Mhuan.Oaklltv. and Perf~onnnce


The Floilda Ouailv Evalation Project


Sample Measures of Service (relevance of measures depends on field)
* Leadership in professional and scientific associations
* Participation in civic/community activities related to the profession including State or
Federal government committees
Leadership in college and university committees and activities
Participation in public and private K-12 programs
For IFAS, measures of quality extension activities
For Health Center, measures of quality clinical activities
Any other service activity deemed of importance by the unit






MiAson, Quality, and Performance


The Florida Quality Evaluation Project


Appendix C: Nine Measures in State University System Accountability Plan

1. Total student credit hours produced, by institution and by discipline;
2. Total number of degrees awarded, by institution and by discipline;
3. Total number of contact hours of instruction produced by faculty, institution, rank, and
course level;
4. Pass rates on professional licensure examinations, by institution;
5. Institutional quality as assessed by follow-up surveys of alumni, parents, clients, and
employers;
6. Length of time and number of academic credits required to complete an academic
degree, by institution and by degree;
7. Enrollment, progression, retention, and graduation rates by race, gender, and disability;
8. Student course demand analysis; and
9. Classroom utilization.









Notes

1 This paper on the Florida Quality Evaluation Project is the result of a considerable amount
of discussion within the University of Florida community. A draft distributed to the faculty,
students, and staff of the university in Spring 1992 prompted widespread comment that informs
this final version. John V. Lombardi is Professor of History and President, and Elizabeth D.
Capaldi is Professor of Psychology and Special Assistant to the President for the FQEP. The
theme of competition and choices for the University of Florida appears in Lombardi, The
University ofFlorida: Competition and Choices (October 12,1990). Others, in a larger context,
address this issue through research and publication about university management. One
example comes in George Keller's Academic Strategy: The Management Revolution in American
Higher Education (Baltimore: The Johns Hopkins University Press, 1983). Another approach
from a business management perspective appeared over a decade and a half ago in the quirky
but illuminating study by Barry M. Richman and Richard N. Farmer, Leadership, Goals, and Power
in Higher Education: A Contingency and Open-Systems Approach to Effective Management (San
Francisco: Josey-Bass Publishers, 1974). A more specific effort to understand the processes of
university life appear in William G. Bowen and Neil L. Rudenstine, In Pursuit of the PhD
(Princeton: Princeton University Press, 1992), whose work analyzes the process of acquiring the
American university's highest degree. University presidents and other leaders often believe
themselves under-appreciated and frequently offer personal reflections on their achievements.
The following represent just a few examples of a large literature and serve as a useful antidote to
hostile attacks on the university. Henry Rosovsky, The University: An Owner's Manual (New
York: W.W. Norton, 1990); Derek Bok, Universities and the Future ofAmerica (Durham: Duke
University Press, 1990); Howard R. Bowen, Academic Recollections (Washington, D.C.: AAHE
and ACE, 1988); Peter T. Flawn, A Primerfor University Presidents: Managing the Modern
University (Austin: University of Texas Press, 1990); and, for an example from an earlier
generation, Clark Kerr, The Uses of the University (Cambridge, Mass: Harvard University Press,
1963).
2 An endless literature of complaint and controversy accompanies the changing mission of
the university. Allan Bloom worries that this generation of faculty and students lacks
seriousness of purpose and blames the university for multiple cultural and intellectual failings in
The Closing of the American Mind: How Higher Education has Failed Democracy and Impoverished the
Souls of Today's Students (New York: Simon and Schuster, 1987); Lynne V. Cheney takes the
educational establishment to task in Tyrannical Machines: A Report on Educational Practices Gone
Wrong and Our Best Hopes for Setting Them Right (Washington, DC: National Endowment for the
Humanities, 1990); the prize for polemical enthusiasm, however, goes to Charles J. Sykes,
Profscam: Professors and the Demise of Higher Education (Washington, DC: Regnery Gateway,
1988). Other classics in this genre include Page Smith, Killing the Spirit: Higher Education in
America (New York: Viking, 1990); E.D. Hirsch, Jr., Cultural Literacy: What Every American Needs
to Know (Boston: Houghton Mifflin, 1987); and the reminder that this topic predates the
critically minded 1980s and 1990s in Richard B. Freeman's The Over-Educated American (New
York: Academic Press, 1976). Universities themselves memorialize their achievements in official
histories ranging from the modest to the encyclopedic. The Johns Hopkins University, as eager
as any to promote its achievements, has a long string of self-congratulatory volumes from
Abraham Flexner's Daniel Coit Gilman, Creator of the American Type of University (New York:
Harcourt, Brace and Company, 1946) or John C. French's A History of the University Founded by
Johns Hopkins (Baltimore: The Johns Hopkins University Press, 1946) to the more recent
publication of John C. Schmidt's Johns Hopkins: Portrait of a University (Baltimore: The Johns
Hopkins University Press, 1986). A major example of a scholarly, multivolume university
history appears in Thomas D. Clark's Indiana University, Midwestern Pioneer (4 vols.,
Bloomington: Indiana University Press, 1970-77).
3 Constituencies and the university community itself often splinter into many microgroups,
each believing itself the prime focus of university effort and attention. Students and faculty;


Mission Oality, and Performnce


The Florida O~altY Evaluation Prmoect






Missin. Oale,.t and Peyforinane


agricultural industry and environmentalists; doctors and trial lawyers; each of the many groups
required to sustain the institution often believe the institution exists primarily to serve that
subgroup's special concerns. In public universities, the state bureaucracies of governance and
budgeting and many other state regulatory agencies become major constituencies because they
assert the right to intervene and regulate.
4 The university community has many interlocking programs. Admissions provides
students to programs, accounting provides services to research, computing provides services to
administration and research, and so on. Some university programs, such as student computer
support offices or staff training programs, exist primarily to serve the university community
while other programs, such as agricultural extension, exist primarily to serve one of the
university's external constituencies.
5 Legislators and students often take this perspective, believing that the university exists
solely to serve students by educating them for degrees. Yet even as they articulate this belief,
students and their parents often choose universities with high reputations based on the quality
of research over those with a focus on teaching, and legislators ask the university's research
enterprise to work for state economic development.
6 Faculty who support this long view of university development recognize the century-long
tradition that underlies our current institutional structures. A few individuals adopt this
perspective because it avoids evaluation and the permanent pursuit of quality.
Evaluation for higher education serves much the same purpose as quality control does for
industrial or commercial enterprises. Evaluation in universities, however, cannot follow the
exact techniques laid out for profit-making enterprises that have precise definitions of products,
markets, and customers. The current enthusiasm for Total Quality Management and its many
variants addresses the need for evaluation within business enterprises, but it lacks the flexibility
and sophistication to deal with the non-market-driven character of much of higher education.
TQM formulas cannot capture the complexity of the full range of university products, markets,
and consumers. Nonetheless, many of the principles, if not the implementation, of TQM apply
to universities just as well as to businesses. The literature on TQM and its various derivatives in
business contexts is extensive. The guru of the quality movement, a prophet in our competitor's
land, W. Edwards Deming, deserves credit for popularizing statistical control of quality and for
promoting a series of management styles that focus on the product and the customer rather than
on internal competition among workers, managers, and suppliers. For two admiring
discussions of his method and personality see Mary Walton, The Deming Management Method
(New York: G. P. Putnam, 1988; 1986 original) and her subsequent book, Deming Management at
Work: Six Successful Companies that Use the Quality Principles of the World-Famous W. Edwards
Deming (New York: G.P. Putnam, 1990). The United States, slow to recognize the benefits of
this method until Japanese firms began producing better-quality products, has its own official
version of TQM enshrined in the instructions and guidelines for the national quality award,
Malcolm Baldrige National Quality Award. 1991 Application Guidelines (Washington, DC: U.S.
Department of Commerce, 1990?). Corporate implementations in the United States produce
brochures and descriptions. The following illustrate the genre: IBM, Unveiling a Quality Way of
Life: Senior Managers Briefed on Plan to Virtually Eliminate Defects, IBM Management Report,
April 1990; Eastman Chemicals, Quality Management (brochure published by Eastman
Chemicals Division about their Quality Management Program), (Kingsport, Tennessee: Eastman
Kodak, 1989). Academics have also studied and taught this method, reflected in George M.C.
Fisher, The Dimensions of Quality (Arlington: -University of Texas, 1990); lan Hau, Teaching
Quality Improvement by Quality Improvement in Teaching (Report Series in Quality and
Productivity, Report 59) (Madison: University of Wisconsin, 1991); Jack Elzinga,. "Continuous
Quality Improvement at the University of Florida" (paper on teaching and research about CQI)
(Gainesville: University of Florida Department of Industrial and Systems Engineering, 1991,
Xerox). University implementations remain scarce, but L. Edwin Coate reports success in
applying TQM principles to non-academic service areas of a university in "TQM on Campus:
Implementing Total Quality Management in a University Setting," NACUBO, November 1990,


__


The FlorIda Q alit P






MsIcia Quliy an Pefnuac Th lrd ult Eauto rj


26-35. Other university leaders work to adapt TQM.or similar strategies to academic
institutions as displayed in The Total Quality Forum: Forging Strategic Links with Higher Education.
A Report of Proceedings (Cincinnati: Procter& Gamble, 1992), which collects comments during a
meeting hosted by the Procter & Gamble Company
8 The AAU has 59 institutional members, 30 public and 28 private universities plus the
University of California system. The U.S. public university members are as follows, with an
asterisk for those public universities with a land-grant mission: Arizona, California at Berkeley,
California at Los Angeles, California at San Diego, Colorado, Florida*, llinois*, Indiana at
Bloomington, Iowa State*, Iowa, Kansas, Maryland at College Park*, Michigan, Michigan State*,
Minnesota*, Missouri*, Nebraska*, North Carolina at Chapel Hill, Ohio State*, Oregon,
Pennsylvania State*, Pittsburgh, Purdue*, Rutgers, State University of New York at Buffalo,
Texas at Austin, Virginia, Washington, and Wisconsin*. The University of Toronto, a Canadian
public university, is a member of the AAU. In addition, Cornell University, a private
institution, has a complete public land-grant division, and the Massachusetts Institute of
Technology has a land-grant function in engineering only. The University of California system
carries a land-grant designation, but the function is delivered primarily through UC-Davis,
which is not a member of the AAU.
9 The involvement of program faculty, students, and staff in the development, specification,
and improvement of evaluation criteria and process is essential. Groups that deliver program
results and who produce the quality own the evaluation, for they know best how to improve
quality and they know what constrains the achievement of quality. Similarly, programs need to
identify whom their program serves and what those people expect.
10 While mission statements can become complex and detailed, every program should
reduce its mission to a short statement: no longer than half a page is highly desirable.
11 The development of quality indicators offers an opportunity for each program or group to
describe the elements of quality in its work. Sometimes it is how quickly a service unit can
respond to a constituent request; at other times it may be the proportion of students placed in
highly competitive graduate schools. In the discussion, members of a program often find that
they do not all share the same definitions of quality. One faculty member may see quality
teaching being reflected in much written work while another member of the same unit may find
it in the acquisition of information as demonstrated in machine-scored tests. Both may be right,
but the discussion will permit the people with expertise to resolve this kind of issue. Some
national indicators of quality exist for certain parts of university work. The National Research
Council's Committee on an Assessment of Quality-Related Characteristics of Research-Doctorate
Programs in the United States produced An Assessment ofResearch-Doctorate Programs in the
United States, with volumes on the Biological Sciences, Engineering, Humanities, the
Mathematical and Physical Sciences, and the Behavioral Sciences (eds: Lyle V. Jones, Gardner
Lindzey, and Porter E. Coggeshall; sponsored by the Conference Board of Associated Research
Councils) (Washington, D.C.: National Academy Press, 1982), with an update to be published in
1994. Private enterprise and academic journals do a thriving business publishing rankings of
universities and programs. John A. Byrne, Business Week's Guide to the Best Business Schools, (2nd
ed., New York: McGraw-Hill, 1991) and "Runner-up Business Schools that make the Grade,"
Business Week (January 28,1991), 96-97, show a business perspective while Albert W. Miemi,
"Research Productivity of American Business Schools, 1975-1985," The Review of Business and
Economic Research, 23(Spring 1988), 1-17; Glenn Petry and John Settle, "A Comprehensive
Analysis of Worldwide Scholarly Productivity in Selected U.S. Business Journals," Quarterly
Review of Economics and Business, 28 (Autumn, 1988), 88-104; and William W. Williams,
"Institutional Propensities to Publish in Academic Journals of Business Administration,"
Quarterly Review of Economics and Business, 27( Spring 1987), 77-94, illustrate the academic
perspective. U.S. News and World Report has made an annual tradition of ranking academic
programs and institutions. For example see "America's Best Graduate Schools," U.S. News and
World Report (March 23, 1992).
0


Mission. Om~at. and Performance


The Florida Onalot Evahurtion Project






MMUCU pafly ma I'ronneTeFo_*Odt ~nu~nP-I


12 The national reference group substitutes for the absence of simple marketplace references
for academic work. While some marketplace information exists about salaries and other
institutional prices, few good marketplace references exist for most teaching, basic research, or
service activities, fewer still for internal administrative and support work. Universities exist in
a competitive context with other universities, and the best national practice represents a
reasonable standard of comparison. Quality exists independently. We can have a low quality
program that has high productivity, or we can have a high-quality program with low
productivity. This project seeks to help programs succeed with both high quality and high
productivity, recognizing that the constraints of inadequate resources may restrain
improvements in both quality and productivity.
13 University data often do not lend themselves to program-based resource accounting as
many expense and income items disappear into utility, space, capital, library, maintenance,
tuition, and other central budgets. The resource balance requires careful attention to issues of
data consistency and completeness. The data and tables of the resource balance currently exist
in detail to the college level. Deans will need to construct departmental resource balances to
expand their analytical tools. Taken alone, these data and tables tell little about quality, but
when combined with qualitative and comparative data, the resource balance helps us
understand the limits on success. Examples of the data categories available for the resource
balance at the University of Florida appear in Appendix A.
14 Because many constraints come from the university's position within the political
structure of state and federal government agencies, dear identification of these constraints lets
us focus our political energy on the worst of the constraints rather than wasting scarce political
capital on minor irritants.
15 The State University System Board of Regents requires various planning exercises: a five-
year Master Plan which includes the mission statements of each of the state's public universities;
a five-year Academic Master Plan that specifies all new academic programs that the university
expects to establish within the five-year period; an Enrollment Plan that the university submits
to project enrollment through academic year 1999-2000; a five-year Facilities Master Plan, which
provides the building plan for the institution and must be linked to the enrollment and academic
master plans; and finally, an Information Resource Management Plan, which sets standards and
criteria for purchasing computer and telecommunications facilities. In addition, the legislature
frequently requests various reports on planning or special data analysis relative to university
topics of legislative interest on such issues as affirmative action, faculty productivity, or
graduation rates. The universities are also currently engaged in a two-step accountability
process mandated by the legislature in the following terms as "the systematic, ongoing
evaluation of quality and effectiveness in the State University System ...[to] monitor
performance at the system level in each of the major areas of instruction, research, and public
service, while recognizing the differing missions of each of the state agencies." The first stage of
that process produced nine accountability measures focused mostly on quantitative issues of
student academic progress and faculty productivity (Accountability Plan Implementation Proposal
DRAFT, Presented to the Board of Regents, State University System of Florida pursuant to
Chapter 91-55, Sections 507, Laws of Florida by the University of Florida, December 31, 1991).
The second stage is an institution-specific quality evaluation and improvement process that
references each university's quality to an appropriate national standard. The FQEP will meet
this objective.
16 Quality improvement and enhancement require constant effort and commitment and will
become part of every one of the university's institutional processes.
17 Much difficulty with intercollegiate athletic programs comes from misunderstanding the
relationship of athletics to the university's missions. By clearly identifying the auxiliary nature
of intercollegiate athletics, especially in big-money programs, we can clarify the issues and make
better decisions about the costs and benefits of these programs. The literature on sports in
American universities is endless, but a perspective relevant to the University of Florida based on


Mvusso, Quaflty, md Performance


The Flarid OmWv Evr~nntk~n Pmir~






Misam,. nafl,. ad PeformnceThe Florida Ouaity Evaluation Project


a reading of some of that material appears in Lombardi, "Sports Medicine," The Journal of the
Association of Governing Boards of Universities & Colleges (34:1, January/February 1992).
18 Many colleges and programs in the university have already implemented or begun
evaluation processes of this kind. Arts and Sciences, Medicine, and Engineering represent only
a few of the units that have begun this process. The university itself responds to many outside
requests for various evaluations, in addition to the Board of Regents and legislative activities
outlined above. Colleges, departments, and programs have external advisory boards and
numerous accreditation agencies review our programs. As a result, many programs have
already done much of the work required to implement this evaluation strategy.
19 This process requires the complete involvement of everyone in the university community.
It requires the strongest support from the administration, and it demands high levels of
participation by faculty, students, and staff. Professor Capaldi serves as Special Assistant to the
President to coordinate this project, although the evaluation of quality and the plans for
improvement occur at the unit level. The evaluation process reports to the president and
provost. The president also chairs a special committee of vice presidents that periodically
reviews the progress of the project.


-18-


Madon. Oumlt. amd Performnce


The Florld OceW(r Evhntloa Proket




University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs