AN EVALUATION OF CHILDREN'S SERVICES IN
COMMUNITY MENTAL HEALTH CENTERS
LEWIS A. FABRIC, II
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF
THE UNIVERSITY OF FLORIDA
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
With love to my mother
For the completion of this project I am indebted to a number of
people. Each of my committee members (Drs. Myrick, Fitzgerald, and
D'Amico) contributed their personal time to read and edit this disser-
tation. I am most grateful to each one of them and to their families
for their assistance and encouragement.
I want to also thank my friends and colleagues over the years
at the Child Youth and Family Center and at the North Central Florida
Community Mental Health Center, Inc., for their assistance in my pro-
fessional development. Their work as clinicians has given this project
its inspiration and enabled its conclusion.
My family has been extremely patient and helpful. Through this
process, my wife, Finnette, has often had to function as a single
parent. She has ably assisted me in editing the manuscript and in many
other ways. My children, Elena and Nathan, have sacrificed much family
time in the completion of this project. I am also appreciative of my
in-laws' interest and support.
Finally, my parents have assisted me through their support of my
education over the years and their caring and assistance. They have
served as examples for me in perseverance and dedication. For this,
I thank them.
TABLE OF CONTENTS
ACKNOWLEDGEMENTS. . . . . . .
ABSTRACT. . . . . . . . .
ONE INTRODUCTION . . . .
Need for the Study . . .
Purpose. . . . . ..
General Research Questions .
Definition of Terms .. .. ..
Organization of the Study. ..
TWO REVIEW OF LITERATURE . . . . . . . . .
History of the Community Mental Health Movement. . .
Development of Children's Mental Health Services .
The Community Mental Health Centers and Children . .
Evaluation and Community Mental Health Centers . .
Outcome Studies in Community Mental Health Centers .
Effectiveness of Evaluation . . . . . .
Outcome Studies Related to Specific Variables
in the Current Study . . . . . . . .
Summary . . . . . . . . . . .
THREE RESEARCH METHODOLOGY . . . . . . . . .
Population and Sample . . . . . . . .
Variables Investigated . . . . . . . .
Criterion Measure . . . . . . . . .
Sampling Procedures . . . . . . . .
Research Hypotheses . . . . . . . .
Method of Data Collection . . . . . . .
Data Collection Visits . . . . . . . .
FOUR ANALYSIS OF DATA . . . . . . . . .
Parent Involvement .
Referral Source. .
Severity of Diagnosis.
Type of Treatment. ..
. . . . . . i
. . . . . .. vi
. . . 1
. . . 5
. . . g
. . . I9
. . . 10
. . . 11
Duration of Treatment. . . . . . . . ... 97
Other Findings . . . . . . . .... 99
FIVE SUMMARY, CONCLUSIONS, LIMITATIONS, AND RECOMMENDATIONS 116
Discussion and Conclusions . . . . . .... 116
Type of Treatment. . . . . . . .... 121
Limitations. . . . . . . . . ... .. 126
Recommendations for Further Research . . . ... 127
A ADMISSION DATA FORM. . . . . . . . ... 130
B RECORDING INSTRUMENT . . . . .. ...... 138
C INSTRUCTIONS FOR FILLING OUT QUESTIONNAIRE . . .. 142
D PROPORTION OF AGREEMENT. . . . . . . ... 147
REFERENCES . . . . . . . . ... ...... 150
BIOGRAPHICAL SKETCH . . . . . . . .... ..... 159
Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the Requirements
for the Degree of Doctor of Philosophy
AN EVALUATION OF CHILDREN'S SERVICES IN
COMMUNITY MENTAL HEALTH CENTERS
Lewis A. Fabrick, II
Chairperson: Robert Myrick
Major Department: Counselor Education
This study investigated clinical variables in relation to the
effectiveness of children's services in community mental health centers
in north central Florida. These variables included parental involvement
in treatment, referral source, diagnostic classification, type of treat-
ment, and duration of treatment.
Information was obtained from 277 closed medical record charts.
The charts were sampled at random from a total of 1352 cases seen during
the period 1980-1983.
Using a cross-tabulation procedure, measures of the variables
studied were examined according to treatment outcomes. Cases were
classified as successful or unsuccessful according to therapist ratings
of progress on specific treatment goals.
Analysis of data suggested that treatment success was positively
related to duration of treatment (p < .001). There were no differences
between cases rated as successful or unsuccessful with regard to
referral source, diagnostic classification, or type of treatment.
The results for parental involvement in its effect on treatment
success were inconclusive.
It was concluded that children's services in community mental
health centers were equally successful regardless of who referred the
client, diagnostic classification, or the type of treatment involved.
However, remaining in treatment longer than five sessions appeared to
increase the chances of being rated as improved by the therapist.
Implications of the study and recommendations for future research
For many years, mental health services were provided through the
efforts of the local communities across the United States. Quality of
care was far from uniform and unavailable to many segments of the
population. Many communities chose to house their mentally ill in far
In 1963, President Kennedy called for a new approach to mental
health care and Congress responded with the Community Mental Health
Center Act (P.L. 88-164). This Act created community mental health
centers throughout the United States. These centers were to provide
alternatives to state and county mental hospitals for the prevention
and treatment of mental illness (Naierman, 1978).
Community mental health centers (CMHCs) tripled in number between
1967 and 1976, increasing to a total of 528-centers (ADAMHA National
Data Book, 1980). By 1983 there were 35 mental health centers in the
state of Florida established with the help of federal funding. As a
result of the increase in community mental health programs, care was
made available to rural and poor populations as well as the urban and
rich (Jones, Wagenfeld, & Robins, 1976).
The mental health center act as amended in 1965 required the centers
to provide five basic services without regard to a client's ability to
pay: inpatient treatment, outpatient care, partial hospitalization
services, emergency treatment and consultation, and education services
(Naierman, 1978). These mental health centers were established as
private, non-profit organizations with links to local, state, and
federal governments. Local matching funds were required to attract
A center's staff usually consisted of an executive director, ad-
ministrative and support personnel, and both professional and non-
professional clinical staff. A board of directors from the community
were responsible for policy guidelines and employment of the director.
Among those professionals who were employed, there were psychiatrists
(9%), psychologists (5%), social workers (9%), registered nurses (14%),
and other mental health professionals (14%) (ADAMHA National Data Book,
Consequent with the establishment of a nationwide network of ser-
vices came the need for large scale evaluation efforts, but mental health
centers were not seen as basic research facilities. They were generally
unconcerned with the discovery of new information. The primary objec-
tive of evaluation or research within centers was to determine the
extent to which services were delivered. Studies to examine the results
achieved were limited. The success of evaluative research depended
ultimately on its usefulness to the administrator (Suchman, 1967).
In recognition of the importance of making mental health centers
publicly accountable, as well as increasing their effectiveness,
legislators amended the Mental Health Center Act in 1975 to include
program evaluation. Centers were required to collect data which showed
1) patterns of use of services; 2) availability, accessibility, and
acceptability of services; and 3) impact of services on clientele
Evaluation procedures have tended to favor simple outcome measures.
This is as opposed to more complex evaluation procedures which explain
why a program may have had effect. Some concern was expressed that
outcome research is too difficult to conduct (Keppler-Seid, Windle, &
Woy, 1980),and others have cautioned that such research studies empha-
size certain outcomes to the exclusion of others (Schulberg, 1981).
Because of confusion, lack of knowledge and resources, and resistance,
research in mental health centers has been limited. Subsequently,
knowledge gaps have developed in community mental health centers,
especially as to how different client types are differentially affected
Relatively little was known from early evaluations as to which
services were most effective for different recipients. Beginning with
Eysenyk's conclusion in 1952 that psychotherapy had no proven efficacy,
researchers became more specific in the types of information they sought.
New investigations looked at different types of treatment, demographic
factors, and client personality. They also examined clinical information,
therapist factors, and therapist-client interactions (Strupp & Bergin,
1969). Even so, there was insufficient research to direct the estab-
lishment of programs and implementation of services in the community
mental health centers.
In 1975 it was mandated that centers provide services to children
and adolescents. Yet, as with other populations, there was a question
about how to provide young people these services. Both Meltzoff and
Kornreich (1970) and Levitt (1971) noted the lack of research regarding
the effectiveness of mental health treatment for children. Several
factors were singled out for blame: research interest in the child
guidance movement by psychiatrists and social workers was not evident
(Levitt, 1971); psychiatric treatment for children was less widespread
than for adults (Baekeland & Lundwall, 1975); methodological problems exist
in research, such as separating out changes in children and adults be-
cause they are often seen together in treatment (Levitt, 1971).
The lack of research on the treatment of children is unfortunate.
Mental health centers are facing economic problems and there is a greater
need to demonstrate their value. Due to less money from the federal
government, centers are giving a greater part of their resources to
clients who are the most seriously disturbed and who are at greater-
risk for institutional placement. Children and adolescents are less
likely to be defined as a service priority when services are limited.
This low priority and a decline in services compound the problem of
research and evaluation as there is less opportunity to do research and,
subsequently, less opportunity to build a documented case for more
Mental health centers need more help in directing their resources
for the maximum benefit for clients. More information is needed about
specialized client groups, such as children and elders. Supportive
research is necessary in order to improve services and build a case for
mental health centers.
Need for the Study
It was estimated 10 years ago that approximately 1.5 million
children in the U.S. had profound emotional disturbances which pre-
cluded their living in normal family, school, and community circum-
stances (Smith & Hobbs, 1966). More recently it has been estimated
that each year more than 6 million children, ages 5 to 19, have mental
health problems severe enough to require professional treatment (Berlin,
1975). Such increasing magnitude suggests a need for specialized treat-
ment programs which have proven efficacy.
Yet, Levitt (1971) noted that after six decades of the child guidance
movement evaluative research still failed to show that child psycho-
therapy was effective. Outcome studies illustrating improvement rates
for treatment groups summed across different categories of emotional
disturbance were approximately the same as for those who did not re-
ceive treatment. Examining the issue of outcome more closely also
revealed large variations in improvement rates between agencies. This
suggested that investigators needed to study more specific treatment
programs for different populations (Strupp & Bergin, 1969) and find
where treatment was effective.
Although research efforts increased in adult psychotherapy, little
research was done in child psychotherapy. Research with children when
compared to adults is inadequate (Baekeland & Lundwall, 1975). Levitt
pointed out that only six reviewers of psychotherapy research in the
Annual Review of Psychology since its inception found it necessary to
use a subheading for research with child patients (Levitt, 1971).
The child therapy researcher has several methodological problems
to contend with. First, he has to consider the status of the child as
a developing organism. Sometimes it is difficult to tell whether a
symptom is a part of the normative growth process which will be taken
care of in time or a truly deviant process in need of treatment. In a
similar vein, one symptom of emotional disturbance may be replaced by
another. Thus, the researcher may have to decide whether treatment for
the original symptom was successful or whether the symptom remitted
through the child's developmental process (Levitt, 1971).
Another difficulty involves the child's comparatively greater
dependence on his environment, especially his family. Children are
usually a captive audience brought to treatment by adults for problems
that have been defined by adults. Thus treatment goals may be more a
factor of adult rather than child expectations. This may be one reason
why investigators of child psychotherapy have focused on the personality,
psychopathology, and attitudes of the child's parents (Baekeland &
Lundwall, 1975). Dropping out of treatment, for example, has been
positively related to negative parental attitudes and behavior toward
the treatment (Lake & Levinger, 1960).
To elaborate further on the special research difficulties created
by the child's dependence on his environment, there is the problem that
the remainder of the family is simultaneously in treatment with the child.
When the child guidance movement was first started, it was common for the
parents to be seen by a psychiatric social worker while their child was
seen separately by a child psychiatrist. As conjoint family therapy
came into vogue in the late 1960s and early 1970s, it became commonplace
to see the family together. Whether seen separately or together, it is
difficult to say at the conclusion of therapy that positive results in
the child were a result of his/her individual treatment. Separating
out treatment effects has become a challenge to researchers.
If specific variables related to treatment success with children were
established through basic research,then effective programs could be de-
vised to meet children's mental health needs. However, it has been noted
that such variables have not been isolated and that such ideal programs
do not exist. An unfortunate consequence of the lack of empirical
support for program interventions is the difficulty in obtaining public
Particularly affected are prevention programs such as mental health
centers which intervene early in the development of emotional dis-
turbance. Friedman (1983) and Kiesler (1982) have noted the preference
in public policy given to institutional/residential programs over-early
intervention models. Such programs are often funded in response to the
most immediate need while preventive services, which meet a less visible
A similar lack of supportive research for children's mental health
programming exists in the area of program evaluation research. Such
research attempts to determine whether specific program activities attain
desired results, thus establishing their usefulness in particular situa-
tions. Unfortunately, mental health center research studies have focused
on the population which receives the greatest part of its resources:
adults (Fortney, 1981; Kline & King, 1973; Sue, McKinney, & Allen, 1976).
In a recent statewide outcome study of mental health centers, only adult
outpatient and day treatment clients were evaluated (Cox, Brown,
Petersen, & Rowe, 1982). Successful outcomes with adults found in some
studies do not necessarily generalize to children, however, for reasons
that have been described.
One study which attempted to evaluate mental health centers' efforts
to work with children has received special attention. Fiester (1978)
developed an automated system to measure specific goal attainment. He
was attempting to comply with federal accountability standards and also
gather data on treatment effectiveness. Treatment was found to be
successful in the areas of social and academic skills. The data analysis
provided feedback to therapists as to their effectiveness with particular
clients, making it a good example of institutional self-study.
The main problem of the Fiester study was that only outcome data
were presented without the specification of process. It did not measure
and partial out alternative explanations for the results, such as the
effects of parent participation, referral source, or demographic status.
Although Fiester used data to develop criteria for the assignment of
clients to therapists, he did so only on the basis of a few limited
variables, such as an individual therapist's success in a specific prob-
lem area. Without the specification of other process variables, ques-
tions about the effectiveness of centers with distinctive populations
could not be answered.
The need for data-based studies of mental health centers' services
is evident. They are needed to assure consumers, taxpayers, and health
providers that services are valuable and warranted. In order to secure
funds for programs, administrators of outpatient children's mental
health outpatient programs need more information regarding outcomes,
which treatment is most successful under which conditions. The need
to relate treatment success to treatment variables has also been recog-
nized (Strupp & Bergin, 1969; Suchman, 1967).
It was the purpose of this study to investigate mental health
center outpatient services to children, adolescents, and their families
and to examine successful and non-successful cases as described in offi-
cial medical records on file in mental health centers. More specifically,
this study examined client records to 1) identify which children's cases
were classified as successful or not successful and 2) examine specific
variables related to the cases. It was then possible to study these
variables in terms of treatment outcomes.
Two northern Florida mental health centers provided data for the
study. Approximately 275 medical charts were systematically reviewed
for data, including information on initial diagnosis, referral source,
type of treatment, parental involvement, and the duration of treatment.
Other information on both client and treatment variables was gathered
for use in secondary analyses.
General Research Questions
1. Does the presence or absence of parental involvement during
treatment of children have an effect on success or reported success?
2. Are referral sources related to successful or non-successful
3. Are some diagnostic categories (based on the Diagnostic and
Statistical Manual III) more responsive to treatment than others?
4. Does the length of treatment have any effect on outcome?
5. Does the type of treatment have any effect on outcome?
Definition of Terms
Community Mental Health Center: A mental health agency or institu-
tion established under guidelines set by the Mental Health Act of 1963
Specialized Children's Program: A program within a mental health
center designed to deliver mental health services to children (0-18 years
old) and their families. Such programs were mandated in 1975 to be part
of federally supported mental health centers.
Medical Record: A client's file, started when he/she seeks treat-
ment. Also referred to as medical chart or case, it contains 1) an
admission data sheet, 2) an intake summary, 3) a treatment plan, 4) a
discharge summary, 5) progress notes, and 6) medication record. Other
available information generally includes 7) referral source, 8) who was
included in treatment, 9) clinical diagnosis, 10) treatment goals,
11) number of sessions client attended, and 12) treatment outcome
rating by therapist.
Diagnostic Category: Refers to the clinical diagnosis re-
corded on the client's records and is based upon the Diagnostic and
Statistical Manual III for the Classification of Psychiatric Disorders
Referral Source: The primary source which directed the client/family
to seek treatment at the Center.
Organization of the Study
This study is organized into five chapters. Chapter One includes
the introduction, need for the study, purpose of the study, research
questions, rationale, and definitions of terms. Chapter Two contains
a review of the literature related to evaluative studies of mental health
treatment programs for children and adolescents. A discussion of the
methodology and data collection is presented in Chapter Three. Results
of this study are presented in Chapter Four, including analysis of
the data. A summary, conclusions, and recommendations made from the
investigation are included in Chapter Five.
REVIEW OF LITERATURE
In this chapter the community mental health movement will be traced
through its historical origins beginning with the establishment of the
National Institute of Mental Health and culminating in the 1980s with
the recision of the Community Mental Health Act by President Reagan.
Services to children will be examined, emphasizing the status of chil-
dren's mental health services prior to the establishment of community
mental health programs and then noting the impact of the centers on child
mental health services. Types of services delivered to children in
mental health centers, their appropriateness and adequacy, will be
described. In separate sections, evaluative research as it applies
to community mental health centers will be presented. Finally, research
related to the main hypotheses will be discussed.
History of the Community Mental Health Movement
Mental health care prior to the twentieth century was largely the
responsibility of the family. Large mental institutions and jails,
usually far removed from the patients' communities, were the primary
sources of containment and treatment of the "mentally ill." Dorthea
Lynde Dix, in the mid-nineteenth century, crusaded for more humane
treatment of patients in these institutions. She helped increase public
awareness of the problem in treating patients. While more mental
hospitals were constructed as a result of her efforts, extreme over-
crowding soon followed. Later in the 1890s the work of Sigmund Freud
added an air of scientific respectability for the first time to the
field of mental illness. Attitudes toward treatment of the mentally
ill slowly began to change.
Mental health care before the 1940s was under the direction of
state authorities. However, during World War II the large number of
draft rejections for mental health problems increased public concern.
This in turn led to the National Mental Health Act of 1946 which in-
volved the federal government in the problem for the first time. At
this time there were fewer than 10,000 full time mental health profes-
sionals in the United States (ADAMHA, 1980).
Several important changes in the mental health field occurred during
the 1950s. Despite the fact that 75% of mental health services still
occurred in inpatient facilities (National Institute of Mental Health,
1977), psychotropic medications were developed to make hospital stays
shorter and enable patients to be maintained in their own communities.
This kind of treatment was supplemented by an increase in the number of
local outpatient psychiatric clinics (Miller, 1974). Congress estab-
lished a Joint Committee on Mental Illness and Health in 1955 to study
the possibility of adopting a new approach to the treatment of mental
illness. Among the recommendations were the expansion of services,
reduction in size of state hospitals, and increased provision of services
in local communities (Joint Commission on Mental Illness and Health,
Community Mental Health Center Construction Act
The recommendations of the Joint Committee on Mental Illness and
Health were used by President Kennedy to establish the Community Mental
Health Center and Mental Retardation Facilities Construction Act of
1963 (P.L. 88-164). Thirty-three states acted immediately, passing
legislation to authorize the construction of mental health centers
(National Institute of Mental Health, 1969). The implementation of
community mental health center programs was given an additional boost
in 1965 with the passage of federal legislation, which provided staffing
assistance to cover personnel costs (P.L. 89-105).
The primary intent of the legislation was to assist communities
in developing alternatives to state and county hospitals in the treatment
of mental illness (Naierman, 1978). A variety of mental health services
were, for the first time, to be available to the entire population
regardless of an individual's ability to pay or geographic location
(Jones, Wagenfeld, & Robin, 1976).
In order to qualify for federal funds, community mental health
centers had to provide few basic services. The minimum services in-
cluded inpatient and outpatient care, partial hospitalization and
emergency services, and consultation and education services. Children
and adolescents were not, at this time, given special services. They
were instead served in programs with adults (Naierman, 1978).
The required basic services required development of new treatment
techniques and a more diversified professional base. Assistance in
manpower training came from the National Institute of Mental Health
and the Veteran's Administration. There was more than a thousand-fold
increase in the number of mental health professionals (125,000 in 1980)
in the thirty years between 1950 and 1980 (Vandenbos, 1980).
Initially, federal staffind grants to centers were awarded for a
fifty-one month period. It was expected that centers would develop suf-
ficient additional funding during this time and would become independent
from federal funding. Despite varied funding from state and local
governments, however, the centers had not yet gained financial inde-
pendence from federal grants by 1970. Hence, federal grants were
extended to eight years. Dependency on federal funds was attributed to
a number of factors, including the following: 1) income to centers was
reduced because payment was on a sliding fee scale based on family income
and number of dependents; 2) many third party insurers did not cover com-
munity based mental health service; and 3) there was a lack of sophisti-
cated financial management (Naierman, 1978).
By 1975 community mental health centers were well established
across the country. Yet there were still problems with securing alterna-
tive funds to replace the federal staffing grants. Public Law 94-63 was
established in partial response to this need (Despard, 1983). It pro-
vided a new sliding scale of federal and state funding for beginning
centers, but one which declined at a faster rate than previously. This
was intended to encourage centers to obtain alternative funding sooner
in their development. It also supported established centers which had
not succeeded in finding replacement funding.
At the same time, however, P.L. 94-63 called for an expansion df
already existing services to include evaluation, screening, follow-up
and transitional services. In addition, specialized services for
children, elders, alcohol and substance abusers wererequired. Grants
were made available to finance these additional costs.
The implementation of evaluation and accountability standards had
several positive effects. For the first time, mental health centers
were accredited by the Joint Commission on American Hospitals. Quality
of care standards were established (Joint Commission on the Accredita-
tion of Hospitals, 1976). Better program planning and evaluation
techniques were encouraged through staff developmenttraining and pub-
lications supported by the federal government (Ellsworth, 1979).
Effects of CMHC Legislation
As early as 1972 the community mental health program was being
heralded as a focal point of the national effort to improve patterns
of mental health care in the United States (Brown, 1975). Centers had
grown in only six years (1965 to 1971) from 1 to 493 and they were in
all 50 states, the District of Columbia, and Puerto Rico. It was esti-
mated that the centers would eventually be serving areas with a combined
population of 73 million (Brown, 1975).
From 1969 to 1971 the services delivered through the community
mental health system nearly doubled from serving 372,000 persons to
659,000, not counting those served through consultation and education
services. Public program emphasis was now based on prevention, early
detection and treatment, as opposed to institutionalization. Mental
health centers thus seemed to be succeeding in making mental health care
available and accessible to the general public (Miller, 1974).
A survey conducted by the Joint Commission on Mental Illness and
Health in 1957 investigated attitudes of the public toward mental health
care as well as utilization rates (Vandenbos, 1980). Only 15% of the
respondents in a poll were found to have ever used any mental health
services while only 28% of this figure had received services from a
trained mental health professional. When the study was repeated in
1976, by Douvan, Kulka, and Veroff (1978), the number of people using
services had grown to 26%, with 47% of these having received treatment
from a qualified mental health professional. In addition, 60% of those
surveyed said they would use mental health services in connection with
an emotional problem.
Mental health centers were especially successful in serving lower
and middle income families. Over 50% of CMHC patients had incomes below
$5,000, while only 6% earned over $16,000 per year (Brown, 1975). About
one-half of all centers in 1972 were located in areas of poverty. In
one Florida mental health center 60-70% of the clients (in 1982) were
eligible for full reduction in fees. Only 18% of the total mental
health caseloads had a family income that exceeded $10,339 (District
III Mental Health Board, 1981-1982).
In addition to serving many lower income clients, the centers
also assisted persons who were formerly in public mental hospitals.
Of all persons treated at mental health centers, 13% were at one time
confined to public mental health hospitals as of 1972 (Brown, 1975).
Recision of Mental Health Systems Act
Mental health centers would have benefitted in 1980 from passage
of the Mental Health Systems Act (H.R. 4156 and S. 1177) based on
findings of the President's Commission on Mental Health (President's
Commission on Mental Health, 1978). This bill would have strengthened
mental health planning and resource allocations at the state level. It
also sought to improve services for special groups, such as the
chronically mentally ill and severely disturbed children. There was a
general intention to make services accountable at the state and local
level by encouraging services to high priority (most disturbed) clients,
basing funding on a performance system, and emphasizing third party
The proposed 1980 Mental Health Systems Act, however, was rescinded
in 1980 under the Reagan administration. This left CMHCs on uncertain
status. The basic federal direction at the time was the provision and
monitoring of block grant funds for alcohol, drug abuse, and mental
health services. Thus the states now had a much larger voice in the
implementation of these services. While many of the states preserved
the essential CMHC components (including the state of Florida), there
is now large variation from state to state and center to center as to
which services are emphasized. These differ according to varying
The unofficial national mental health policy at present is deinsti-
tutionalization with an emphasis on outpatient treatment. Actual
expenditure of funds, however, disputes this premise. Kiesler (1982)
notes that medicaid expenditures in 1977 for state, county, and private
institutional care for mental illness was greater than the total federal
cost of the entire community mental health program (600 centers).
Recent figures suggest that something similar is occurring for
children and youth in the state of Florida (Friedman, 1983). Of the
total funds expended for mental health in 1983, 66.55% was allocated
for residential care, while only 26.56% was designated for non-
residential, including outpatient services. Prevention services re-
ceived the least funding at 6.89%. It has been argued (Friedman, 1983)
that more money should go to those services which intervene before
residential care becomes necessary.
As first conceived, CMHCs were to deliver services to the general
population in hopes of preventing institutionalization. Such preventive
services were funded by the federal government on the assumption that
services would eventually become financially self-sustaining through
private fees and/or state or local stipends. That this has not been the
case is noted by Naierman (1978). This study recommended that centers
should continue to receive maintenance support from the federal govern-
ment if they were to continue to provide all basic services without
regard for the client's ability to pay. With the higher priority given
to residential care, it is possible that preventive services will
The history of community mental health reveals a pattern of in-
creasing federal responsibility for provision of services from the
early 1950s until 1980. During the past three decades scientific
advances and changed social perceptions have been observed in the area
of mental illness. These new developments supported federal involvement
in this area. Community mental health legislation was one important
product of this involvement. In general, there was public support for
a policy of early intervention and prevention in the field of mental
illness in order to avoid institutionalization. This has changed, how-
ever, since 1980 with the smaller federal role in the mental health
movement. Without federal monitoring and financial support, mental
health centers may not be able to maintain their unique characteristics
as originally mandated.
Development of Children's Mental Health Services
Prior to community mental health services including children, the
Joint Commission on Mental Health of Children described mental health
services to children as a national disaster. It was estimated in 1965
that there was a total of 4.6 million emotionally disturbed children
between 5 and 19 years old. At the same time, children receiving ser-
vices from psychiatric facilities were estimated in 1964 to be only
632,000 persons (Joint Commission on Mental Health of Children,
There were seven White House conferences on children since 1909;
however, the number of children admitted to mental hospitals in 1973
increased faster than their increase in the overall population. Admis-
sion to public mental hospitals doubled in the ten years after 1963
for patients under 25 though there was an overall reduction in the
hospital population for the general population (Taschman, 1975).
Treatment at these facilities was limited to custodial care, with
approximately 20 attendants to every one professional (Joint Informa-
tion Services of the American Psychiatric Association and the National
Association for Mental Health, 1968).
In 1973, evaluation of existing services found them often to be
directed at exclusive populations while underserving specific groups
such as the poor and persons living in rural areas (JCMHC, 1973a,
Task Force V). These latter groups were often those most "at risk" to
develop emotional disorders. When services were provided they were
often inappropriate and/or uncoordinated (Gorman, 1968; Hollingshead &
The poor were especially disenfranchised. Furman, Sweat, and
Crocetti (1965) found public clinics tended to curtail their assistance
to the poor when faced with an overdemand for their services. This
compounded the problem from the fact that the poor were less likely in
the first place to seek care prior to a mental health crisis (Myers,
Bean, & Pepper, 1965).
It is the lower socio-economic status (SES) child who is most at
risk to develop mental illness. Poverty is characterized by adverse
physical circumstances, disorganized and often chaotic family structures,
and inappropriate child rearing practices, described as authoritarian,
fatalistic, and lacking positive stimulation. There is an absence of
child centeredness in parental attitudes (Beiser, 1965; Malone, 1966).
Delinquents, especially prevalent among the poor, were another
group that appeared to receive inadequate services. Few juvenile courts
(17%) had any regular access to psychological assistance. Placements
were frequently made to training schools or reformatories, even adult
prisons. High recidivism rates have proven such placements to be in-
effective (U.S. Senate Subcommittee on Employment, Manpower, and Poverty,
Services to rural children were also inadequate. Rosen,
Kramer, and Reddick (1968) noted that only one-tenth of the outpatient
clinics opened in 1965 were in rural areas and these served only 8% of
the total clinic cases. These figures were highly discrepant from actual
needs as one-third of children under 18 lived in rural settings where
outpatient psychiatric facilities were the only treatment available
because of transportation problems (Rosen et al., 1968). Services were
characterized by brevity and were often diagnostic and prescriptive in
nature rather than treatment oriented.
In addition to problems in service availability to the underserved
groups, there were also problems with adequacy and appropriateness of
services. Clients who appeared to be too severely disturbed or un-
motivated for traditional treatment were screened out (Hunt, 1962).
Only about one-third of the clients referred to child psychiatric clinics
ever received treatment (Levitt, 1971). There was an expectation for
clients to come to the clinic and articulate a psychological problem.
This was discrepant from the actual needs of the poor and/or minorities
which tended to be for increased social and occupational skills
Services were also fragmented. One source stated that this frag-
mentation occurred at every level: local, state, and federal (Taschman,
1975). On the national level alone at least 100 separately administered
programs for children existed.
The Joint Commission on the Mental Health of Children concluded
in 1973 that the mental health system a) tended to be oriented more to
professional than to client priorities, b) was concerned with remedial
services and crisis resolution rather than prevention, c) served only
a small portion of those who needed help, d) provided services that
were fragmented and uncoordinated. New technologies were needed to
make service more accessible and available to those in need. There
was also a need to increase the number of non-professionals who could
deliver services to indigenous groups (JCMHC, 1973a).
The Committee recommended a number of changes to resolve these
insufficiencies. Services should be coordinated and need-specific.
They were to include educational, recreational, and vocational plan-
ning, as well as psychotherapy. Thus, mental health services should
be integrated into the larger service agency network. Lower socio-
economic groups and minorities needed to have services established
specifically for them and their special needs, utilizing the input of
these groups to design such services. Particularly of relevance to
the community mental health program were the suggestions that mental
health services be community based whenever possible with close ties
to residents. It was also recommended that greater use be made of
parent education in mental health consultation as part of preventive
services. In addition, paraprofessionals were suggested as personnel
who could help provide more mental health services (JCMHC, 1973a).
The Community Mental Health Centers and Children
The period of the mid-seventies was a major turning point for
mental health services to children. The U.S. Congress in 1975 mandated
the addition of specialized children's services to the federally funded
centers. Already children and youth comprised 23% of the total clients
served by CMHCs (ADAMHA, 1980). Services to children were to be monitored
separately from the adult population, encouraging planning and account-
ability from the point of view of this group (NIMH, 1969). Professional
staff familiar with the special needs of youth were also added. Funds
were made available through conversion grants contingent upon imple-
mentation of these new services (National Institute of Mental Health,
The NIMH specified a set of principles on which CMHC services for
children and youth should be based. These included administrative and
clinical guidelines. Administratively, services should be delivered
on a confidential basis. They should also include the involvement of
community youth and adults in planning. Facilities should be adequate
and accessible to all segments of the community. Clinical services
should be delivered in cooperation with other community agencies,- be
designed to insure continuity of care, and emphasize prevention, early
identification, and early intervention (NIMH, 1978). Children's mental
health in general is distinguished from adult services by the inclusion
of the child's social environment as a primary focus (President's
Commission on Mental Health, 1978). This considers the child's family,
the school system, the courts, and various social agencies. As a result
of these guidelines, there are many similarities across centers for
children's services including staffing and assessment, treatment and
Staffing of Mental Health Centers
Mental health center staffing represents a full range of the core
mental health professions. The ADAHMA (1980) gave a general description
of personnel within mental health centers: psychiatrists, 5%; psycholo-
gists, 9%; psychiatric social workers, 14%; registered nurses, 9%;
other mental health professionals, 14%; non-professionals involved in
patient care (e.g., licensed practical nurses), 22%; administrative
personnel, 26%. As noted already, there has been an increase in the
number of mental health professionals in the field from the 1940s
(10,500) to the 1980s (125,000) (Vandenbos, 1982).
The NIMH recommends that a children's staff include developmental
specialists trained to be primary evaluators, back-up experts in such
areas as pediatrics and speech therapy, specially trained mental health
professionals, psycho-educational therapists, therapists, and para-
professionals. Each functions in one or more of the mandatory service
areas, such as emergency services or outpatient care (NIMH, 1978).
Referrals may come from a variety of sources. The judicial system,
social services, family physicians, school, and the family itself
frequently refer children to mental health services. Rarely is the
child self-referred. Rather, someone else determines that a child has
a problem (Baekeland & Lundwall, 1975; Levitt, 1971) and initiates the
Referrals for the poor are most likely to come from the courts or
schools. They also reflect the ethnic and social class bias of middle-
class institutions. The lower the social class of the child, the longer
referrals are delayed (McDermott, Harrison, Schrager,& Wilson, 1965).
Children referred for services fall into two categories according
to Behar (1981). There are profound differences between children who
are intelligent, willing, and able to talk about themselves and those
who suffer from poor impulse control, environmental deprivation, and
are less receptive to verbal therapies. The former group has potential
to benefit from occasional services. The latter group, however, is
chronically impaired. They need long-term care, more intensive treat-
ment, and more follow-up.
Assessment is a multi-faceted process consisting of interviews and
observations of a child with and separately from his/her family. Family
interactions are noted. Interviews are structured in order to obtain
such information as family, medical, educational, and personal histories
(Appendix A). Another source of information may be the child's teacher.
Objectives of assessment procedures are several. In order to
foster healthy psychosocial development and plan a timely intervention,
it is important to make early identification of mental health problems
(NIMH, 1978). A good assessment gives the clinician a total picture
of the person and his/her environment and enables treatment planning.
After obtaining relevant information the clinician files one or
more diagnoses according to the DSM III (APA, 1980). The most commonly
used classification for children in both inpatient and outpatient settings
(60%) in 1980 was the transient stress disorder. This is a non-adaptive
emotional reaction to environmental stress (ADAMHA, 1980). It may be
inferred from this statistic that a diagnostic label for children and
adolescents is not as useful as for adults in determining treatment
planning. Such planning must be done with the child's family as well
as with the presenting problem in mind. Diagnosis should be used for
appropriate guidance and follow-up rather than categorization only.
While assessment may take up to three sessions and involve several
professionals, typically some kind of treatment plan is agreed on at
the time of the intake. The treatment plan may specify specific prob-
lems and identify goals and types of treatment that are applicable. If
services at the center are not appropriate for the client's problem,
a referral may be made to another agency.
Family financial information is also obtained during the early clinical
interviews. Such information is required to set fees. A sliding scale
balances the number of dependents against total income and helps estab-
lish fees paid by clients.
The National Institute for Mental Health specifies that outpatient
services should be timely, coordinated with other social services, and should
attempt to prevent institutionalization (NIMH, 1978).
A full range of approaches and modalities should exist to allow
for individualization of services. The therapeutic possibilities are
many, including individual, group, and family therapy, among others.
The form of therapy varies to some extent based on age. For example,
individual therapy for younger children may use symbolic play (Axline,
1967) while adolescents benefit from more verbal forms of therapy.
Special diagnostic tools such as psychological testing and learning
assessments should be available. Another important treatment component
is medical consultation for purposes of assessment and prescription of
psychoactive medication. Therapy sessions are held once weekly for a
period of several months on the average.
Family involvement in treatment has become more popular in recent
years, as evidence indicates that children's problems are related to
family functioning (Minuchin, 1974; Haley, 1976). When relationships
are impaired, dialogue between parent and child is supervised by the
therapist in order to help clarify and resolve issues and problem
Parents are also involved in treatment through parent education
classes. These classes help parents to understand their child's be-
havior. An example of such an approach is Thomas Gordon's program,
Parent Effectiveness Training (1975).
Some youngsters require more intensive services than a once a week
visit because of the severity of disturbance. If unable to be main-
tained in a regular school or home placement on a full-time basis, day
or evening treatment is an alternative. Such services often involve
collaboration between CHMCs and local school systems (Behar, 1981).
These programs offer high staff-to-patient ratios and are oriented
toward developing social and academic skills as well as meeting physical
needs (NIMH, 1978).
Residential services may become necessary when a child is severely
disturbed or the family is unable to provide for the emotional needs of
the child. Such placements are typically made after less intensive
services have been tried and found to be inadequate. In the state of
Florida, for example, the Baker Act governing the rights of the mentally
ill provides for the care of the client in the least restrictive alter-
native. Placements are ideally short (3 months) to intermediate term
(6 months to 1 year).
Mental health programs focus on the educational, recreational,
social, and physical needs of the client (NIMH, 1979). A variety of
programs exist to meet the needs of different clients. These include
traditional psychiatric hospitalization, placement in a therapeutic
group or foster home, and wilderness camping (A. Gray, 1983).* Rarely
do mental health centers provide such services independently, however.
Emergency treatment should be available to deal with crisis situa-
tions 24 hours a day, 7 days a week. It should attempt to prevent
further complications including hospitalization. Adolescents may pre-
sent with a variety of problems including suicidal attempts or enraged
states related often to family disturbances. Mental health professionals
attempt to defuse a family crisis through discussion thereby avoiding
hospitalization (Behar, 1981).
Centers provide services which help to prevent the development of
emotional illness. Mental health education classes are aimed at special
problems. Training and consultation are available to other caregivers.
Content may include, for example, preparation for parenthood, sex educa-
tion, or family relations (NIMH, 1979).
While consultation and education (C&E) services are required, how-
ever, they are rarely financially self-sustaining. Such services
usually do not attract state dollars in replacing lost federal grants
as state funding often goes to higher priorities of treatment such as
chronic mental illness or crisis treatment (Naierman, 1978).
Outcomes of the Community Mental Health Movement for Children
Greater numbers of children and youth have received more services
following the establishment of community mental health centers. Approxi-
mately 24% of all mental health care done in CMHCs in 1975 involved
children and youth for a total of 1,627,000 clients (ADAMHA, 1980).
There was a reverse shift in proportion of inpatient to outpatient
care for children between 1955 and 1975 (ADAMHA, 1980). Inpatient care
consisted of approximately 77% of the total in 1955, while outpatient
care consisted of 72% of the total in 1975. Hospital admissions for
children had become less frequent proportionatelythan for adults. There
was as well an overall increase in the number of outpatient treatment
episodes between 1971 and 1975 of 154% (ADAMHA, 1980), with the majority
of young people diagnosed as transient stress disorders.
There is evidence that lower socio-economic status (SES) and
minority children were equitably served by CMHCs (Brown, 1975). Data
cited in connection with the general mental health center population
indicate that lower socio-economic groups were well represented among
CMHC clients. In addition, it is known that at least 17% of service
recipients were non-white in 1975 (ADAMHA, 1980). Though services are
not equally utilized by all minorities (Miller, 1974), the overall
representation rate of minorities in the CMHC inpatient population
(17%) surpassed the percent of minorities treated in private hospitals
and in general hospital psychiatric units (10%) (ADAMHA, 1980).
Thus, there is some basis to conclude that mental health centers
succeeded in making specialized services for children available to the
general public by providing a greater volume of care to more clients.
These clients were less severely disturbed (most had stress disorders)
and were treated in outpatient settings in their local communities.
Evaluation and Community Mental Health Centers
Prior to the 1950s there was little research related to mental
illness. The National Institute of Mental Health was founded in the
late 1940s but it was not until the 1970s that funds were made available
for meaningful and comprehensive program evaluation (Vandenbos, 1980).
Evaluation was mandated as an essential CMHC function through P.L.
94-63, Title 3 (1975). Centers were to spend at least two percent of
their overall budget for evaluation (Congressional Record, 1975). They
were to make an annual self-evaluation study of six areas: cost of
operations, quality and patterns of service, accessibility and aware-
ness of services, and availability of services to the public. Proof of
effective treatment was not required.
The sum of these endeavors required centers to keep a wide variety
of data. Included were client demographics and types and amounts of
services provided to individual clients. This involved extensive record
keeping (Keppler-Seid, Windle, & Woy, 1980). Caution was urged to avoid
burdening centers with extra costs, redundant measurement systems, and
useless and/or poor quality data. Government sources were warned to
avoid overzealous requirements and rigid government standards which
might be irrelevant to local needs.
Need for Evaluation
The 1960s had been many social programs created by enthusiastic
professionals. They failed, however, to obtain empirical support for
their efforts and thus had difficulty in convincing state legislators
and consumer groups of the efficacy of services (Kappler-Seid et al.,
1980; Flaherty & Olson, 1982). Competition for federal dollars made
demonstrated performance a critical issue for mental health centers as
well. Though centers were largely successful in meeting federal
monitoring requirements, there was also pressure to demonstrate suc-
cessful outcomes and link these with planning and service delivery
Program Evaluation Defined
Program evaluation is a process within organizational management
rather than an independent research enterprise for making new scientific
discoveries (Attkisson, Hargreaves,Horowitz, & Sorenson, 1978). Such
evaluation should be rigorous and based on a systematic data collection
system. Suchman, for example, advocated an extended definition of
the study of planned programs for producing social change
through social experiments. These experiments test the
validity of the hypothesis that the action program has
within it elements that will effect certain causal factors
in the development of the desired objective. What is
evaluated in the action hypothesis is that defined program
activities will achieve specific desired objectives through
their ability to influence those intervening processes that
affect that occurrence of these objectives. (Suchman, 1967,
Outcome Research Defined
Outcome research is one of the most highly favored kinds of program
evaluation (Schulberg, 1981; Attkisson et al., 1978). It can help to
determine if a change was made in connection with a program, although
it does not necessarily specify that the change was caused by the pro-
gram. Such concrete effects related to program goals can include such
indicators as hospital recidivism rates, improvement on standardized
tests, or changes in individual program areas, e.g., Goal Attainment
Scaling (Kiresuk & Sherman, 1968).
Outcome research has been criticized from different points of view.
Schulberg (1981) noted that instruments used to measure changes fre-
quently lack validity and that evaluation of specific outcomes often
excludes many relevant aspects of care. Keppler-Seid et al. (1982)
questioned whether the types of client outcomes were too exhaustive to
feasibly measure on a programmatic basis. They also questioned the
variability between therapists, clients, and client-therapist
interactions. In addition, the lack of experimental studies in which
control groups are used for comparison purposes has proven to be a
problem in evaluating treatment effectiveness. Mental health evaluation
should not be reduced to a series of simplistic criteria such as
profit/loss ratios and gross measures of client improvement (Schulberg,
Process Evaluation Defined
Process evaluation is an alternative to outcome research. De-
scribing how a program attempts to affect outcomes has been
referred to as "process measurement" (Suchman, 1967). The steps involved
in process measurement are a) specification of the program character-
istics, b) analysis of effectiveness between target populations served,
c) specification of conditions under which the program is most success-
ful, d) control or explanation of factors that pose alternative explana-
tions for the results, and e) generalization of results to other
situations (Suchman, 1967). This approach attempts to illustrate the rela-
tionship between service processes and outcomes more specifically than
would be the case in simple outcome research where only one variable is
studied. If such linkages can be validated for different client groups,
then for the future, measurement of program effort, i.e., number of
sessions, alone will suffice.
One's choice evaluation approaches is determined primarily by
intent whether it be evaluation of effort, performance, or process
(Suchman, 1967). Another variable is the degree of experimental control
in a situation. This influences research design possibilities. Finally,
there are practical issues, such as expense, to be considered. However,
some general guidelines for evaluation have been recommended:
1. Evaluation procedures should be linked closely with evaluative
objectives in order to avoid redundant or irrelevant data.
2. Criterion measures should be valid and reliable.
3. Support systems required to obtain basic information for the
evaluation must be available.
4. The costs of the procedure should be considered and balanced
against its value to the agency.
5. The impact of evaluation data on performance should be monitored
and assessed (Coursey, Specter, Munell, & Hurst, 1977).
Chronologically, one usually identifies program goals and objec-
tives. Indicators of performance are developed. These are operation-
alized as a routine data collection system and then the data are used
to assess program functioning. It has been suggested that an integrated
evaluation program under one computerized system eases data collection
and analysis (Coursey et.al., 1977).
An example of such a system is Fiester's (1978) Automated Compre-
hensive Children's Evaluation System (ACCESS) for evaluating a children's
mental health program. Evaluation procedures were designed to measure
all basic areas mandated by P.L. 94-63 (availability, accessibility,
patterns of utilization, quality of services, and client satisfaction).
In addition, a modified version of Goal Attainment Scaling (Kiresuk &
Sherman, 1968) was used to identify goal commonalities across clients
and assist in programmatic comparisons. A specific form was devised
to collect demographic data at the outset of treatment. Shortly after
case termination, a request for more information was made to clients.
As therapists' outcome ratings, demographic data, and follow-up
interviews were completed, they were forwarded to data processing. A
computer program analyzed the data. Print-outs were made available to
each therapist. The system was flexible, standardized, inexpensive,
and minimally intrusive to clinical staff.
Experimental Design in Program Evaluation
Program evaluation is not limited by research methodology. Depend-
ing upon the situation,descriptive, correlational, experimental, and
quasi-experimental methods have their place in evaluation.
There are several considerations in choosing the best method for
a given situation. Rigorous experimental procedures are sometimes
impossible to implement within broadly aimed social programs (Weiss,
1970). Random assignment to different treatment conditions with the
use of control groups is, for example, difficult and may not be feasible
in most mental health centers. Quasi experiments (Cook & Campbell,
1979) are frequently more practical, as long as they are well-planned,
carefully implemented, and cautiously evaluated. Schulberg (1981)
suggested using small group samples in a manner analogous to a single-
case design. If program quality is monitored through simple outcome
measures then more detailed assessments may be pursued with smaller
Aside from overall considerations of experimental method in
designing a method of program evaluation, there are other aspects that
need to be taken into account. First, does one measure individual or
ecological change? Second, is it best to assess change through
standardized measures or through subjective individual goal attainment?
Third, from what point of view should the assessment be made?
Crucial to the issue of what is to be assessed is what constitutes
effective psychotherapy. Most current outcome studies look at such
variables as the psychiatric status of the client, his/her level of
social functioning, and the client's satisfaction with treatment
(Schulberg, 1981). There exist a variety of well-established instru-
ments for these purposes, such as the Global Assessment Scale
(Endicott, Spitzer, Fliess, & Cohen, 1976).
On the other hand, Cyntrynbaum, Snow, Phillips, Goldblatt, and
Tischler (1976) call for evaluation to take a more ecological perspec-
tive. As well as measures of individual change, it is proposed that
family' and community data be collected (e.g., socio-demographic
information, physical health, school and peer adjustment, and family
and individual perspectives). They advocated that successful treatment
should involve environmental as well as individual change and, thus,
should be a part of evaluation procedures.
Closely related to the issue of what to measure is how to measure
it. This usually involves a decision between standardized measures and
subjective goal attainment scales. Arguments favoring the use of
standardized measures emphasize that CMHCs should focus on a few valid
and reliable generic measures that are limited to a few areas of
personal functioning. This approach encourages comparability of treat-
ments and results across centers. For example clients might be given an
activities of daily living scale as a pre and post treatment measure.
Progress would be determined on the improved levels of functioning as
revealed by this instrument.
Another approach favors goal attainment measures which set idio-
syncratic goals for each client (Kiresuk & Sherman, 1968). Change is
not assessed by norms on a standardized instrument, but according to a
client's observed functioning and compared to an initial rating. The
latter approach is commended for its specificity with regard to the
individual client. Individual results are less likely to get lost in
group aggregate data.
Both approaches can provide useful information about programs. But
it is difficult, if one does not limit the number of goals employed, to
compare results between programs if one uses a goal attainment scale
From what point of view should the assessment take place? The
clinician, the client, the family, and the independent observer all
have different perspectives. There seems to be some agreement that
multiple vantage points are desirable. Some points of view are more
suited than others to different outcome measures. The therapist, for
instance, may not be the most objective observer; however, he/she is
frequently in the best position to rate clinical indicators of pathology
and subsequently any progress that may be made. Such observations may
not be evident to others. Again, objectivity, reliabiTity, salience,
and cost are taken into account in answering the question of who should
observe and make ratings (Rydman, 1981).
Topics of Special Concern in Evaluating Children's Issues
When children's services were added to CMHCs in 1975 it became
necessary to evaluate them. In addition to the general problems con-
cerning evaluation of mental health services previously discussed,
children's services have some additional complexities. Levitt (1971)
noted that a child is a developing organism and many symptoms are sub-
ject to spontaneous remission. This makes it difficult to determine
if treatment or developmental factors alone contributed significantly
to success. The tendency of certain symptoms to occur at particular
ages, remit spontaneously, and then be replaced by new symptoms is also
a consideration. This tends to make the researcher more cautions when
interpreting results and concluding that the intervention truly bene-
fitted the client. Finally, when family members are in treatment con-
current with the client,treatment is compounded and variables related
to change are more difficult to assess. It is thus difficult to attribute
change to a particular technique or strategy when multiple treatment
The maturational threat to internal validity might be eliminated
through use of controlled research, but Walker and Zinober (1978) noted
the ethical difficulty involved in identifying children with unmet
emotional needs and postponing their treatment for research purposes.
They suggest, instead, the use of measures that have normative data in
order that tested results might be compared to similar groups who were
part of a normed population. Yet, few, if any, instruments exist in the
field of emotional development.
The question of who is the best informant to report change is
especially crucial for children. Therapists are often selected to
perform assessment because of training and experience. However, as was
noted to be true with adults, personal involvement in the success of
treatment may present an inherent bias. Problems include perceptual
distortion, bias due to theoretical orientation, and awareness of
administrative consequences (Kiesler, 1966). Fiester (1978) concluded
that therapists' ratings of goal attainment in child clients were higher
in some areas than the child and/or parent reports. This is not a
general conclusion, however, as in other cases parents and children
either tended to rate progress higher or about the same as did the
therapist (Fiester, 1978).
Parents as informants may have many of the same problems as thera-
pists. They observe their child's behavior and are aware of pre and
post differences,but there is also an inherent lack of objectivity in
making ratings. Other informants, such as teachers or relatives, may
be more reliable and provide more valid information with respect to the
client's functioning, but they too may be unable to give unbiased in-
formation in certain situations.
Similar complexities exist when children report data. Children
may be unable or reluctant to supply information, especially about
internal states. Furthermore, the self-reports of children may be
seen as unreliable and irrelevant by adults who are more likely to be
the judge of success.
Perhaps the best approach is to rely upon several sources of
information. "The multi-informant approach assumes that each informant
has access to different kinds of valid information on the patient and
the effectiveness of treatment . combining the [different] perspec-
tives can result in a general index of treatment effectiveness"
(Walker & Zinober, 1978, p. 174).
Outcome Studies in Community Mental Health Centers
Research studies have been conducted for individual centers and
statewide systems. Most often they have focused on broad issues of
efficacy, such as whether or not CMHCs can keep clients out of state
hospitals (Becker, 1974). Some studies have given attention to
P.L. 94-63 requirements to assess accessibility and adequacy of service
(Cox et al., 1982; Fiester, 1978). Others (e.g., Fortney, 1981; Kline
& King, 1973; McGrogan, 1976; Sue, McKinney, & Allen, 1976) have been
interested only in specific treatment outcomes.
In a review of the Community Mental Health Journal by Cox et al.
(1982) few studies were found which measured treatment effectiveness.
Reported research was usually descriptive rather than experimental.
None of the studies reviewed were methodologically rigorous enough to
establish causal relationships between dependent and independent
Descriptive data relating antecedents and outcomes have been re-
ported on some variables. These include data obtained from the clinical
record (i.e., age, sex, occupation, and diagnosis); data related to
treatment (i.e., length, type, and intensity); and data from psycho-
metric tests (e.g., the MMPI).
A study in Washington State provided one example of accountability
procedures that served several different purposes (Cox et al., 1982).
This study was concerned with the problem of statewide measurement of
CMHC populations. Two samples of subjects were defined. One sample
from current caseloads was used to give a descriptive cross-section of
clients receiving services. Another sample, drawn from new admissions,
was assessed prior to and after receiving treatment. Assessment in-
cluded work and leisure activities, relationships with others, psycho-
logical distress level, and other areas of functioning. The study
revealed gaps in service delivery patterns. For example, clients who
scored high on a psychological distress scale received the fewest
services. The study helped convince legislators of the importance of
One study (Becker, 1971) typified another kind of evaluative
effort, measuring program achievement in a specific CMHC. It related
admissions to a state mental hospital to service at a CMHC in Oregon.
Records were used from both institutions to document either increase
or decrease in admissions to the hospital following implementation of
the CMHC. It was concluded that the center had little impact on admis-
sion to the state hospital. But, the center and hospital seemed to
effect different populations. While the center provided services to
clients who presumably needed them, such services were not equally
available to all clients who came to the center, as indicated by the
lack of state hospital clients in outpatient caseloads.
Another group of researchers (Fortney, 1981; Kline & King, 1973;
McGrogan, 1976; Sue, McKinney, & Allen, 1976) studied outpatients and
investigated length of treatment, type of treatment outcome, and/or status
of termination. McGrogan (1976), for example, examined forty independent
variables and their potential use as predictors. Information from
clinical records was found to be the most predictive of four variables
(duration of therapy, termination status, termination condition, and
cost of treatment) as opposed to other information related to type of
treatment or psychological testing. However, overall results were dis-
appointing. It was concluded that a heterogeneous population used
center services and that a variety of variables contributed to
terminating prematurely or remaining in therapy.
Dropping out of therapy at a CMHC is a significant problem and
may range between 35 and 40% (Kline & King, 1973; Sue et al., 1976).
Centers were intended to provide care to a variety of patients using
whatever services may be necessary to meet their needs, but possibly
this expectation is unrealistic given the limited resource of most CMHCs.
Sue et al. (1976) noted the value of identifying and describing patients
who rejected a center's services. Who benefits? What needs are not
met? Early termination is also an important consideration because
seeing clients who immediately drop out postpones services to others
Generalizations about early termination in CMHCs have been diffi-
cult to make because the definition of premature termination varies from
study to study. Studies also vary in terms of population characteristics
and the nature of the setting. For example, Sue et al. (1976) found
clinical diagnosis not to'be predictive of dropping out while Kline
and King (1973) found diagnosis to predict termination.
In one study,data were obtained from over 3,000 clients in 17 CMHC
facilities (Sue et al., 1976). Defining premature termination as drop-
ping out after only one session, it was concluded that ethnic minority
clients with lower incomes and educational levels, who were diagnosed
as psychotic and who were not assigned to individual therapy, were more
likely to terminate prematurely. Such clients were more likely to be
seen by paraprofessionals. They were most often assigned for diagnosis
and evaluation only rather than regular treatment.
Kline and King (1972) through a cluster analysis found eight sets
of variables, inclusive of 39 separate variables, to identify dropouts.
Different clusters were associated with depression, danger to self,
thought impairment, and danger to others. Dropouts tended to be younger
or recently married. Life histories of broken homes and unresolved
psychological issues pertaining to childhood also were a factor. The
researchers concluded that dropping out of treatment was another
adjustment failure similar to those of the past.
Significantly, however, the dropout group in Kline and King's
study were not distinguished by symptoms of the chronically mentally
ill. Being younger than other clients, this group presented a picture
of a group at risk for development of more serious problems.
These conclusions were corroborated by Fortney (1981). He sug-
gested that dropping out of treatment was a failure on the clients' part
to fit into the psychotherapeutic system. In effect, dropping out was
an indicator of the selection processes used by the center. Ten out
of sixteen demographic and treatment variables were found to be pre-
dictive of dropping out. Factor analysis isolated four elements:
social stability (age, family size, income, contact with significant
other); socio-cultural experience (age, race, education); therapeutic
stance (presence of a treatment plan, professional therapist); severity
of diagnosis (use of medication, diagnostic category). The results of
the preliminary analysis were not confirmed in a sample one year later.
This failure was attributed to changes within the center which created
new and different factors and, subsequently, affected who stayed in
Dropping out of treatment is a complex process involving many
variables. However, most studies test a great number of hypotheses.
This increases the probability that conclusions could be spurious.
Increasing the number of variables studied reduces the probability of
finding a true effect when it is present (Levitt, 1971). Cohen and
Cohen (1975) suggested that using a large number of variables, in the
interest of thoroughness, is seldom justified. If a large number of
variables is to be considered, multiple regression theory holds that
such research is appropriate only when the research goal is predictive,
the N is very large, the results are to be substantively interpreted,
and if a cross-validation study is undertaken (Cohen & Cohen, 1975).
Another theoretical problem with studies that have used premature ter-
mination as an outcome criterion was cited by Luborsky, Auerbach, Chandler,
Cohen, and Bachrach (1971). They observed that there is no explicit evi-
dence that dropping out of treatment is consistently related to lack of
client improvement. Although length of treatment was usually related in a
positive fashion to outcome, the conclusion that length itself was a
curative factor was questioned for several reasons. Therapists may over-
estimate positive change in longer-term patients with whom they may be
assumed to have some personal investment. Conversely, therapists may
rate early dropouts negatively, assuming a minimum number of sessions
are necessary before change can occur. Thus while premature termination
is a frequently used indicator of outcome, it is not always clear what
this variable involves.
Notably, all the aforementioned studies concerned with treatment
outcome in mental health centers were concerned with adults. No data
were collected on children and/or adolescents. One study (Kline &
King, 1973) noted the importance of childhood experience in contributing
to dropping out of treatment as an adult. In some studies (e.g., Kline
& King, 1973; Sue et al., 1976), age was a variable, but only in
increments over 18.
There is one significant exception. Fiester (1978) designed the
Automated Comprehensive Children's Evaluation System (ACCESS) to fill
a void in evaluating children's services. Based on a modification of
goal attainment scaling (Kiresuk & Sherman, 1968), the therapist and
client establish from one to three goals for treatment. Goals are
broken down into different levels of attainment from best possible to
worst possible outcome and are selected from 14 possible standardized
skill areas to enable comparisons among therapists. At the end of
treatment, the therapist and child or parent rate their perception of
the child's level of attainment with respect to each goal.
Fiester described changes associated with the program in 12 out of
14 skill areas. His study encouraged rational planning of new children's
services and the development of staff inservice training programs. It
did not, however, relate either client (e.g., demographic character-
istics) or treatment (e.g., duration of treatment) variables to outcome.
Thus, program evaluation attempts in CHMCs: 1) seldom separate
effects for adults from those for children, 2) seldom study institu-
tional processes beyond a single level (i.e., studying goal attainment
in a specific area for a specific therapist), and 3) seldom investigate
specific treatment outcomes as a function of situational as well as
client related variables (where such studies exist, they are often
subject to a criticism of making a large number of non-independent
statistical analyses). Overall, little outcome research on program
effectiveness with children exists.
Effectiveness of Evaluation
Pressures for more accountability have created several problems
for centers. One was the demands for similar information from different
monitoring groups. Another was that accountability efforts diverted
energy and resources from service delivery. Also, evaluation criteria
prescribed by external sources are not necessarily useful to local
programs (Keppler-Seid et al., 1980).
Prior to the enactment of P.L. 94-63 requiring centers to maintain
accountability data, Rocheleau (1974) reviewed Florida mental health
centers and noted that CMHCs did not assess the impact of evaluation
upon later performance. Furthermore, evaluation efforts, although con-
suming significant amounts of resources and staff time, appeared to have
little effect on the organizations themselves. Directors of centers
undertaking evaluation expected little useful information from such
efforts and results were used infrequently in decision making.
One of the desired outcomes of P.L. 94-63 was that accountability
studies would not only produce data helpful to NIMH in monitoring
legislative intent,but would serve as a stimulus to improve services
(Flaherty & Olsen, 1982). Yet, Rossman, Hober, and Ciarlo (1979) found
that program evaluation ranked eighth or ninth out of ten sources of
input with respect to relevance for decision making by CMHC managers.
This is in contrast to "input from immediate supervisors" which ranked
first. Ellsworth (1973) found that when feedback on treatment efficacy
was provided to staff in two separate programs, only one program improved
its performance in response to the data. This outcome eventually sub-
sided to the prestudy level.
An exception to these findings was Fortney's (1981) results of an
evaluation study of a mental health center in Gainesville, Florida. This
study sought to link client, therapist, and treatment variables to
outcomes. The preliminary study noted four factors which discriminated
between positive and negative outcomes. However, a cross validation
study a year later did not confirm the original findings. Fortney
speculated, in explaining the results, that the center had improved its
practices partially in response to the initial study so that the
original factors no longer applied.
There is general agreement (e.g., Bigelow, 1975; Flaherty & Olson,
1982; Rossman et al., 1979) that there are factors which determine
the usefulness of evaluation efforts to the agency. They are
1. The study should result from the felt needs of the center and
the desire to monitor its own performance, as opposed to requirements
being imposed from the outside.
2. The focus of the study (i.e., cost, amounts of service delivered,
etc.) should consider the audience addressed. For instance, in one
survey (Rossman et al., 1979) studies related to cost were preferred
by administrators while studies related to performance were felt to be
most relevant by clinicians.
3. The dissemination of findings and their usefulness to the agency
4. Program evaluation data are most likely to have an impact
when management receives feedback that is directly involved with cur-
rent issues facing the center and when information is presented in a
simple and personal manner.
Outcome Studies Related to Specific Variables
in the Current Study
There are several studies which relate client variables to therapy
outcomes in different treatment situations (e.g., Baekeland & Lundwall,
1975; Levitt, 1971; Luborsky et al., 1971). However, there is little
research on the type of treatment, parental involvement in therapy,
duration of treatment, referral source, and severity of diagnosis,
especially as they relate to children in a mental health center setting.
It is important in understanding the function of a mental health
center not only to demonstrate whether changes occurred in clientele
as a result of treatment, but also how different client groups are
affected by treatment (Suchman, 1967). For example, are children whose
parents participate in therapy more positively affected? Are children
who are referred by the court less positively affected by treatment? If
a center can answer such questions, then it can make institutional
changes to better serve clients (Cox et al., 1982).
A review of related research influenced the choice of variables
selected for study in this investigation. The decision to focus on a
discrete number of variables was made on the basis of Cohen and Cohen's
(1975) caution regarding the chances of spurious significance when
simultaneously studying a large number of variables. The following
variables received specific attention.
Many people believe that parent involvement is important if thera-
peutic changes are to occur in a child client. This is in accord with
a systemic view of family functioning which emphasizes that a change
within one family member will affect the entire system and vice versa
(Haley, 1974; Minuchin, 1974). If an individual's role in a family
helps maintain a particular family balance, then family members can
maintain or help change a child's behavior.
Three studies reviewed by Levitt (1971) supported parmtal involve-
ment in treatment. His review suggested that treatment was least effec-
tive when the focus was an the child alone and most effective when
mother, father, and child were included. Two studies (D'Angelo &
Walsh, 1967; Gluck, 1964) clearly favored the addition of the father
Parental involvement in therapy may not in itself be a curative
factor, however. There are examples of treatment programs with chil-
dren which have been successful without any parent involvement (Brody,
1978; Seeman, 1964). It is also possible that the child's age may act
as a moderating variable on the effect of parent participation in therapy.
Hartmann (1968) found that either group or individual therapy for parents
separately from their children was unrelated to outcome with hospital-
ized adolescents. The negative implication of this finding for parental
involvement is mitigated by the fact that the parents were seen
separately. There is evidence in a related field that conjoint (seen
together) marital therapy is more effective than concurrent (seen
separately) marital therapy (Cookerly, 1974). The same effect may
apply for parents and children.
Studies of specific dimensions of parent involvement generally
indicate their importance in successfully treating the child. Lake
and Levinger (1960) found parental disagreement with therapists' problem
assessment, poor cooperation in the initial interview, a parental
tendency to locate the problem in the child rather than in the parent,
and external coercion in bringing the child for treatment, to negatively
affect treatment outcome. Another study concluded that in unsuccessful
cases, parents were less likely to be in treatment (Ross & Lacey, 1961).
Finally, it was noted by Lessing and Schilling (1966) that improvement
was more likely to occur if the mother recognized and accepted her
responsibility in the child's problem.
These findings are not universal however. Levitt (1957) and
Williams and Pollack (1964) found no significant difference between
dropouts and those who remained in treatment on the basis of different
dimensions of parent involvement.
Knowing more about parent involvement in treatment could influence
children's programs. If it were found to be a significant variable,
therapists might require or work harder to obtain parent participation
as a condition of providing services.
Type of Treatment
Type of treatment, as defined in this study, focuses upon individual
and family therapy separately and in combination, couples therapy, use
of parent and child groups, and use of medication. These are the dif-
ferent modalities of treatment frequently used with children.
Overall success rates have been established for child therapy summing
across different treatment modalities (Levitt, 1971). However, there
are few studies comparing the effect of different treatments. Malouf
and Alexander (1976) noted that with many mental health facilities
adopting non-traditional approaches, such as family therapy, it is
important to answer the question: Which treatments are most effective
Most comparative outcome studies consider the issue of whether
family therapy is more effective than individual therapy for children and
adolescents. Wellischet al.(1976) studied a group of hospitalized
adolescents and recorded a rehospitalization (return to hospital after
discharge) rate of 43% for the group that received individual therapy
while none of the group receiving family therapy were rehospitalized.
Minuchin (1974) described positive results achieved in working with
families of diabetic, asthmatic, and anorexic youngsters. Structural
family therapy was able to effect substantial improvement in 83-100%
of patients and these changes endured in 91% of the cases. Alexander
and Parsons (1973) found recidivism rates for juvenile offenders who
received short-term behavioral family therapy to be substantially less
(26.1% versus 49% to 73%) than for youngsters who received non-specific
family group therapy or church sponsored eclectic therapies. One recent
review of family therapy literature indicated that every comparative
study showed non-behavioral family therapy either equal or superior to
other programs which included such approaches as individual therapy,
probation programs, parent counseling, methadone programs, or milieu
therapy (Gurman & Kniskern, 1978).
Programs which attempt to teach communication, behavioral, or
parenting skills have been termed "skill training" by Malouf and Alex-
ander (1976). All studies reviewed which in some way attempted
parent training in reinforcement techniques were successful. Outcomes
were maintained over time.
In addition, Parent Effectiveness Training (Gordon, 1971) and
Systematic Training in Effective Parenting (Dinkmeyer & MacKay, 1976)
have also been shown to be effective in changing either parent attitudes
or behaviors, relationships, or child behaviors. Few studies involving
these approaches include comparisons between different treatments,
If centers were able to establish which types of therapy were most
effective, they could better plan their services. For example, if
parenting groups were shown to be as effective as individual therapy
for children, then it would be less costly to serve clients in such
groups. Similarly, if family therapy is no more effective than indi-
vidual therapy, why offer it as an alternative since it is more costly
Duration of Treatment
Length of treatment may or may not be related to successful out-
comes. It has been argued that there should be a positive relationship
between outcome and number of treatment sessions (Levitt, 1971). Others
have argued that the relationship between length of treatment and success
is not a linear one (Meltzoff & Kornreich, 1970). Some clients may be
helped by very brief interventions, while others do not show any appre-
Length of treatment and outcome for children is generally un-
related. Lessing and Schilling (1965) concluded that neither frequency,
number of sessions, nor total time in treatment were related to outcome
in a large outpatient clinic serving children and adolescents. The
same conclusion was reached by Hartmann, Glasser, Greenblatt, and
Solomon (1968), Phillips(1960), and Sheppard (1966), all of whom found
no correlation between outcome and number of treatment hours.
For adults, however, the case may be different. Luborsky et al.
(1971) found that in 20 of 22 studies of time-unlimited treatment,
length of therapy, as indicated by number of sessions, was positively
correlated with outcome. The authors stopped short of concluding that
the length of treatment itself was a causative factor, noting the
tendency to rate long term clients positively and produce skewed
therapist ratings. It has also been argued that those who stayed in
treatment longer may simply have "felt" helped rather than making any
Time-limited therapies appear to be no less effective than time-
unlimited ones (Goldstein, Heller, & Sechrest, 1966). One study con-
cluded that limiting therapy was associated with a more positive outcome
(Schlein, Mosak, & Dreikeurs, 1963). Further evidence for the efficacy
of short-term therapy comes from Rosenthal and Frank (1958) who dis-
cussed the possibility of positive change occurring within five sessions.
Frank (1963) found that positive change could occur in the first session
after the administration of a placebo. Heilbrun (1974) concluded,
based on a series of investigations, that termination after the first
interview could signal satisfaction. Overall, adult patients with acute
situational problems derive little additional benefit from long-term
treatment and may drop out after their needs have been met. Dropping
out does not necessarily signify treatment failure, however.
These findings are especially pertinent for mental health clinics
that have high dropout rates (Fortney, 1981; Smith & Kline, 1973;
Sue et al., 1976). The fact that CMHCs possess high dropout rates which
come early in treatment and have an overall low median number of
sessions makes it important to document success of treatment by
measures other than premature termination. It is questionable to use
dropping out of treatment as a sole measure of success or failure until
an unequivocal relationship between dropping out and treatment outcome
Duration of treatment, as it relates to children's programs in
community mental health centers, is still an important issue. If an
optimal duration of treatment can be identified, centers could more
efficiently plan their services.
Severity of Diagnosis
Most research indicates a close relationship between severity of
diagnosis and/or level of personality functioning and lack of treatment
success. Ross and Lacey (1961) noted that the child who drops out of
treatment was more likely to have had developmental difficulties, un-
usual behavior and/or non-specific symptoms, and to have been a school
truant. The Institute of Juvenile Research (Lessing & Schilling,
1966) reported that degree of emotional disturbance was inversely
related to improvement in cases where mothers and children were treated.
A review of outcome research with children (Levitt, 1971) cate-
gorized children into neurotic, psychotic, and acting-out domains ("neurotic"
was defined as a child who was neither a psychotic, a behavior dis-
order, nor a special symptom). Neurotics improved at a rate of 67.5%,
psychotics at a rate of 65.1%, while children with acting-out symptoms
improved at a rate of only 55%. It therefore appeared that children
with behavior disorders were the most resistant to treatment. This
finding is alarming as aggressive, destructive, and anti-social behaviors
carry over into adult life with greater frequency than other emotional
disorders. High proportions of these children are arrested, hospital-
ized, or abused, and experience alcoholism and divorce as adults
Research with adults corroborates findings with children with
respect to severity of diagnosis. Barr (1977) concluded that psychotics
averaged the longest in attendance while persons with marital problems
left most quickly. One review found initial level of personality
functioning to be positively related to treatment success in 14 out of
15 studies (Luborsky et al., 1971). Similar results were reported for
the more severe diagnoses (e.g., psychosis), noting that these clients
benefitted less from psychotherapy. It was concluded that the healthier
the patient was to begin with, the better the outcome. The converse
also appeared to be true.
Another review substantiated this position (Baekeland & Lundwall,
1975). It was reported that sociopathy, paranoia, alcoholism, and
extreme depression were generally characterized by premature termination.
Anxiety and moderate depression, on the other hand, improved the chances
of remaining in therapy.
There is some research on this topic specific to mental health
centers. Anxiety neurosis was found to be the strongest predictor of
termination status in a multi-factor study (McGrogan, 1976). In another
study adult dropouts tended to be more psychologically impaired and had
a greater tendency to act out (Kline & King, 1973). Significantly,
presence of childhood problems as recalled by adult clients was asso-
ciated with dropping out of treatment. Such clients more often came
from broken homes and had conflicts in the areas of comfort and
dependency. This group also included more suicidal persons and clients
who were generally more psychologically impaired (Kline & King, 1973).
It is important to determine whether severity of diagnosis affects
outcome in mental health centers. Given this information, informed
decisions could be made about serving different diagnostic groups.
Resources could be concentrated in areas where outpatient treatment was
most successful. On the other hand, services could be improved for
diagnostic groups not benefitting from current treatments. Referrals
could be more justifiably made if resources were not adequate to provide
Research indicates that referral source is correlated with treat-
ment outcome. In hospital medical settings, leaving treatment against
medical advice was more likely for clients who were court referred
(Altman, Brown, & Sletten, 1972). In another study in an outpatient
clinic it was found that refuserss," defined as those who were accepted for
treatment but did not show up, were more likely to be referred from
the school systems (Williams & Pollack, 1964).
Motivation for treatment has consistently been demonstrated to
affect treatment outcome. One review suggested that in 34 of 41 studies
lack of motivation was the most important factor in defection from
treatment (Baekeland & Lundwall, 1975). Although motivation may have
been defined differently in the studies, the conclusion was a patient's
personal reasons for seeking treatment determined whether or not he or
she stayed in it.
In the treatment of children it is the adult's motivation rather
than the child's that is most salient. At least one study investigated
outcomes as a function of parental attitudes toward treatment (Lake &
Levinger, 1960). Parents of treatment dropouts were more likely to have
submitted to institutional pressures suggesting that they take their
children for treatment. This conclusion was challenged, however, by
Mindess' finding that working with court-referred clients was as suc-
cessful as with self-referred (Mindess, 1953). It may be that if the
client perceives the agency as favorable to him regardless of referral
source, the effects of an initial referral may be mitigated.
It is apparent that motivation for treatment on the part of the
parent may be an important discriminator of treatment outcome in a
community mental health center. The study of referral source as an
indicator of motivation may help shed light on barriers to treat-
A review of the professional literature indicates that program evalu-
ation research is needed to validate mental health center services for
particular client types. This study is designed to examine the attributes
of treatment success or failure in relation to variables of involvement in
treatment, referral source, severity of disturbance, and duration and type
of treatment. These have all been shown to be relevant to treatment
outcome in other settings. The question is whether these can be con-
nected to treatment outcome with predictive implications in the mental
health centers studied. This will afford direct benefits to patients
in terms of improvement of treatment as well as to agencies in terms of
defining the most effective and efficient ways to utilize their resources.
The purpose of this study was to investigate variables related to
the effectiveness of mental health center outpatient services to
children. Using information from medical records, case data were
gathered on five variables: referral source, type of treatment,
severity of diagnosis, parent participation in treatment, and duration
of treatment. The relationship of these variables to one another were
examined by comparing cases described as successful and cases described
as unsuccessful. Data on other variables believed to affect treatment
outcome were also gathered.
This chapter is organized into the following sections: population,
description of data source, sampling procedures, research hypotheses,
research design, method of data collection, investigative procedures,
analysis of data, and the limitations of the study.
Population and Sample
The population consisted of approximately 1300 children and
adolescents, ages 0-17, who had received help from two northern Florida
community mental health centers during the years of 1980 to 1983. The
sample for the study consisted of approximately 275 randomly drawn
child and adolescent cases.
The Mental Health Centers
The District III Mental Health Board, Inc., from which the sample
was drawn, was located in one of the fastest growing population areas
in the United States. It was mainly a rural area with all the counties
served by the District Board below the state average in terms of popu-
lation density. Racially, 84% of the population was white, 13.3% was
black, and 1% other races, which was the approximate racial balance
throughout Florida. Regarding age, 80% of the counties in District III
had a higher percentage of young people (17 and under) than the state
as a whole and a below average percentage of older (55 and over)
citizens. Average educational attainment was below the state average,
although there were several centers of higher education in the dis-
trict (District III Mental Health Plan, 1979-1980).
The children's programs (District III Mental Health Plan, 1979-1980)
selected for this study served a three county area in Northern Florida.
Two of the programs were administered under the North Central Florida
Community Mental Health Center, and served Alachua and Levy counties.
Service locations were in Gainesville (the Child, Youth and Family
Center) and in Bronson (the Community Counseling Center). The other
children's program, administered under the North Florida Community
Mental Health Center, was located in Lake City, Florida (the Columbia
Counseling Center), and served Columbia County.
Originally the two centers were combined, having been established
as a non-profit organization in 1972, under a federal staffing grant
from the National Institute of Mental Health. The original service area
included ten rural counties in the north central part of the state. In
1980 the center split into the North Central Florida Community Mental
Health Center, comprising the four southern counties of the original
catchment area, and the North Florida Community Mental Health Center,
comprising the six northern counties. This transition marked the end
of an eight year federal staffing grant for the North Central Florida
Community Mental Health Center and the beginning of a Federal grant for
the North Florida Community Mental Health Center.
Both centers received funding from county, state, and federal
governments, third party payors and client fees. State dollars flowed
to each center through the District III Mental Health Board.
The District III Mental Health Board was a private, non-profit
corporation. In addition to the disbursement of funds to the center,
its responsibilities included the identification, review, and evaluation
of mental health/substance abuse needs, services, and facilities. The
Board monitored the centers to assure contractual compliance. It also
served as the liaison between the Department of Health and Rehabilita-
tive Services, the District's 15 county governing bodies, and providers
of mental health and substance abuse services (District III
Mental Health Plan, 1979-1980).
The individual centers operated under the direct control of Boards
of Directors composed of residents of the service area. The executive
directors for the centers were responsible for implementing District
Board policy, managing center operations, and maintaining positive
relationships with government agencies.
Whereas the centers received direct federal support during and
prior to 1980, the federal government changed its funding pattern in
1980. Money to the centers now came through the state government in
the form of block grants. The state of Florida chose to maintain mental
health funding at 85-90% of the previous levels. Mental health centers
were thus under the authority of the states and local communities.
As a consequence of the expiration of federal legislation (in 1980)
governing community mental health centers, no national requirements for
monitoring or evaluating centers existed. For the two centers in this
study, monitoring was done jointly by the state Health and Rehabilita-
tive Services department and its contractor for mental health services,
the District III Mental Health Board.
Catchment Area 3A. Catchment Area 3A of District III was geo-
graphically large and included Columbia, Hamilton, Lafayette, and
Suwannee counties. However, the area was sparsely populated (about
73,000) with all counties falling below the state mean of population
density. It also had the slowest population growth rate of the District.
About one-third of the service population was 17 years old or younger,
a percentage higher than both the district and state averages. The
census count by sex was consistent with the state averages. The per-
centage of black residents was, however, higher than either the
District or state averages. The entire catchment area had been
designated a poverty area in 1979-80 by the then Department of
Health, Education and Welfare. Other special problems included a
high rate of teenage pregnancy, substandard housing, and a low number
of school years completed (District III Mental Health Plan, 1979-
In 1979-80, the child and adolescent population in need of mental
health services was estimated to be 3,100 individuals. At that time,
the center was meeting approximately 16% of the estimated needs.
Children's needs were considered a high priority as evidenced by
the results of a mental health needs survey done in 1979-80. Child
abuse was ranked second (out of five choices) as a problem in two of
the counties while children with emotional difficulties requiring out-
patient counseling was ranked second, fourth, or fifth, in three of the
counties (District III Mental Health Plan, 1979-1980). Out-
patient services to meet the need of family problems was cited in 1980
to be a "most needed" service as no private sector resources existed in
the area from which to obtain private psychiatric care.
Service delivery was based on an organizational concept in which an
Area Director and a team of clinical and support staff were responsible
for the delivery and coordination of services within a geographic area.
The four counties were served by two clinics, Columbia Counseling Center
and the Suwannee River Counseling Center. The clinic of interest to
this study, the Columbia Counseling Center, employed two full time
mental health professionals specifically serving children during the
period 1980-1983. One of these was a professional and the other a
paraprofessional. The paraprofessional was replaced in the later part
of 1983 by a professional.
Catchment Area 3B. Catchment Area 3B was the largest area in
District II in terms of population. Approximately 190,000 residents
lived in the service area. The Gainesville urban area comprised 62%
of the population while the remaining 38% resided in small, rural com-
munities. Services were often limited in rural areas because of trans-
portation difficulties for clients. Approximately 29% of the population
was seventeen years of age or younger. Census counts by race and sex
were consistent with state averages. However, the area had a low
median income with over 50% of persons in poverty. There were also
high rates of teenage pregnancies and students dropping out of school.
In addition, Alachua County had a high mobility rate, a large population
of youth, and one of the highest per capital crime rates in the nation.
These factors indicated a relatively unstable community at risk for
developing mental health problems.
In 1979-1980, the child and adolescent population in need of mental
health services was estimated to be 4500 persons. About 16% of this
group received services. Needs assessments for Alachua and Levy counties
included the following priorities concerning children: child abuse,
seriously emotionally disturbed children requiring outpatient counsel-
ing, the need for living facilities to prevent dependent children from
becoming emotionally disturbed.
Service delivery in the 3B area was again based on an area concept
of organization in which a team of clinical and support staff delivered
services to children in a geographic area not necessarily consistent
with county lines. Alachua County was served by the Child, Youth, and
Family Center (CYFC) located in Gainesville, while Levy, Gilchrist,
and Dixie Counties were served by the Community Counseling Center (CCC)
located in Bronson. Additional Community Counseling Center outreach
clinics were located in Trenton and Cross City. The clinics of interest
to this study possessed the following staffing patterns: During the
period of the study the Child, Youth and Family Center had two MSW
social workers, one Ph.D. counselor, one part-time clinical psychologist,
and one paraprofessional counselor. The CCC employed six professional
therapists (master's degree or above).
Closed Medical Records
A closed medical record contains information related to three
phases of patient care: intake/diagnosis, treatment, and discharge.
During the admissions interview with each client, demographic and
clinical information are recorded, including race, sex, age, family
size, income, marital status of parent, referral source, and history
of prior psychological services. (See Appendix A.)
Following the initial or intake interview, a clinician writes a
summary and a treatment plan. The summary includes the presenting
problem, medical, psychiatric, educational, social, and family histories,
and a case summation with both a diagnosis and prognosis. The clinical
diagnosis follows categories of emotional illness as described in the
Diagnostic and Statistical Manual III (APA, 1980). The treatment plan
is intended to complement the intake summary. It contains a brief
description of the client's problems, specific treatment goals, and how
these goals will be achieved.
A clinician's observations or chart notes describe the middle phase of
thetreatment process. The notes show who was involved in the session,
the prominent events which occurred, and goals for future sessions.
These notes are designed to be concise and theme oriented rather than
descriptive and factual.
Finally, when a chart or medical record is closed, the clinician
fills out both a discharge summary and a discharge sheet. The discharge
summary contains a brief summation of the course of treatment. It
includes the number of treatment sessions, who attended, and reports
any progress that was made toward the desired goals. The clinician
also rates the status of termination in such categories as "refused
treatment or dropped out after one session" or "referred to other mental
health center" on a discharge sheet. This evaluation reflects whether
the termination was by mutual agreement or viewed as premature. A final
rating of "much improved," "somewhat improved," "unchanged," "slightly
regressed," or "significantly regressed" is given at termination to each
treatment goal listed on the treatment plan.
It has been noted (Chapter Two) that a study of both treatment (type
and duration of treatment and parent involvement) and client (severity
of diagnosis and referral source) related variables was necessary to
understand the effectiveness of a CMHC. The following variables were
selected from a review of related research.
Parent involvement was defined as parent participation in a child
or adolescent therapy session beyond the intake interview. A parent
or guardian must have been present during the intake and given per-
mission for treatment. However, some parents out of fear, misunder-
standing, or personal uninvolvement preferred to be uninvolved and hoped
that the center would assume full responsibility for treatment. Other
parents wanted to be involved and willingly participated in the sessions.
Most clinicians in most cases recommended parental involvement in the
treatment of a child or adolescent. For various reasons, parent involve-
ment in some cases was non-existent or limited. Did parent involvement
make a positive difference? To help answer this question, it was
necessary to differentiate cases according to whether or not the
One session beyond the intake interview constituted parent parti-
cipation in treatment. The mean number of therapy visits for centers
was approximately 7.5 sessions per case. Thus proportionately, even one
session beyond that required for the intake attended by the parents was
Information concerning parent involvement was obtained from chart
notes where the therapist routinely reported who was involved in any
given session. If information was vague or ambiguous, the case was
recorded as undetermined and not used in the analysis of data.
Referral source was grouped into seven categories: self, school
system, Health and Rehabilitative Services (HRS) recommendation, com-
munity agency or church, physician or medical clinic, law enforcement
or court order or HRS performance agreement (see Appendix B). These
categories included the primary referral sources and also differentiated
voluntary, agency, and involuntary categories. Voluntary referrals
were those in which there was no element of coercion. Agency
referrals were those where the impetus for the referral came from an
agency, but no court order prevailed. Involuntary referrals were those
where a client was directed by police or court order or probation
officer to receive treatment at the Center.
This information was routinely recorded during the intake inter-
view on the admissions data sheet. If the referral source was unclear,
it was recorded as undetermined. If two or more sources were involved,
the most recent source was used.
Severity of Diagnosis
The diagnostic category of a client was related to the Diagnostic
and Statistical Manual III (ARA, 1980). Several codes may have existed
for a client.
Classification of psychiatric disorders in the DSM III is made
on two primary axes: Axis I includes clinical problems, i.e., conduct
disorders, developmental disorders, etc., and conditions not attributable
to a mental disorder that are a focus of attention or treatment, i.e.,
parent-child problems. Axis II includes personality disorders and
specific developmental disorders. Each axis may include more than one
diagnostic category. Other axes mentioned in the DSM III were not
utilized in this study as they were not uniformly used by the centers.
For purposes of analysis, recorders were instructed to use only
the first diagnosis listed on Axis I and the first diagnosis listed on
Axis II. A case was classified as "mild" if it was a condition not
attributable to a mental disorder or an adjustment disorder. Cases to
be classified as moderate-severe included conduct, thought, affective,
developmental, and personality disorders, or substance abuse. Cases
in this category were distinguished by their longer duration and/or a
specific pattern of behavior.
Type of Treatment
Type of treatment included 1) individual, 2) family/couple,
3) combination individual/family/couple, 4) group, and 5) medication.
For all but medication, two or more sessions in a modality were necessary
in order to qualify for the category.
Recorders identified the treatment modality(s) from the chart
notes. If there was no specific mention of who was involved in the
sessions the treatment category was marked undetermined. For a dis-
tinction of family/couple therapy, it is not necessary for family
members to be seen conjointly with the client. The needed only to be
present at the same session. Family treatment could not be separated
from couple treatment because of ambiguous references in the chart
Duration of Treatment
Duration of treatment was measured by counting the total number of
sessions attended either by the client and/or other family members.
Recorders noted the number of sessions attended from chart notes and
used the discharge summary for corroboration.
For each specific problem originally listed on the treatment plan,
the therapist specified outcome goals or objectives. Progress toward
these goals, at termination or other specific times, was rated on a five
point scale: significant progress (+2), some progress (+1), no progress
(-1), some regression (-1), severe regression (-2). To obtain an
overall rating for an individual chart, the specific goal ratings were
summed. Positive outcome was classified as any positive sum while a
negative outcome was classified as any negative sum. Undetermined
outcome resulted in cases being disqualified from the study.
Two hundred and seventy-five closed medical records charts (cases)
from three separate specialized children's programs within two separate
mental health centers were randomly*drawn from the study. The three
programs were 1) the Child, Youth and Family Center, Alachua County;
2) the Community Counseling Center, Levy County; and 3) the Columbia
Counseling Center, Columbia County. These programs were interested in
and willing to cooperate with the study.
Permission to access medical records for research purposes without
the client's express consent is allowed by both state and federal laws
if certain conditions are met. State statutes concerning confidentiality
*Additional unsuccessful cases (27) were sampled in order to elevate the
proportion of such cases to 40% in the sample.
usually require a client's written permission to use a case record for
research purposes. However, an exception is made if the client's
identity can not be deduced from the data collected. Such was the case
in this study.
Medical records librarians from each unit selected closed medical
records from the period of January, 1980, until December, 1983. All
cases available for review were numbered and then sampled according to
a table of random numbers. Approximately 1300 charts were available
for review in the three clinics.
Criteria for Inclusion Cases in the Study
In order for a case to be included in the study the chart had to
contain a) an admission data form, b) an intake summary or intake form,
c) a diagnosis according to the DSM III Manual, d) a treatment plan,
e) chart notes, f) a discharge summary, and g) therapist rating of
treatment goals. Records were rejected for the purpose of the study
if the outcome was indeterminant, outcome not rated, the child classi-
fied as mentally retarded or over 17 years of age, or in the event that
the child was not in treatment with the parent or legal guardian (e.g.,
foster care), or if the family had not returned for further sessions
after the initial interview. However, all cases rejected for the study
were counted (see Table 1).
Cases were divided between 162 (58.5%) positive outcomes and 112
(40.4%) negative outcomes. Approximately 65% of the total cases sampled
were rejected. The following reasons applied: 10% were discarded in
order to ensure sufficient negative outcomes for the sample; 44% had no evaluation
SAMPLING DISTRIBUTION BY CLINIC AND BY YEAR
1980 1981 1982 1983 Total
Total Cases Available 1352
of progress; 16% were cases where clients were older than 17; 22% were
seen for only the intake interview; 4% of the cases involved the child
being in foster care; 4% of the cases did not have a DSM III diagnosis
or had an inappropriate diagnosis (e.g., retardation). See Table 1 for
a summary of total cases available, cases sampled, cases accepted for
the study, and cases rejected.
Demographic Description of Sample
Approximately equal numbers of eases came from the Gainesville and
Lake City clinics while a smaller percentage (see Table 1) of cases
came from the Bronson Clinic. This was due to the smaller number of
cases available from this location. Frequency of cases by year (see
Table 2) were approximately equal, except for a smaller frequency in
1980. This was explained by the introduction of the DSM III Diagnostic
System in the middle of the year, making cases seen earlier in the year
ineligible for the study. Thus, cases were distributed equitably by
location and year.
Other demographic categories of interest included age of client,
race, sex, and income. Clients ranged in age from 2 to 17 years old.
This was divided into four categories corresponding with school status:
preschool, 2 to 5; elementary school, 6 to 10; middle school, 11 to 13;
high school, 14 to 17 (see Table 3). The data suggest that the younger
children were served less often by the mental health centers. The
racial balance evident in the sample (see Table 4) approximated the
racial balance throughout Florida. Males and females were represented
equally in the sample (see Table 5). When race and sex were grouped
together (see Table 6), white males were in the majority while black
females were in the minority. There was a range in income from $0 to
$32,000 a year. Income was divided into quintiles according to annual
earnings. These included the following categories: lower-lower, lower-
middle, middle-middle, upper-middle, and upper-upper (see Table 7).
Almost one-half of cases fell in the lower-lower category. This is
consistent with Brown's (1975) finding that over 50% of clients served
in community mental health centers had incomes below $5,000. As the
poverty index for a family of four was $9,862 (Beeghley, 1983) most
clients served in the sample could be classified as living at, below,
or slightly above the poverty level. In sum, distribution of cases for
race and sex were close to population averages while distribution of
cases for age and annual income was not representative.
Where family composition was concerned, almost 3/4 of the clients
sampled lived with only one natural parent, while 1/4 did not (see
Table 8). Approximately 2/3 of the clients came from families where
divorce or separation had occurred (see Table 9). These data reflect
a higher than normal distribution of cases involving divorce as the
percentage of divorce. involving children under the age of 13 in
Alachua County (the county in which one of the clinics was located)
was 33 to 40%'(Arey, 1979). Of those divorced, approximately
1/3 had remarried, which was close to the 35% figure cited for
all of Alachua County (Florida Public Health Statistics, 1980).
The percentages for divorce and separation and remarriage (see
Table 10) reflect approximately equal percentages of
PERCENTAGE OF CASES ACCEPTED FOR
THE STUDY BY YEAR
PERCENTAGE OF CASES BY AGE
PERCENTAGE OF CASES BY RACE
PERCENTAGE OF CASES BY SEX
PERCENTAGE OF CASES BY RACE AND SEX
White 44 39
Black 10 6
PERCENTAGE OF CASES BY ANNUAL INCOME
Upper- Middle- Upper- Upper-
Lower- lower middle middle upper
lower 7600- 11,000- 15,100- 25,000-
0-7500 10,800 15,000 24,800 32,000
48.4 19.5 16.2 11.2 4.7
(134) (53) (48) (30) (12)
PERCENTAGE OF CASES LIVING WITH ONLY ONE NATURAL PARENT
one parent Other
PERCENTAGE OF CLIENTS EXPERIENCING DIVORCE OR SEPARATION
divorced or separated Other
PERCENTAGE OF CASES BY MARITAL HISTORY
Parents divorced Parents divorced,
Intact families and remarried single
30.3 30.7 38.3
(84) (86) (107)
cases from families where no divorce had occurred or remarriage had
taken place. A higher percentage of cases were from single parent homes.
The data suggest that children from single parent homes were the group
most likely to receive help from the mental health centers.
1. There is no significant difference between successful and un-
successful cases regarding parent involvement in treatment.
2. There is no significant difference between successful and
unsuccessful cases in terms of referral source.
3. There is no significant difference between successful and
unsuccessful cases regarding severity of diagnosis.
4. There is no significant difference between successful and
unsuccessful cases for type of treatment used by the therapist.
5. There is no significant difference between successful and
unsuccessful cases in terms of duration of treatment.
Method of Data Collection
Chart Review Procedures
The principle investigator developed a procedure to review closed
medical records charts. Data included information about the client,
treatment, outcome, and the therapist. Information gathered about the
client included family income, client age, client diagnosis, race, sex,
family composition, parents' marital status, school performance, and
medical history. Information related to treatment included duration,
type of treatment, and parent participation in treatment. Information
on outcome included specific outcome ratings and whether or not the
termination was premature. Information related to the therapist in-
cluded the therapist's educational level (BA, MA or higher, parapro-
fessional) (see Appendix B).
Data were entered directly from the medical record onto a recording
instrument (see Appendix B) according to the recorder's guide sheet
(see Appendix C). The guide sheet listed the information to be ob-
tained and explained how and where this information was to be found in
a medical record. It also showed where the data were to be recorded
on the recording instrument. The items on the recording instrument
followed the natural order in which information appeared in the chart.
No identifying information was retained to identify the charts in
order to protect client confidentiality. The code numbers assigned by
the principal investigator and medical records librarian correlated any
given case with the medical records chart number. The correlative list
remained with the medical records librarian and was retained for the
purposes of 1) data retrieval in the future, and 2) insurance against
duplication in this or a future study.
A pilot study with 20 cases, not to be included in the study,
tested the recording and data collection procedures. It provided
information as to 1) average time to record a chart, 2) incidence of
missing or ambiguous information, 3) estimated the number of charts
that were required in a sampling in order to obtain the minimum number
of successful and unsuccessful cases.
Permission from the Centers
A letter was sent to each center inviting its participation in the
study. A proposed time schedule for the data review and recording to
occur and instructions for the medical records librarian was included.
Also included was an explanation of how the research might be expected
to advise center programs and an offer to share results. Both centers
gave permission to use their closed medical records for the purposes
of the study.
Selection and Training of Recorders
Employees of the three mental health clinics in the study who had
familiarity with the medical records of the agency were recruited to
assist in the study. The purpose of using employees was to ensure
confidentiality as well as take advantage of their familiarity with
the medical records and agency procedures.
One employee of the North Florida Community Mental Health Center,
Inc., reviewed the charts at that center (Lake City). One employee of
the North Central Florida Community Mental Health Center, Inc., reviewed
charts at the two locations (Gainesville and Bronson) at that center.
At a training session the general purpose of the study was pre-
sented. Recorders were not told the specific hypotheses involved.
They were given the study's time schedule and compensation for their
services was agreed upon. The recorders were made familiar with the
recorder's guide sheet and recording instrument. They were 1) given
precise definitions of each variable on which data were to be recorded,
2) shown how and where information was to be obtained from the medical
record, and 3) instructed as to how each item was to be recorded on
the data sheet.
Recorders practiced data gathering on the same medical records
charts. An inter-judge reliability coefficient or proportion of agree-
ment was computed to determine rater accuracy and reliability (see
Appendix D). Kappas (Ks) were used as an indicator of reliability
between judges for nominal level variables (Cohen, 1960). Items with
lower correlation coefficients were more fully explained or revised to
Data Collection Visits
All data were collected at the medical records library for each
respective children's program during a time that minimized disruption
to the center's regular operation (e.g., after hours). The investigator
was available initially and thereafter at intervals to answer questions
and clarify issues. He participated in the initial numbering and
random selection of charts, using a table of random numbers. Recorders
were able to review about 4 cases per hour. In the event that a
medical record was incomplete or did not meet the criteria for inclusion
in the study, another suitable chart also chosen on a random basis was
substituted. In the case of the Bronson Clinic, all records available
were sampled in the study.
One recorder reviewed a total of approximately 100 cases while
the other reviewed approximately 177 cases. Collection of data at each
site required no more than 10 weeks to complete.
Upon the submission of recording instruments, the investigator
checked to be sure they were properly filled out. When the total number
of cases was reviewed, data from the recording instruments were key
punched onto cards.
1. The investigator developed recording procedures to obtain
needed information from charts.
2. The investigator visited clinic sites to explore with adminis-
trators the feasibility of collecting the desired information.
3. Permission to do the research was obtained.
4. Data recorders were recruited and trained.
5. Approximately 10 cases were randomly drawn to test and refine
the recording procedures and obtain measures of rater reliability. The
recording instrument was changed to improve reliability.
6. A schedule was set for data collection.
7. The data were collected during a ten week period with the
principal investigator consulting with recorders to resolve questions.
8. Data were key punched and analyzed.
Limitations of the Study
The major limitation of this study was that the information obtained
was restricted to medical records. No corroborative information was
available, though at times it was possible to check data from one sec-
tion of the chart against data in another section. The accuracy of the
data was thus dependent on the preciseness, correctness, and objec-
tivity of the clinicians especially as concerned ratings of outcome.
Another limitation of the study was the precision of measurement
with some variables of interest. For example, parent involvement was
measured only as present/not present.
Analysis of Data
Data were analyzed through cross tabulation on a joint frequency
distribution of cases according to two or more classificatory variables.
The joint frequency distributions were tested for strength of associa-
tion by chi-square. Other measures of association such as phi or
Cramer's V described the degree to which one variable predicted another
(Nie, Hull, Jenkins, Steinbrenner, & Bent, 1975). The addition of other
variables as controls allowed study of intervening or interacting
ANALYSIS OF DATA
This study investigated variables related to the effectiveness of
community mental health outpatient services to children. Specifically,
case data were collected on five primary variables: parent involvement
in treatment, referral source, severity of diagnosis, type of treatment,
and duration of treatment. The relationship of these variables to
cases defined as successful or unsuccessful was explored.
Closed medical records were randomly sampled from three clinics
in two mental health centers. Information on 274 cases was collected
systematically by trained recorders who followed the same written
instructions (Appendices B and C).
Successful or unsuccessful outcomes for the cases were defined
by therapist ratings on specific treatment goals. A total of 274 cases
were reviewed. Of these, 162 (58.5%) had successful outcomes and 112
(40.4%) had unsuccessful ones. Data on other variables believed to
affect the primary variables were also collected (Appendix B).
Analysis of the data was accomplished through a cross tabulation
procedure using a joint frequency distribution on two variables. The
strength of association was measured by chi-square (Nie, Hull, Jenkins,
Steinbrenner, & Bent, 1975). This statistic enables one to determine
the probability that the observed relationship between variables could
have occurred by chance. First, row and column marginal totals were
used to calculate expected cell frequencies (if no relationship is
present between variables). The expected cell frequencies are then
compared to the actual values. As the discrepancy between expected
and actual frequencies increases, the chi-square value becomes larger.
Chi-square only assists in determining whether or not variables
are independent. It does not state the magnitude of this relationship.
Either phi or Cramer's V indexes the degree of correlation between two
variables much as does Pearson's R. These statistics also adjust for
the fact that chi-square is directly proportional to the number of
cases in the sample as well as the table size. Phi or Cramer's V are
included in the tables of this study when the chi-square analysis was
found to be significant or approaching significance for a primary
The .05 level of significance was used in all tests of significance
as the criterion for rejection of a hypothesis. However, significance
levels equal or greater than p < .25 were reported. Below p < .25,
results were described simply as nonsignificant. The results of the
study will be reported in this chapter. More specifically, data were
analyzed for each of five variables and related hypotheses.
Parent involvement in child therapy is often thought to be an
important factor and is highly recommended (e.g., Levitt, 1971;
Minuchin, 1974). The systemic view of family functioning (Haley,
1976) asserts that a symptom within one family member may serve a
function for the entire family. The parents may be unknowingly
supporting a child's behavior. Thus, their involvement in the change
process is assumed to be necessary.
Parent involvement in this study was defined specifically as parent
participation in treatment after the intake interview. Parents present
at one or more sessions after the intake interview were considered to
be "involved" in the treatment of their child.
Data were available on all 274 cases. In 81 (29.5%) cases, both
parents were included in at least one session following intake. In the
remaining cases (70.4%) both parents were not included. Of the cases
where both parents were included in post intake sessions, 75 (92.5%)
of the parents attended sessions together. There were 6 cases (7.5%)
where both parents participated in treatment, but did not do so together.
Of parents who participated in therapy alone, without the other parent,
there were 177 females (91.7%) and 10 males (9.3%). Single parents partici-
pating thus comprised 68% (187 cases) of the sample. There was a
remainder of 6 cases (2%) of the sample where neither parent par-
ticipated in sessions after the intake.
HO1. There is no significant difference between successful and
unsuccessful cases regarding parent involvement in treatment.
The frequency of 6 cases in which there was no parent involvement
in treatment was so small that further analysis seemed inappropriate.
Chi-square assumes that for any category, sample frequencies are dis-
tributed normally about the population value. The assumption of
normality is not met when expected population values are close to zero.
Thus, with very small expected frequencies, the validity of x2 results
is questioned (Welkowitz, Ewen, & Cohen, 1971). For this reason,
analysis was not pursued and the results remain inconclusive.
Examination of the data reveals that considerable numbers of
parents, 193 (70.4%),participated in treatment without another
adult. This is consistent with the representation of single
parents in the sampling distribution as reported in Chapter Three.
Referral source may be a likely indicator of attitudes and moti-
vation toward treatment (Lake & Levinger, 1960). Parents of children
who drop out of treatment are more likely to have experienced institu-
tional pressure and resisted treatment.
Information about the source of referral was first coded into
seven categories. These were self, 103 cases (37.6%); school, 60 cases
(21.9%); Health and Rehabilitative Services (HRS) suggestion, 63 cases
(22.7%); community agency or church, 12 cases (4.3%); physician or
medical clinic, 17 cases (6.1%); police or law enforcement agency,
15 cases (5.4%); judicial court, 15 cases (5.5%) (see Table 11).
These referral sources were then combined to yield three major
categories used for analysis: 1) voluntary and personal referral
(self, community agency or church, and medical clinic or private
doctor); 2) agency referral (included school or HRS suggestion);
3) involuntary (included law enforcement and court ordered/HRS per-
Voluntary cases were those where little or no coercion could be
presumed to exist. Of a total of 277 cases, 132 (47.7%) were in
FREQUENCY DISTRIBUTION FOR REFERRAL SOURCES
Relative Adjusted Cum
Referral Absolute Freq Freq Freq
Source Code Freq (PCT) (PCT) (PCT)
Self 1. 103 37.2 37.6 37.6
School 2. 60 21.7 21.9 59.5
HRS Suggestion 3. 63 22.7 23.0 82.5
Agency 4. 12 4.3 4.4 86.9
Medical Clinic 5. 17 6.1 6.2 93.1
Police 6. 4 1.4 1.5 94.5
Judicial 7. 15 5.4 5.5 100.0
9. 3 1.1 Missing 100.0
Total 277 100.0 100.0
FREQUENCY DISTRIBUTION FOR REFERRAL SOURCE RECLASSIFIED
Relative Adjusted Cum
Referral Absolute Freq Freq Freq
Source Code Freq (PCT) (PCT) (PCT)
Referral 1. 132 47.7 48.2 48.2
Suggestion 2. 123 44.4 44.9 93.1
Involuntary 3. 19 6.9 6.9 100.0
9. 3 1.1 Missing 100.0
Total 277 100.0 100.0
this category. Agency referral cases were those where the child's problem
was presumably noticed by someone in a position of authority and it
was suggested that counseling be sought. One hundred and twenty-three
cases (44.4%) were in this category. Ordered cases were defined as
those cases where substantial coercion was employed. There were 19
such cases (6.9%) (see Table 12).
HO2. There is no significant difference between successful and
unsuccessful cases in terms of referral source.
Analysis for the three categories of referral source by type of
outcome is presented in Table 13. Cases where data were either missing
or the outcome was undetermined were excluded from the analysis. A
total of 271 cases were included in the analysis.
Of the voluntary cases, 77 (59.7%) were considered successful
while 52 (40.3%) were seen as unsuccessful. These were the same per-
centages for successful and unsuccessful cases as in the sample as a
whole. Similar frequencies and percentages were evident for agency
referral cases as well indicating no difference from sample values.
There were smaller numbers of cases in the ordered category (19); 57.9%
of these cases were successful while 42.1% were unsuccessful. Again,
this was similar to sample values. Observed and expected frequencies
were approximately equal. The chi-square statistic was .0224 with 2
degrees of freedom. This was not significant. Thus, the null hypothesis
for referral source was accepted.
Severity of Diagnosis
Severity of diagnosis is associated with lack of treatment success
(Lessing & Schilling, 1966; Ross & Lacy, 1961). This variable was
defined by using the DSM-III (APA, 1980) diagnostic categories listed
for the children.
Cases classified as "mild" included those which were given diagnoses
involving adjustment problems (e.g., maladaptive reactions to stress
and/or conditions not attributable to a mental disorder; parent-child
disagreements). One hundred and sixty cases, or 49.3% of the sample,
fell into this category. A total of 110 cases were classified as
"moderate/severe," or 40.7% of the distribution. Cases classified in
this category included substance abuse, attention deficit disorder with
hyperactivity, and conduct disorders, among others. Only one case was
labeled psychotic and the child was diagnosed as schizophrenic.
HO3. There is no significant difference between successful and
unsuccessful cases regarding severity of diagnosis.
A cross tabulation procedure using a joint frequency distribution
for severity of diagnosis and type of outcome (successful or unsuccess-
ful) was used for analysis. The chi-square value was .68 with 1 degree
of freedom. This value was not significant. Of cases classified as
mild, 98 cases (61.3%) had successful outcomes while 62 cases (38.8%)
had unsuccessful outcomes. Of cases in the moderate/severe classifi-
cation, 55.5% or 61 cases had successful outcomes while 44.5% had nega-
tive outcomes. Percentages of successful and unsuccessful cases for
each category closely resembled sample values (58.9% successful,
41.4% unsuccessful). Since observed values were thus very close to
expected values from the overall sample, the null hypothesis was
accepted (see Table 14).
CHI SQUARE ANALYSIS FOR TYPE OF OUTCOME BY REFERRAL SOURCE
Type of Col PCT Agency Row
Outcome Tot PCT Voluntary Suggestion Involuntary Total
Successful 1. 77 73 11 161
47.8 45.3 6.8 59.4
59.7 59.3 57.9
28.4 26.9 4.1
Unsuccessful 2. 52 50 8 110
47.3 45.5 7.3 40.6
40.3 40.7 42.1
19.2 18.5 3.0
Column 129 123 19 271
Total 47.6 45.4 7.0 100.0
Chi square = 0.0224 with 2 degrees of freedom Significance = 0.9888
CHI SQUARE ANALYSIS FOR TYPE OF OUTCOME BY DIAGNOSTIC CLASSIFICATION
Type of Col PCT Mild Mod-Severe Row
Outcome Tot PCT 1 2 Total
Successful 1. 98 61 159
61.6 38.4 58.9
Unsuccessful 2. 62 49 111
55.9 44.1 41.1
Column 160 110 270
Corrected chi square = .680 with 1 degree of freedom Significance = .4093