Anevaluation of children's services in community mental health centers

AN EVALUATION OF CHILDREN'S SERVICES IN
COMMUNITY MENTAL HEALTH CENTERS

By

LEWIS A. FABRIC, II

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF
THE UNIVERSITY OF FLORIDA
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
DEGREE OF DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

1984

With love to my mother

ACKNOWLEDGEMENTS

For the completion of this project I am indebted to a number of

people. Each of my committee members (Drs. Myrick, Fitzgerald, and

D'Amico) contributed their personal time to read and edit this disser-

tation. I am most grateful to each one of them and to their families

for their assistance and encouragement.

I want to also thank my friends and colleagues over the years

at the Child Youth and Family Center and at the North Central Florida

Community Mental Health Center, Inc., for their assistance in my pro-

fessional development. Their work as clinicians has given this project

its inspiration and enabled its conclusion.

My family has been extremely patient and helpful. Through this

process, my wife, Finnette, has often had to function as a single

parent. She has ably assisted me in editing the manuscript and in many

other ways. My children, Elena and Nathan, have sacrificed much family

time in the completion of this project. I am also appreciative of my

in-laws' interest and support.

Finally, my parents have assisted me through their support of my

education over the years and their caring and assistance. They have

served as examples for me in perseverance and dedication. For this,

I thank them.

TABLE OF CONTENTS

Page

ACKNOWLEDGEMENTS. . . . . . .

ABSTRACT. . . . . . . . .

CHAPTER

ONE INTRODUCTION . . . .

Need for the Study . . .
Purpose. . . . . ..
General Research Questions .
Definition of Terms .. .. ..
Organization of the Study. ..

TWO REVIEW OF LITERATURE . . . . . . . . .

History of the Community Mental Health Movement. . .
Development of Children's Mental Health Services .
The Community Mental Health Centers and Children . .
Evaluation and Community Mental Health Centers . .
Outcome Studies in Community Mental Health Centers .
Effectiveness of Evaluation . . . . . .
Outcome Studies Related to Specific Variables
in the Current Study . . . . . . . .
Summary . . . . . . . . . . .

THREE RESEARCH METHODOLOGY . . . . . . . . .

Population and Sample . . . . . . . .
Variables Investigated . . . . . . . .
Criterion Measure . . . . . . . . .
Sampling Procedures . . . . . . . .
Research Hypotheses . . . . . . . .
Method of Data Collection . . . . . . .
Data Collection Visits . . . . . . . .

FOUR ANALYSIS OF DATA . . . . . . . . .

Parent Involvement .
Referral Source. .
Severity of Diagnosis.
Type of Treatment. ..

. . . . . . i

. . . . . .. vi

5
9
. . . 1
. . . 5
. . . g
. . . I9
. . . 10
. . . 11

12

Page
Duration of Treatment. . . . . . . . ... 97
Other Findings . . . . . . . .... 99

FIVE SUMMARY, CONCLUSIONS, LIMITATIONS, AND RECOMMENDATIONS 116

Discussion and Conclusions . . . . . .... 116
Type of Treatment. . . . . . . .... 121
Limitations. . . . . . . . . ... .. 126
Recommendations for Further Research . . . ... 127

APPENDIX

A ADMISSION DATA FORM. . . . . . . . ... 130

B RECORDING INSTRUMENT . . . . .. ...... 138

C INSTRUCTIONS FOR FILLING OUT QUESTIONNAIRE . . .. 142

D PROPORTION OF AGREEMENT. . . . . . . ... 147

REFERENCES . . . . . . . . ... ...... 150

BIOGRAPHICAL SKETCH . . . . . . . .... ..... 159

Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the Requirements
for the Degree of Doctor of Philosophy

AN EVALUATION OF CHILDREN'S SERVICES IN
COMMUNITY MENTAL HEALTH CENTERS

by

Lewis A. Fabrick, II

August, 1984

Chairperson: Robert Myrick
Major Department: Counselor Education

This study investigated clinical variables in relation to the

effectiveness of children's services in community mental health centers

in north central Florida. These variables included parental involvement

in treatment, referral source, diagnostic classification, type of treat-

ment, and duration of treatment.

Information was obtained from 277 closed medical record charts.

The charts were sampled at random from a total of 1352 cases seen during

the period 1980-1983.

Using a cross-tabulation procedure, measures of the variables

studied were examined according to treatment outcomes. Cases were

classified as successful or unsuccessful according to therapist ratings

of progress on specific treatment goals.

Analysis of data suggested that treatment success was positively

related to duration of treatment (p < .001). There were no differences

between cases rated as successful or unsuccessful with regard to

referral source, diagnostic classification, or type of treatment.

The results for parental involvement in its effect on treatment

success were inconclusive.

It was concluded that children's services in community mental

health centers were equally successful regardless of who referred the

client, diagnostic classification, or the type of treatment involved.

However, remaining in treatment longer than five sessions appeared to

increase the chances of being rated as improved by the therapist.

Implications of the study and recommendations for future research

were discussed.

CHAPTER ONE
INTRODUCTION

For many years, mental health services were provided through the

efforts of the local communities across the United States. Quality of

care was far from uniform and unavailable to many segments of the

population. Many communities chose to house their mentally ill in far

away institutions.

In 1963, President Kennedy called for a new approach to mental

health care and Congress responded with the Community Mental Health

Center Act (P.L. 88-164). This Act created community mental health

centers throughout the United States. These centers were to provide

alternatives to state and county mental hospitals for the prevention

and treatment of mental illness (Naierman, 1978).

Community mental health centers (CMHCs) tripled in number between

1967 and 1976, increasing to a total of 528-centers (ADAMHA National

Data Book, 1980). By 1983 there were 35 mental health centers in the

state of Florida established with the help of federal funding. As a

result of the increase in community mental health programs, care was

made available to rural and poor populations as well as the urban and

rich (Jones, Wagenfeld, & Robins, 1976).

The mental health center act as amended in 1965 required the centers

to provide five basic services without regard to a client's ability to

pay: inpatient treatment, outpatient care, partial hospitalization

services, emergency treatment and consultation, and education services

(Naierman, 1978). These mental health centers were established as

private, non-profit organizations with links to local, state, and

federal governments. Local matching funds were required to attract

federal dollars.

A center's staff usually consisted of an executive director, ad-

ministrative and support personnel, and both professional and non-

professional clinical staff. A board of directors from the community

were responsible for policy guidelines and employment of the director.

Among those professionals who were employed, there were psychiatrists

(9%), psychologists (5%), social workers (9%), registered nurses (14%),

and other mental health professionals (14%) (ADAMHA National Data Book,

1980).

Consequent with the establishment of a nationwide network of ser-

vices came the need for large scale evaluation efforts, but mental health

centers were not seen as basic research facilities. They were generally

unconcerned with the discovery of new information. The primary objec-

tive of evaluation or research within centers was to determine the

extent to which services were delivered. Studies to examine the results

achieved were limited. The success of evaluative research depended

ultimately on its usefulness to the administrator (Suchman, 1967).

In recognition of the importance of making mental health centers

publicly accountable, as well as increasing their effectiveness,

legislators amended the Mental Health Center Act in 1975 to include

program evaluation. Centers were required to collect data which showed

1) patterns of use of services; 2) availability, accessibility, and

acceptability of services; and 3) impact of services on clientele

(Fiester, 1978).

Evaluation procedures have tended to favor simple outcome measures.

This is as opposed to more complex evaluation procedures which explain

why a program may have had effect. Some concern was expressed that

outcome research is too difficult to conduct (Keppler-Seid, Windle, &

Woy, 1980),and others have cautioned that such research studies empha-

size certain outcomes to the exclusion of others (Schulberg, 1981).

Because of confusion, lack of knowledge and resources, and resistance,

research in mental health centers has been limited. Subsequently,

knowledge gaps have developed in community mental health centers,

especially as to how different client types are differentially affected

by services.

Relatively little was known from early evaluations as to which

services were most effective for different recipients. Beginning with

Eysenyk's conclusion in 1952 that psychotherapy had no proven efficacy,

researchers became more specific in the types of information they sought.

New investigations looked at different types of treatment, demographic

factors, and client personality. They also examined clinical information,

therapist factors, and therapist-client interactions (Strupp & Bergin,

1969). Even so, there was insufficient research to direct the estab-

lishment of programs and implementation of services in the community

mental health centers.

In 1975 it was mandated that centers provide services to children

and adolescents. Yet, as with other populations, there was a question

about how to provide young people these services. Both Meltzoff and

Kornreich (1970) and Levitt (1971) noted the lack of research regarding

the effectiveness of mental health treatment for children. Several

factors were singled out for blame: research interest in the child

guidance movement by psychiatrists and social workers was not evident

(Levitt, 1971); psychiatric treatment for children was less widespread

than for adults (Baekeland & Lundwall, 1975); methodological problems exist

in research, such as separating out changes in children and adults be-

cause they are often seen together in treatment (Levitt, 1971).

The lack of research on the treatment of children is unfortunate.

Mental health centers are facing economic problems and there is a greater

need to demonstrate their value. Due to less money from the federal

government, centers are giving a greater part of their resources to

clients who are the most seriously disturbed and who are at greater-

risk for institutional placement. Children and adolescents are less

likely to be defined as a service priority when services are limited.

This low priority and a decline in services compound the problem of

research and evaluation as there is less opportunity to do research and,

subsequently, less opportunity to build a documented case for more

services.

Mental health centers need more help in directing their resources

for the maximum benefit for clients. More information is needed about

specialized client groups, such as children and elders. Supportive

research is necessary in order to improve services and build a case for

mental health centers.

Need for the Study

It was estimated 10 years ago that approximately 1.5 million

children in the U.S. had profound emotional disturbances which pre-

cluded their living in normal family, school, and community circum-

stances (Smith & Hobbs, 1966). More recently it has been estimated

that each year more than 6 million children, ages 5 to 19, have mental

health problems severe enough to require professional treatment (Berlin,

1975). Such increasing magnitude suggests a need for specialized treat-

ment programs which have proven efficacy.

Yet, Levitt (1971) noted that after six decades of the child guidance

movement evaluative research still failed to show that child psycho-

therapy was effective. Outcome studies illustrating improvement rates

for treatment groups summed across different categories of emotional

disturbance were approximately the same as for those who did not re-

ceive treatment. Examining the issue of outcome more closely also

revealed large variations in improvement rates between agencies. This

suggested that investigators needed to study more specific treatment

programs for different populations (Strupp & Bergin, 1969) and find

where treatment was effective.

Although research efforts increased in adult psychotherapy, little

research was done in child psychotherapy. Research with children when

compared to adults is inadequate (Baekeland & Lundwall, 1975). Levitt

pointed out that only six reviewers of psychotherapy research in the

Annual Review of Psychology since its inception found it necessary to

use a subheading for research with child patients (Levitt, 1971).

The child therapy researcher has several methodological problems

to contend with. First, he has to consider the status of the child as

a developing organism. Sometimes it is difficult to tell whether a

symptom is a part of the normative growth process which will be taken

care of in time or a truly deviant process in need of treatment. In a

similar vein, one symptom of emotional disturbance may be replaced by

another. Thus, the researcher may have to decide whether treatment for

the original symptom was successful or whether the symptom remitted

through the child's developmental process (Levitt, 1971).

Another difficulty involves the child's comparatively greater

dependence on his environment, especially his family. Children are

usually a captive audience brought to treatment by adults for problems

that have been defined by adults. Thus treatment goals may be more a

factor of adult rather than child expectations. This may be one reason

why investigators of child psychotherapy have focused on the personality,

psychopathology, and attitudes of the child's parents (Baekeland &

Lundwall, 1975). Dropping out of treatment, for example, has been

positively related to negative parental attitudes and behavior toward

the treatment (Lake & Levinger, 1960).

To elaborate further on the special research difficulties created

by the child's dependence on his environment, there is the problem that

the remainder of the family is simultaneously in treatment with the child.

When the child guidance movement was first started, it was common for the

parents to be seen by a psychiatric social worker while their child was

seen separately by a child psychiatrist. As conjoint family therapy

came into vogue in the late 1960s and early 1970s, it became commonplace

to see the family together. Whether seen separately or together, it is

difficult to say at the conclusion of therapy that positive results in

the child were a result of his/her individual treatment. Separating

out treatment effects has become a challenge to researchers.

If specific variables related to treatment success with children were

established through basic research,then effective programs could be de-

vised to meet children's mental health needs. However, it has been noted

that such variables have not been isolated and that such ideal programs

do not exist. An unfortunate consequence of the lack of empirical

support for program interventions is the difficulty in obtaining public

support.

Particularly affected are prevention programs such as mental health

centers which intervene early in the development of emotional dis-

turbance. Friedman (1983) and Kiesler (1982) have noted the preference

in public policy given to institutional/residential programs over-early

intervention models. Such programs are often funded in response to the

most immediate need while preventive services, which meet a less visible

need,are neglected.

A similar lack of supportive research for children's mental health

programming exists in the area of program evaluation research. Such

research attempts to determine whether specific program activities attain

desired results, thus establishing their usefulness in particular situa-

tions. Unfortunately, mental health center research studies have focused

on the population which receives the greatest part of its resources:

adults (Fortney, 1981; Kline & King, 1973; Sue, McKinney, & Allen, 1976).

In a recent statewide outcome study of mental health centers, only adult

outpatient and day treatment clients were evaluated (Cox, Brown,

Petersen, & Rowe, 1982). Successful outcomes with adults found in some

studies do not necessarily generalize to children, however, for reasons

that have been described.

One study which attempted to evaluate mental health centers' efforts

to work with children has received special attention. Fiester (1978)

developed an automated system to measure specific goal attainment. He

was attempting to comply with federal accountability standards and also

gather data on treatment effectiveness. Treatment was found to be

successful in the areas of social and academic skills. The data analysis

provided feedback to therapists as to their effectiveness with particular

clients, making it a good example of institutional self-study.

The main problem of the Fiester study was that only outcome data

were presented without the specification of process. It did not measure

and partial out alternative explanations for the results, such as the

effects of parent participation, referral source, or demographic status.

Although Fiester used data to develop criteria for the assignment of

clients to therapists, he did so only on the basis of a few limited

variables, such as an individual therapist's success in a specific prob-

lem area. Without the specification of other process variables, ques-

tions about the effectiveness of centers with distinctive populations

could not be answered.

The need for data-based studies of mental health centers' services

is evident. They are needed to assure consumers, taxpayers, and health

providers that services are valuable and warranted. In order to secure

funds for programs, administrators of outpatient children's mental

health outpatient programs need more information regarding outcomes,

which treatment is most successful under which conditions. The need

to relate treatment success to treatment variables has also been recog-

nized (Strupp & Bergin, 1969; Suchman, 1967).

Purpose

It was the purpose of this study to investigate mental health

center outpatient services to children, adolescents, and their families

and to examine successful and non-successful cases as described in offi-

cial medical records on file in mental health centers. More specifically,

this study examined client records to 1) identify which children's cases

were classified as successful or not successful and 2) examine specific

variables related to the cases. It was then possible to study these

variables in terms of treatment outcomes.

Two northern Florida mental health centers provided data for the

study. Approximately 275 medical charts were systematically reviewed

for data, including information on initial diagnosis, referral source,

type of treatment, parental involvement, and the duration of treatment.

Other information on both client and treatment variables was gathered

for use in secondary analyses.

General Research Questions

1. Does the presence or absence of parental involvement during

treatment of children have an effect on success or reported success?

2. Are referral sources related to successful or non-successful

cases?

3. Are some diagnostic categories (based on the Diagnostic and

Statistical Manual III) more responsive to treatment than others?

4. Does the length of treatment have any effect on outcome?

5. Does the type of treatment have any effect on outcome?

Definition of Terms

Community Mental Health Center: A mental health agency or institu-

tion established under guidelines set by the Mental Health Act of 1963

and/or 1975.

Specialized Children's Program: A program within a mental health

center designed to deliver mental health services to children (0-18 years

old) and their families. Such programs were mandated in 1975 to be part

of federally supported mental health centers.

Medical Record: A client's file, started when he/she seeks treat-

ment. Also referred to as medical chart or case, it contains 1) an

admission data sheet, 2) an intake summary, 3) a treatment plan, 4) a

discharge summary, 5) progress notes, and 6) medication record. Other

available information generally includes 7) referral source, 8) who was

included in treatment, 9) clinical diagnosis, 10) treatment goals,

11) number of sessions client attended, and 12) treatment outcome

rating by therapist.

Diagnostic Category: Refers to the clinical diagnosis re-

corded on the client's records and is based upon the Diagnostic and

Statistical Manual III for the Classification of Psychiatric Disorders

(APA, 1980).

Referral Source: The primary source which directed the client/family

to seek treatment at the Center.

Organization of the Study

This study is organized into five chapters. Chapter One includes

the introduction, need for the study, purpose of the study, research

questions, rationale, and definitions of terms. Chapter Two contains

a review of the literature related to evaluative studies of mental health

treatment programs for children and adolescents. A discussion of the

methodology and data collection is presented in Chapter Three. Results

of this study are presented in Chapter Four, including analysis of

the data. A summary, conclusions, and recommendations made from the

investigation are included in Chapter Five.

CHAPTER TWO
REVIEW OF LITERATURE

In this chapter the community mental health movement will be traced

through its historical origins beginning with the establishment of the

National Institute of Mental Health and culminating in the 1980s with

the recision of the Community Mental Health Act by President Reagan.

Services to children will be examined, emphasizing the status of chil-

dren's mental health services prior to the establishment of community

mental health programs and then noting the impact of the centers on child

mental health services. Types of services delivered to children in

mental health centers, their appropriateness and adequacy, will be

described. In separate sections, evaluative research as it applies

to community mental health centers will be presented. Finally, research

related to the main hypotheses will be discussed.

History of the Community Mental Health Movement

Mental health care prior to the twentieth century was largely the

responsibility of the family. Large mental institutions and jails,

usually far removed from the patients' communities, were the primary

sources of containment and treatment of the "mentally ill." Dorthea

Lynde Dix, in the mid-nineteenth century, crusaded for more humane

treatment of patients in these institutions. She helped increase public

awareness of the problem in treating patients. While more mental

hospitals were constructed as a result of her efforts, extreme over-

crowding soon followed. Later in the 1890s the work of Sigmund Freud

added an air of scientific respectability for the first time to the

field of mental illness. Attitudes toward treatment of the mentally

ill slowly began to change.

Mental health care before the 1940s was under the direction of

state authorities. However, during World War II the large number of

draft rejections for mental health problems increased public concern.

This in turn led to the National Mental Health Act of 1946 which in-

volved the federal government in the problem for the first time. At

this time there were fewer than 10,000 full time mental health profes-

sionals in the United States (ADAMHA, 1980).

Several important changes in the mental health field occurred during

the 1950s. Despite the fact that 75% of mental health services still

occurred in inpatient facilities (National Institute of Mental Health,

1977), psychotropic medications were developed to make hospital stays

shorter and enable patients to be maintained in their own communities.

This kind of treatment was supplemented by an increase in the number of

local outpatient psychiatric clinics (Miller, 1974). Congress estab-

lished a Joint Committee on Mental Illness and Health in 1955 to study

the possibility of adopting a new approach to the treatment of mental

illness. Among the recommendations were the expansion of services,

reduction in size of state hospitals, and increased provision of services

in local communities (Joint Commission on Mental Illness and Health,

1961).

Community Mental Health Center Construction Act

The recommendations of the Joint Committee on Mental Illness and

Health were used by President Kennedy to establish the Community Mental

Health Center and Mental Retardation Facilities Construction Act of

1963 (P.L. 88-164). Thirty-three states acted immediately, passing

legislation to authorize the construction of mental health centers

(National Institute of Mental Health, 1969). The implementation of

community mental health center programs was given an additional boost

in 1965 with the passage of federal legislation, which provided staffing

assistance to cover personnel costs (P.L. 89-105).

The primary intent of the legislation was to assist communities

in developing alternatives to state and county hospitals in the treatment

of mental illness (Naierman, 1978). A variety of mental health services

were, for the first time, to be available to the entire population

regardless of an individual's ability to pay or geographic location

(Jones, Wagenfeld, & Robin, 1976).

In order to qualify for federal funds, community mental health
centers had to provide few basic services. The minimum services in-

cluded inpatient and outpatient care, partial hospitalization and

emergency services, and consultation and education services. Children

and adolescents were not, at this time, given special services. They

were instead served in programs with adults (Naierman, 1978).

The required basic services required development of new treatment
techniques and a more diversified professional base. Assistance in

manpower training came from the National Institute of Mental Health

-15-

and the Veteran's Administration. There was more than a thousand-fold

increase in the number of mental health professionals (125,000 in 1980)

in the thirty years between 1950 and 1980 (Vandenbos, 1980).

Initially, federal staffind grants to centers were awarded for a

fifty-one month period. It was expected that centers would develop suf-

ficient additional funding during this time and would become independent

from federal funding. Despite varied funding from state and local

governments, however, the centers had not yet gained financial inde-

pendence from federal grants by 1970. Hence, federal grants were

extended to eight years. Dependency on federal funds was attributed to

a number of factors, including the following: 1) income to centers was

reduced because payment was on a sliding fee scale based on family income

and number of dependents; 2) many third party insurers did not cover com-

munity based mental health service; and 3) there was a lack of sophisti-

cated financial management (Naierman, 1978).

By 1975 community mental health centers were well established

across the country. Yet there were still problems with securing alterna-

tive funds to replace the federal staffing grants. Public Law 94-63 was

established in partial response to this need (Despard, 1983). It pro-

vided a new sliding scale of federal and state funding for beginning

centers, but one which declined at a faster rate than previously. This

was intended to encourage centers to obtain alternative funding sooner

in their development. It also supported established centers which had

not succeeded in finding replacement funding.

At the same time, however, P.L. 94-63 called for an expansion df

already existing services to include evaluation, screening, follow-up

and transitional services. In addition, specialized services for

children, elders, alcohol and substance abusers wererequired. Grants

were made available to finance these additional costs.

The implementation of evaluation and accountability standards had

several positive effects. For the first time, mental health centers

were accredited by the Joint Commission on American Hospitals. Quality

of care standards were established (Joint Commission on the Accredita-

tion of Hospitals, 1976). Better program planning and evaluation

techniques were encouraged through staff developmenttraining and pub-

lications supported by the federal government (Ellsworth, 1979).

Effects of CMHC Legislation

As early as 1972 the community mental health program was being

heralded as a focal point of the national effort to improve patterns

of mental health care in the United States (Brown, 1975). Centers had

grown in only six years (1965 to 1971) from 1 to 493 and they were in

all 50 states, the District of Columbia, and Puerto Rico. It was esti-

mated that the centers would eventually be serving areas with a combined

population of 73 million (Brown, 1975).

From 1969 to 1971 the services delivered through the community

mental health system nearly doubled from serving 372,000 persons to

659,000, not counting those served through consultation and education

services. Public program emphasis was now based on prevention, early

detection and treatment, as opposed to institutionalization. Mental

health centers thus seemed to be succeeding in making mental health care

available and accessible to the general public (Miller, 1974).

A survey conducted by the Joint Commission on Mental Illness and

Health in 1957 investigated attitudes of the public toward mental health

care as well as utilization rates (Vandenbos, 1980). Only 15% of the

respondents in a poll were found to have ever used any mental health

services while only 28% of this figure had received services from a

trained mental health professional. When the study was repeated in

1976, by Douvan, Kulka, and Veroff (1978), the number of people using

services had grown to 26%, with 47% of these having received treatment

from a qualified mental health professional. In addition, 60% of those

surveyed said they would use mental health services in connection with

an emotional problem.

Mental health centers were especially successful in serving lower

and middle income families. Over 50% of CMHC patients had incomes below

$5,000, while only 6% earned over $16,000 per year (Brown, 1975). About

one-half of all centers in 1972 were located in areas of poverty. In

one Florida mental health center 60-70% of the clients (in 1982) were

eligible for full reduction in fees. Only 18% of the total mental

health caseloads had a family income that exceeded $10,339 (District

III Mental Health Board, 1981-1982).

In addition to serving many lower income clients, the centers

also assisted persons who were formerly in public mental hospitals.

Of all persons treated at mental health centers, 13% were at one time

confined to public mental health hospitals as of 1972 (Brown, 1975).

Recision of Mental Health Systems Act

Mental health centers would have benefitted in 1980 from passage

of the Mental Health Systems Act (H.R. 4156 and S. 1177) based on

findings of the President's Commission on Mental Health (President's

Commission on Mental Health, 1978). This bill would have strengthened

mental health planning and resource allocations at the state level. It

also sought to improve services for special groups, such as the

chronically mentally ill and severely disturbed children. There was a

general intention to make services accountable at the state and local

level by encouraging services to high priority (most disturbed) clients,

basing funding on a performance system, and emphasizing third party

collections.

The proposed 1980 Mental Health Systems Act, however, was rescinded

in 1980 under the Reagan administration. This left CMHCs on uncertain

status. The basic federal direction at the time was the provision and

monitoring of block grant funds for alcohol, drug abuse, and mental

health services. Thus the states now had a much larger voice in the

implementation of these services. While many of the states preserved

the essential CMHC components (including the state of Florida), there

is now large variation from state to state and center to center as to

which services are emphasized. These differ according to varying

legislative priorities.

The unofficial national mental health policy at present is deinsti-

tutionalization with an emphasis on outpatient treatment. Actual

expenditure of funds, however, disputes this premise. Kiesler (1982)

notes that medicaid expenditures in 1977 for state, county, and private

institutional care for mental illness was greater than the total federal

cost of the entire community mental health program (600 centers).

Recent figures suggest that something similar is occurring for

children and youth in the state of Florida (Friedman, 1983). Of the

total funds expended for mental health in 1983, 66.55% was allocated

for residential care, while only 26.56% was designated for non-

residential, including outpatient services. Prevention services re-

ceived the least funding at 6.89%. It has been argued (Friedman, 1983)

that more money should go to those services which intervene before

residential care becomes necessary.

As first conceived, CMHCs were to deliver services to the general

population in hopes of preventing institutionalization. Such preventive

services were funded by the federal government on the assumption that

services would eventually become financially self-sustaining through

private fees and/or state or local stipends. That this has not been the

case is noted by Naierman (1978). This study recommended that centers

should continue to receive maintenance support from the federal govern-

ment if they were to continue to provide all basic services without

regard for the client's ability to pay. With the higher priority given

to residential care, it is possible that preventive services will

decline.

The history of community mental health reveals a pattern of in-

creasing federal responsibility for provision of services from the

early 1950s until 1980. During the past three decades scientific

advances and changed social perceptions have been observed in the area

-20-

of mental illness. These new developments supported federal involvement

in this area. Community mental health legislation was one important

product of this involvement. In general, there was public support for

a policy of early intervention and prevention in the field of mental

illness in order to avoid institutionalization. This has changed, how-

ever, since 1980 with the smaller federal role in the mental health

movement. Without federal monitoring and financial support, mental

health centers may not be able to maintain their unique characteristics

as originally mandated.

Development of Children's Mental Health Services

Prior to community mental health services including children, the

Joint Commission on Mental Health of Children described mental health

services to children as a national disaster. It was estimated in 1965

that there was a total of 4.6 million emotionally disturbed children

between 5 and 19 years old. At the same time, children receiving ser-

vices from psychiatric facilities were estimated in 1964 to be only

632,000 persons (Joint Commission on Mental Health of Children,

1973a).

There were seven White House conferences on children since 1909;

however, the number of children admitted to mental hospitals in 1973

increased faster than their increase in the overall population. Admis-

sion to public mental hospitals doubled in the ten years after 1963

for patients under 25 though there was an overall reduction in the

hospital population for the general population (Taschman, 1975).

Treatment at these facilities was limited to custodial care, with

approximately 20 attendants to every one professional (Joint Informa-

tion Services of the American Psychiatric Association and the National

Association for Mental Health, 1968).

In 1973, evaluation of existing services found them often to be

directed at exclusive populations while underserving specific groups

such as the poor and persons living in rural areas (JCMHC, 1973a,

Task Force V). These latter groups were often those most "at risk" to

develop emotional disorders. When services were provided they were

often inappropriate and/or uncoordinated (Gorman, 1968; Hollingshead &

Redlich, 1958).

The poor were especially disenfranchised. Furman, Sweat, and

Crocetti (1965) found public clinics tended to curtail their assistance

to the poor when faced with an overdemand for their services. This

compounded the problem from the fact that the poor were less likely in

the first place to seek care prior to a mental health crisis (Myers,

Bean, & Pepper, 1965).

It is the lower socio-economic status (SES) child who is most at

risk to develop mental illness. Poverty is characterized by adverse

physical circumstances, disorganized and often chaotic family structures,

and inappropriate child rearing practices, described as authoritarian,

fatalistic, and lacking positive stimulation. There is an absence of

child centeredness in parental attitudes (Beiser, 1965; Malone, 1966).

Delinquents, especially prevalent among the poor, were another

group that appeared to receive inadequate services. Few juvenile courts

(17%) had any regular access to psychological assistance. Placements

were frequently made to training schools or reformatories, even adult

prisons. High recidivism rates have proven such placements to be in-

effective (U.S. Senate Subcommittee on Employment, Manpower, and Poverty,

1967).

Services to rural children were also inadequate. Rosen,

Kramer, and Reddick (1968) noted that only one-tenth of the outpatient

clinics opened in 1965 were in rural areas and these served only 8% of

the total clinic cases. These figures were highly discrepant from actual

needs as one-third of children under 18 lived in rural settings where

outpatient psychiatric facilities were the only treatment available

because of transportation problems (Rosen et al., 1968). Services were

characterized by brevity and were often diagnostic and prescriptive in

nature rather than treatment oriented.

In addition to problems in service availability to the underserved

groups, there were also problems with adequacy and appropriateness of

services. Clients who appeared to be too severely disturbed or un-

motivated for traditional treatment were screened out (Hunt, 1962).

Only about one-third of the clients referred to child psychiatric clinics

ever received treatment (Levitt, 1971). There was an expectation for

clients to come to the clinic and articulate a psychological problem.

This was discrepant from the actual needs of the poor and/or minorities

which tended to be for increased social and occupational skills
(Gladwin, 1967).

Services were also fragmented. One source stated that this frag-

mentation occurred at every level: local, state, and federal (Taschman,

1975). On the national level alone at least 100 separately administered

programs for children existed.

The Joint Commission on the Mental Health of Children concluded

in 1973 that the mental health system a) tended to be oriented more to

professional than to client priorities, b) was concerned with remedial

services and crisis resolution rather than prevention, c) served only

a small portion of those who needed help, d) provided services that

were fragmented and uncoordinated. New technologies were needed to

make service more accessible and available to those in need. There

was also a need to increase the number of non-professionals who could

deliver services to indigenous groups (JCMHC, 1973a).

The Committee recommended a number of changes to resolve these

insufficiencies. Services should be coordinated and need-specific.

They were to include educational, recreational, and vocational plan-

ning, as well as psychotherapy. Thus, mental health services should

be integrated into the larger service agency network. Lower socio-

economic groups and minorities needed to have services established

specifically for them and their special needs, utilizing the input of

these groups to design such services. Particularly of relevance to

the community mental health program were the suggestions that mental

health services be community based whenever possible with close ties

to residents. It was also recommended that greater use be made of

parent education in mental health consultation as part of preventive

services. In addition, paraprofessionals were suggested as personnel

who could help provide more mental health services (JCMHC, 1973a).

The Community Mental Health Centers and Children

The period of the mid-seventies was a major turning point for

mental health services to children. The U.S. Congress in 1975 mandated

the addition of specialized children's services to the federally funded

centers. Already children and youth comprised 23% of the total clients

served by CMHCs (ADAMHA, 1980). Services to children were to be monitored

separately from the adult population, encouraging planning and account-

ability from the point of view of this group (NIMH, 1969). Professional

staff familiar with the special needs of youth were also added. Funds

were made available through conversion grants contingent upon imple-

mentation of these new services (National Institute of Mental Health,

1977).

The NIMH specified a set of principles on which CMHC services for

children and youth should be based. These included administrative and

clinical guidelines. Administratively, services should be delivered

on a confidential basis. They should also include the involvement of

community youth and adults in planning. Facilities should be adequate

and accessible to all segments of the community. Clinical services

should be delivered in cooperation with other community agencies,- be

designed to insure continuity of care, and emphasize prevention, early

identification, and early intervention (NIMH, 1978). Children's mental

health in general is distinguished from adult services by the inclusion

of the child's social environment as a primary focus (President's

Commission on Mental Health, 1978). This considers the child's family,

the school system, the courts, and various social agencies. As a result

of these guidelines, there are many similarities across centers for

children's services including staffing and assessment, treatment and

consultation procedures.

Staffing of Mental Health Centers

Mental health center staffing represents a full range of the core

mental health professions. The ADAHMA (1980) gave a general description

of personnel within mental health centers: psychiatrists, 5%; psycholo-

gists, 9%; psychiatric social workers, 14%; registered nurses, 9%;

other mental health professionals, 14%; non-professionals involved in

patient care (e.g., licensed practical nurses), 22%; administrative

personnel, 26%. As noted already, there has been an increase in the

number of mental health professionals in the field from the 1940s

(10,500) to the 1980s (125,000) (Vandenbos, 1982).

The NIMH recommends that a children's staff include developmental

specialists trained to be primary evaluators, back-up experts in such

areas as pediatrics and speech therapy, specially trained mental health

professionals, psycho-educational therapists, therapists, and para-

professionals. Each functions in one or more of the mandatory service

areas, such as emergency services or outpatient care (NIMH, 1978).

Referrals

Referrals may come from a variety of sources. The judicial system,

social services, family physicians, school, and the family itself

frequently refer children to mental health services. Rarely is the

child self-referred. Rather, someone else determines that a child has

a problem (Baekeland & Lundwall, 1975; Levitt, 1971) and initiates the

referral.

Referrals for the poor are most likely to come from the courts or

schools. They also reflect the ethnic and social class bias of middle-

class institutions. The lower the social class of the child, the longer

referrals are delayed (McDermott, Harrison, Schrager,& Wilson, 1965).

Children referred for services fall into two categories according

to Behar (1981). There are profound differences between children who

are intelligent, willing, and able to talk about themselves and those

who suffer from poor impulse control, environmental deprivation, and

are less receptive to verbal therapies. The former group has potential

to benefit from occasional services. The latter group, however, is

chronically impaired. They need long-term care, more intensive treat-

ment, and more follow-up.

Diagnostic Interview

Assessment is a multi-faceted process consisting of interviews and

observations of a child with and separately from his/her family. Family

interactions are noted. Interviews are structured in order to obtain

such information as family, medical, educational, and personal histories

(Appendix A). Another source of information may be the child's teacher.

Objectives of assessment procedures are several. In order to

foster healthy psychosocial development and plan a timely intervention,

it is important to make early identification of mental health problems

(NIMH, 1978). A good assessment gives the clinician a total picture

of the person and his/her environment and enables treatment planning.

After obtaining relevant information the clinician files one or

more diagnoses according to the DSM III (APA, 1980). The most commonly

used classification for children in both inpatient and outpatient settings

(60%) in 1980 was the transient stress disorder. This is a non-adaptive

emotional reaction to environmental stress (ADAMHA, 1980). It may be

inferred from this statistic that a diagnostic label for children and

adolescents is not as useful as for adults in determining treatment

planning. Such planning must be done with the child's family as well

as with the presenting problem in mind. Diagnosis should be used for

appropriate guidance and follow-up rather than categorization only.

While assessment may take up to three sessions and involve several

professionals, typically some kind of treatment plan is agreed on at

the time of the intake. The treatment plan may specify specific prob-

lems and identify goals and types of treatment that are applicable. If

services at the center are not appropriate for the client's problem,

a referral may be made to another agency.

Family financial information is also obtained during the early clinical

interviews. Such information is required to set fees. A sliding scale

balances the number of dependents against total income and helps estab-

lish fees paid by clients.

Treatment Methods

The National Institute for Mental Health specifies that outpatient

services should be timely, coordinated with other social services, and should

attempt to prevent institutionalization (NIMH, 1978).

A full range of approaches and modalities should exist to allow

for individualization of services. The therapeutic possibilities are

many, including individual, group, and family therapy, among others.

The form of therapy varies to some extent based on age. For example,

individual therapy for younger children may use symbolic play (Axline,

1967) while adolescents benefit from more verbal forms of therapy.

Special diagnostic tools such as psychological testing and learning

assessments should be available. Another important treatment component

is medical consultation for purposes of assessment and prescription of

psychoactive medication. Therapy sessions are held once weekly for a

period of several months on the average.

Family involvement in treatment has become more popular in recent

years, as evidence indicates that children's problems are related to

family functioning (Minuchin, 1974; Haley, 1976). When relationships

are impaired, dialogue between parent and child is supervised by the

therapist in order to help clarify and resolve issues and problem

situations.

Parents are also involved in treatment through parent education

classes. These classes help parents to understand their child's be-

havior. An example of such an approach is Thomas Gordon's program,

Parent Effectiveness Training (1975).

Some youngsters require more intensive services than a once a week

visit because of the severity of disturbance. If unable to be main-

tained in a regular school or home placement on a full-time basis, day

or evening treatment is an alternative. Such services often involve

collaboration between CHMCs and local school systems (Behar, 1981).

These programs offer high staff-to-patient ratios and are oriented

toward developing social and academic skills as well as meeting physical

needs (NIMH, 1978).

Residential services may become necessary when a child is severely

disturbed or the family is unable to provide for the emotional needs of

the child. Such placements are typically made after less intensive

services have been tried and found to be inadequate. In the state of

Florida, for example, the Baker Act governing the rights of the mentally

ill provides for the care of the client in the least restrictive alter-

native. Placements are ideally short (3 months) to intermediate term

(6 months to 1 year).

Mental health programs focus on the educational, recreational,

social, and physical needs of the client (NIMH, 1979). A variety of

programs exist to meet the needs of different clients. These include

traditional psychiatric hospitalization, placement in a therapeutic

group or foster home, and wilderness camping (A. Gray, 1983).* Rarely

do mental health centers provide such services independently, however.

Emergency treatment should be available to deal with crisis situa-

tions 24 hours a day, 7 days a week. It should attempt to prevent

further complications including hospitalization. Adolescents may pre-

sent with a variety of problems including suicidal attempts or enraged

states related often to family disturbances. Mental health professionals

attempt to defuse a family crisis through discussion thereby avoiding

hospitalization (Behar, 1981).

Centers provide services which help to prevent the development of

emotional illness. Mental health education classes are aimed at special

*Personal communication.

problems. Training and consultation are available to other caregivers.

Content may include, for example, preparation for parenthood, sex educa-

tion, or family relations (NIMH, 1979).

While consultation and education (C&E) services are required, how-

ever, they are rarely financially self-sustaining. Such services

usually do not attract state dollars in replacing lost federal grants

as state funding often goes to higher priorities of treatment such as

chronic mental illness or crisis treatment (Naierman, 1978).

Outcomes of the Community Mental Health Movement for Children

Greater numbers of children and youth have received more services

following the establishment of community mental health centers. Approxi-

mately 24% of all mental health care done in CMHCs in 1975 involved

children and youth for a total of 1,627,000 clients (ADAMHA, 1980).

There was a reverse shift in proportion of inpatient to outpatient

care for children between 1955 and 1975 (ADAMHA, 1980). Inpatient care

consisted of approximately 77% of the total in 1955, while outpatient

care consisted of 72% of the total in 1975. Hospital admissions for

children had become less frequent proportionatelythan for adults. There

was as well an overall increase in the number of outpatient treatment

episodes between 1971 and 1975 of 154% (ADAMHA, 1980), with the majority

of young people diagnosed as transient stress disorders.

There is evidence that lower socio-economic status (SES) and

minority children were equitably served by CMHCs (Brown, 1975). Data

cited in connection with the general mental health center population

indicate that lower socio-economic groups were well represented among

CMHC clients. In addition, it is known that at least 17% of service

recipients were non-white in 1975 (ADAMHA, 1980). Though services are

not equally utilized by all minorities (Miller, 1974), the overall

representation rate of minorities in the CMHC inpatient population

(17%) surpassed the percent of minorities treated in private hospitals

and in general hospital psychiatric units (10%) (ADAMHA, 1980).

Thus, there is some basis to conclude that mental health centers

succeeded in making specialized services for children available to the

general public by providing a greater volume of care to more clients.

These clients were less severely disturbed (most had stress disorders)

and were treated in outpatient settings in their local communities.

Evaluation and Community Mental Health Centers

Prior to the 1950s there was little research related to mental

illness. The National Institute of Mental Health was founded in the

late 1940s but it was not until the 1970s that funds were made available

for meaningful and comprehensive program evaluation (Vandenbos, 1980).

Evaluation was mandated as an essential CMHC function through P.L.

94-63, Title 3 (1975). Centers were to spend at least two percent of

their overall budget for evaluation (Congressional Record, 1975). They

were to make an annual self-evaluation study of six areas: cost of

operations, quality and patterns of service, accessibility and aware-

ness of services, and availability of services to the public. Proof of

effective treatment was not required.

The sum of these endeavors required centers to keep a wide variety

of data. Included were client demographics and types and amounts of

services provided to individual clients. This involved extensive record

keeping (Keppler-Seid, Windle, & Woy, 1980). Caution was urged to avoid

burdening centers with extra costs, redundant measurement systems, and

useless and/or poor quality data. Government sources were warned to

avoid overzealous requirements and rigid government standards which

might be irrelevant to local needs.

Need for Evaluation

The 1960s had been many social programs created by enthusiastic

professionals. They failed, however, to obtain empirical support for

their efforts and thus had difficulty in convincing state legislators

and consumer groups of the efficacy of services (Kappler-Seid et al.,

1980; Flaherty & Olson, 1982). Competition for federal dollars made

demonstrated performance a critical issue for mental health centers as

well. Though centers were largely successful in meeting federal

monitoring requirements, there was also pressure to demonstrate suc-

cessful outcomes and link these with planning and service delivery

(Schulberg, 1982).

Program Evaluation Defined

Program evaluation is a process within organizational management

rather than an independent research enterprise for making new scientific

discoveries (Attkisson, Hargreaves,Horowitz, & Sorenson, 1978). Such

evaluation should be rigorous and based on a systematic data collection

system. Suchman, for example, advocated an extended definition of

program evaluation:

the study of planned programs for producing social change
through social experiments. These experiments test the
validity of the hypothesis that the action program has
within it elements that will effect certain causal factors
in the development of the desired objective. What is
evaluated in the action hypothesis is that defined program
activities will achieve specific desired objectives through
their ability to influence those intervening processes that
affect that occurrence of these objectives. (Suchman, 1967,
p. 177)

Outcome Research Defined

Outcome research is one of the most highly favored kinds of program

evaluation (Schulberg, 1981; Attkisson et al., 1978). It can help to

determine if a change was made in connection with a program, although

it does not necessarily specify that the change was caused by the pro-

gram. Such concrete effects related to program goals can include such

indicators as hospital recidivism rates, improvement on standardized

tests, or changes in individual program areas, e.g., Goal Attainment

Scaling (Kiresuk & Sherman, 1968).

Outcome research has been criticized from different points of view.

Schulberg (1981) noted that instruments used to measure changes fre-

quently lack validity and that evaluation of specific outcomes often

excludes many relevant aspects of care. Keppler-Seid et al. (1982)

questioned whether the types of client outcomes were too exhaustive to

feasibly measure on a programmatic basis. They also questioned the

variability between therapists, clients, and client-therapist

interactions. In addition, the lack of experimental studies in which

control groups are used for comparison purposes has proven to be a

problem in evaluating treatment effectiveness. Mental health evaluation

should not be reduced to a series of simplistic criteria such as

profit/loss ratios and gross measures of client improvement (Schulberg,

1981).

Process Evaluation Defined

Process evaluation is an alternative to outcome research. De-

scribing how a program attempts to affect outcomes has been

referred to as "process measurement" (Suchman, 1967). The steps involved

in process measurement are a) specification of the program character-

istics, b) analysis of effectiveness between target populations served,

c) specification of conditions under which the program is most success-

ful, d) control or explanation of factors that pose alternative explana-

tions for the results, and e) generalization of results to other

situations (Suchman, 1967). This approach attempts to illustrate the rela-

tionship between service processes and outcomes more specifically than

would be the case in simple outcome research where only one variable is

studied. If such linkages can be validated for different client groups,

then for the future, measurement of program effort, i.e., number of

sessions, alone will suffice.

General Considerations

One's choice evaluation approaches is determined primarily by

intent whether it be evaluation of effort, performance, or process

(Suchman, 1967). Another variable is the degree of experimental control

in a situation. This influences research design possibilities. Finally,

there are practical issues, such as expense, to be considered. However,

some general guidelines for evaluation have been recommended:

1. Evaluation procedures should be linked closely with evaluative

objectives in order to avoid redundant or irrelevant data.

2. Criterion measures should be valid and reliable.

3. Support systems required to obtain basic information for the

evaluation must be available.

4. The costs of the procedure should be considered and balanced

against its value to the agency.

5. The impact of evaluation data on performance should be monitored

and assessed (Coursey, Specter, Munell, & Hurst, 1977).

Chronologically, one usually identifies program goals and objec-

tives. Indicators of performance are developed. These are operation-

alized as a routine data collection system and then the data are used

to assess program functioning. It has been suggested that an integrated

evaluation program under one computerized system eases data collection

and analysis (Coursey et.al., 1977).

An example of such a system is Fiester's (1978) Automated Compre-

hensive Children's Evaluation System (ACCESS) for evaluating a children's

mental health program. Evaluation procedures were designed to measure

all basic areas mandated by P.L. 94-63 (availability, accessibility,

patterns of utilization, quality of services, and client satisfaction).

In addition, a modified version of Goal Attainment Scaling (Kiresuk &

Sherman, 1968) was used to identify goal commonalities across clients

and assist in programmatic comparisons. A specific form was devised

to collect demographic data at the outset of treatment. Shortly after

case termination, a request for more information was made to clients.

As therapists' outcome ratings, demographic data, and follow-up

interviews were completed, they were forwarded to data processing. A

computer program analyzed the data. Print-outs were made available to

each therapist. The system was flexible, standardized, inexpensive,

and minimally intrusive to clinical staff.

Experimental Design in Program Evaluation

Program evaluation is not limited by research methodology. Depend-

ing upon the situation,descriptive, correlational, experimental, and

quasi-experimental methods have their place in evaluation.

There are several considerations in choosing the best method for

a given situation. Rigorous experimental procedures are sometimes

impossible to implement within broadly aimed social programs (Weiss,

1970). Random assignment to different treatment conditions with the

use of control groups is, for example, difficult and may not be feasible

in most mental health centers. Quasi experiments (Cook & Campbell,

1979) are frequently more practical, as long as they are well-planned,

carefully implemented, and cautiously evaluated. Schulberg (1981)

suggested using small group samples in a manner analogous to a single-

case design. If program quality is monitored through simple outcome

measures then more detailed assessments may be pursued with smaller

client samples.

-37-

Aside from overall considerations of experimental method in

designing a method of program evaluation, there are other aspects that

need to be taken into account. First, does one measure individual or

ecological change? Second, is it best to assess change through

standardized measures or through subjective individual goal attainment?

Third, from what point of view should the assessment be made?

Crucial to the issue of what is to be assessed is what constitutes

effective psychotherapy. Most current outcome studies look at such

variables as the psychiatric status of the client, his/her level of

social functioning, and the client's satisfaction with treatment

(Schulberg, 1981). There exist a variety of well-established instru-

ments for these purposes, such as the Global Assessment Scale

(Endicott, Spitzer, Fliess, & Cohen, 1976).

On the other hand, Cyntrynbaum, Snow, Phillips, Goldblatt, and

Tischler (1976) call for evaluation to take a more ecological perspec-

tive. As well as measures of individual change, it is proposed that

family' and community data be collected (e.g., socio-demographic

information, physical health, school and peer adjustment, and family

and individual perspectives). They advocated that successful treatment

should involve environmental as well as individual change and, thus,

should be a part of evaluation procedures.

Closely related to the issue of what to measure is how to measure

it. This usually involves a decision between standardized measures and

subjective goal attainment scales. Arguments favoring the use of

standardized measures emphasize that CMHCs should focus on a few valid

and reliable generic measures that are limited to a few areas of

-38-

personal functioning. This approach encourages comparability of treat-

ments and results across centers. For example clients might be given an

activities of daily living scale as a pre and post treatment measure.

Progress would be determined on the improved levels of functioning as

revealed by this instrument.

Another approach favors goal attainment measures which set idio-

syncratic goals for each client (Kiresuk & Sherman, 1968). Change is

not assessed by norms on a standardized instrument, but according to a

client's observed functioning and compared to an initial rating. The

latter approach is commended for its specificity with regard to the

individual client. Individual results are less likely to get lost in

group aggregate data.

Both approaches can provide useful information about programs. But

it is difficult, if one does not limit the number of goals employed, to

compare results between programs if one uses a goal attainment scale

(Fishman, 1975).

From what point of view should the assessment take place? The

clinician, the client, the family, and the independent observer all

have different perspectives. There seems to be some agreement that

multiple vantage points are desirable. Some points of view are more

suited than others to different outcome measures. The therapist, for

instance, may not be the most objective observer; however, he/she is

frequently in the best position to rate clinical indicators of pathology

and subsequently any progress that may be made. Such observations may

not be evident to others. Again, objectivity, reliabiTity, salience,

and cost are taken into account in answering the question of who should

observe and make ratings (Rydman, 1981).

Topics of Special Concern in Evaluating Children's Issues

When children's services were added to CMHCs in 1975 it became

necessary to evaluate them. In addition to the general problems con-

cerning evaluation of mental health services previously discussed,

children's services have some additional complexities. Levitt (1971)

noted that a child is a developing organism and many symptoms are sub-

ject to spontaneous remission. This makes it difficult to determine

if treatment or developmental factors alone contributed significantly

to success. The tendency of certain symptoms to occur at particular

ages, remit spontaneously, and then be replaced by new symptoms is also

a consideration. This tends to make the researcher more cautions when

interpreting results and concluding that the intervention truly bene-

fitted the client. Finally, when family members are in treatment con-

current with the client,treatment is compounded and variables related

to change are more difficult to assess. It is thus difficult to attribute

change to a particular technique or strategy when multiple treatment

conditions exist.

The maturational threat to internal validity might be eliminated

through use of controlled research, but Walker and Zinober (1978) noted

the ethical difficulty involved in identifying children with unmet

emotional needs and postponing their treatment for research purposes.

They suggest, instead, the use of measures that have normative data in

order that tested results might be compared to similar groups who were

part of a normed population. Yet, few, if any, instruments exist in the

field of emotional development.

The question of who is the best informant to report change is

especially crucial for children. Therapists are often selected to

perform assessment because of training and experience. However, as was

noted to be true with adults, personal involvement in the success of

treatment may present an inherent bias. Problems include perceptual

distortion, bias due to theoretical orientation, and awareness of

administrative consequences (Kiesler, 1966). Fiester (1978) concluded

that therapists' ratings of goal attainment in child clients were higher

in some areas than the child and/or parent reports. This is not a

general conclusion, however, as in other cases parents and children

either tended to rate progress higher or about the same as did the

therapist (Fiester, 1978).

Parents as informants may have many of the same problems as thera-

pists. They observe their child's behavior and are aware of pre and

post differences,but there is also an inherent lack of objectivity in

making ratings. Other informants, such as teachers or relatives, may

be more reliable and provide more valid information with respect to the

client's functioning, but they too may be unable to give unbiased in-

formation in certain situations.

Similar complexities exist when children report data. Children

may be unable or reluctant to supply information, especially about

internal states. Furthermore, the self-reports of children may be

seen as unreliable and irrelevant by adults who are more likely to be

the judge of success.

Perhaps the best approach is to rely upon several sources of

information. "The multi-informant approach assumes that each informant

has access to different kinds of valid information on the patient and

the effectiveness of treatment . combining the [different] perspec-

tives can result in a general index of treatment effectiveness"

(Walker & Zinober, 1978, p. 174).

Outcome Studies in Community Mental Health Centers

Research studies have been conducted for individual centers and

statewide systems. Most often they have focused on broad issues of

efficacy, such as whether or not CMHCs can keep clients out of state

hospitals (Becker, 1974). Some studies have given attention to

P.L. 94-63 requirements to assess accessibility and adequacy of service

(Cox et al., 1982; Fiester, 1978). Others (e.g., Fortney, 1981; Kline

& King, 1973; McGrogan, 1976; Sue, McKinney, & Allen, 1976) have been

interested only in specific treatment outcomes.

In a review of the Community Mental Health Journal by Cox et al.

(1982) few studies were found which measured treatment effectiveness.

Reported research was usually descriptive rather than experimental.

None of the studies reviewed were methodologically rigorous enough to

establish causal relationships between dependent and independent

variables.

Descriptive data relating antecedents and outcomes have been re-

ported on some variables. These include data obtained from the clinical

record (i.e., age, sex, occupation, and diagnosis); data related to

-42-

treatment (i.e., length, type, and intensity); and data from psycho-

metric tests (e.g., the MMPI).

A study in Washington State provided one example of accountability

procedures that served several different purposes (Cox et al., 1982).

This study was concerned with the problem of statewide measurement of

CMHC populations. Two samples of subjects were defined. One sample

from current caseloads was used to give a descriptive cross-section of

clients receiving services. Another sample, drawn from new admissions,

was assessed prior to and after receiving treatment. Assessment in-

cluded work and leisure activities, relationships with others, psycho-

logical distress level, and other areas of functioning. The study

revealed gaps in service delivery patterns. For example, clients who

scored high on a psychological distress scale received the fewest

services. The study helped convince legislators of the importance of

CMHC services.

One study (Becker, 1971) typified another kind of evaluative

effort, measuring program achievement in a specific CMHC. It related

admissions to a state mental hospital to service at a CMHC in Oregon.

Records were used from both institutions to document either increase

or decrease in admissions to the hospital following implementation of

the CMHC. It was concluded that the center had little impact on admis-

sion to the state hospital. But, the center and hospital seemed to

effect different populations. While the center provided services to

clients who presumably needed them, such services were not equally

available to all clients who came to the center, as indicated by the

lack of state hospital clients in outpatient caseloads.

Another group of researchers (Fortney, 1981; Kline & King, 1973;

McGrogan, 1976; Sue, McKinney, & Allen, 1976) studied outpatients and

investigated length of treatment, type of treatment outcome, and/or status

of termination. McGrogan (1976), for example, examined forty independent

variables and their potential use as predictors. Information from

clinical records was found to be the most predictive of four variables

(duration of therapy, termination status, termination condition, and

cost of treatment) as opposed to other information related to type of

treatment or psychological testing. However, overall results were dis-

appointing. It was concluded that a heterogeneous population used

center services and that a variety of variables contributed to

terminating prematurely or remaining in therapy.

Dropping out of therapy at a CMHC is a significant problem and

may range between 35 and 40% (Kline & King, 1973; Sue et al., 1976).

Centers were intended to provide care to a variety of patients using

whatever services may be necessary to meet their needs, but possibly

this expectation is unrealistic given the limited resource of most CMHCs.

Sue et al. (1976) noted the value of identifying and describing patients

who rejected a center's services. Who benefits? What needs are not

met? Early termination is also an important consideration because

seeing clients who immediately drop out postpones services to others

in need.

Generalizations about early termination in CMHCs have been diffi-

cult to make because the definition of premature termination varies from

study to study. Studies also vary in terms of population characteristics

and the nature of the setting. For example, Sue et al. (1976) found

clinical diagnosis not to'be predictive of dropping out while Kline

and King (1973) found diagnosis to predict termination.

In one study,data were obtained from over 3,000 clients in 17 CMHC

facilities (Sue et al., 1976). Defining premature termination as drop-

ping out after only one session, it was concluded that ethnic minority

clients with lower incomes and educational levels, who were diagnosed

as psychotic and who were not assigned to individual therapy, were more

likely to terminate prematurely. Such clients were more likely to be

seen by paraprofessionals. They were most often assigned for diagnosis

and evaluation only rather than regular treatment.

Kline and King (1972) through a cluster analysis found eight sets

of variables, inclusive of 39 separate variables, to identify dropouts.

Different clusters were associated with depression, danger to self,

thought impairment, and danger to others. Dropouts tended to be younger

or recently married. Life histories of broken homes and unresolved

psychological issues pertaining to childhood also were a factor. The

researchers concluded that dropping out of treatment was another

adjustment failure similar to those of the past.

Significantly, however, the dropout group in Kline and King's

study were not distinguished by symptoms of the chronically mentally

ill. Being younger than other clients, this group presented a picture

of a group at risk for development of more serious problems.

These conclusions were corroborated by Fortney (1981). He sug-

gested that dropping out of treatment was a failure on the clients' part

to fit into the psychotherapeutic system. In effect, dropping out was

an indicator of the selection processes used by the center. Ten out

-45-

of sixteen demographic and treatment variables were found to be pre-

dictive of dropping out. Factor analysis isolated four elements:

social stability (age, family size, income, contact with significant

other); socio-cultural experience (age, race, education); therapeutic

stance (presence of a treatment plan, professional therapist); severity

of diagnosis (use of medication, diagnostic category). The results of

the preliminary analysis were not confirmed in a sample one year later.

This failure was attributed to changes within the center which created

new and different factors and, subsequently, affected who stayed in

therapy.

Dropping out of treatment is a complex process involving many

variables. However, most studies test a great number of hypotheses.

This increases the probability that conclusions could be spurious.

Increasing the number of variables studied reduces the probability of

finding a true effect when it is present (Levitt, 1971). Cohen and

Cohen (1975) suggested that using a large number of variables, in the

interest of thoroughness, is seldom justified. If a large number of

variables is to be considered, multiple regression theory holds that

such research is appropriate only when the research goal is predictive,

the N is very large, the results are to be substantively interpreted,

and if a cross-validation study is undertaken (Cohen & Cohen, 1975).

Another theoretical problem with studies that have used premature ter-

mination as an outcome criterion was cited by Luborsky, Auerbach, Chandler,

Cohen, and Bachrach (1971). They observed that there is no explicit evi-

dence that dropping out of treatment is consistently related to lack of

client improvement. Although length of treatment was usually related in a

positive fashion to outcome, the conclusion that length itself was a

curative factor was questioned for several reasons. Therapists may over-

estimate positive change in longer-term patients with whom they may be

assumed to have some personal investment. Conversely, therapists may

rate early dropouts negatively, assuming a minimum number of sessions

are necessary before change can occur. Thus while premature termination

is a frequently used indicator of outcome, it is not always clear what

this variable involves.

Notably, all the aforementioned studies concerned with treatment

outcome in mental health centers were concerned with adults. No data

were collected on children and/or adolescents. One study (Kline &

King, 1973) noted the importance of childhood experience in contributing

to dropping out of treatment as an adult. In some studies (e.g., Kline

& King, 1973; Sue et al., 1976), age was a variable, but only in

increments over 18.

There is one significant exception. Fiester (1978) designed the

Automated Comprehensive Children's Evaluation System (ACCESS) to fill

a void in evaluating children's services. Based on a modification of

goal attainment scaling (Kiresuk & Sherman, 1968), the therapist and

client establish from one to three goals for treatment. Goals are

broken down into different levels of attainment from best possible to

worst possible outcome and are selected from 14 possible standardized

skill areas to enable comparisons among therapists. At the end of

treatment, the therapist and child or parent rate their perception of

the child's level of attainment with respect to each goal.

Fiester described changes associated with the program in 12 out of

14 skill areas. His study encouraged rational planning of new children's

services and the development of staff inservice training programs. It

did not, however, relate either client (e.g., demographic character-

istics) or treatment (e.g., duration of treatment) variables to outcome.

Thus, program evaluation attempts in CHMCs: 1) seldom separate

effects for adults from those for children, 2) seldom study institu-

tional processes beyond a single level (i.e., studying goal attainment

in a specific area for a specific therapist), and 3) seldom investigate

specific treatment outcomes as a function of situational as well as

client related variables (where such studies exist, they are often

subject to a criticism of making a large number of non-independent

statistical analyses). Overall, little outcome research on program

effectiveness with children exists.

Effectiveness of Evaluation

Pressures for more accountability have created several problems

for centers. One was the demands for similar information from different

monitoring groups. Another was that accountability efforts diverted

energy and resources from service delivery. Also, evaluation criteria

prescribed by external sources are not necessarily useful to local

programs (Keppler-Seid et al., 1980).

Prior to the enactment of P.L. 94-63 requiring centers to maintain

accountability data, Rocheleau (1974) reviewed Florida mental health

centers and noted that CMHCs did not assess the impact of evaluation

upon later performance. Furthermore, evaluation efforts, although con-

suming significant amounts of resources and staff time, appeared to have

little effect on the organizations themselves. Directors of centers

undertaking evaluation expected little useful information from such

efforts and results were used infrequently in decision making.

One of the desired outcomes of P.L. 94-63 was that accountability

studies would not only produce data helpful to NIMH in monitoring

legislative intent,but would serve as a stimulus to improve services

(Flaherty & Olsen, 1982). Yet, Rossman, Hober, and Ciarlo (1979) found

that program evaluation ranked eighth or ninth out of ten sources of

input with respect to relevance for decision making by CMHC managers.

This is in contrast to "input from immediate supervisors" which ranked

first. Ellsworth (1973) found that when feedback on treatment efficacy

was provided to staff in two separate programs, only one program improved

its performance in response to the data. This outcome eventually sub-

sided to the prestudy level.

An exception to these findings was Fortney's (1981) results of an

evaluation study of a mental health center in Gainesville, Florida. This

study sought to link client, therapist, and treatment variables to

outcomes. The preliminary study noted four factors which discriminated

between positive and negative outcomes. However, a cross validation

study a year later did not confirm the original findings. Fortney

speculated, in explaining the results, that the center had improved its

practices partially in response to the initial study so that the

original factors no longer applied.

There is general agreement (e.g., Bigelow, 1975; Flaherty & Olson,

1982; Rossman et al., 1979) that there are factors which determine

the usefulness of evaluation efforts to the agency. They are

1. The study should result from the felt needs of the center and

the desire to monitor its own performance, as opposed to requirements

being imposed from the outside.

2. The focus of the study (i.e., cost, amounts of service delivered,

etc.) should consider the audience addressed. For instance, in one

survey (Rossman et al., 1979) studies related to cost were preferred

by administrators while studies related to performance were felt to be

most relevant by clinicians.

3. The dissemination of findings and their usefulness to the agency

are important.

4. Program evaluation data are most likely to have an impact

when management receives feedback that is directly involved with cur-

rent issues facing the center and when information is presented in a

simple and personal manner.

Outcome Studies Related to Specific Variables
in the Current Study

There are several studies which relate client variables to therapy

outcomes in different treatment situations (e.g., Baekeland & Lundwall,

1975; Levitt, 1971; Luborsky et al., 1971). However, there is little

research on the type of treatment, parental involvement in therapy,

duration of treatment, referral source, and severity of diagnosis,

especially as they relate to children in a mental health center setting.

It is important in understanding the function of a mental health

center not only to demonstrate whether changes occurred in clientele

as a result of treatment, but also how different client groups are

affected by treatment (Suchman, 1967). For example, are children whose

parents participate in therapy more positively affected? Are children

who are referred by the court less positively affected by treatment? If

a center can answer such questions, then it can make institutional

changes to better serve clients (Cox et al., 1982).

A review of related research influenced the choice of variables

selected for study in this investigation. The decision to focus on a

discrete number of variables was made on the basis of Cohen and Cohen's

(1975) caution regarding the chances of spurious significance when

simultaneously studying a large number of variables. The following

variables received specific attention.

Parent Involvement

Many people believe that parent involvement is important if thera-

peutic changes are to occur in a child client. This is in accord with

a systemic view of family functioning which emphasizes that a change

within one family member will affect the entire system and vice versa

(Haley, 1974; Minuchin, 1974). If an individual's role in a family

helps maintain a particular family balance, then family members can

maintain or help change a child's behavior.

Three studies reviewed by Levitt (1971) supported parmtal involve-

ment in treatment. His review suggested that treatment was least effec-

tive when the focus was an the child alone and most effective when

mother, father, and child were included. Two studies (D'Angelo &

Walsh, 1967; Gluck, 1964) clearly favored the addition of the father

to treatment.

Parental involvement in therapy may not in itself be a curative

factor, however. There are examples of treatment programs with chil-

dren which have been successful without any parent involvement (Brody,

1978; Seeman, 1964). It is also possible that the child's age may act

as a moderating variable on the effect of parent participation in therapy.

Hartmann (1968) found that either group or individual therapy for parents

separately from their children was unrelated to outcome with hospital-

ized adolescents. The negative implication of this finding for parental

involvement is mitigated by the fact that the parents were seen

separately. There is evidence in a related field that conjoint (seen

together) marital therapy is more effective than concurrent (seen

separately) marital therapy (Cookerly, 1974). The same effect may

apply for parents and children.

Studies of specific dimensions of parent involvement generally

indicate their importance in successfully treating the child. Lake

and Levinger (1960) found parental disagreement with therapists' problem

assessment, poor cooperation in the initial interview, a parental

tendency to locate the problem in the child rather than in the parent,

and external coercion in bringing the child for treatment, to negatively

affect treatment outcome. Another study concluded that in unsuccessful

cases, parents were less likely to be in treatment (Ross & Lacey, 1961).

Finally, it was noted by Lessing and Schilling (1966) that improvement

was more likely to occur if the mother recognized and accepted her

responsibility in the child's problem.

These findings are not universal however. Levitt (1957) and

Williams and Pollack (1964) found no significant difference between

dropouts and those who remained in treatment on the basis of different

dimensions of parent involvement.

Knowing more about parent involvement in treatment could influence

children's programs. If it were found to be a significant variable,

therapists might require or work harder to obtain parent participation

as a condition of providing services.

Type of Treatment

Type of treatment, as defined in this study, focuses upon individual

and family therapy separately and in combination, couples therapy, use

of parent and child groups, and use of medication. These are the dif-

ferent modalities of treatment frequently used with children.

Overall success rates have been established for child therapy summing

across different treatment modalities (Levitt, 1971). However, there

are few studies comparing the effect of different treatments. Malouf

and Alexander (1976) noted that with many mental health facilities

adopting non-traditional approaches, such as family therapy, it is

important to answer the question: Which treatments are most effective

and/or efficient?

Most comparative outcome studies consider the issue of whether

family therapy is more effective than individual therapy for children and

adolescents. Wellischet al.(1976) studied a group of hospitalized

adolescents and recorded a rehospitalization (return to hospital after

discharge) rate of 43% for the group that received individual therapy

while none of the group receiving family therapy were rehospitalized.

Minuchin (1974) described positive results achieved in working with

families of diabetic, asthmatic, and anorexic youngsters. Structural

family therapy was able to effect substantial improvement in 83-100%

of patients and these changes endured in 91% of the cases. Alexander

and Parsons (1973) found recidivism rates for juvenile offenders who

received short-term behavioral family therapy to be substantially less

(26.1% versus 49% to 73%) than for youngsters who received non-specific

family group therapy or church sponsored eclectic therapies. One recent

review of family therapy literature indicated that every comparative

study showed non-behavioral family therapy either equal or superior to

other programs which included such approaches as individual therapy,

probation programs, parent counseling, methadone programs, or milieu

therapy (Gurman & Kniskern, 1978).

Programs which attempt to teach communication, behavioral, or

parenting skills have been termed "skill training" by Malouf and Alex-

ander (1976). All studies reviewed which in some way attempted

parent training in reinforcement techniques were successful. Outcomes

were maintained over time.

In addition, Parent Effectiveness Training (Gordon, 1971) and

Systematic Training in Effective Parenting (Dinkmeyer & MacKay, 1976)

have also been shown to be effective in changing either parent attitudes

or behaviors, relationships, or child behaviors. Few studies involving

these approaches include comparisons between different treatments,

however.

If centers were able to establish which types of therapy were most

effective, they could better plan their services. For example, if

parenting groups were shown to be as effective as individual therapy

for children, then it would be less costly to serve clients in such

groups. Similarly, if family therapy is no more effective than indi-

vidual therapy, why offer it as an alternative since it is more costly

to provide?

Duration of Treatment

Length of treatment may or may not be related to successful out-

comes. It has been argued that there should be a positive relationship

between outcome and number of treatment sessions (Levitt, 1971). Others

have argued that the relationship between length of treatment and success

is not a linear one (Meltzoff & Kornreich, 1970). Some clients may be

helped by very brief interventions, while others do not show any appre-

ciable gain.

Length of treatment and outcome for children is generally un-

related. Lessing and Schilling (1965) concluded that neither frequency,

number of sessions, nor total time in treatment were related to outcome

in a large outpatient clinic serving children and adolescents. The

same conclusion was reached by Hartmann, Glasser, Greenblatt, and

Solomon (1968), Phillips(1960), and Sheppard (1966), all of whom found

no correlation between outcome and number of treatment hours.

For adults, however, the case may be different. Luborsky et al.

(1971) found that in 20 of 22 studies of time-unlimited treatment,

length of therapy, as indicated by number of sessions, was positively

correlated with outcome. The authors stopped short of concluding that

the length of treatment itself was a causative factor, noting the

tendency to rate long term clients positively and produce skewed

therapist ratings. It has also been argued that those who stayed in

treatment longer may simply have "felt" helped rather than making any

objective changes.

Time-limited therapies appear to be no less effective than time-

unlimited ones (Goldstein, Heller, & Sechrest, 1966). One study con-

cluded that limiting therapy was associated with a more positive outcome

(Schlein, Mosak, & Dreikeurs, 1963). Further evidence for the efficacy

of short-term therapy comes from Rosenthal and Frank (1958) who dis-

cussed the possibility of positive change occurring within five sessions.

Frank (1963) found that positive change could occur in the first session

after the administration of a placebo. Heilbrun (1974) concluded,

based on a series of investigations, that termination after the first

interview could signal satisfaction. Overall, adult patients with acute

situational problems derive little additional benefit from long-term

treatment and may drop out after their needs have been met. Dropping

out does not necessarily signify treatment failure, however.

These findings are especially pertinent for mental health clinics

that have high dropout rates (Fortney, 1981; Smith & Kline, 1973;

Sue et al., 1976). The fact that CMHCs possess high dropout rates which

come early in treatment and have an overall low median number of

sessions makes it important to document success of treatment by

measures other than premature termination. It is questionable to use

dropping out of treatment as a sole measure of success or failure until

an unequivocal relationship between dropping out and treatment outcome

is demonstrated.

Duration of treatment, as it relates to children's programs in

community mental health centers, is still an important issue. If an

optimal duration of treatment can be identified, centers could more

efficiently plan their services.

Severity of Diagnosis

Most research indicates a close relationship between severity of

diagnosis and/or level of personality functioning and lack of treatment

success. Ross and Lacey (1961) noted that the child who drops out of

treatment was more likely to have had developmental difficulties, un-

usual behavior and/or non-specific symptoms, and to have been a school

truant. The Institute of Juvenile Research (Lessing & Schilling,

1966) reported that degree of emotional disturbance was inversely

related to improvement in cases where mothers and children were treated.

A review of outcome research with children (Levitt, 1971) cate-

gorized children into neurotic, psychotic, and acting-out domains ("neurotic"

was defined as a child who was neither a psychotic, a behavior dis-

order, nor a special symptom). Neurotics improved at a rate of 67.5%,

psychotics at a rate of 65.1%, while children with acting-out symptoms

improved at a rate of only 55%. It therefore appeared that children

with behavior disorders were the most resistant to treatment. This

finding is alarming as aggressive, destructive, and anti-social behaviors

carry over into adult life with greater frequency than other emotional

disorders. High proportions of these children are arrested, hospital-

ized, or abused, and experience alcoholism and divorce as adults

(Robbins, 1966).

Research with adults corroborates findings with children with

respect to severity of diagnosis. Barr (1977) concluded that psychotics

averaged the longest in attendance while persons with marital problems

left most quickly. One review found initial level of personality

functioning to be positively related to treatment success in 14 out of

15 studies (Luborsky et al., 1971). Similar results were reported for

the more severe diagnoses (e.g., psychosis), noting that these clients

benefitted less from psychotherapy. It was concluded that the healthier

the patient was to begin with, the better the outcome. The converse

also appeared to be true.

Another review substantiated this position (Baekeland & Lundwall,

1975). It was reported that sociopathy, paranoia, alcoholism, and

extreme depression were generally characterized by premature termination.

Anxiety and moderate depression, on the other hand, improved the chances

of remaining in therapy.

There is some research on this topic specific to mental health

centers. Anxiety neurosis was found to be the strongest predictor of

termination status in a multi-factor study (McGrogan, 1976). In another

study adult dropouts tended to be more psychologically impaired and had

a greater tendency to act out (Kline & King, 1973). Significantly,

presence of childhood problems as recalled by adult clients was asso-

ciated with dropping out of treatment. Such clients more often came

from broken homes and had conflicts in the areas of comfort and

dependency. This group also included more suicidal persons and clients

who were generally more psychologically impaired (Kline & King, 1973).

It is important to determine whether severity of diagnosis affects

outcome in mental health centers. Given this information, informed

decisions could be made about serving different diagnostic groups.

Resources could be concentrated in areas where outpatient treatment was

most successful. On the other hand, services could be improved for

diagnostic groups not benefitting from current treatments. Referrals

could be more justifiably made if resources were not adequate to provide

successful treatment.

Referral Source

Research indicates that referral source is correlated with treat-

ment outcome. In hospital medical settings, leaving treatment against

medical advice was more likely for clients who were court referred

(Altman, Brown, & Sletten, 1972). In another study in an outpatient

clinic it was found that refuserss," defined as those who were accepted for

treatment but did not show up, were more likely to be referred from

the school systems (Williams & Pollack, 1964).

Motivation for treatment has consistently been demonstrated to
affect treatment outcome. One review suggested that in 34 of 41 studies

lack of motivation was the most important factor in defection from

treatment (Baekeland & Lundwall, 1975). Although motivation may have

been defined differently in the studies, the conclusion was a patient's

personal reasons for seeking treatment determined whether or not he or

she stayed in it.

In the treatment of children it is the adult's motivation rather

than the child's that is most salient. At least one study investigated

outcomes as a function of parental attitudes toward treatment (Lake &

Levinger, 1960). Parents of treatment dropouts were more likely to have

submitted to institutional pressures suggesting that they take their

children for treatment. This conclusion was challenged, however, by

Mindess' finding that working with court-referred clients was as suc-

cessful as with self-referred (Mindess, 1953). It may be that if the

client perceives the agency as favorable to him regardless of referral

source, the effects of an initial referral may be mitigated.

It is apparent that motivation for treatment on the part of the

parent may be an important discriminator of treatment outcome in a

community mental health center. The study of referral source as an

indicator of motivation may help shed light on barriers to treat-

ment.

Summary

A review of the professional literature indicates that program evalu-

ation research is needed to validate mental health center services for

particular client types. This study is designed to examine the attributes

of treatment success or failure in relation to variables of involvement in

treatment, referral source, severity of disturbance, and duration and type

of treatment. These have all been shown to be relevant to treatment

-60-

outcome in other settings. The question is whether these can be con-

nected to treatment outcome with predictive implications in the mental

health centers studied. This will afford direct benefits to patients

in terms of improvement of treatment as well as to agencies in terms of

defining the most effective and efficient ways to utilize their resources.

CHAPTER THREE
RESEARCH METHODOLOGY

The purpose of this study was to investigate variables related to

the effectiveness of mental health center outpatient services to

children. Using information from medical records, case data were

gathered on five variables: referral source, type of treatment,

severity of diagnosis, parent participation in treatment, and duration

of treatment. The relationship of these variables to one another were

examined by comparing cases described as successful and cases described

as unsuccessful. Data on other variables believed to affect treatment

outcome were also gathered.

This chapter is organized into the following sections: population,

description of data source, sampling procedures, research hypotheses,

research design, method of data collection, investigative procedures,

analysis of data, and the limitations of the study.

Population and Sample

The population consisted of approximately 1300 children and

adolescents, ages 0-17, who had received help from two northern Florida

community mental health centers during the years of 1980 to 1983. The

sample for the study consisted of approximately 275 randomly drawn

child and adolescent cases.

The Mental Health Centers

The District III Mental Health Board, Inc., from which the sample

was drawn, was located in one of the fastest growing population areas

in the United States. It was mainly a rural area with all the counties

served by the District Board below the state average in terms of popu-

lation density. Racially, 84% of the population was white, 13.3% was

black, and 1% other races, which was the approximate racial balance

throughout Florida. Regarding age, 80% of the counties in District III

had a higher percentage of young people (17 and under) than the state

as a whole and a below average percentage of older (55 and over)

citizens. Average educational attainment was below the state average,

although there were several centers of higher education in the dis-

trict (District III Mental Health Plan, 1979-1980).

The children's programs (District III Mental Health Plan, 1979-1980)

selected for this study served a three county area in Northern Florida.

Two of the programs were administered under the North Central Florida

Community Mental Health Center, and served Alachua and Levy counties.

Service locations were in Gainesville (the Child, Youth and Family

Center) and in Bronson (the Community Counseling Center). The other

children's program, administered under the North Florida Community

Mental Health Center, was located in Lake City, Florida (the Columbia

Counseling Center), and served Columbia County.

Originally the two centers were combined, having been established

as a non-profit organization in 1972, under a federal staffing grant

from the National Institute of Mental Health. The original service area

included ten rural counties in the north central part of the state. In

1980 the center split into the North Central Florida Community Mental

Health Center, comprising the four southern counties of the original

catchment area, and the North Florida Community Mental Health Center,

comprising the six northern counties. This transition marked the end

of an eight year federal staffing grant for the North Central Florida

Community Mental Health Center and the beginning of a Federal grant for

the North Florida Community Mental Health Center.

Both centers received funding from county, state, and federal

governments, third party payors and client fees. State dollars flowed

to each center through the District III Mental Health Board.

The District III Mental Health Board was a private, non-profit

corporation. In addition to the disbursement of funds to the center,

its responsibilities included the identification, review, and evaluation

of mental health/substance abuse needs, services, and facilities. The

Board monitored the centers to assure contractual compliance. It also

served as the liaison between the Department of Health and Rehabilita-

tive Services, the District's 15 county governing bodies, and providers

of mental health and substance abuse services (District III

Mental Health Plan, 1979-1980).

The individual centers operated under the direct control of Boards

of Directors composed of residents of the service area. The executive

directors for the centers were responsible for implementing District

Board policy, managing center operations, and maintaining positive

relationships with government agencies.

-64-

Whereas the centers received direct federal support during and

prior to 1980, the federal government changed its funding pattern in

1980. Money to the centers now came through the state government in

the form of block grants. The state of Florida chose to maintain mental

health funding at 85-90% of the previous levels. Mental health centers

were thus under the authority of the states and local communities.

As a consequence of the expiration of federal legislation (in 1980)

governing community mental health centers, no national requirements for

monitoring or evaluating centers existed. For the two centers in this

study, monitoring was done jointly by the state Health and Rehabilita-

tive Services department and its contractor for mental health services,

the District III Mental Health Board.

Catchment Area 3A. Catchment Area 3A of District III was geo-

graphically large and included Columbia, Hamilton, Lafayette, and

Suwannee counties. However, the area was sparsely populated (about

73,000) with all counties falling below the state mean of population

density. It also had the slowest population growth rate of the District.

About one-third of the service population was 17 years old or younger,

a percentage higher than both the district and state averages. The

census count by sex was consistent with the state averages. The per-

centage of black residents was, however, higher than either the

District or state averages. The entire catchment area had been

designated a poverty area in 1979-80 by the then Department of

Health, Education and Welfare. Other special problems included a

high rate of teenage pregnancy, substandard housing, and a low number

of school years completed (District III Mental Health Plan, 1979-

1980).:

In 1979-80, the child and adolescent population in need of mental

health services was estimated to be 3,100 individuals. At that time,

the center was meeting approximately 16% of the estimated needs.

Children's needs were considered a high priority as evidenced by

the results of a mental health needs survey done in 1979-80. Child

abuse was ranked second (out of five choices) as a problem in two of

the counties while children with emotional difficulties requiring out-

patient counseling was ranked second, fourth, or fifth, in three of the

counties (District III Mental Health Plan, 1979-1980). Out-

patient services to meet the need of family problems was cited in 1980

to be a "most needed" service as no private sector resources existed in

the area from which to obtain private psychiatric care.

Service delivery was based on an organizational concept in which an

Area Director and a team of clinical and support staff were responsible

for the delivery and coordination of services within a geographic area.

The four counties were served by two clinics, Columbia Counseling Center

and the Suwannee River Counseling Center. The clinic of interest to

this study, the Columbia Counseling Center, employed two full time

mental health professionals specifically serving children during the

period 1980-1983. One of these was a professional and the other a

paraprofessional. The paraprofessional was replaced in the later part

of 1983 by a professional.

Catchment Area 3B. Catchment Area 3B was the largest area in

District II in terms of population. Approximately 190,000 residents

lived in the service area. The Gainesville urban area comprised 62%

of the population while the remaining 38% resided in small, rural com-

munities. Services were often limited in rural areas because of trans-

portation difficulties for clients. Approximately 29% of the population

was seventeen years of age or younger. Census counts by race and sex

were consistent with state averages. However, the area had a low

median income with over 50% of persons in poverty. There were also

high rates of teenage pregnancies and students dropping out of school.

In addition, Alachua County had a high mobility rate, a large population

of youth, and one of the highest per capital crime rates in the nation.

These factors indicated a relatively unstable community at risk for

developing mental health problems.

In 1979-1980, the child and adolescent population in need of mental

health services was estimated to be 4500 persons. About 16% of this

group received services. Needs assessments for Alachua and Levy counties

included the following priorities concerning children: child abuse,

seriously emotionally disturbed children requiring outpatient counsel-

ing, the need for living facilities to prevent dependent children from

becoming emotionally disturbed.

Service delivery in the 3B area was again based on an area concept

of organization in which a team of clinical and support staff delivered

services to children in a geographic area not necessarily consistent

with county lines. Alachua County was served by the Child, Youth, and

Family Center (CYFC) located in Gainesville, while Levy, Gilchrist,

and Dixie Counties were served by the Community Counseling Center (CCC)

located in Bronson. Additional Community Counseling Center outreach

clinics were located in Trenton and Cross City. The clinics of interest

to this study possessed the following staffing patterns: During the

period of the study the Child, Youth and Family Center had two MSW

social workers, one Ph.D. counselor, one part-time clinical psychologist,

and one paraprofessional counselor. The CCC employed six professional

therapists (master's degree or above).

Closed Medical Records

A closed medical record contains information related to three

phases of patient care: intake/diagnosis, treatment, and discharge.

During the admissions interview with each client, demographic and

clinical information are recorded, including race, sex, age, family

size, income, marital status of parent, referral source, and history

of prior psychological services. (See Appendix A.)

Following the initial or intake interview, a clinician writes a

summary and a treatment plan. The summary includes the presenting

problem, medical, psychiatric, educational, social, and family histories,

and a case summation with both a diagnosis and prognosis. The clinical

diagnosis follows categories of emotional illness as described in the

Diagnostic and Statistical Manual III (APA, 1980). The treatment plan

is intended to complement the intake summary. It contains a brief

description of the client's problems, specific treatment goals, and how

these goals will be achieved.

A clinician's observations or chart notes describe the middle phase of

thetreatment process. The notes show who was involved in the session,

the prominent events which occurred, and goals for future sessions.

These notes are designed to be concise and theme oriented rather than

descriptive and factual.

Finally, when a chart or medical record is closed, the clinician

fills out both a discharge summary and a discharge sheet. The discharge

summary contains a brief summation of the course of treatment. It

includes the number of treatment sessions, who attended, and reports

any progress that was made toward the desired goals. The clinician

also rates the status of termination in such categories as "refused

treatment or dropped out after one session" or "referred to other mental

health center" on a discharge sheet. This evaluation reflects whether

the termination was by mutual agreement or viewed as premature. A final

rating of "much improved," "somewhat improved," "unchanged," "slightly

regressed," or "significantly regressed" is given at termination to each

treatment goal listed on the treatment plan.

Variables Investigated

It has been noted (Chapter Two) that a study of both treatment (type

and duration of treatment and parent involvement) and client (severity

of diagnosis and referral source) related variables was necessary to

understand the effectiveness of a CMHC. The following variables were

selected from a review of related research.

Parent Involvement

Parent involvement was defined as parent participation in a child
or adolescent therapy session beyond the intake interview. A parent

or guardian must have been present during the intake and given per-

mission for treatment. However, some parents out of fear, misunder-

standing, or personal uninvolvement preferred to be uninvolved and hoped

that the center would assume full responsibility for treatment. Other

parents wanted to be involved and willingly participated in the sessions.

Most clinicians in most cases recommended parental involvement in the

treatment of a child or adolescent. For various reasons, parent involve-

ment in some cases was non-existent or limited. Did parent involvement

make a positive difference? To help answer this question, it was

necessary to differentiate cases according to whether or not the

parents participated.

One session beyond the intake interview constituted parent parti-

cipation in treatment. The mean number of therapy visits for centers

was approximately 7.5 sessions per case. Thus proportionately, even one

session beyond that required for the intake attended by the parents was

significant.

Information concerning parent involvement was obtained from chart

notes where the therapist routinely reported who was involved in any

given session. If information was vague or ambiguous, the case was

recorded as undetermined and not used in the analysis of data.

Referral Source

Referral source was grouped into seven categories: self, school

system, Health and Rehabilitative Services (HRS) recommendation, com-

munity agency or church, physician or medical clinic, law enforcement

or court order or HRS performance agreement (see Appendix B). These

categories included the primary referral sources and also differentiated

voluntary, agency, and involuntary categories. Voluntary referrals

were those in which there was no element of coercion. Agency

referrals were those where the impetus for the referral came from an

agency, but no court order prevailed. Involuntary referrals were those

where a client was directed by police or court order or probation

officer to receive treatment at the Center.

This information was routinely recorded during the intake inter-

view on the admissions data sheet. If the referral source was unclear,

it was recorded as undetermined. If two or more sources were involved,

the most recent source was used.

Severity of Diagnosis

The diagnostic category of a client was related to the Diagnostic

and Statistical Manual III (ARA, 1980). Several codes may have existed

for a client.

Classification of psychiatric disorders in the DSM III is made

on two primary axes: Axis I includes clinical problems, i.e., conduct

disorders, developmental disorders, etc., and conditions not attributable

to a mental disorder that are a focus of attention or treatment, i.e.,

parent-child problems. Axis II includes personality disorders and

specific developmental disorders. Each axis may include more than one

diagnostic category. Other axes mentioned in the DSM III were not

utilized in this study as they were not uniformly used by the centers.

For purposes of analysis, recorders were instructed to use only

the first diagnosis listed on Axis I and the first diagnosis listed on

Axis II. A case was classified as "mild" if it was a condition not

attributable to a mental disorder or an adjustment disorder. Cases to

be classified as moderate-severe included conduct, thought, affective,

developmental, and personality disorders, or substance abuse. Cases

in this category were distinguished by their longer duration and/or a

specific pattern of behavior.

Type of Treatment

Type of treatment included 1) individual, 2) family/couple,

3) combination individual/family/couple, 4) group, and 5) medication.

For all but medication, two or more sessions in a modality were necessary

in order to qualify for the category.

Recorders identified the treatment modality(s) from the chart

notes. If there was no specific mention of who was involved in the

sessions the treatment category was marked undetermined. For a dis-

tinction of family/couple therapy, it is not necessary for family

members to be seen conjointly with the client. The needed only to be

present at the same session. Family treatment could not be separated

from couple treatment because of ambiguous references in the chart

notes.

Duration of Treatment

Duration of treatment was measured by counting the total number of

sessions attended either by the client and/or other family members.

Recorders noted the number of sessions attended from chart notes and

used the discharge summary for corroboration.

Criterion Measure

For each specific problem originally listed on the treatment plan,

the therapist specified outcome goals or objectives. Progress toward

these goals, at termination or other specific times, was rated on a five

point scale: significant progress (+2), some progress (+1), no progress

(-1), some regression (-1), severe regression (-2). To obtain an

overall rating for an individual chart, the specific goal ratings were

summed. Positive outcome was classified as any positive sum while a

negative outcome was classified as any negative sum. Undetermined

outcome resulted in cases being disqualified from the study.

Sampling Procedures

Two hundred and seventy-five closed medical records charts (cases)

from three separate specialized children's programs within two separate

mental health centers were randomly*drawn from the study. The three

programs were 1) the Child, Youth and Family Center, Alachua County;

2) the Community Counseling Center, Levy County; and 3) the Columbia

Counseling Center, Columbia County. These programs were interested in

and willing to cooperate with the study.

Permission to access medical records for research purposes without

the client's express consent is allowed by both state and federal laws

if certain conditions are met. State statutes concerning confidentiality
*Additional unsuccessful cases (27) were sampled in order to elevate the
proportion of such cases to 40% in the sample.

usually require a client's written permission to use a case record for

research purposes. However, an exception is made if the client's

identity can not be deduced from the data collected. Such was the case

in this study.

Medical records librarians from each unit selected closed medical

records from the period of January, 1980, until December, 1983. All

cases available for review were numbered and then sampled according to

a table of random numbers. Approximately 1300 charts were available

for review in the three clinics.

Criteria for Inclusion Cases in the Study

In order for a case to be included in the study the chart had to

contain a) an admission data form, b) an intake summary or intake form,

c) a diagnosis according to the DSM III Manual, d) a treatment plan,

e) chart notes, f) a discharge summary, and g) therapist rating of

treatment goals. Records were rejected for the purpose of the study

if the outcome was indeterminant, outcome not rated, the child classi-

fied as mentally retarded or over 17 years of age, or in the event that

the child was not in treatment with the parent or legal guardian (e.g.,

foster care), or if the family had not returned for further sessions

after the initial interview. However, all cases rejected for the study

were counted (see Table 1).

Cases were divided between 162 (58.5%) positive outcomes and 112

(40.4%) negative outcomes. Approximately 65% of the total cases sampled

were rejected. The following reasons applied: 10% were discarded in

order to ensure sufficient negative outcomes for the sample; 44% had no evaluation

TABLE 1

SAMPLING DISTRIBUTION BY CLINIC AND BY YEAR

1980 1981 1982 1983 Total

Gainesville

Cases
Cases
Cases
Cases

Available
Sampled
Accepted
Rejected

211 (30)
106 (29)
30 (30)
76 (28)

Bronson

Cases
Cases
Cases
Cases

Available
Sampled
Accepted
Rejected

Lake City

Cases
Cases
Cases
Cases

Available
Sampled
Accepted
Rejected

Total Cases Available 1352

Cases Sampled

Cases Accepted

Cases Rejected

of progress; 16% were cases where clients were older than 17; 22% were

seen for only the intake interview; 4% of the cases involved the child

being in foster care; 4% of the cases did not have a DSM III diagnosis

or had an inappropriate diagnosis (e.g., retardation). See Table 1 for

a summary of total cases available, cases sampled, cases accepted for

the study, and cases rejected.

Demographic Description of Sample

Approximately equal numbers of eases came from the Gainesville and

Lake City clinics while a smaller percentage (see Table 1) of cases

came from the Bronson Clinic. This was due to the smaller number of

cases available from this location. Frequency of cases by year (see

Table 2) were approximately equal, except for a smaller frequency in

1980. This was explained by the introduction of the DSM III Diagnostic

System in the middle of the year, making cases seen earlier in the year

ineligible for the study. Thus, cases were distributed equitably by

location and year.

Other demographic categories of interest included age of client,

race, sex, and income. Clients ranged in age from 2 to 17 years old.

This was divided into four categories corresponding with school status:

preschool, 2 to 5; elementary school, 6 to 10; middle school, 11 to 13;

high school, 14 to 17 (see Table 3). The data suggest that the younger

children were served less often by the mental health centers. The

racial balance evident in the sample (see Table 4) approximated the

racial balance throughout Florida. Males and females were represented

-76-

equally in the sample (see Table 5). When race and sex were grouped

together (see Table 6), white males were in the majority while black

females were in the minority. There was a range in income from $0 to

$32,000 a year. Income was divided into quintiles according to annual

earnings. These included the following categories: lower-lower, lower-

middle, middle-middle, upper-middle, and upper-upper (see Table 7).

Almost one-half of cases fell in the lower-lower category. This is

consistent with Brown's (1975) finding that over 50% of clients served

in community mental health centers had incomes below $5,000. As the

poverty index for a family of four was $9,862 (Beeghley, 1983) most

clients served in the sample could be classified as living at, below,

or slightly above the poverty level. In sum, distribution of cases for

race and sex were close to population averages while distribution of

cases for age and annual income was not representative.

Where family composition was concerned, almost 3/4 of the clients

sampled lived with only one natural parent, while 1/4 did not (see

Table 8). Approximately 2/3 of the clients came from families where

divorce or separation had occurred (see Table 9). These data reflect

a higher than normal distribution of cases involving divorce as the

percentage of divorce. involving children under the age of 13 in

Alachua County (the county in which one of the clinics was located)

was 33 to 40%'(Arey, 1979). Of those divorced, approximately

1/3 had remarried, which was close to the 35% figure cited for

all of Alachua County (Florida Public Health Statistics, 1980).

The percentages for divorce and separation and remarriage (see

Table 10) reflect approximately equal percentages of

-77-

TABLE 2
PERCENTAGE OF CASES ACCEPTED FOR

1980 1981

16.6 31.4
(46) (87)

THE STUDY BY YEAR

1982 1983

27.8 24.2
(77) (67)

TABLE 3
PERCENTAGE OF CASES BY AGE

6-10 11-13

33 23
(91) (63)

14-17

36
(100)

TABLE 4
PERCENTAGE OF CASES BY RACE

White Black

83 17
(230) (47)

2-5

~

r

-78-

TABLE 5

PERCENTAGE OF CASES BY SEX

Male Female

55 45
(152) (125)

TABLE 6

PERCENTAGE OF CASES BY RACE AND SEX

Males Females

White 44 39
(123) (109)
Black 10 6
(28) (17)

TABLE 7

PERCENTAGE OF CASES BY ANNUAL INCOME

Upper- Middle- Upper- Upper-
Lower- lower middle middle upper
lower 7600- 11,000- 15,100- 25,000-
0-7500 10,800 15,000 24,800 32,000

48.4 19.5 16.2 11.2 4.7
(134) (53) (48) (30) (12)

TABLE 8
PERCENTAGE OF CASES LIVING WITH ONLY ONE NATURAL PARENT

Living with
one parent Other

72.6 27.4
(201) (76)

TABLE 9

PERCENTAGE OF CLIENTS EXPERIENCING DIVORCE OR SEPARATION

Natural parents
divorced or separated Other

65.7 34.3
(182) (95)

TABLE 10

PERCENTAGE OF CASES BY MARITAL HISTORY

Parents divorced Parents divorced,
Intact families and remarried single

30.3 30.7 38.3
(84) (86) (107)

cases from families where no divorce had occurred or remarriage had

taken place. A higher percentage of cases were from single parent homes.

The data suggest that children from single parent homes were the group

most likely to receive help from the mental health centers.

Research Hypotheses

1. There is no significant difference between successful and un-

successful cases regarding parent involvement in treatment.

2. There is no significant difference between successful and

unsuccessful cases in terms of referral source.

3. There is no significant difference between successful and

unsuccessful cases regarding severity of diagnosis.

4. There is no significant difference between successful and

unsuccessful cases for type of treatment used by the therapist.

5. There is no significant difference between successful and

unsuccessful cases in terms of duration of treatment.

Method of Data Collection

Chart Review Procedures

The principle investigator developed a procedure to review closed

medical records charts. Data included information about the client,

treatment, outcome, and the therapist. Information gathered about the

client included family income, client age, client diagnosis, race, sex,

family composition, parents' marital status, school performance, and

medical history. Information related to treatment included duration,

type of treatment, and parent participation in treatment. Information

on outcome included specific outcome ratings and whether or not the

termination was premature. Information related to the therapist in-

cluded the therapist's educational level (BA, MA or higher, parapro-

fessional) (see Appendix B).

Data were entered directly from the medical record onto a recording

instrument (see Appendix B) according to the recorder's guide sheet

(see Appendix C). The guide sheet listed the information to be ob-

tained and explained how and where this information was to be found in

a medical record. It also showed where the data were to be recorded

on the recording instrument. The items on the recording instrument

followed the natural order in which information appeared in the chart.

No identifying information was retained to identify the charts in

order to protect client confidentiality. The code numbers assigned by

the principal investigator and medical records librarian correlated any

given case with the medical records chart number. The correlative list

remained with the medical records librarian and was retained for the

purposes of 1) data retrieval in the future, and 2) insurance against

duplication in this or a future study.

A pilot study with 20 cases, not to be included in the study,

tested the recording and data collection procedures. It provided

information as to 1) average time to record a chart, 2) incidence of

missing or ambiguous information, 3) estimated the number of charts

that were required in a sampling in order to obtain the minimum number

of successful and unsuccessful cases.

Permission from the Centers

A letter was sent to each center inviting its participation in the

study. A proposed time schedule for the data review and recording to

occur and instructions for the medical records librarian was included.

Also included was an explanation of how the research might be expected

to advise center programs and an offer to share results. Both centers

gave permission to use their closed medical records for the purposes

of the study.

Selection and Training of Recorders

Employees of the three mental health clinics in the study who had

familiarity with the medical records of the agency were recruited to

assist in the study. The purpose of using employees was to ensure

confidentiality as well as take advantage of their familiarity with

the medical records and agency procedures.

One employee of the North Florida Community Mental Health Center,

Inc., reviewed the charts at that center (Lake City). One employee of

the North Central Florida Community Mental Health Center, Inc., reviewed

charts at the two locations (Gainesville and Bronson) at that center.

At a training session the general purpose of the study was pre-

sented. Recorders were not told the specific hypotheses involved.

They were given the study's time schedule and compensation for their

services was agreed upon. The recorders were made familiar with the

recorder's guide sheet and recording instrument. They were 1) given

precise definitions of each variable on which data were to be recorded,

2) shown how and where information was to be obtained from the medical

record, and 3) instructed as to how each item was to be recorded on

the data sheet.

Recorders practiced data gathering on the same medical records

charts. An inter-judge reliability coefficient or proportion of agree-

ment was computed to determine rater accuracy and reliability (see

Appendix D). Kappas (Ks) were used as an indicator of reliability

between judges for nominal level variables (Cohen, 1960). Items with

lower correlation coefficients were more fully explained or revised to

increase precision.

Data Collection Visits

All data were collected at the medical records library for each

respective children's program during a time that minimized disruption

to the center's regular operation (e.g., after hours). The investigator

was available initially and thereafter at intervals to answer questions

and clarify issues. He participated in the initial numbering and

random selection of charts, using a table of random numbers. Recorders

were able to review about 4 cases per hour. In the event that a

medical record was incomplete or did not meet the criteria for inclusion

in the study, another suitable chart also chosen on a random basis was

substituted. In the case of the Bronson Clinic, all records available

were sampled in the study.

One recorder reviewed a total of approximately 100 cases while

the other reviewed approximately 177 cases. Collection of data at each

site required no more than 10 weeks to complete.

Upon the submission of recording instruments, the investigator

checked to be sure they were properly filled out. When the total number

of cases was reviewed, data from the recording instruments were key

punched onto cards.

General Procedures

1. The investigator developed recording procedures to obtain

needed information from charts.

2. The investigator visited clinic sites to explore with adminis-

trators the feasibility of collecting the desired information.

3. Permission to do the research was obtained.

4. Data recorders were recruited and trained.

5. Approximately 10 cases were randomly drawn to test and refine

the recording procedures and obtain measures of rater reliability. The

recording instrument was changed to improve reliability.

6. A schedule was set for data collection.

7. The data were collected during a ten week period with the

principal investigator consulting with recorders to resolve questions.

8. Data were key punched and analyzed.

Limitations of the Study

The major limitation of this study was that the information obtained

was restricted to medical records. No corroborative information was

available, though at times it was possible to check data from one sec-

tion of the chart against data in another section. The accuracy of the

data was thus dependent on the preciseness, correctness, and objec-

tivity of the clinicians especially as concerned ratings of outcome.

Another limitation of the study was the precision of measurement

with some variables of interest. For example, parent involvement was

measured only as present/not present.

Analysis of Data

Data were analyzed through cross tabulation on a joint frequency

distribution of cases according to two or more classificatory variables.

The joint frequency distributions were tested for strength of associa-

tion by chi-square. Other measures of association such as phi or

Cramer's V described the degree to which one variable predicted another

(Nie, Hull, Jenkins, Steinbrenner, & Bent, 1975). The addition of other

variables as controls allowed study of intervening or interacting

influences.

CHAPTER FOUR
ANALYSIS OF DATA

This study investigated variables related to the effectiveness of

community mental health outpatient services to children. Specifically,

case data were collected on five primary variables: parent involvement

in treatment, referral source, severity of diagnosis, type of treatment,

and duration of treatment. The relationship of these variables to

cases defined as successful or unsuccessful was explored.

Closed medical records were randomly sampled from three clinics

in two mental health centers. Information on 274 cases was collected

systematically by trained recorders who followed the same written

instructions (Appendices B and C).

Successful or unsuccessful outcomes for the cases were defined

by therapist ratings on specific treatment goals. A total of 274 cases

were reviewed. Of these, 162 (58.5%) had successful outcomes and 112

(40.4%) had unsuccessful ones. Data on other variables believed to

affect the primary variables were also collected (Appendix B).

Analysis of the data was accomplished through a cross tabulation

procedure using a joint frequency distribution on two variables. The

strength of association was measured by chi-square (Nie, Hull, Jenkins,

Steinbrenner, & Bent, 1975). This statistic enables one to determine

the probability that the observed relationship between variables could

have occurred by chance. First, row and column marginal totals were

-87-

used to calculate expected cell frequencies (if no relationship is

present between variables). The expected cell frequencies are then

compared to the actual values. As the discrepancy between expected

and actual frequencies increases, the chi-square value becomes larger.

Chi-square only assists in determining whether or not variables

are independent. It does not state the magnitude of this relationship.

Either phi or Cramer's V indexes the degree of correlation between two

variables much as does Pearson's R. These statistics also adjust for

the fact that chi-square is directly proportional to the number of

cases in the sample as well as the table size. Phi or Cramer's V are

included in the tables of this study when the chi-square analysis was

found to be significant or approaching significance for a primary

variable.

The .05 level of significance was used in all tests of significance

as the criterion for rejection of a hypothesis. However, significance

levels equal or greater than p < .25 were reported. Below p < .25,

results were described simply as nonsignificant. The results of the

study will be reported in this chapter. More specifically, data were

analyzed for each of five variables and related hypotheses.

Parent Involvement

Parent involvement in child therapy is often thought to be an

important factor and is highly recommended (e.g., Levitt, 1971;

Minuchin, 1974). The systemic view of family functioning (Haley,

1976) asserts that a symptom within one family member may serve a

function for the entire family. The parents may be unknowingly

supporting a child's behavior. Thus, their involvement in the change

process is assumed to be necessary.

Parent involvement in this study was defined specifically as parent

participation in treatment after the intake interview. Parents present

at one or more sessions after the intake interview were considered to

be "involved" in the treatment of their child.

Data were available on all 274 cases. In 81 (29.5%) cases, both

parents were included in at least one session following intake. In the

remaining cases (70.4%) both parents were not included. Of the cases

where both parents were included in post intake sessions, 75 (92.5%)

of the parents attended sessions together. There were 6 cases (7.5%)

where both parents participated in treatment, but did not do so together.

Of parents who participated in therapy alone, without the other parent,

there were 177 females (91.7%) and 10 males (9.3%). Single parents partici-

pating thus comprised 68% (187 cases) of the sample. There was a

remainder of 6 cases (2%) of the sample where neither parent par-

ticipated in sessions after the intake.

HO1. There is no significant difference between successful and

unsuccessful cases regarding parent involvement in treatment.

The frequency of 6 cases in which there was no parent involvement

in treatment was so small that further analysis seemed inappropriate.

Chi-square assumes that for any category, sample frequencies are dis-

tributed normally about the population value. The assumption of

normality is not met when expected population values are close to zero.

Thus, with very small expected frequencies, the validity of x2 results

is questioned (Welkowitz, Ewen, & Cohen, 1971). For this reason,

analysis was not pursued and the results remain inconclusive.

Examination of the data reveals that considerable numbers of

parents, 193 (70.4%),participated in treatment without another

adult. This is consistent with the representation of single

parents in the sampling distribution as reported in Chapter Three.

Referral Source

Referral source may be a likely indicator of attitudes and moti-

vation toward treatment (Lake & Levinger, 1960). Parents of children

who drop out of treatment are more likely to have experienced institu-

tional pressure and resisted treatment.

Information about the source of referral was first coded into

seven categories. These were self, 103 cases (37.6%); school, 60 cases

(21.9%); Health and Rehabilitative Services (HRS) suggestion, 63 cases

(22.7%); community agency or church, 12 cases (4.3%); physician or

medical clinic, 17 cases (6.1%); police or law enforcement agency,

15 cases (5.4%); judicial court, 15 cases (5.5%) (see Table 11).

These referral sources were then combined to yield three major

categories used for analysis: 1) voluntary and personal referral

(self, community agency or church, and medical clinic or private

doctor); 2) agency referral (included school or HRS suggestion);

3) involuntary (included law enforcement and court ordered/HRS per-

formance agreement).

Voluntary cases were those where little or no coercion could be

presumed to exist. Of a total of 277 cases, 132 (47.7%) were in

TABLE 11

FREQUENCY DISTRIBUTION FOR REFERRAL SOURCES

Relative Adjusted Cum
Referral Absolute Freq Freq Freq
Source Code Freq (PCT) (PCT) (PCT)

Self 1. 103 37.2 37.6 37.6
School 2. 60 21.7 21.9 59.5
HRS Suggestion 3. 63 22.7 23.0 82.5
Agency 4. 12 4.3 4.4 86.9
Medical Clinic 5. 17 6.1 6.2 93.1
Police 6. 4 1.4 1.5 94.5
Judicial 7. 15 5.4 5.5 100.0
9. 3 1.1 Missing 100.0

Total 277 100.0 100.0

TABLE 12

FREQUENCY DISTRIBUTION FOR REFERRAL SOURCE RECLASSIFIED

Relative Adjusted Cum
Referral Absolute Freq Freq Freq
Source Code Freq (PCT) (PCT) (PCT)

Voluntary and
Personal
Referral 1. 132 47.7 48.2 48.2

Agency
Suggestion 2. 123 44.4 44.9 93.1

Involuntary 3. 19 6.9 6.9 100.0

9. 3 1.1 Missing 100.0

Total 277 100.0 100.0

this category. Agency referral cases were those where the child's problem

was presumably noticed by someone in a position of authority and it

was suggested that counseling be sought. One hundred and twenty-three

cases (44.4%) were in this category. Ordered cases were defined as

those cases where substantial coercion was employed. There were 19

such cases (6.9%) (see Table 12).

HO2. There is no significant difference between successful and

unsuccessful cases in terms of referral source.

Analysis for the three categories of referral source by type of

outcome is presented in Table 13. Cases where data were either missing

or the outcome was undetermined were excluded from the analysis. A

total of 271 cases were included in the analysis.

Of the voluntary cases, 77 (59.7%) were considered successful

while 52 (40.3%) were seen as unsuccessful. These were the same per-

centages for successful and unsuccessful cases as in the sample as a

whole. Similar frequencies and percentages were evident for agency

referral cases as well indicating no difference from sample values.

There were smaller numbers of cases in the ordered category (19); 57.9%

of these cases were successful while 42.1% were unsuccessful. Again,

this was similar to sample values. Observed and expected frequencies

were approximately equal. The chi-square statistic was .0224 with 2

degrees of freedom. This was not significant. Thus, the null hypothesis

for referral source was accepted.

Severity of Diagnosis

Severity of diagnosis is associated with lack of treatment success
(Lessing & Schilling, 1966; Ross & Lacy, 1961). This variable was

defined by using the DSM-III (APA, 1980) diagnostic categories listed

for the children.

Cases classified as "mild" included those which were given diagnoses

involving adjustment problems (e.g., maladaptive reactions to stress

and/or conditions not attributable to a mental disorder; parent-child

disagreements). One hundred and sixty cases, or 49.3% of the sample,

fell into this category. A total of 110 cases were classified as
"moderate/severe," or 40.7% of the distribution. Cases classified in

this category included substance abuse, attention deficit disorder with

hyperactivity, and conduct disorders, among others. Only one case was

labeled psychotic and the child was diagnosed as schizophrenic.

HO3. There is no significant difference between successful and

unsuccessful cases regarding severity of diagnosis.

A cross tabulation procedure using a joint frequency distribution

for severity of diagnosis and type of outcome (successful or unsuccess-

ful) was used for analysis. The chi-square value was .68 with 1 degree

of freedom. This value was not significant. Of cases classified as

mild, 98 cases (61.3%) had successful outcomes while 62 cases (38.8%)

had unsuccessful outcomes. Of cases in the moderate/severe classifi-

cation, 55.5% or 61 cases had successful outcomes while 44.5% had nega-

tive outcomes. Percentages of successful and unsuccessful cases for

each category closely resembled sample values (58.9% successful,

41.4% unsuccessful). Since observed values were thus very close to

expected values from the overall sample, the null hypothesis was

accepted (see Table 14).

TABLE 13

CHI SQUARE ANALYSIS FOR TYPE OF OUTCOME BY REFERRAL SOURCE

Count
Row PCT
Type of Col PCT Agency Row
Outcome Tot PCT Voluntary Suggestion Involuntary Total

Successful 1. 77 73 11 161
47.8 45.3 6.8 59.4
59.7 59.3 57.9
28.4 26.9 4.1

Unsuccessful 2. 52 50 8 110
47.3 45.5 7.3 40.6
40.3 40.7 42.1
19.2 18.5 3.0

Column 129 123 19 271
Total 47.6 45.4 7.0 100.0

Chi square = 0.0224 with 2 degrees of freedom Significance = 0.9888

TABLE 14

CHI SQUARE ANALYSIS FOR TYPE OF OUTCOME BY DIAGNOSTIC CLASSIFICATION

Count
Row PCT
Type of Col PCT Mild Mod-Severe Row
Outcome Tot PCT 1 2 Total

Successful 1. 98 61 159
61.6 38.4 58.9
61.3 55.5
36.6 22.6

Unsuccessful 2. 62 49 111
55.9 44.1 41.1
38.8 44.5
23.0 18.1

Column 160 110 270
Total

Corrected chi square = .680 with 1 degree of freedom Significance = .4093

MILLISECOND	CLASS.METHOD	MESSAGE
0	sobekcm_page_globals.constructor
0	sobekcm_page_globals.constructor	Application State validated or built
0	sobekcm_database.verify_item_lookup_object
0	sobekcm_page_globals.constructor	Navigation Object created from URI query string
0	sobekcm_database.verify_item_lookup_object
0	sobekcm_page_globals.display_item	Retrieving item or group information
0	sobekcm_page_globals.get_entire_collection_hierarchy	Retrieving hierarchy information
0	sobekcm_assistant.get_entire_collection_hierarchy
0	cached_data_manager.retrieve_item_aggregation
0	cached_data_manager.retrieve_item_aggregation	Found item aggregation on local cache
0	item_aggregation_builder.get_item_aggregation	Found 'all' item aggregation in cache
0	system.web.ui.page.page_load (ufdc.page_load)
0	sobekcm_page_globals.constructor.on_page_load
0	html_echo_mainwriter.add_style_references	Adding style references to HTML
0	html_echo_mainwriter.add_text_to_page	Reading the text from the file and echoing back to the output stream
52	html_echo_mainwriter.add_text_to_page	Finished reading and writing the file