Effects of variable diagnostic conditions on medical problem solving

MISSING IMAGE

Material Information

Title:
Effects of variable diagnostic conditions on medical problem solving
Physical Description:
Book
Creator:
Wilson, Eugenie
Publication Date:

Record Information

Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
oclc - 18808381
System ID:
AA00022411:00001

Table of Contents
    Title Page
        Page i
    Acknowledgement
        Page ii
    Table of Contents
        Page iii
    Abstract
        Page iv
        Page v
    Chapter 1. Introduction
        Page 1
        Page 2
        Page 3
        Page 4
        Page 5
        Page 6
        Page 7
        Page 8
        Page 9
        Page 10
        Page 11
        Page 12
        Page 13
        Page 14
        Page 15
        Page 16
        Page 17
        Page 18
        Page 19
        Page 20
        Page 21
        Page 22
        Page 23
        Page 24
        Page 25
        Page 26
        Page 27
        Page 28
        Page 29
        Page 30
        Page 31
    Chapter 2. Methods
        Page 32
        Page 33
        Page 34
        Page 35
    Chapter 3. Results
        Page 36
        Page 37
        Page 38
        Page 39
        Page 40
        Page 41
        Page 42
        Page 43
        Page 44
        Page 45
        Page 46
        Page 47
        Page 48
        Page 49
        Page 50
    Chapter 4. Discussion
        Page 51
        Page 52
        Page 53
        Page 54
        Page 55
        Page 56
    Appendix A. Cue-clusters
        Page 57
        Page 58
        Page 59
        Page 60
        Page 61
        Page 62
        Page 63
        Page 64
        Page 65
        Page 66
        Page 67
        Page 68
        Page 69
    Appendix B. U.S. Bureau of the Census major occupational groups
        Page 70
        Page 71
    Appendix C. Resident problem book
        Page 72
        Page 73
        Page 74
        Page 75
        Page 76
        Page 77
        Page 78
        Page 79
        Page 80
        Page 81
        Page 82
        Page 83
        Page 84
    References
        Page 85
        Page 86
        Page 87
        Page 88
        Page 89
        Page 90
    Biographical sketch
        Page 91
        Page 92
        Page 93
        Page 94
Full Text










EFFECTS OF VARIABLE DIAGNOSTIC CONDITIONS
ON MEDICAL PROBLEM SOLVING:
AN ASSESSMENT OF CLINICAL JUDGEMENT






BY

EUGENIE WILSON


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN
PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF DOCTOR OF PHILOSOPHY




UNIVERSITY OF FLORIDA


1987















ACKNOWLEDGMENTS


I would like to thank the members of my committee for their time and support, with

special appreciation to Dr. Blashfleld and Dr. Mealiea for their effort and patience.

This work was supported in part by a grant from the Houssell's Gift Committee of

Memorial Medical Center, Long Beach. California.
















TABLE OF CONTENTS

PAGE


ACKNOW L.EGEMENTS ...................................................... i

ABSTR ACT ................................................................ iv

CHAPTERS

I INTRODUCTION .................................................. 1

Bayesian Analysis in Decision Making............................ 3
Clinical Judgement in Medical Settings........................... 11
Factors Affecting Clinical Judgement............................ 19
The Problem ..................................................... 26

II METHODS ..................................................... 32

Development of Medical Cue-Clusters..............................32
Development of Case-specific SES Information....................33
Bayesian Inferential Problems................................... 33
Subjects ......................................................... 34
Procedure ....................................................... 34

III RESULTS ........................... ............................ 36

Problem Development Analysis .................................. 36
Resident Physician Data Analysis................................ 36
Analysis by Disease Categories................................... 39
Subjects' Use of Information Provided............................ 39

IV DISCUSSION...................... ........................ 51

APPENDICES

A CUE-CLUSTERS ............................ ...................... B

B U.S. BUREAU OF THE CENSUS MAJOR
OCCUPATIONAL GROUPS....................................... 71

C RESIDENT PROBLEM BOOK..................................... 73

REFERENCES ............................................................ 85

BIOGRAPHI-CAL SKETCH................................................. 91


ii1











Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfilment
of the Requirements for the Degree of
Doctor of Philosophy

EFFECTS OF VARIABLE DIAGNOSTIC CONDITIONS
ON MEDICAL PROBLEM SOLVING:
AN ASSESSMENT OF CLINICAL JUDGEMENT


by

Eugenie Wilson

December 1987



Chairman: Roger K. Blashfleld

Major Department: Clinical and Health Psychology



Conditions of uncertainty under which medical decisions are often made require

subjective estimates of diagnostic probabilities. Bayes's theorem provides an accurate

method of estimating probabilities. However, decision makers are most often

insensitive to factors affecting estimated outcome and have been shown to yield to

systematic biases. As well, characteristics of the medical problem itself may change its

information value and predispose diagnosticians to error. The purpose of this study

was to examine three categories of problem characteristics for their effects on estimates

of probability of medical diagnosis. These three characteristics were the manner in

which medical cues are presented (cue-clustering conditions), socioeconomic status of

the patient (SES), and type of disease (gastrointestinal or rheumatic).

Classic Bayesian inference problems, integrated with medical cues and SES

information, were used to assess the effects of problem variables. Each problem

contained enough information for subjects to attempt a normative decision by

Bayesian analysis. Medical residents in four family practice residency training











programs were asked to estimate in percentages the likelihood that the described

patient had a medical disorder.

Subjects' estimates of medical diagnosis consistently underestimated the Bayesian

estimates. Mean scores tended to be influenced by validity information, demonstrating

the common phenomenon of Ignoring base rates in decision making (the base rate

fallacy).

In summary, cue-clusters were recognized by subjects as high and low in the same

manner they had been rated by board certified physicians. This made possible problem

comparisons to examine the effects of SES information on estimates. The presence of

any SES information lowered estimates of medical disorder in both low and high cue-

clustering conditions in the gastrointestinal disease category. Effects of SES

information in the rheumatologic disease category were mixed. Equivalent problems

compared across disease categories drew different predictions in any category where

SES information was present, but not when it was absent. SES of the patient can act as

a biasing variable in analysis of patient management problems. As well, it interacts

with disease content to affect outcome.















CHAPTER I
INTRODUCTION

Unlike conditions in the early history of medicine, modem physicians are faced

with a bewildering array of information about almost every patient (Thomas. 1979;

Schwartz and Griffin, 1986). This mass of available knowledge requires a systematic

means of understanding if it is to be used (Schwartz and Griffin. 1986). Analysis of so

large a body of facts may lead physicians to make intuitive judgments subject to biases

of which they are unaware. They may also use unarticulated decision rules they have

learned by clinical experience only (Pauker, 1982). Little time for formal training in

decision making has been available in an already crowded medical curriculum (Elstein,

1982).

Psychology has taken an active research interest in areas bearing on decision

making in medicine, but little of this knowledge has yet come to bear on physicians

actively engaged in clinical diagnosis (Elstein and Bordage, 1979). Decision making

research as applied to medicine falls broadly into two categories (Pauker. 1982).

Descriptive studies attempt to plot the reasoning processes used by physicians, while

prescriptive studies examine the method by which a correct decision can be made in a

particular clinical circumstance. Both bear on increasing understanding of how

physicians select and integrate information in order to generate diagnostic hypotheses

so crucial to treatment decisions.

Prescriptive processes are normative. That is. a standardized technique and rules

are applied to a variety of clinical problems. These rules may be derived from texts or

journals, or may be mathematical in nature, relying specifically on statements of

probability (that is, likelihood of the occurrence of certain events). Bayes's theorem









(described in detail below) is the accepted method for revising probabilities in light of

new information (Schwartz and Griffin. 1986. Phillips, 1973). However, in a series of

papers, Kahneman and Tversky (1972. 1973) and Tversky and Kahneman (1974)

have demonstrated that man is not likely to use Bayesian analysis in decision making

and is more likely to rely on certain judgmental heuristics (or strategies) which are

influenced by biasing factors.

This study examines how physicians make decisions when different amounts and

types of information are provided to them. Overall, very little information was

provided, a condition known as Judgment under uncertainty. Under conditions of

uncertainty, Tversky and Kahneman (1974) have noted that biasing factors are most

likely to enter into a decision.

Alleged biasing factors such as race, socioeconomic status, and sex have been

examined in psychodiagnostic judgment (see Abramovitz and Dokecki, 1977, for a

review). Potential biasing factors in medical diagnosis have not been well investigated.

For the purposes of this study, variations in socioeconomic status (SES) have been

mixed with variations in amounts of biomedical information provided to determine

whether subjects use a normative analysis, in this case Bayes's theorem. As well, the

study examines the effects of socioeconomic status of the patient on estimates of

likelihood of medical illness when that information is combined with varying

amounts of biomedical information. Thereby, both a potential source of bias and

conditions under which it might operate have been examined.

In order to improve the quality of medical decision making, more needs to be

understood both about the process itself and about factors which may influence

Judgment. Before examining the methods and outcome of this study and how it

increases understanding of Judgmental heuristics and potential for bias in medical

decision making, this paper will review salient aspects of current knowledge in

decision making, clinical Judgment, and factors known to affect it.











Bayesian Analysis in Decision Making

The knowledge base in the area of judgment and decision making is steadily

developing and its importance is increasing (Shulman and Elstein. 1975; Slovic and

Lichtensteln. 1971; Slovic, Fischhoff and ULchtenstein, 1977). Much of the inquiry in

the field seems directed toward asking how individuals make decisions without

complete information, since rarely are all facts available whether in research or in the

real world (Schwartz and Griffin, 1986). Prior to 1971, few researchers sought to

explain decision making behavior by means of strategies, or heuristics (Slovic and

Lichtenstein, 1971). Rather, focus was on attempts to compare behavior to normative

models, essentially to those derived from Bayes's theorem. Pauker (1982) has described

ways in which normative models are attractive as a means of accurate decision

making. They are explicit, and underlying assumptions and data can be examined. As

such, they can be taught clearly and concisely without dependence on the accidental

experience of the individual. Third, they help avoid common Judgment errors described

below.

Bayes's theorem is the most widely accepted normative model and decision making

rule for making and revising decisions in ways which avoid common errors of

judgment such as the ignoring of prior information while integrating new information.

It is the mathematically correct and, therefore, optimal method for evaluating and

revising Judgments based on probabilistic information (Phillips, 1973).

The concept of probability revision by Bayes's theorem is actually quite simple, but

the process of mathematical calculation coupled with the difficulty most people have

thinking probabilistically increases its apparent complexity. Essentially, Bayes's

theorem calculates the probability, or likelihood, of the occurrence of an event taking

into consideration available information. This is done by combining the prior

probability of the event's occurrence with relevant diagnostic information to yield the










revised, or posterior odds of the events. Bayes's theorem and its use are best illustrated

by the following example, originally from Kahneman and Tversky (1972), and quoted

by Bar-Hillel (1980a, p. 212).


Two cab companies operate in a given city, the Blue and the Green (according to
the color of cab they run). Eighty-five percent of the cabs in the city are Blue and
the remaining 15% are Green.
A cab was involved in a hit-and-run accident at night.
A witness later Identified the cab as a Green cab.
The court tested the witness' ability to distinguish between Blue and Green cabs
under nighttime visibility conditions. It found that the witness was able to
identify each color correctly about 80% of the time, but confused it with the
other color about 20% of the time. What do you think are the chances that the
errant cab was indeed Green, as the witness claimed?

The background data on the percentage of cabs of each color in the city provide

population base-rate information, or prior probability of occurrence. The second

information (the witness' reliability) is the diagnostic information, or that which

relates to the specific cab in question.

The correct, Bayesian manner to combine the two types of evidence is as follows:

p(G/g)= ,p(g/G) p (G)
p(G/g)= ------ = =
(p(g/G) p(G)) + (p(g/G) p (G)

where p(G) = the prior probability the cab is Green.

p(G) = the prior probability the cab is not Green
(that is, it is Blue).

p(g/G) = the probability that, given the cab is
Green, the witness says it is green.

p(g/G) = the probability that, given the cab is
not Green (i.e., Blue), the witness says it is
Green.

p(G/g) = the posterior odds the cab is Green,
given the witness' testimony.

Population base rates provide prior odds and diagnostic information to produce the

following likelihood ratio for the occurrence.











p(G/g)= (.8 x .15) = 12 = .41
p(G/g .. .." =- .41
(.8 x .15) + (.2 x .85) .29


The tendency among problem solvers Is to ignore the base rates and estimate by

indicators alone, overestimating the reliability of Indicators. In the case of the

problem above, this error would yield a probability of 80% instead of 41%.

Of course. information whereby prior probability is revised to produce posterior

probability may be objective and based on known frequencies of occurrence. However,

to the degree it is not, subjective probabilities must be generated based on beliefs about

the true state of nature. This introduces increased opportunity for Judgmental error

even If a Bayesian analysis was undertaken.

Following a review of the research in decision making, Peterson and Beach (1967)

concluded that man functions well as an "intuitive statistician," although with a

tendency to conservatism. By conservatism, it is meant that he revises his judgment

less than he legitimately might, thus taking less than full advantage of new

information. Kahneman and Tversky (1972) maintained that non-optimal decision

making was more than conservative and in so doing discouraged reliance on the

Bayesian normative approach to understanding human decision making. They held

that the Bayesian approach failed to capture crucial aspects of Judgment process and,

"In his evaluation of evidence, man is apparently not a conservative Bayesian: he is not

Bayesian at all" (p.450). In a series of papers (Tversky and Kahneman, 1971, 1973, 1974;

Kahneman and Tversky, 1972, 1973), they described and demonstrated several

systematic biases in decision making under conditions of uncertainty. These

judgmental heuristics, or strategies, exert an important effect on the accuracy of

outcome.

Tversky and Kalhneman (1974) described subjects' reliance on a limited number of

heuristics to reduce judgmental complexity to simpler operations. One of these










strategies, "representativeness," is often used to make categorical predictions

(Kahneman and Tversky, 1972. 1973).

In a categorical prediction, an individual simply names a choice made between

alternatives. Predicting a patient's diagnosis or a person's future occupation are both

examples of this type of prediction. Kahneman and Tversky (1973) asked subjects to

make categorical predictions under different instructions. The first group of 69

subjects was asked to estimate percentages of all first year graduate students in the

United States enrolled in each of nine listed specializations. This, the base rate group,

identified the beliefs of the subject population being sampled about the prior

probability that any randomly selected graduate student would belong to one specialty

or another. The second group (65 subjects) was given a brief personality sketch of Tom

W. and asked to rank him according to his similarity to typical graduate students in

each of the nine fields of study. This, the similarity group, sampled how subjects

ranked the diagnostic Information about Tom W. in terms of its descriptive power for

each specialty. A third group (114 subjects) was told the personality sketch was based

on projective tests done during Tom W.'s senior year in high school. This, the

likelihood group, was asked to predict the likelihood that Tom W. had become a

graduate student in each of the nine fields of study.

Product-moment correlation between predictions of likelihood (group 3) and

similarity (group 2) was .97; that between likelihood and the estimated base rates of

group 1 was -.65. In fact, subjects ignored the base rates (prior probability) and

predicted outcome (likelihood) by similarity to the personality sketch demonstrating

that their representativeness heuristic governed predictive behavior more powerfully

than beliefs about prior probability.

Further assessment of the prediction group about the accuracy and reliability of

projective tests indicated that as sources of information they were held in low regard.











Although the diagnostic accuracy of the information thus acquired was seen as low, it

was still preferred to base rates. In fact, Kahneman and Tversky (1972) were correct in

that the Bayesian approach to understanding human decision making does not capture

crucial aspects of the judgment process and that other strategies affect accuracy of

predictive outcome.

To further explore the sensitivity of judgments by representativeness to prior

probabilities, Kahneman and Tversky (1973) presented subjects with a more classic

Bayesian inference problem. Again, subjects were given two types of information. Base

rate information provided background about the distribution of the phenomenon in

nature. Diagnostic information related specifically to that about which the prediction

was to be made. Two groups of subjects were told that a group of 100 engineers and

lawyers had been interviewed by a panel of psychologists, who had written a thumbnail

description of each. For one group of subjects, base rates of 30 engineers and 70 lawyers

were provided. These base rates were reversed for the other group of subjects. All

subjects were than presented with five brief descriptions. Two of them follow.


Jack is a 45-year-old man. He is married and has four children. He is generally
conservative, careful and ambitious. He shows no interest in political and
social issues and spends most of his free time on his many hobbies which
include home carpentry, sailing and mathematical puzzles. The probability
that this man is one of the 30 engineers in the sample of 100 is -%. (p. 241)
Dick is a 30-year-old man. He is married with no children. Aman of high
ability and high motivation, he promises to be quite successful in his field. He is
well liked by his colleagues. The probability that this man is one of the 30
engineers in the sample of 100 is %. (p. 242)

After rating likelihood for five descriptions, subjects were given a null description:

Suppose now that you are given no information whatsoever about an individual
chosen at random from the sample. The probability that this man is one of the
30 engineers in the sample is ___%. (p. 241)

While prior probability was found to have a slight effect, judgments of the five

descriptive problems In no way approximated those predicted by Bayes's formula.










Judges responded to specific Information as predicted by the representativeness

hypothesis even when it was in no way diagnostic. Responses to the null description

matched those predicted by the normative model, indicating that base rates had been

utilized in that case only. In all, predictions were dominated by representativeness and

relatively insensitive to prior probabilities.

Representativeness as a judgmental heuristic risks serious error because it is

insensitive to factors that should affect probability judgment, such as known prior

probability, the nature of random processes, and the effects of sample size (Tversky and

Kahneman, 1974). Perhaps most important to real world predictions is the illusion of

validity to which representativeness is subject. Confidence in a prediction rests

heavily on the quality of the match between input and outcome (degree of perceived

representativeness) without regard to the reliability of input information. In fact,

internal consistency remains the prime determiner of confidence in Judgment even

when the Judge is aware of factors limiting predictive accuracy, as in the example of the

projective test data. Clearly, people tend to be too confident in their Judgments overall,

disregarding the extent and unreliability of their data base (Kahneman and Tversky.

1972, 1973).

This tendency to ignore base rates in favor of specific information rather than to

integrate the two has been found to be a robust phenomenon, stable and replicable even

when aspects of the problem content or base rates are manipulated (Lyon and Slovic,

1976; Bar-Hillel, 1980a; Hammerton, 1973). Reliance on the representativeness

heuristic has been demonstrated with expert judges as well as with laymen (Kahneman

and Tversky, 1973; Tversky and Kahneman, 1971).

That individuals will demonstrate cognitive bias leading to judgmental error has

been clearly demonstrated by studies cited above. The cause of the error is less well

understood. Insensitivity to the effects of sample size and prior probabilities has been

cited by Tversky and Kahneman (1974) as resulting injudgmental error through use of a











faulty heuristic. Newell and Simon (1972) have stressed characteristics of the problem

space itself as determiners of choice of problem solving strategy. Using a fault tree

problem representation, Fischhoff, Slovic and Lichtenstein (1978) found that changing

the way the problem was presented altered their subjects' behavior. A fault tree is a

common representation technique for troubleshooting problems. It organizes sources

of difficulty into a branched, tree-like structure. Fischhoff. et al. (1978) used a fault tree

to study subjects' analysis of an automobile's failure to start. They found both laymen

and experienced mechanics were insensitive to possible problem sources left out of the

fault tree (i.e. battery failure), and that changes in detail of the tree or any branches

altered the perceived importance of the information.

Kahneman and Tversky (1972) have suggested that representativeness is governed

by two factors. First, an event seen as similar in essential characteristics to a given

population will be determined to be more representative of that population. For

example, when asked to estimate the likelihood of two different birth orders among all

families of six children in a given city, Kahneman and Tversky's (1972) subjects

estimated that GBGBBG (where G indicated female birth and B a male birth) was more

likely than BGBBBB, although both birth sequences were of about equally likely to

occur in the population. The former is more representative of characteristics of the

population as a whole. Second, an event will be seen as representative if it reflects

salient features of the uncertain process which could account for it. In the case of

random coin tosses, more regular sequences (e.g. HTHTHTHT) were seen as unlikely

because more Irregularity was expected. Tversky and Kahneman (1971, p. 105) have

labeled this phenomenon of local representativeness the "belief in the law of small

numbers," in which small samples drawn randomly from a population are labeled as

descriptive of the whole population in its essential characteristics.

Bar-Hillel (1980a) has argued that appearance of the representativeness error and











consequent tendency to Ignore base rates in favor of case-specific information (the base

rate fallacy) is governed in part by the perceived relevance of information involved.

She has proposed that judges order information by its perceived relevance to the

problem In question with high-relevance information dominating that which is

perceived as low-relevance. Thus, subjects ignore certain classes of information when

problem solving because they feel it is less than relevant to the task at hand. To test the

relevance hypothesis, Bar-Hillel (1980a) presented subjects with a number of problems

which manipulated the relevance of information, beginning with problems designed to

manifest the base-rate fallacy, and closing with two problem categories In which the

base rate fallacy did not appear in problems where base rates and diagnostic

information were equated in apparent relevance to the problem solving task. Although

the joint impact of information was not accurately calculated by subjects according to

the Bayesian model, there were clear attempts to consider both sources. This supports

Bar-Hillel's (1980a) contention that perceived relevance of information will influence

decision making strategy. Also as predicted, problems in which information presented

was not clearly relevant to the task at hand were more error prone.

If information available Is relevant to a problem solving task and is perceived as

relevant, judgment will be enhanced. If information is not relevant, but is perceived as

relevant conditions supporting cognitive bias and error may result. That which

appears most relevant will take the foreground, even if the perception of relevance is in

error (Tversky and Kahneman. 1980. cited in Einhorn and Hogarth. 1981).

Availability is a judgmental heuristic in which an event's probability is assessed by

the ease with which it can be retrieved from memory (Tversky and Kahneman, 1973,

1974). Thus, classes of events more easily remembered will be judged more probable

than those less easily recalled. In a study ofjudgment of word frequency, subjects were

asked to estimate whether each of a given set of letters appeared more often in the first











or third position in words. In fact, all the letters appeared more often in the third

position. Still, subjects' bias significantly favored the first position (p<.001. by sign

test), since it Is easier to recall words beginning with a given letter than those in which

the letter takes third place. As with representativeness, availability can lead to

accurate judgment since memory may be correlated with actual frequency of events.

However, its sensitivity to such factors as saliency, familiarity and recency can lead to

systematic bias through their effects on recall, as in the study described above. At those

times, probability of an event may be overestimated.

Some form of judgmental heuristic would seem to be essential given the complexity

of most real task environments and the basic psychological principle of bounded

rationality (Newell and Simon, 1972). This principle describes a limited human

capacity for information processing which is not the result of any unconscious

dynamic motivations. A complex task environment must be simplified and organized

strategies serving this end are inevitable. While these heuristics render problem-

solving at least possible, they must inevitably do so at the price of some lost

information and risk of error. Exploration of circumstances under which error is most

likely would certainly benefit problem-solving outcomes in a variety of settings.

Clinical Judgment in Medical Settings

The use of clinical judgment in diagnosis is inevitable. Given that the limits of

knowledge and information create a climate of uncertainty, the act of differentiating

some disordered state of the organism from other states of existence requires the

choosing and combining of information of varying reliability in order to make a

judgment. Bounded rationality requires that the task be simplified by heuristics if

judgment is to be possible at all, especially within a limited time frame. The use of

Bayes's theorem has been suggested to calculate posterior probabilities following the

acquisition of diagnostic test data in order to introduce a more controlled and objective









element Into the judgment process (Lusted. 1968: Schwartz, Gorry, Kassirer and Essig,

1973; McNeil, Keeler and Adelstein, 1975; Pauker, 1976; Sisson. Schoomaker and Ross,

1976), Schwartz et al. (1973) have asserted that clinicians actually employ formal

decision-making strategies similar to optimal models, but are unable to communicate

them explicitly. McNeil et al. (1975) described in detail methods for employing Bayes's

theorem to medical decision making both for tests with binary outcomes and those to

which cutting scores must be applied. The utility of this method has been explored for

decisions regarding coronary artery surgery (Pauker, 1976), differential diagnosis of

thyroid problems (Gustafson, Kestley. Geist and Jansen, 1971), diagnostic work-up of

renal lesions (Fryback, 1974), diagnosis of abdominal pain (Leaper, Horrocks,

Staniland and deDombal, 1972), and treatment of cancer of the pancreas (Sisson et al.,

1976). However. Elstein (1976) has noted that, overall, the impact of psychological

research on clinical judgment In medicine has been minimal.

Where statistical analysis of large numbers of clinical cases is available, objective

probability data can be provided for use in decision making. However, Leaper et al.

(1972) found that even computer assisted diagnoses generated substantial error when

clinicians' subjective probabilities regarding the association of certain symptoms with

diseases were employed. This suggests that, in the absence of objective probability data,

the generation of subjective probabilities may be subject to bias.

How clinical judgments are actually made has been studied by Elstein. Kagan.

Shulman, Jason and Loupe (1972) and by Elstein. Shulman and Sprafka (1978) in their

five year Medical Inquiry Project. They had noted that literature in the field

admonished clinicians on how they should do their work or described how computers

might do it, but had left untouched any systematic empirical enquiry into how

physicians actually solve medical problems. Using several methodologies, they

undertook an extensive descriptive analysis of the clinical reasoning of experienced

physicians, and experimentally tested selected hypotheses. Subjects In the study were











board certified or board eligible in internal medicine. Peer nominations were used to

select the group. Members of medical staffs of three hospitals were asked to list the four

best diagnosticians known to them. Physicians labeled criteriall" (expert) received at

least five votes. Those labeled "non-criterial" (average) received one or none. This

permitted comparison of the two groups on problem-solving strategies and skills.

Initially, high fidelity simulations of diagnostic patient contacts were done using

"programmed patients" of the type developed by Barrows and Abrahamson (1964).

These were actors trained to simulate patients and given extensive information about

their characters' medical and personal history. Patient and physician were videotaped

during sessions, beginning with a history taking. Then, the "patient" left the room and

an individual serving as a "data bank" entered to supply information requested by the

physician during the physical examination period. Laboratory tests could be requested

at any time, with results supplied by the data bank Extent of the patient work-up was

at the physician's discretion and no time limits were applied. Intermittently, the

physician was asked to sum up an account of his process. This introspective method

providing verbal process protocols can be criticized since the degree to which such

artificial reporting changes process is unknown. However, it has been employed

fruitfully in such studies as deGroot's (1965) analysis of problem solving in chess, and

Newell and Simon (1972) have argued for the utility of such process tracing approaches

in fully understanding information processing.

Elstein et al. (1978) concluded that most physicians problem solve by generating

and testing a series of hypotheses affected in part by probability estimates of disease

likelihood. Working hypotheses were generated in the first few minutes of the

encounter. Elstein et al. (1978) also found that subjects generated these hypotheses even

when instructed to refrain from doing so. The early generation of hypotheses has been

independently described by Barrows and Bennett (1972) in neurology residents. Both










Barrows and Bennett (1972) and Estein et al. (1978) found that the single most frequent

cause of incorrect diagnosis among their sample was the failure to generate and follow-

up a relevant hypothesis. Clearly, the hypotheses considered are crucial to correct

diagnosis. This importance is compounded by Wallsten's (1978) finding that

information gathered later in the diagnostic process may be system-atically distorted

to be consistent with earlier hypotheses.

The generation and evaluation of hypotheses observed by Elstein et al. (1978)

followed the sequence diagrammed in Figure 1. Cues may take the form of Information

acquired from patient history, physical finding, diagnostic tests or other sources. Cues

are interpreted toward processing current hypotheses under consideration. In this

manner, more and more specific questions may be asked toward defining and

transforming an open-ended problem in a more manageable set of closed problems

(Bartlett, 1958). Cues selected define the internal representation of the problem space

and delineate strategies to be used in seeking the solution (Newell and Simon. 1972).

In their sample, choice of hypotheses was determined from associations between

very few cues. They concluded that simple association and retrieval from memory may

be involved since a single salient cue was a more frequent hypothesis generator than

cue combinations and noted that hypothesis generation may be mediated by

availability to recall, as was demonstrated later by Schiffman, Cohen, Nowik and

Selinger (1978). Elstein and Bordage (1980) have suggested that the use of single or

limited numbers of cues to generate diagnostic alternatives may represent statements

of conditional probability. This use of limited cues to generate hypotheses may be

problematic, since the reliability of cues themselves in physical diagnosis has been

questioned (Koran, 1975). However, no research has yet been done on the accuracy with

which physicians discriminate reliable from unreliable cues (Elstein and Bordage.

1979).






















































Figure 1. Generation and evaluation of hypotheses according to the
hypothetico-deductive process











A second, more controlled methodology used by Elstein et al. (1978) was that of

written patient management problems (PMPs). Beginning with a brief verbal

description of the patient's problem, the PMP task required subjects to decide how to

evaluate the patient, what information was required, and what treatments should be

undertaken. Subjects were 15 of the original 24 physicians who had been studied in the

original life simulations. As with the simulation study, early generation of a correct

hypothesis took three forms. First, physicians who erred formed premature closure on

a hypothesis and failed to gather further (potentially disconfirming) information.

Second, there was a tendency to try to cluster cues too parsimoniously, leading to

overinterpretation or underinterpretation of data to fit with a current hypothesis,

when more than a single hypothesis would have better represented the data. A third cue

interpretation error occurred when one physician frankly misinterpreted a normal

serum value to support a favored hypothesis. Thoroughness of cue acquisition and

interpretation accuracy were both related to accuracy of diagnosis, but the latter was

found to be most important, since both successful and unsuccessful problem solvers

were thorough. The importance of accurate interpretation and integration of data has

been stressed by Gill, Leaper, Guillou, Staniland, Horrocks and deDombal (1973).

As in the earlier simulation study, physician performance varied across problems,

suggesting that characteristics of tasks are important to problem solving strategy and

outcome. The similarity of results of the simulation and PMP format problems led

Elstein et al. (1978) to conclude that the latter, lower-fidelity problem solving task was

in many ways comparable to the higher-fidelity simulations, and may accurately

represent some aspects of real world task environments.

Using more controlled experimental designs, Elstein et al. (1978) studied selected

hypotheses, such as whether counter-instruction would limit physicians' generation of

early hypotheses, whether medical students could be trained in specific problem











solving heuristics to any advantage, and the effects of variations in problem structure

on generation of hypotheses. For the purposes of this review, how variations in

problem structure affect problem solving is of interest. Twenty-two of the 24

physicians enrolled in the study were presented with "fixed order problems." While

further removed from real diagnostic encounter, these problems had the advantage of

permitting systematic variation of problem structure and also permitted exploration of

the potential utility of less expensive simulations in studies of medical problem

solving.

Two dimensions were used to vary problem structure, that of diagnostic specificity

(whether cues given converged on a single hypothesis or could be applied to more than

one), and cue consistency (degree to which cues were consistent with a classic

description of the disease). Cues were presented on six cards, with approximately two

cues per card. Cards were presented always In the same order. A thinking aloud method

was used following each card presentation to track number of hypotheses generated,

number of early hypotheses generated, and number of cues used to generate hypotheses.

Repeated measures analysis of variance replicated the high-fidelity simulation

results in that expert and average physician groups were not distinguishable from one

another. Elstein et al. (1978) maintain that the fixed-order simulation is as valid an

exploratory tool in medical decision making as higher-fidelity methods.

Where significant differences were found across both hypothesis and cue-related

variables, they were attributable to the structural dimensions of problems and not to

physician effects, supporting earlier findings that characteristics of the task affected

diagnostic outcome. Subjects generated more hypotheses when cues clustered

consistently around a single diagnosis than when some data were inconsistent,

suggesting that inconsistencies tended to be ignored or underinterpreted. Elstein et al.

(1978) suggested that the heuristic of clustering cues to generate hypotheses and the











ignoring of inconsistencies would fit well with some medical problems, but will be less

adaptive strategies in situation where cues cluster less well.

Hypotheses were generated early and tended to be limited in number. The upper

limits were from five to seven hypotheses, consistent with the theory of chunkingg" of

information as a memory organizer (Mandler. 1976; Wortman, 1970). This finding of

an upper limit to working hypotheses maintained was consistent throughout the

Medical Inquiry Project and supports Newell and Simon's (1972) concept of bounded

rationality.

To briefly summarize the extensive data of Elstein et al. (1978), the process of

medical problem solving revolves around the early generation of hypotheses by means

of associating a small number of (or possibly single) cues with known patterns. Less

consistent cues may increase limits on the number of hypotheses generated. A small

number of hypotheses are selected and tested against cues in a hypothetico-deductive

manner. Diagnostic error accrued primarily from failure to generate the correct

hypothesis and from problems in cue interpretation. By far the most common error in

cue interpretation was that of over interpretation, that is, assigning relevance to

noncontributory information. Underinterpretation and occasional frank

misinterpretation of cues occurred as well. Criteria diagnosticians were not

distinguishable from noncriterial ones, with physicians' performances quite variable

across problems. Data fit Newell and Simon's (1972) theory of problem solving in

emphasizing characteristics of the problem space and its internal representation as a

determiner of activities employed in a quest of a solution. The result found by Elstein

et al. (1978) counter the more traditional model dividing physicians into more and less

competent groups and raise questions about what task variables increase risk of error.

and how those errors actually occur.










Factors Affecting Clinical Judgment

Error resulting from alleged bias in diagnostic Judgment has been examined from

the epidemiological and social-psychological points of view in psychodiagnosis.

Interest in the effect of social values on clinical Judgment was spurred by reports that

lower class patients were disproportionately represented among mental hospital

populations (Hollingshead and Redlich. 1958; Dohrenwend and Dohrenwend, 1969).

Although the effects of Judgmental bias in these and other correlational studies were

confounded by the possibility of true variation in mental health status across

socioeconomic populations, the reports served to sensitize researchers to the

possibility that clinical judgment might not always be objective and value free, but,

rather, subject to error from unexpected sources.

Direct study of patient characteristics influencing clinical Judgment has been

undertaken using a clinical analogue methodology based on the Impression formation

paradigm ofAsch (1946). In this method, case specific material is presented to subjects

with the characteristic of interest systematically varied. Using variants of the method,

several potential biasing factors have been studied in psychodiagnosis and treatment.

Patient social class has been studied by Lee and Temerlln (1970), Kurtz, Kurtz and

Hoffnung (1970), Del Gaudio, Stein, Ansley and Carpenter (1976), and Nalven, Hofmann

and Bierbryer (1969). The effects of racial factors have been studied by Benefee,

Abramovitz, Weitz and Armstrong (1976), Nalven et al. (1969), and Smith (1974).

Broverman, Broverman. Clarkson, Rosenkrantz and Vogel (1970), Abramovitz.

Abramovitz, Jackson and Gomes (1973), and Persely. Johnson and Hornsby (1975) have

studied the effects of sex as a potential biasing factor in psychodiagnosis and treatment.

These analogue studies have primarily employed written case material. Lee and

Temerlin (1970) varied the written case method somewhat by using an audiotaped

interview, with experimental groups hearing an audiotaped social history as well,

varied with respect to socioeconomic status (SES). In their procedure Lee and Temerlin











(1970) recorded an audiotaped interview with a trained actor who presented himself as a

mentally healthy man. His script contained no socioeconomic indices. Indicators of

socioeconomic status were presented as three case histories: upper, middle or lower

SES. Experimental groups heard a case history and the interview. Controls heard only

the latter. Ratings of both diagnosis and prognosis varied with SES in the experimental

groups. Controls diagnosed the patient as normal with a good prognosis. Those who

heard the lower SES history rated the patient as mentally ill with only a fair prognosis.

Median ratings were significantly different (for diagnosis, chi square = 9.80. p< .01: for

prognosis, chi square = 5.00. p<.05).

Abramovltz and Dokecki (1977) have noted that the issue of a" reality expectation"

based on a "cumulative wisdom" regarding the relationship of certain attributes

complicates interpretations of social class bias in clinical Judgment since the true

distribution of mental health across socioeconomic strata is not known. For example,

increased stresses of poverty have been hypothesized to increase incidence of

depression among poor women (Srole. Langner, Michael, Opler and Rennie. 1962) and

such factors could conceivably produce variability across strata. Since mental illness

cannot be determined without clinical judgment and diagnosis, the actual distribution

of mental health would be difficult to assess. However, whatever the true state of

nature, it is clear that subjective estimates of the probability of mental illness and

subsequent readiness to diagnose it is increased by SES factors.

The extent of social class biasing effects in psychodiagnosis is a complex issue since

it has been found to interact with such variables as clinician experience (Levy and

Kahn. 1970; Routh and King, 1972). clinician social class (Briar. 1961) and clinician

values (Del Gaudio et al.. 1976) with variable effects.

Bias-related judgmental variability was addressed by Broverman et al. (1970) in

their study of sex-role stereotypes and clinical Judgments of mental health. Using a











sex-role questionnaire of bipolar items, three groups of clinically trained

psychologists, psychiatrists and social workers were asked to rate the questionnaire

under one of three instructional sets. Subjects were asked to describe a mature and

psychologically healthy man, woman, or adult of unspecified sex. They found that the

characteristics of good mental health varied by sex, and that those attributed to the sex-

unspecified adult more closely resembled those for men. Subjects were less likely to

attribute mature adult characteristics to a woman. Characteristics attributed to males

and to sex-unspecified adults were similar to those held to be socially desirable by

college students (Cowen, 1961). Put another way, males would be described as more

similar to and more likely to belong to a population of mentally healthy adults than

would women. Whether demonstrated stereotypic conceptions of women affect actual

clinical Judgment has remained unclear. The classic analogue studies have not

supported this (Abramowltz et al., 1973); Nalven et al., 1969). Abramowitz and Dokecki

(1977) have suggested that feminism may have increased the transparency of this

analogue to the clinical populations of interest.

In the area of medical practice, factors affecting clinical judgment have not been

well investigated. The issue of bias due to sex-role stereotyping has been raised, based

mostly on anecdotal evidence (Howell, 1974: Safilios-Rothschild. 1974). In an analysis

of drug advertisements in medical journals, Prather and Fidel (1975) found that

advertising for psychotropic medications primarily depicted women. Those for more

classic organic illness depicted men more frequently. As well, symptoms attributed to

males were depicted in a straightforward and humorless way. while those attributed to

women were depicted more often with humor. They interpreted this an an

encouragement of physicians to denigrate the seriousness of women's complaints and

to view them as emotional in origin.

McCranie. Horowitz and Martin (1978) used a clinical simulation strategy, a form of










the patient management problem, to explore the presence of sex-bias in medical

judgments more systematically. With case descriptions held constant and sex of a

patient varied, problems were presented to physicians who responded to a series of

written questions on diagnosis and treatment. Cases were presented such that they

were open to alternative diagnoses. Mailing of problem books to 300 physicians yielded

a return of 117 (39%). Physicians In the sample responded, as well, to a checklist by

which they predicted the likelihood of other symptoms which had been rated for

"psychogenicity" suggestivenesss of emotional distress). Results did not support the

influence of sex-role stereotypes on initial medical diagnosis or on attribution of

psychogenic symptoms.

Since potential for sampling bias exists in this study and since simulation Is not

real life, the issue remains unresolved. However, if patient's gender influences clinical

Judgment in medicine to the degree that those in advertising may well believe, it is not

reflected here.

Raynes (1979), in a descriptive study, examined the characteristics of information-

gathering processes of physicians leading to prescriptions of psychotropic drugs. Her

sample of ten English general practitioners were observed for four consultation hours

each. Patients presenting problems and the physicians questions were independently

rated by two raters for focus (psychological, social, physical), yielding coefficients of

concordance of .98 for both symptoms and questions. Psychotropic drugs were

prescribed in consultations involving psychological symptoms with or without

psychiatric diagnosis. Among consultations during which psychotropic drugs were

prescribed, the physicians's questions focused on social and emotional aspects of

patient condition in 10 of 13 consultations with purely physical presenting symptoms.

Data did not permit analysis of characteristics of the patient presentation and

diagnostic problem that led the physician to explore other than physical aspects of the

problem in these cases and not in others.











Feinstein (1967) has noted that the personal and environmental characteristics of

the patient should enter into clinical judgment and do so in an objective manner. In a

study of medical diagnosis in the United Kingdom, Fielding and Evered (1978) explored

the potential biasing influence of non-medical cues on diagnosis by varying social

characteristics attributed to a patient during a tape-recorded interview. The patient

presented with an ambiguous medical problem involving physical, emotional and

behavioral symptoms. The patient adopted a middle-class accent for one interview and

southwest regional, lower-class accent for the other. The interviewing physician

maintained a middle class accent in both interviews. Two groups of pre-clinical

medical students listened to the tapes and completed a questionnaire. The

questionnaire contained a psychosomatic scale and a seriousness scale permitting

subjects to rate their perceptions across several dimension (i.e.. trivial/serious;

reasonable/unreasonable; psychosomatic/not psychosomatic). As well, the

questionnaire permitted dimensional ratings of aspects of the patient's personality, the

interview itself, and patient's inferred behavior prior to the interview. Analysis of

rated social class differences permitted a manipulation check.

In fact, symptoms were perceived as more clearly psychosomatic when presented by

a middle-class than by a lower-class patient as predicted. In an attempt to understand

the stereotype characteristics resulting in differential diagnostic ratings, a factor

analysis was done. Factors extracted accounted for about 60% of the variance and

reflected scales associated with 1) social standing. 2) behavioral competence, 3)

standard-nonstandard pronunciation, and 4) an emotional-unemotional distinction.

The first, third and fourth factors reflected significant differences. Although almost

40% of the variance remained unaccounted for by labeled factors, the study lends

support to the possibility that medical diagnosis is open to non-objective Influences, at











least among less experienced diagnosticians. Generalizability of these results would

require further study.

To briefly review the work of Elstein et al. (1978), expert and average diagnosticians

were not distinguishable from one another. Physicians' performances varied widely

across problems, suggesting that characteristics of the problems themselves influenced

clinical judgment and and outcome. This is clear from a close inspection of the high-

fidelity simulation problem designed by Elstein et al. (1978) in which a 21 year old,

unmarried female was brought to an emergency room suffering from acute paralysis in

her legs. She could not sit up without assistance. Information available to inquiry was

both social and physical. It included a variety of symptoms typical of multiple

sclerosis, including transient blindness in one eye and urinary urgency, peculiar bodily

sensations, muscle weakness, and positive bilateral Babinski's sign (an abnormal

reflex response of the toes indicating a lower motor neuron lesion). Social history

included history of sexual activity in a single patient whose menstrual period was

overdue. The patient was very upset by the abrupt loss of function and appeared highly

agitated and tearful. She repeatedly questioned the diagnostician about the meaning of

her condition.

The characteristics of the problem were such that it was not a difficult one for those

who maintained readily available knowledge of neurology. However, for those without,

initial cues did not cluster as meaningfully. Twenty out of 23 physicians generated

conversion hysteria as their first hypothesis, although the patient's dramatic anxiety

was inconsistent with classical descriptions of that condition. Eighteen maintained

the hypothesis at the quarter mark. 17 at the halfway mark. and 10 at the conclusion of

the problem. Twenty-two physicians generated this hypothesis at any point. In

analyzing the responses of their physicians. Elstein et al. (1978) concluded that the case

was difficult for those who closed prematurely on the psychiatric hypothesis and either











failed to gather or misinterpreted additional data. Justification for the diagnosis was

often found in the patient's possible pregnancy. Although gross motor deficit is a very

rare presentation even in a diagnosed conversion hysteria (Freedman. Kaplan and

Sadock, 1975), it was seen as a high probability diagnosis by this group. The problem

may have been solved by relying on the representativeness heuristic (TIversky and

Kahneman. 1974) rather than through Elstein et al.'s (1978) hypothetlco-deductive

problem solving strategy, with non-contributory evidence taken as diagnostic since it

appeared more relevant than the physical cues elicited (Bar-Hillel, 1980a).

Physicians' assessment of subjective probability of a diagnosis has been determined

by Elstein et al. (1972, 1978) to influence the generation of hypotheses in early

diagnostic reasoning. Treatability of a disease, its seriousness and, for some, novelty

will influence hypothesis generation as well. Schiffman et al. (1978) found that the

available heuristic of Tversky and Kahneman (1973. 1974) distorted physicians'

judgments of the probability of diagnostic hypotheses. They presented eight case

simulations followed by a series of questions to a sample of 34 internists. Cases

included demographic patient information and a few symptoms to permit diagnostic

formulation. Subjects were divided into three groups. Group A generated four to six

diagnostic hypotheses and listed them in the order recalled (a measure of availability).

Group A also rated probability of each diagnosis in percentages and the seriousness of

each. Group B rated probability and seriousness of four to six differential diagnoses per

case, listed for them by the experimenters. Two years later, Group C replicated the

performance of Group A. In order to compare subjective assessments to actual

frequency of diagnoses, hospital records were examined and true probabilities for that

setting were obtained.

Spearman rank correlations between availability of hypotheses and judgments of

probability revealed a high level of similarity between order of recall and judged










probability for both Groups A and C. When simply asked to judge probability of a

provided diagnostic alternative, Judgment of Group B subjects reflected a similarity to

objective probabilities for the setting in which they worked. Having to recall diagnostic

possibilities themselves--a demand more like their real task environment--left

subjects vulnerable to the biasing effects of availability in which probability

estimation was influenced by ease of recall. Seriousness of disease did not bias

probability estimates in the same way.

The Problem

Medical problem-solving requires that judgments be made under conditions of

uncertainty. Under such conditions, estimates of subjective probability are required

which may yield to systematic biases (Tversky and Kahneman, 1971, 1973, 1974;

Kahneman and Tversky. 1972, 1973). That one of these errors, availability, can affect

medical decision-making has been demonstrated by Schiffman et al. (1978).

Characteristics of the medical problem itself may predispose diagnosticians to error,

especially if cues or symptom patterns do not cluster together clearly (Elstein et al.,

1978). In such problems, the physician may fall to generate the correct early diagnostic

hypotheses, or alter their estimated probability of accuracy. The manner in which

medical cues are presented (cue-clustering conditions) may change the information

value of other available, non-medical data. In the absence of other cues, information

which would ordinarily be ignored may be used even if it has little diagnostic value.

This creates a risk of distortion and alteration of diagnostic probability estimates

which could ultimately affect generation and investigation of hypotheses.

Examination of problem characteristics' effects on probability estimates would clarify

situations in which medical decision makers are at risk for error.

Cue-clusters, forming the basis of diagnostic information, are made up of a mixture

of specific, or sign, data (the concrete, observable indicators such as white blood cell

counts) and non-specific, or symptom. information (such as subjective reports of aches











and pains). Clearly, where all medical cue indicators are highly specific, memory

rather than Judgment is what is involved in clear decision making. In fact, the

judgment continuum from most difficult to simplest problems as defined by the

relationship of specific to non-specific indicators is such that the most difficult

problems are those posed by all non-specific symptoms, the easiest are those with all

sign data, and an intermediate, mixed category in the middle. With all sign data

producing the clearest diagnostic problems, any competent physician should not fail to

diagnose accurately medical conditions known to him. It is hypothesized that under

more difficult diagnostic conditions the physician is more at risk for judgmental error

from the effects of non-objective influences on clinical Judgment.

A variety of investigations of the effects of non-objective factors on clinical

judgment have been reviewed above. They suggest that the effects ofSES may be to

increase likelihood of assignment of psychiatric diagnosis and lower estimates of

prognosis. Interpretation of these findings is somewhat confused by the findings of

Hollingshead and Redlich (1958) and Dohrenwend and Dohrenwend (1969) that lower

class patients are disproportionately represented among psychiatric clinic and mental

hospital populations. It is difficult to tell whether the tendency to assign labels and

more guarded prognoses to lower SES patients increases their proportion in hospital

populations, or whether true SES variation in mental health status has resulted in a

"cumulative wisdom" (Abramowitz and Dokecki, 1977, p. 462) which under some

circumstances might come to be inappropriately seen as a bias. Interestingly, as in

many of the diagnostic bias studies. Lee and Temerlin's (1970) cases contained little

pertinent information (a variety of low cue-clustering), probably creating an uncertain

condition ideal for stimulating judgmental error. This, coupled with the demand

characteristics inherent in this situation (request for a "diagnosis" in a psychiatric

setting), was no doubt highly influential in the outcome.











Fielding and Evered's (1978) British study found that SES attributions as provided

by the patient's taped speaking accent affected physicians'Judgments of the

psychosomatic origin in an ambiguous medical problem involving physical, emotional

and behavioral components in a male patient. Higher social class of the patient was

associated with a tendency to rate symptoms as psychosomatic and as less serious,

reversing results found in American studies. Once again, Judgment error appeared in an

ambiguous situation.

Whether Fielding and Evered's (1978) results reflect a "clinical lore" stemming from

some true difference in tendency of some social classes to somaticize psychological

problems to a greater degree than others is difficult to ascertain. Reports of pain have

been found to be influenced by ethnic status (Zborowski, 1952), but SES often cuts

across ethnic groups. Freedman and Hollingshead (1957) found more frequent somatic

complaints among some classes of psychiatric patients, but the distribution of

complaints was opposite to that which would have been predicted by Fielding and

Evered's (1978) results, with more frequent somatic complaints among lower than

among higher class patients. As well. generalization from Freedman and

Hollingshead's (1957) sample to that of medical patient populations in general would be

tenuous, and any data of this kind are potentially confounded by incidence of true

illness and the limits it sets on life activity and income potential. Illness is reported to

be somewhat higher among those of lower income (U.S. Department of Health and

Human Services, 1981).

Currently, then, no solid information exists regarding differential tendency across

SES to report somatic symptoms unrelated to illness. However. Fielding and Evered's

(1978) results support the existence of a bias among physicians under certain

circumstances.

It is proposed here that the characteristics of the diagnostic problems themselves










alter the effects of other available information on diagnostic outcome. Conditions of

low cue-clustering create a risk situation in which biasing factors may become

influential. Under conditions of high cue-clustering, risk for judgmental error will be

reduced, and less relevant information will be more likely to be ignored.

It possible that type of disease itself affects decision making in medical problems.

In reporting on that portion of their sample of 19,000 clinic patients who had mental

disorder as their primary diagnosis, the percentage of them with secondary medical

diagnoses varied somewhat across diagnostic categories. As well, recalling the

interesting experience of Elstein et al. (1978) with their high fidelity simulation

problem featuring multiple sclerosis in a young woman, one must consider the

influence of problem content on diagnostic outcome. For this reason, the range of

medical problems used to study judgment in medical populations should be those with

whom the sample population would be readily familiar. Of the variety of disorders

frequently seen in American training hospitals, certain disorders lend themselves to

problem formulation for the study of medical judgment as a result of their tendency to

produce natural diagnostic puzzles. The symptomatic pictures associated with

rheumatic and gastrointestinal diseases include objective signs in widely varying

degrees or not at all. For this reason, the difficulty of differential diagnosis when

"functional" complaints are involved has been noted (Harrison, 1980). Using these

conditions, test cases can be developed in which conditions of cue-clustering and SES

can be systematically varied.

The optimal decision making model of the Bayesian inference problem provides a

framework in which the effect of social information and cue-clustering conditions can

be evaluated. (See page 8 for a review of the paradigmatic Bayesian problem.)

Kahneman and Tversky's (1973) representativeness error Is, in fact, an example of

the base-rate fallacy, in which subjects overestimated the reliability of personal

descriptions which contained little information. At those times, the correspondingly











less likely membership in the smaller subclass, that is, the likelihood that the

individual in question was an engineer (with prior odds at .30) was estimated at .50.

This was because brief, unrelated descriptors were believed "to represent essential

features of the evidence" (Kahneman and Tversky. 1973, p. 237), insensitive to such

factors as the prior probability of outcome and sample size. Bar-Hillel (1980a) has

demonstrated that this tendency to ignore base rates is governed in part by perceived

relevance of sets of information. That is, base rates will be ignored if available

diagnostic information is perceived to be more relevant for making estimates. At other

times, estimates will fall closer to population base rates.

Using Bayesian problems constructed such that the true probability of medical

disorder is calculable, the degree to which physicians' estimates approach this true

estimate and the manner in which their estimates vary from it can be used to assess the

effects of variables on their clinical judgment under controlled conditions. If the

problem design is such that actual probability of a medical disorder is quite high,

probability estimates for conditions seen as less clearly medical should be

significantly lower than for those seen as more clearly medical. The latter should yield

estimates closer to population base rates than to the diagnostic base rates.

Problem formulations were varied on three dimensions. These were 1. content of

disease, 2. socioeconomic status of the patient, and 3. cue-clustering. Medical and SES

problem units were formulated as detailed in the methods section and integrated in

Bayesian problems to test the following questions, using medical resident physicians

as a sample.

1. Does the availability of social information change physician's estimates of

probability of a medical diagnosis under low cue-clustering conditions but not under

high cue-clustering conditions?

2. Will physicians lower estimated probability of medical diagnoses more when






31


given high SES information in a low cue-clustering condition but not under high cue-

clustering conditions?

To test these questions, four hypotheses have been generated. These are as follows:

1. Medical residents' estimates of probability of a medical diagnosis will be lower

under low cue-clustering conditions than under high cue-clustering conditions.

2. Under low cue-clustering conditions, residents will estimate a lower probability

of a medical diagnosis when social information is available than when it is not.

3. Under low cue-clustering conditions, the presence of high SES information will

produce lower estimates of probability of medical diagnosis than will the presence of

low SES information.

4. There will be no effects of SES under high cue-clustering conditions.















CHAPTER II
METHODS

Development of Medical Cue-clusters

Medical cue-clusters were developed with the assistance of a board certified.

family practice physician and a physician board certified in rheumatology and

internal medicine. The cue-clusters produced were to be combined with socioeconomic

information about the described patient to form the basis of the Bayesian judgment

problems. Cue-clusters were a combination of sign and symptom indicators, which

varied the amounts of specific and less specific information available. Signs are

defined as concrete, observable indicators (e.g., white blood cell count). Symptoms are

the more subjective reports of the patient (e.g., joint or muscle pain). Combinations of

these form the kind of diagnostic picture presented to physicians in real life and were

believed to strike a reasonable balance between producing realistic cue-clusters while

strictly controlling the amounts of information made available to subjects.

Cue-clusters were designed in the diagnostic areas of rheumatic disease and

gastrointestinal disease, and to fit into two categories:

1. Low cue-clustering formulations were problems providing less specific, less

definitive information, to provide a low information category for evaluating the effects

of SES information. These clusters were designed to be less readily identifiable as

representing disease states while suggesting a need for further information before a

clear diagnosis could rule out medical problems.

2. High cue-clustering formulations were problems providing more specific and

more definitive information. These clusters would be more readily identified as











representing diseases without need for additional information. This provided the high

information condition for evaluating the effects of SES information.

A group of 12 family practice attending physicians (all board certified) were

asked to perform a rating task in which they were presented with a set of cue-clusters.

For each problem, the physicians were asked to rate the likelihood of disease as low or

high. Cue-clusters which reached 70% agreement across raters were used to form the

low and high categories. Cue-clusters with instructions are in Appendix A.

Development of Case-specific SES Information

Using data from the U. S. Bureau of the Census (1982). low ("blue collar") and high

("white collar") Job descriptions were used to define low and high SES status. This list

(see Appendix B) provided a selection of different but equivalently rated job categories.

As with the cue-clusters, this enabled the researcher to construct apparently different.

but controlled problems to fit the four categories of SES and cue-cluster combinations

used in each disease type. For purposes of control, each case description was of a female

whose spouse held either a blue or white collarJob. Patients' ages were kept within a

limited range, and race (white) was held constant for potentially contaminating

variables.

Bayesian Inferential Problems

The cue-clusters and SES oriented case descriptions were combined to form four

basic problem types. (See Table 3 for problem assignment and disease category.) These

were

1. High SES, low cue

2. High SES, high cue

3. Low SES, high cue

4. Low SES, low cue

Using the rated cue-clusters and the U.S. Bureau of the Census (1982) job lists, the case











descriptions produced were different, but rated equal both in their medical cues and

SES indicators. These case descriptions were nested in the classic Bayesian inference

problem. The second set of base rates provided in each problem varied slightly for

credibility. However, actual calculated percentile outcomes for probability of medical

illness were almost exactly the same and were very high in all problems. (See Appendix

C for the problem set administered to all subjects, with actual calculated probabilities.)

Each subject solved 10 problems. Problem order was varied using a random

numbers table to control for effects of problem order on problem solving.

Subjects

Subjects were sampled from second and third year residents at four family practice

residency training programs. These were at University of California, Irvine. School of

Medicine (UCI); San Pedro Peninsula Hospital (SP); Santa Monica Hospital (SM); and

Memorial Medical Center of Long Beach (MMC). A total of 44 male and 17 female

residents acted as subjects in the study, for a total of 61 subjects. The total number of

residents potentially available in the four programs was 77. However, not all were

available on sampling days. usually because of their participation in off-site rotations

or due to vacation schedules. Altogether, 79% of the potential resident population was

sampled. Each subject was paid $10.00 for participating in the research.

Procedure

The experimenter met with subjects at the traditional luncheon conferences

routinely held in all of the research settings and was introduced as a researcher from a

local family practice residency training program. Subjects at the Memorial Medical

Center of Long Beach knew the researcher as a teacher and clinician but were unaware

that this research project was underway.

At the luncheon conference residents were given problem books which included a

full page of instructions providing them with a context for the research, instructions on






35


what they were to do, and an assurance that their performance would remain

confidential. The final page of the booklet collected selected demographic data and

contained a check for $10.00. (Again, see Appendix C.)
















CHAPTER II
RESULTS

Problem Development Analysis

Level of agreement between raters was determined by per cent of agreement.

Agreement at 70% or above discriminated low and high cue-clusters for inclusion in the

problems. Percentages of agreement for cue-clusters are listed in Tables 1 and 2.

Because of an error in compiling the problem books to be rated, two cue-clusters

were rated only by 11 of the 12 raters. These were cue-clusters which were used in

problems number 2 and 9. Problems were assigned to SES conditions and numbered as

found in Table 3.

Resident Physician Data Analysis

Total subject sample was 61. with 44 males and 17 females sampled from the four

training programs. Distribution by sex and program is found in Table 4.

Before data for male and female subjects could be collapsed and analyzed together.

possible sex differences in estimated probability of medical diagnosis were analyzed for

each of the ten problems. To do so, ten one-way analyses of variance were performed on

the problems. These used arc sine transformations to correct for irregular distribution

ofthedata. Only three of the 10 F values were above 1.0 and none was above 2.0. Since

no significant differences were found, data were collapsed and analyzed as a unit.

The data are not normally distributed. This can be seen by inspection of the

descriptive statistics in Table 5. The means, medians, and modal scores are often

different which suggests non-normal distributions. The non-normative distribution of

the data is also exemplified from inspection of Figure 2, which graphs the bimodal

distribution of problem 1. Because data are not normally distributed, the Wilcoxon sign











test for differences in related samples was chosen as a conservative measure to evaluate

differences in probability estimates made in response to the ten problems. This non-

parametric test does not rely on an assumed normal distribution of data.

For clarity, the hypotheses will be restated here along with the predicted

relationships between problems. For convenience, significance tables are provided for

each hypothesis. Since specific predictions about direction of differences were made.

probabilities were calculated one-tailed. Note that problems one through five are in the

rheumatologic disease category, and 6 through 10 are in the gastrointestinal disease

category (see Table 3).

Hypothesis 1

The first hypothesis is that medical residents' estimates of probability of a medical

diagnosis will be lower under low cue-clustering conditions than under high cue-

clustering conditions. For hypothesis 1 to be supported, high SES, high cue problems

must draw higher estimated probabilities than high SES, low cue problems. Therefore,

estimates for problem 2 should be greater than those for problem 1, and those for

problem 7 should be greater than those for problem 6. Also estimates for low SES, high

cue problems should be higher than for low SES, low cue problems. That is. estimates

for problem 3 should be greater than those for problem 4, and those for problem 8

should be greater than those for problem 9. Data analysis indicated that all predicted

relationships held, as reported in Table 6.

Hvypothesis 2

Under low cue-clustering conditions, the next hypothesis states that residents will

estimate a lower probability of a medical diagnosis when SES information is available

than when it is not. In support of hypothesis 2, both low SES, low cue and high SES. low

cue problems will elicit lower estimates of probability of medical diagnosis than will

low cue only problems. Thus. estimates for problem 5 should be greater than those for






38



problem 4, and also greater than those for problem 1. In addition, estimates for

problem 10 should be greater than those for problem 9. and greater than those for

problem 6. In the rheumatologic disease category, problem 5 drew significantly higher

estimates than did problem 4. However, the predicted relationship between problem 5

and problem 1 did not reach significance in the desired direction. In the

gastrointestinal disease category, problem estimates were significantly different from

one another and predicted (see Table 7).

Hypothesis 3

Hypothesis 3 expects that, under low cue-clustering conditions, the presence of high

SES information will produce lower estimates of probability of medical diagnosis than

will the presence of low SES information. Support for hypothesis 3 requires that

estimates for problem 4 (low SES. low cue) be higher than those for problem 1 (high

SES. low cue), and those for problem 9 (low SES, low cue) be higher than those for

problem 6 (high SES, low cue). The predicted relationship was found in the

gastrointestinal disease category, but not in the rheumatologic disease category (see

Table 8).

Hypothesis 4

The final hypothesis is that there will be no effects of SES under high cue-clustering

conditions. Hypothesis 4 is supported only if problems in the high cue-clustering

conditions are not significantly different from one another. Estimates for problem 2

should not differ from those for problem 3, and those for problem 7 should not differ

from those for problem 8. However, problems in both categories were significantly

different from one another and the hypothesis is not supported (see Table 9). The

directions of differences between these problem sets were as follows: 2 greater than 3.

and 8 greater than 7.






39


Analysis by Disease Categories

Estimates made in response to like problems across the two disease categories were

significantly different in all comparisons except those between low cue only problems

nubered 5 and 10 (see Table 10). That is. problems drew like estimates only when SES

information was absent. Higher estimates of probability of medical diagnosis were

made in response to problems in the rheumatic disease category than to those in the

gastrointestinal disease category in three of four significant comparisons.

Subjects' Use of Information Provided

When calculated by Bayes's theorem, the probabilities of medical diagnosis are all

quite high (see Table 5). Modal scores, representing 25% or less of subjects in any given

problem, consistently underestimate these figures. Mean scores underestimate them

even more and show a much wider range than do the Bayesian estimates.
















Table 1. Percent of Raters' Agreement for Cue-Clusters In Rheumatic
Disease Category



Rating

Problem Number Low High

1 .75 .25

2 .00 1.00

3 .17 .83

4 .75 .25

5 .75 .25






41









Table 2. Percent of Raters' Agreement for Cue-Clusters in Gastrointestinal
Disease Category



Rating

Problem Number Low High

6 .83 .17

7 .00 1.00

8 .00 1.00

9 .82 .18

10 .75 .25
















Table 3. Problem Assignment by SES and Cue-Cluster Status


Problem Number

1

2

3

4

5

6

7

8

9

10


SES Cue-Cluster Disease Category


High

High

Low

Low

(none)

High

High

Low

Low

(none)
















Table 4. Number of Subjects by Sample Site, by Sex and Total





Residency Program


UCI SP SM MMC TOTAL

Male 10 10 8 16 44


Female 3


Total 13


3 7 4 17

13 15 20 61


a
UCI = Univ. California at Irvine, School of Medicine
SP = San Pedro Peninsula Hospital
SM = Santa Monica Hospital
MMC = Memorial Medical Center at Long Beach





















15






in 10


0


5
n- -










0 .25 .50 .75 1.0
Percentile Estimate

Figure 2. Data Distribution for Problem 1
















Table 5. Problem Information and Descriptive Statistics for Each Problem


Bayesian Validity
Problem Estimate Info Mean Median Mode Range Stan. Dev


HiLo 1 .80 .20 .44 .50 .50 .05-.90 .22

HiHi 2 .94 .65 .81 .83 .90 .50-.99 .12

LoHi 3 .94 .60 .67 .75 .90 .20-.95 .21

LoLo 4 .86 .30 .38 .33 .20 .05-.85 .22

LoCue 5 .85 .28 .46 .50 .50 .10-.80 .21

HiLo 6 .86 .31 .24 .20 .20 .01-.80 .18

HIH 7 .94 .61 .70 .75 .80 .10-.98 .21

LoHli 8 .93 .65 .85 .85 .80 .65-1.0 .09

LoLo 9 .85 .28 .29 .21 .20 .05-.85 .20

LoCue 10 .86 .30 .50 .50 .50 .05-.90 .21
















Table 6. Wilcoxon Sign Test Comparisons Between Problems
for Hypothesis 1


Problems/Predicted Relationshios

Hi SES, Hi Cue 2 > Hi SES, Lo Cue 1

Hi SES. Hi Cue 7 > Hi SES. Lo Cue 6

Lo SES, Hi Cue 3 > Lo SES, Lo Cue 4

Lo SES, Hi Cue 8 > Lo SES, Lo Cue 9


Z Score Significance Level (One Tailed)

-6.96 p< .001

-6.94 p< .001

-6.20 p< .001

-7.06 p< .001















Table 7. Wilcoxon Sign Test Comparisons Between Problems
for Hypothesis 2


rruDiems/ rreclcIeQ


Relationships


Low Cue Only 5 > Low SES, Low Cue 4

Low Cue Only 5 > High SES. Low Cue 1

Low Cue Only 10 > Low SES, Low Cue 9

Low Cue Only 10 > High SES, Low Cue 6


Z Score

-2.46



-5.85

-6.65


Significance Level (One Tailed)

p<.05

ns

p<.001

p<.001
















Table 8. Wilcoxon Sign Test Comparisons Between Problems
for Hypothesis 3


Problems/Predicted Relationships Z Score

Low SES. Low Cue 4 > High SES, Low Cue 1 ---

Low SES, Low Cue 9 > High SES. Low Cue 6 -2.22


Significance Level (One Tailedl

ns

p <.01






49







Table 9. Wilcoxon Sign Test Comparisons Between Problems
for Hypothesis 4


Problems/Predicted Relationships Z Score Significance Level (One Tailed)
High SES, High Cue 2 not significantly
different from Low SES, High Cue 3 -5.48 p< .001


High SES, High Cue 7 not significantly
different from Low SES, High Cue 8 -5.36 p< .001






50







Table 10. Wilcoxon Sign Test Comparisons Between Equivalent Problems
Across Disease Categories



Problems Z Significance Level (Two Tailedl Direction of
Significance

Hi SES Lo Cue 1 with 6 -5.01 p<.001 1>6

HI Hi2wlth7 -4.13 p<.001 2>7

LoH 3 with -5.94 p<.001 8>3

LoLo 4 with9 -2.72 p<.01 4>9

Lo Cue only 5 with 10 -1.13 ns
















CHAPTER IV
DISCUSSION

Elstein (1976) has remarked that strategies for decision making and clinical

inference in medicine have changed little in 30 years although medical practice is

quick to embrace new discoveries and technology. Many factors may enter into

practitioners' resistance to give up a purely clinical approach for the use of

mathematical decision tools. At times when the outcome of statistical decision making

does not coincide with clinicians' experience, they may see it as more risky. Then.

more personalized clinical judgment may seem prudent and conservative (Schwartz

and Griffin. 1986). Unfortunately, this clinical judgment is subject to biases

(Kahneman and Tversky. 1973).

Under conditions of uncertainty, decision makers rely on heuristics rather than on

Bayesian normative logic as had been argued in the past (Kahneman and Tversky,

1972). Such strategies as availability and representativeness seem to capture crucial

aspects of the decision making process more accurately than the model of man as

"intuitive statistician" presented by Peterson and Beach (1967).

Bar-Hillel (1980b) explored what features of a sample make it seem representative of

a given population and found that subjects seemed to draw conclusions based on the

appearance of the sample as a whole. Perceived relevance of information in a problem

has been shown by her to figure heavily in such decision making (Bar-Hillel. 1980a).

However, true relevance and perceived relevance of information may not be the same

thing, although they function similarly in encouraging the use of the

representativeness heuristic.

Characteristics of a problem iave been theorized to influence choice of problem











solving strategies (Newell and Simon, 1972). The purpose of this study was to explore

three categories of problem characteristics for their effects on estimated probabilities

of medical diagnosis. These three characteristics were the manner in which medical

cues are presented (cue-clustering conditions), socioeconomic status of the patient

(SES), and type of disease. Subjects were presented with problem configurations which

included enough information for subjects to attempt a normative decision by Bayesian

analysis. Mean response estimates to each of the ten problems varied widely and were

not consistent with those predicted by Bayes's theorem. The manner in which each of

the three categories of problem characteristics affected estimated probability of

medical diagnosis was examined by selective comparison of the judgment problems

within a framework of four hypotheses.

SES of the patient has been suggested to be a biasing factor in decision making. It

has not been clear under what circumstances this might be so. The four hypotheses

examined in the study were designed to evaluate whether SES has an effect on medical

decision making, and under what circumstances it might operate as a biasing factor.

Hypothesis 1 compared subjects' responses to problems when only cue-clustering

conditions were varied (see Table 6). As for all hypotheses, comparisons were made

only within a disease category. In all four comparisons, higher estimates of probability

of medical disorder were given under high than low cue-clustering conditions. This

agrees with ratings of cue-clusters as high or low provided in the problem development

phase and provides a manipulation check to support that the case history portion of the

problem was seen as relevant in the manner it was designed to be.

The effects of SES information on diagnosis have not been well studied in medical

problem solving, although they have been hypothesized to affect both diagnosis and

quality of care received by the patient (Schwartz and Griffin, 1986). In their British

study. Fielding and Evered (1978) found that the complaints of higher SES patients were










rated as less serious than those of lower SES patients, and rated as more likely to be

psychosomatic. This reverses findings of Lee and Temerlin (1970) and others in

psychodiagnosis, which suggest that lower SES patients are more likely to be rated as

having psychiatric complaints.

Problem comparisons in hypothesis 2 examined the effects of high and low SES

information on prediction of diagnosis in the low cue-clustering condition. This was

done by comparing combined cue and SES problems with those in the low cue only

condition. The low cue condition is the most likely to induce judgment bias by

providing a low information condition coupled with an implicit demand for diagnosis.

Under very low information conditions, subjects had the best chance of Judgment

accuracy by calculating according to Bayes's theorem to arrive at their conclusion. This

was possible using information provided in the problem. A second option would have

been tojudge by the base rates provided. Data do not support that subjects used either

strategy, even though their written instructions noted that the limited amounts of

patient information provided would greatly reduce the certainty with which they could

make decisions from that data alone.

In the rheumatologic disease category, low SES information significantly reduced

the predicted likelihood of medical disorder. High SES information did not reproduce

this effect. In the gastrointestinal disease category, the presence of any SES

information reduced the estimated likelihood of medical disorder. This SES effect was

not consistent across disease categories and was less significant in the one

rheumatologic comparison (see Table 7). Inspection of Table 10 supports that problems

in the two disease categories operated differently in all comparisons where SES was

present, but not in the condition where only a low cue stimulus was available. These

results suggest that the presence of SES information may impact the appearance of the

problem as a feature making it seem more representative of a population, as described






54


by Bar-Hillel (1980b). This conclusion is guarded, since some variation in quality of

the problems might also have had an unknown effect.

Hypothesis 3 compared the effects of low and high SES information to one another

under low cue-clustering conditions. Again, no effect was found in the rheumatologic

disease condition. The predicted effect was found in the gastrointestinal disease

category, with high SES information drawing lower estimates of medical diagnosis

than did low SES information. Fielding and Evered's (1978) study also found that

complaints of higher SES patients were rated as less serious. Unfortunately, they do

not report the content of patients' complaints, only that they were held constant. It is

difficult to say with certainty why varying disease content produced different outcomes.

Some inconsistency in the quality or recognizability of problems is possible. However,

they performed as raters predicted (see hypothesis 1). Exposure to patient cases in

family practice residency training programs may favor one disease category over the

other. If so, one type of patient management problem may have represented a more

intellectual than clinical exercise. A process oriented methodology would be required

to more fully understand the interaction of patient SES with type of disease.

SES of the patient has been found to interact with such variables as clinician values

(Del Gaudio et al., 1979) in predicting psychodiagnostic outcome and may interact with

any number of factors in medical decision making.

The final hypothesis predicted that no effects of SES should be found under the less

ambiguous, high cue-clustering conditions. In fact, problem comparisons revealed

significant differences between high and low SES problems in both disease categories

(see Table 9). However, the direction of difference was reversed. In the rheumatologic

disease category, high SES produced higher estimates of medical diagnosis than did low

SES. In the gastrointestinal disease category the reverse was true. with low SES

drawing higher estimates. In this latter disease category, high and low cue-clustering

conditions produced the same outcome when combined with SES. This raises the











possibility that the high cue-clustering problems may have been a more ambiguous

stimulus than they were designed to be. Professional ratings may not have controlled

this.

To summarize the effects of SES, in the gastrointestinal disease category the

presence of any SES information in a problem lowered the estimated likelihood of

medical disorder. This was true in both low and high cue problems. High SES

information resulted in lower estimates than did low SES information. Effects of

patient SES in the rheumatologic disease category differed from those found in the

gastrointestinal disease category. Introduction of SES into a problem lowered

estimates of medical diagnosis, but only in the low cue condition. Effects of low versus

high SES were not different in the low cue condition. In the high cue condition, effects

of patient SES differed in the opposite direction to that found in the gastrointestinal

disease category. In this case, low SES produced lower estimates of medical diagnosis

than did high SES.

It is difficult to understand the interaction of SES with disease category since the

limited problem solving time available to subjects restricted the number of problems

done. It would be helpful to have problems in all four pure conditions (low and high

SES and cue states). The current design locks variables together when an upper limit on

number of problems exists. As well, it is difficult to determine the exact focus of a

subject's attention using this problem solving design. In future research, it would be

interesting to combine a patient management problem design with a thinking aloud

method to more effectively track subjects' focus.

Use of a thinking aloud method might also make It possible to define the meaning of

SES to the practicing physician. SES as a variable is loaded with implicit meaning and

covaries with many factors. Little is known about what SES might imply to the

physician for management of a medical practice. These implications might include






56


include beliefs about the patient's understanding of and compliance with advice,

personal characteristics and values. Further exploration of these aspects of the

meaning of SES as a variable in medical diagnosis would be possible using process

tracing techniques or an adjective checklist. Perhaps this would suggest reasons for the

interaction of SES with disease category.

Patient SES can operate as a biasing variable. However, problem characteristics

increasing this likelihood are complex. In this study, the interaction of SES with type

of disease affected the results. Researchers should consider this possible interaction

when developing studies in medical decision making.

















APPENDIX A
CUE-CLUSTERS

















INSTRUCTIONS

Recent research in cognitive aspects of decision making suggests that

characteristics of the problem being evaluated influence the accuracy of decision far

more than variability in subjects' problem solving skills (Elstein. Shulman and

Sprafka, 1978). In other words, people seem to make the same mistakes no matter how

well trained they are when problems have certain characteristics. Our research group

has begun exploration of ways in which this influences decision making in medicine.

To help us explore this problem, we would appreciate your assistance in the

following task. Attached is a set of vignettes containing varying amounts of patient

information. Please read them and decide whether the configuration presented

represents a medical disorder.

Some of the vignettes will very clearly represent disease states. Others will be far

less clear. In those, it may be Impossible to know with any certainty whether or not the

patient is ill without further Information. It is obvious that, in some cases, other

diagnostic tests will be indicated before a definitive diagnosis can be made. We are not

asking you to make a definitive diagnosis. Just make the best guess you can based on

the data. In no case will you have enough information for a definitive answer.

Please decide whether the patient in each case presentation has a high or low

probability of a medical disorder. We define the words "medical disorder" to mean a

well-defined disease verifiable by some laboratory, x-ray, or biopsy procedure and/or

whose definition Is generally agreed upon.

Thank you for your assistance.










Rheumatic Disease (Problem 1)

Low Cue



The patient is a 38-year-old woman who was well until approximately three months

ago. when she began to have diffuse myalgias which would wax and wane. fatigability

and headache in the occipital region. She stated she often feels short of breath.

Pulmonary function tests were within normal limits, as were differential, ESR,

electrolytes, BUN. creatinine and urinalysis. Review of systems was otherwise within

normal limits except that she complains of amennorhea of five months duration.



This patient has

-------- high probability of medical disorder

-------- low probability of medical disorder


Please check one.






60



Rheumatic Disease (Problem 2)

High Cue



A 44-year-old woman is complaining of six weeks of bilateral knee swelling with

pain. Her joints are stiff in the morning for about one hour. She has had pain in her

wrists for about two years.



This patient has

.-.------ high probability of medical disorder

--------- low probability of medical disorder



Please check one.











Rheumatic Disease (Problem 3)

High Cue



The patient is a 42-year-old woman who has had, for the last three or four months, a

more or less continuous aching in herjoints. This repeats, for her. an experience of

intermittent aches and pains over the last two years. At the beginning of that period,

about two years ago, evaluation revealed an ANA of 1:180. However, currently, ANA is

1:140; ESR, BUN, creatinine, and urinalysis were unremarkable, as was x-ray of her

joints. She reports feeling chronically fatigued and quite weak. However, no muscle

weakness was noted on examination.



This patient has

--------- high probability of medical disorder

--------- low probability of medical disorder


Please check one.






62



Rheumatic Disease (problem 4)

Low Cue



A 41-year-old female comes to the doctor to "see about her arthritis." She has never

seen a physician before concerning this problem. She has gained 25 pounds over the

last year. and also has stomach pain. Review of systems other than skeletal and

abdominal is normal.



This patient has

--------- high probability of medical disorder

--------- low probability of medical disorder



Please check one.











Rheumatic Disease (Problem 5)

Low Cue



The patient Is a 38-year-old woman who was well until approximately three months

ago, when she began to have diffuse myalgias which would wax and wane, fatlgability

and headache In the occipital region. Differential. ESR. electrolytes. BUN, creatinine

and urinalysis were all within normal limits. Review of systems was otherwise within

normal limits except that she complains of amennorhea of five months duration.



This patient has

--------- high probability of medical disorder

--------- low probability of medical disorder


Please check one.






64



Gastrointestinal Disease (Problem 6)

Low Cue



A 29-year-old woman has a lifelong complaint of feeling of gas and abdominal

distension, worse on some days than others. It is not painful, and there is no diarrhea,

constipation, nausea, vomiting, or weight loss. Appetite is good. Exam Is remarkable

only for mild obesity (CBC and panel 20 are normal).



This patient has

--------- high probability of medical disorder

.....---- low probability of medical disorder



Please check one.











Gastrointestinal Disease (Problem 7)

High Cue



A 30-year-old woman has had 2 to 3 months of cramps after meals and watery

stools several times a day. She feels weak and has lost weight but does not have nausea

or vomiting. Two weeks ago she developed a rash on her arms and legs. Exam Is

remarkable for a thin female with an abdominal exam demonstrating mild diffuse

tenderness.



This patient has

--------- high probability of medical disorder

--------- low probability of medical disorder


Please check one.











Gastrointestinal Disease (Problem 8)

High Cue



A 41-year-old woman has had epigastric pain for 3 weeks, aching and burning in

quality, coming 1 to 2 hours after meals and sometimes at night. Pain is sometimes

relieved by eating. Pain varies in intensity and is sometimes quite severe.

Examination reveals a blood pressure of 135/90, and mild epigastric tenderness. CBC

and panel 20 are normal. Antacids in full dose are found to relieve the pain.



This patient has

-------- high probability of medical disorder

-------- low probability of medical disorder


Please check one.






67



Gastrointestinal Disease (Problem 9)

Low Cue



A 36-year-old woman complains of "too much gas" and is very concerned about food

allergy. She was told by her mother she was very allergic as a child. Examination of

the abdomen reveals mild obesity.



This patient has

-------- high probability of medical disorder

-------- low probability of medical disorder



Please check one.











Gastrointestinal Disease (Problem 10)

Low Cue



A 33-year-old woman has had bloating and "gas" for many years. Over the last 3 to 5

months she has suffered from constipation and cramps after meals. She also suffers

from epigastric burning at night and burping. Antacids at full dose provide little or no

relief. Complete physical examination Including abdominal and rectal exam Is

normal.



This patient has

----.---- high probability of medical disorder

-------- low probability of medical disorder


Please check one.






69



Name ------------------------------------------Age____

Race____- Sex ____ National Origin ____

Medical School _____ _ _ _ -

Where did you take residency training?



Specialty ________________------

# Years Residency Training __

Date your residency training ended______

Father's occupation (if retired list former occupation):

Mother's occupation (if retired list former occupation):

Are you Board Certified? Yes ___ No ___

Comments

















APPENDIX B
U.S. BUREAU OF THE CENSUS
MAJOR OCCUPATIONAL GROUPS













Major Occupation Groups, United States Bureau of the Census. 1982-83



White Collar:

professional

administrative

technical

clerical

general administrative, clerical, office services

postal

engineering and architecture

accounting

medical, hospital, dental and public health

business and industry related

legal and kindred

social science, psychology, welfare

Blue Collar:

electronic equipment installation and maintenance

electrical installation and maintenance

instrument work

machine tool work

general maintenance, cleaning services and support

metal processing

metal work

painting and paper hanging

plumbing and pipefitting

printing

woodwork


















APPENDIX C
RESIDENT PROBLEM BOOK











INSTRUCTIONS

Recent research in cognitive aspects of decision making suggests that

characteristics of the problem being evaluated influence the accuracy of decision far

more than variability in subjects' problem solving skills (Elstein, Shulman and

Sprafka, 1978). In other words, people seem to make the same mistakes no matter how

well trained they are when problems have certain characteristics. Our research group

has begun exploration of ways in which this influences decision making in medicine.

To help us explore this problem, we would appreciate your assistance in the

following task. Attached is a set of ten problems. Based on the varying amounts of

information in each problem, please estimate the chances that the patient described

has a medical disorder. (State your estimate in percentages.) In no case will you have

enough information to know with certainty. Just do the best you can with the little

information you have. The task will probably take you less than 15 minutes. Since our

information is based on a particular, limited literature, you will note that the first

paragraph of each problem is identical. Thereafter, we have varied available

information. You will be doing only a few of all the problems we are currently

investigating.

An information page is attached to gather some specific factual data about you as a

subject. As soon as this information is returned, you will be assigned a code number.

Thereafter, your data will be kept entirely confidential. Of course, your assistance to us

with this task has no bearing on your residency training. We hope it will assist us to

improve our teaching and the teaching skills of family practice faculty at large.

Thank you for your time.

Please sign the check receipt below.



I have received a check in the amount of $10.00. attached to the last page of this booklet.










PROBLEM 1
Rheumatic Disease
High SES
Low Cue


Pope (1979), in an analysis of the use of services in American medical clinics,

reported that of all patients appearing at medical outpatient clinics for evaluation, 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

The patient is a 38-year-old woman who is brought in by her husband, an

advertising account manager. She reports she was well until approximately three

months ago. when she began to have diffuse myalgias which would wax and wane,

fatigability and headache in the occipital region. She stated she often feels short of

breath. Pulmonary function tests were within normal limits, as were differential. ESR,

electrolytes. BUN, creatinine and urinalysis. Review of systems was otherwise within

normal limits except that she complains of amennorhea of five moths duration

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 20% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 20% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE .801 .










PROBLEM 2
Rheumatic Disease
High SES
High Cue


Pope (1979), in an analysis of the use of services in American medical clinics,

reported that of all patients appearing at medical outpatient clinics for evaluation, 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

A 44-year-old woman, the wife of a local hospital administrator, is complaining of

six weeks of bilateral knee swelling with pain. Her joints are stiff in the morning for

about one hour. She has had pain in her wrists for about two years.

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 65% of patients reporting

symptoms similar to those described above will be determined to have a physical

Illness. 18% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE L.94)










PROBLEM 3
Rheumatic Disease
Low SES
High Cue



Pope (1979). in an analysis of the use of services in American medical clinics,

reported that of all patients appearing at medical outpatient clinics for evaluation. 80%

have a medical disorder and 20% have some primary mental disorder In which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

The patient is a 42-year-old woman who has had, for the last three or four months, a

more or less continuous aching in her Joints. This repeats, for her, an experience of

intermittent aches and pains over the last two years. At the beginning of that period.

about two years ago, evaluation revealed an ANA of 1:180. However, currently. ANA is

1:140; ESR. BUN, creatinine. and urinalysis were unremarkable, as was x-ray of her

joints. She is accompanied by her husband, a food service worker in the hospital itself.

He reports she has described feeling chronically fatigued and quite weak. However, no

muscle weakness was noted on examination.

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 60% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 15% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE (.94)










PROBLEM 4
Rheumatic Disease
Low SES
Low Cue



Pope (1979). in an analysis of the use of services in American medical clinics.

reported that of all patients appearing at medical outpatient clinics for evaluation, 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

A 41-year-old female comes to the Doctor to "see about her arthritis," brought in by

her husband, who works as a course painter throughout most of the South Bay area. She

has never seen a physician before concerning this problem. She has gained 25 pounds

over the last year, and also has stomach pain. Review of systems other than skeletal

and abdominal is normal.

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 30% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 20% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE (.86)






78




PROBLEM 5
Rheumatic Disease
Low Cue Only



Pope 11979). in an analysis of the use of services in American medical clinics.

reported that of all patients appearing at medical outpatient clinics for evaluation, 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

The patient is a 38-year-old woman who was well until approximately three months

ago, when she began to have diffuse myalgias which would wax and wane. fatigability

and headache in the occipital region. Differential, ESR, electrolytes, BUN. creatinine

and urinalysis were all within normal limits. Review of systems was otherwise within

normal limits except that she complains of amennorhea of five months duration.

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 28% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 22% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE 1.851











PROBLEM 6
Gastrointestinal Disease
High SES
Low Cue



Pope (1979), in an analysis of the use of services in American medical clinics,

reported that of all patients appearing at medical outpatient clinics for evaluation. 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

A 29-year-old woman has a lifelong complaint of feeling of gas and abdominal

distension, worse on some days than others. It is not painful, and there is no diarrhea.

constipation, nausea, vomiting, or weight loss, she worries about it sufficiently that her

husband, a tax accountant, suggested she come to see her doctor. Her appetite is good.

Exam is remarkable only for mild obesity (CBC and panel 20 are normal).

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 31% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 22% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE (.86L










PROBLEM 7
Gastrointestinal Disease
High SES
High Cue



Pope (19791, in an analysis of the use of services in American medical clinics.

reported that of all patients appearing at medical outpatient clinics for evaluation, 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

The 30-year-old wife of a local public defender has had 2 to 3 months of cramps after

meals and watery stools several times a day. She feels weak and has lost weight but

does not have nausea or vomiting. Two weeks ago she developed a rash on her arms and

legs. Exam is remarkable for a thin female with an abdominal exam demonstrating

mild diffuse tenderness.

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 61% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 17% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE (.941






81



PROBLEM 8
Gastrointestinal Disease
Low SES
High Cue



Pope (1979), in an analysis of the use of services in American medical clinics,

reported that of all patients appearing at medical outpatient clinics for evaluation. 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

A 41-year-old woman is brought in by her husband, a machinist, complaining of

epigastric pain for 3 weeks, aching and burning in quality, coming 1 to 2 hours after

meals and sometimes at night. Pain is sometimes relieved by eating. Pain varies in

intensity and is sometimes quite severe. Examination reveals a blood pressure of

135/90, and mild epigastric tenderness. CBC and panel 20 are normal. Antacids in full

dose are found to relieve the pain.

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 65% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 19% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE 1.931










PROBLEM 9
Gastrointestinal Disease
Low SES
Low Cue



Pope (1979), in an analysis of the use of services in American medical clinics.

reported that of all patients appearing at medical outpatient clinics for evaluation, 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

A 36-year-old woman complains of "too much gas" and is very concerned about food

allergy. She was told by her mother she was very allergic as a child. Examination of

the abdomen reveals mild obesity. The patient is a housewife and married to self-

employed plumber.

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 28% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 21% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate In percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE (.85)











PROBLEM 10
Gastrointestinal Disease
Low Cue Only



Pope (1979), in an analysis of the use of services in American medical clinics.

reported that of all patients appearing at medical outpatient clinics for evaluation, 80%

have a medical disorder and 20% have some primary mental disorder in which

psychological symptoms are presented in somatic form. The patient described below

was recently evaluated at Harbor-UCLA Medical Center.

A 33-year-old woman has had bloating and "gas" for many years. Over the last 3 to 5

months she has suffered from constipation and cramps after meals. She also suffers

from epigastric burning at night and burping. Antacids at full dose provide little or no

relief. Complete physical examination including abdominal and rectal exam is

normal.

A recent article in the Annals of Internal Medicine reported a study of the

frequency with which certain symptom clusters are found in medical disease states and

in somatoform disorders. Data reported suggested that 30% of patients reporting

symptoms similar to those described above will be determined to have a physical

illness. 22% of patients whose distress is primarily psychological will report such

symptoms.

What do you think the chances are that this patient actually has a medical

disorder? Please state your estimate in percentages.

THE CHANCES THAT THE PATIENT ABOVE HAS A MEDICAL DISORDER

ARE (.861






84



N am e _ _ _ _ _ _ _ A ge_ _ _ _

Race Sex ____ National Origin --- -

Medical School

Residency training Program ______----------......

Residency Training Year (circle one) 1 2 3

Father's occupation (if retired list former occupation):

Mother's occupation (if retired list former occupation):

Comments
















REFERENCES


Abramowltz, S. I., Abramowitz, C. V.. Jackson, C., and Gomes. B. (1973). The politics of
clinical judgment: What nonliberal therapists infer about women who do not stifle
themselves. Journal of Consulting and Clinical Psychologv, 41, 385-391.

Abramowitz. C. V.. and Dokecki, P. R. (1977). The politics of clinical judgment: Early
empirical returns. Psychological Bulletin, 84, 460-476.

Asch, S. E. (1946). Forming impressions of personality. Journal of Abnormal and
Social Psychology. 41, 258-290.

Bar-Hillel, M. (1980a). The base-rate fallacy in probability Judgment. Acta
Psvchologica. 44, 211-233.

Bar-Hillel, M. (1980b). What features make samples seem representative? Journal of
Experimental Psychology: Human Perception and Performance. 6. 578-589.

Barrows, H. S., and Abrahamson, S. (1964). The programmed patient: A technique for
appraising student performance in clinical neurology. Journal of Medical
Education. 39, 802-805.

Barrows, H. S., and Bennett, K. (1972). The diagnostic (problem solving) skill of the
neurologist: Experimental studies and their implications for neurological training.
Archives of Neurology, 26. 273-277.

Bartlett, F. D. (1958). Thinking New York: Basic Books.

Benefee. L. M., Abramowitz, S. I., Weitz. L. J.. and Armstrong, S. H. (1976). Effects of
patient racial attribution on black clinicians' inferences. American Journal of
Community Psychology. 4. 268-273.

Briar, S. (1961). Use of theory in studying effects of client social class on students'
Judgment. Social Work 6, 91-97.

Broverman, I. K., Broverman. D. M., Clarkson, F. E., Rosenkrantz. P. S., and Vogel. S.
(1970). Sex-role stereotypes and clinical Judgments of mental health. Journal of
Clinical and Consulting Psychology, 34. 1-7.

Cowen, E. L. (1961). The social desirability of trait descriptive terms: Preliminary
norms and sex differences. Journal of Social Psychology 53 225-233.

deGroot, A. C. (1965). Thought and choice in chess. The Hague: Mouton.










Del Gaudio. A. C., Stein, L. S.. Ansley, M. Y., and Carpenter, P. J. (1976). Attitudes of
therapists varying in community mental health Ideology and democratic values.
Journal of Consulting and Clinical Psychology. 44. 646-655.

Dohrenwend. B. P.. and Dohrenwend, B. S. (1969). Social status and psychological
disorder. New York: Wiley.

Einhornm, H. J., and Hogarth. RM. (1981). Behavioral decision theory: Processes of
judgment and choice. Annual Review of Psychology, 32, 53-88.

Elstein, A S. (1976). ClinicalJudgment: Psychological research and medical practice.
Science, 194, 696-700.

Elstein, A. S. (1982). Decision making as educational subject matter. Medical Decision
Making, 2, 1-5.

Elstein, A. S.. and Bordage, G. (1979). Psychology of clinical reasoning. In G. C. Stone,
F. Cohen, and N. E. Adler (Eds.). Health psychology: A handbook. (pp. 333-367). San
Francisco: Jossey-Bass.

Elstein, A. S.. Kagen. N., Shulman, L. S., Jason, H., and Loupe, M. J. (1972). Methods
and theory in the study of medical inquiry. Journal of Medical Education. 47, 85-
92.

Elstein, A. S.. Shulman, L. S., and Sprafka, S. A. (1978). Medical problem solving: An
analysis of clinical reasoning. Cambridge: Harvard University Press.

Feinstein, A. R. (1967). Clinical Judgment. Baltimore: Williams and Wilkins.

Fielding. G.. and Evered, C. (1978). An exploratory experimental study of the influence
of patients' social background upon diagnostic process and outcome. Psychiatria
Clinica, 11, 61-86.

Fischhoff, B., Slovic. P., and Lichtenstein, S. (1978). Fault trees: Sensitivity of
estimated failure probabilities to problem representation. Journal of Experimental
Psychology: Human Perception and Performance, 4, 330-344.

Freedman, A. M.. Kaplan, H. I., and Sadock, B. J. (1975). Comprehensive textbook of
psychiatry (second edition). Baltimore: Williams and Wilkins.

Freedman, L. Z., and Hollingshead, A. D. (1957). Neurosis and social class. American
Journal of Psychiatry, 113, 769-775.

FrybackD. G. (1974). Use of radiologists' subjective estimates in a medical decision
making problem. Ann Arbor: University of Michigan.

Gill, B. W., Leaper. D. J., Guillou, P. J.. Stanfland, J.R.. Horrocks, J.C.. and deDombal,
F. T. (1973). Observer variation In clinical diagnosis--A computer-aided assessment
of its magnitude and importance in 552 patients with abdominal pain. Methods of
Information in Medicine, 12. 108-113.










Gustafson. D. H., Kestley, J. J.. Geist, J. H., and Jansen. N. M. (1971). Initial evaluation
of a subjective Bayesian diagnostic system. Health Service Research, 6f, 204-213.

Hammerton, M. (1973). A case of radical probability estimation. Journal of
Experimental Psvchologv, 101. 252-254.

Harrison, T. R (1980). Principles of internal medicine. New York: Blakiston.

Hollingshead, A. B., and Redllch, F. C. (1958). Social class and mental illness. New
York: Wiley.

Howell, M. C. (1974). What medical schools teach about women. New England Journal
of Medicine, 291, 304-310.

Kahneman, D., and Tversky. A. C. (1972). Subjective probability: Ajudgment of
representativeness. Cognitive Psvchologv. 3, 430-454.

Kahneman.. D.. and Tversky, A. C. (1973). On the psychology of prediction.
Psychological Review, 80, 237-251.

Koran, L. M. (1975). The reliability of clinical methods, data and Judgment. New
England Journal of Medicine. 293, 642-646 and 695-701.

Kurtz, N.. Kurtz, R., and Hoffnung. R. (1970). Attitudes toward lower- and middle-class
psychiatric patients as a function of authoritarianism among mental health
students. Journal of Consulting and Clinical Psvchologyv, 35. 338-341.

Leaper, D. J., Horrocks, J. C., Staniland, J. R., and deDombal, F. T. (1972). Computer-
assisted diagnosis of abdominal pain using "estimates" provided by clinicians.
British Medical Journal. 4. 350-354.

Lee. S. D., and Temerlin. M. K. (1970). Social class, diagnosis and prognosis for
psychotherapy. Psychotherapy: Theory. Research and Practice. 7. 181-185.

Levy, M., and Kahn. M. (1970). Interpreter bias on the Rorschach test as a function of
patients' socioeconomic status. Journal of Projectlve Techniaues and Personality
Assessment 34, 106-112.

Lusted, L. B. (1968). Introduction to medical decision making. Springfield. IL:
Thomas.

Lyon, D.. and Slovic. P. (1976). Dominance of accuracy information and neglect of base
rates in probability estimation. Acta Psvchologlca, 40, 287-298.

Mandler. G. A (1967). Organization and memory. In K. W. Spence and J. T. Spence
(Eds.). The psvchology of learning and motivation (Vol. 1. pp. 328-372). New York:
Academic Press.

McCranie, E. W., Horowitz. A J., and Martin, R. M. (1978). Alleged sex-role
stereotyping in the assessment of women's physical complaints: A study of general
practitioners. Social Science and Medicine, 12. 111-116.










McNeil,. B. J., Keeler. E.. and Adelstein, S. J. (1975). Primer on certain elements of
medical decision making. New England Journal of Medicine. 293. 211-215.

Nalven, F. B., Hofmnann.rm, L. J., and Blerbryer, B. (1969). Effects of subjects' age, sex, race
and socioeconomic status on psychologists' estimates of "true IQ" from WISC scores.
Journal of Clinical Psychology, 25. 271-274.

Newell, A.. and Simon, H. A. (1972). Human problem solving. Englewood Cliffs, New
Jersey, Prentice-Hall.

Orne, M. T. (1969). Demand characteristics and the concept of quasi controls. In R
Rosenthal, and R. L. Rosnow (Eds.). Artifact in behavioral research (pp. 143-179).
New York: Academic Press.

Pauker. S. G. (1976). Coronary artery surgery: the use of decision analysis. Annals of
Internal Medicine. 85. 8-18.

Pauker, S. G. (1982). Prescriptive approaches to medical decision making. American
Behavioral Scientist. 25, 507-522.

Persely, G., Johnson, J. H., and Hornsby, L. G. (1975). Effects of profession, sex and
prognostic expectancies on therapists' comments in a psychotherapeutic analogue.
Psychological Reports. 37, 445-459.

Peterson. C. R., and Beach, L. R. (1967). Man as an intuitive statistician. Psychological
Bulletin, 68. 29-46.

Phillips, L. D. (1973). Bayesian statistics for social scientists. New York: Crowell.

Pope. C. R. (1979). Illness with a high emotional component and the use of medical
services. Medical Care, 17. 1182-1195.

Prather, J., and Fidell, L. S. (1975). Sex differences in the content and style of medical
advertisements. Social Science and Medicine. 9, 23-30.

Raynes, N. V. (1979). Factors affecting the prescribing of psychotropic drugs in general
practice. Psychological Medicine. 9,. 671-679.

Routh, D. K., and King, K. M. (1972). Social class bias in clinical judgment. Journal of
Consulting and Clinical Psvchology. 38, 202-207.

Safilios-Rothschild, C. (1974). Women and social policy. Englewood Cliffs, New
Jersey: Prentice-Hall.

Schiffman. D. A., Cohen. S., Nowik, R., and Selminger, D. (1978). Initial diagnostic
hypotheses: Factors which may distort physicians' judgment. Organizational
Behavior and Human Performance. 21, 305-315.

Schwartz. S., and Griffin, T. (1986). Medical thinking: The psychology of medical
judgment and decision making. New York: Springer-Verlag.

Schwartz, W. A., Gorry. G. A., Kassirer, J. P., and Essig, A. (1973). Decision analysis
and clinical judgment. American Journal of Medicine. 55,. 459-472.











Shulman, L. S.,and Elstein, A. S. (1975). Studies of problem solving, Judgment and
decision making: Implications for educational research. In F. N. Kerlinger (Ed.),
Review of research in education (pp. 114-136). Itasca, IL: Peacock.

Sisson, J., Schoomaker. E., and Ross, J. (1976). Clinical decision analysis -- The
hazard of using additional data. Journal of the American Medical Association, 236
1259-1263.

Slovic. P.. Fischhoff, B., and Lichtenstein, S. (1977). Behavioral decision theory.
Annual Review of Psychology. 28. 1-39.

Slovic, P., and Lichtenstein. S. (1971). Comparison of Bayesian and regression
approaches to the study of information processing judgment. Organti7ational
Behavior and Human Performance, 6, 649-744.

Smith, M. L. (1974). Influence of client sex and ethnic group on counselor judgment.
Journal of Counseling Psvchologv 21, 516-521.

Srole, L., Langner, T. S., Michael, S. T.. Opler, M. K., and Rennie, T. A. C. (1962). Mental
health in the metropolis. Thomas A. C. Rennie Series in Social Psychiatry, Vol. 1.
New York: McGraw-Hill.

Thomas, L. (1979). The medusa and the snail. New York: Viking Press.

Tversky, A.. and Kahneman, D. (1971). Belief in the law of small numbers.
Psychological Bulletin. 76. 105-110.

Tversky. A.. and Kahneman. D. (1973). Availability: A heuristic for Judging frequency
and probability. Cognitive Psvchologv. 5. 207-232.

Tversky, A. .and Kahneman, D. (1974). Judgment under uncertainty: Heuristics and
biases. Science. 185, 1124-1131.

Tversky. A. and Kahneman, D. (1980). Causal schemas in Judgments under
uncertainty. In M. Fishbein (Ed.), Progress in social psychology (Vol. 1, pp. 49-72).
Hilsdale, N. J.: Erlbaum.

U. S. Department of Commerce, Bureau of the Census. (1982). Statistical Abstracts of
the United States (103rd ed.), 1982-83. Washington, DC: U. S. Government Printing
Office.

U. S. Department of Health and Human Services. (1981). Current estimates from the
national health interview survey: United States. 1979. Hyattsville. MD: Author.

Wallsten, T. S. (1978). Three biases in the cognitive processing of diagnostic
information. Unpublished paper, Psychometric Laboratory, University of North
Carolina, Chapel Hill.

Weinstein. M. C., Fineberg. H. V., Elstein, A. S., Frazier, H. S., Neuhauser. D., Neutra. R.
R., and McNeil, B. J. (1980). Clinical decision analysis. Philadelphia: W. B.
Saunders.






90



Wortman, P. M. (1970). Cognitive utilization of probabilistic cues. Behavioral Science.
15, 329-336.

Zborowski, M. (1952). Cultural components in response to pain. Journal of Social
Issues, B. 16-30.












BIOGRAPHICAL SKETCH


Eugenie Wilson received her Bachelor of Arts degree, magna cum laude, from

California State University, Fresno. She received a Master of Science degree from the

University of Florida. She currently resides in Long Beach, California, where she

pursues her interests in mystery, music and how things work.











I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Doctor of Philosophy.



oger K. fl
Professor of Clinical and
Health Psychology

I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate. In scope and
quality, as a dissertation for the degree of Doctor of Philosophy.


(7 s, I
Richard S. Panush
Professor of Immunology and
Medical Microbiology

I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Doctor of Philosophy.


-_^^_-Aa&L& __
Eileen B. Fennell
Professor of Clinical and
Health Psychology

I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Doctor of Philosophy.



Hugh C. D-I"s
Professor of Clinical and
Health Psychology











I certify that I have read this study and that in my opinion it conforms to
acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Doctor of Philosophy.




Wallace L. Mealica, Jr.
Professor of Clinical and
Health Psychology

This dissertation was submitted to the Graduate Faculty of the College of Health Related
Professions and to the Graduate School and was accepted as partial fulfillment of the
requirements for the degree of Doctor of Philosophy.


December, 1987 l? .
Dean, College of Health
Related Professions



Dean, Graduate School









































































3 1262 08554 8096