ANOTHER EXPERIMENT IN SAMPLE
JOHN J. HONTGMANN AND RICHAD CARRERA
SOUTHWESTERN JOURNAL OF ANTHROPOLOGY
VOLUME 13 .* NUMBER 1 SPRING
UNIVERSITY OF NEW MEXCICO
e 'Q V..
.. ,. -..-~ -...
., ;,. -
,, ; :
.. '.. .
l- ,. : : .,,. : .. : .
a: : .''i" '.. '
,* ..~1 .- .
ANOTHER EXPERIMENT IN SAMPLE RELIABILITY
JOHN J. HONIGMANN AND RICHARD CARRERA
IT IS A COMMONPLACE that ethnological field work constitutes a scien-
tific undertaking but this can hardly be maintained if the field work process
cannot explicitly be conceptualized and rules formulated concerning how informa-
tion is actually acquired under specific conditions. Anthropology is still far from
these goals. Specifically it would seem desirable to specify (1) how field workers
acquire data in actual field work situations (through analysis of the transactions
occurring between ethnologists and community or informant); (2) the reliability
of informants chosen by different procedures (sampling reliability), and (3) how
anthropologists work up their data in order to develop behavior patterns and con-
ceptions of the overall cultural configuration.
During field work continued in 1955 among the Cree Indians of Attawapiskat,
Ontario, another attempt was made to discover whether information obtained in
response to a controlled stimulus from a sample selected by random methods would
differ significantly from that given to the same stimulus by persons chosen simply
on the bases of convenience and opportunity.1 The hypothesis to be tested was that
the opportunistic sample would give responses significantly different from the per-
sons randomly selected. Such a difference we reasoned would be reflected in other
kinds of information that would be forthcoming if members of these categories
served as ethnographic informants. The stimulus chosen was one that could favor
nobody simply on the basis of age, experience, or degree of acculturation. It con-
sisted of the second card in the Behn Rorschach series, a test normally used to
avoid contaminating subjects who have not yet taken the standard Rorschach
Test.2 The Behn series was chosen for the experiment because many of the indi-
viduals in our samples would also receive, or had already taken, the Rorschach
test. Under such circumstances it was felt that using a Rorschach rather than a
Behn card might influence subjects' performance in the experimental situation
or in the Rorschach test situation when it occurred. The respondents were all
equally unfamiliar with the Behn series. If any particular reason existed for choos-
1 For a similar previous experiment see John and Irma Honigmann, Sampling Reliability in
Ethnological Field Work (Southwestern Journal of Anthropology, vol. 11, pp. 282-287, 1955).
For field work funds grateful acknowledgment is due to the Wenner-Gren Foundation for An-
thropological Research and the Institute for Research in Social Science. The latter also supported
the research project described in the present paper.
2 Hans Zulliger, Einfiihrung in den Behn-Rorschach-Test. Behn-Rorschach-versuch. Tafeln
(Bern: H. Huber, 1939).
VOL. 13, 1957
100 SOUTHWESTERN JOURNAL OF ANTHROPOLOGY
ing the second rather than another card in the Behn set it lay in the rather clearly
structured nature of that card compared to the others. Limitations of time de-
termined the use of only one card rather than employment of the full set. Twenty
adult subjects selected at random included 12 women. The opportunistic sample
contained 23 persons, including 16 women.
ANALYSIS OF DATA
Responses were recorded and each scored for location, determinant, and con-
tent. The scoring of movement went to any movement response, whether it involved
human, animal, or inanimate movement. Color was tabulated only with regard
to its presence in the response and dominance was not considered. In scoring con-
tent, humans and human details were considered together as were animals and
animal details. This procedure seemed justified by the small number of subjects.
The two Attawapiskat samples were then compared on the basis of eight scor-
ing dimensions: animal content, human content, response total, responses involv-
ing color, movement responses, incidence of Detail One,3 incident of Detail Two,
and number of rejections. A mean frequency (x) was computed for each of these
categories. The results are presented in Table 1.
A statistical test of significance using Wilcoxen's Rank Method (Non-Para-
metric) was employed. By this technique the differences between the means of
the random and opportunistic categories turned out to be non-significant. It may
be seen from an inspection of the means (Table 1) that the two samples follow
Comparison of Samples in Terms of Categories of Responses
Attawapiskat x Attawapiskat x Undergraduate x
Random Opportunistic Sample
Category Sample Sample (N = 96)
(N = 20) (N = 23)
Animal content 27 1.35 26 1.13 117 1.22
Human content 3 .15 7 .30 86 .83
Total R 43 2.15 41 1.78 287 2.99
Incidence of color 3 .15 4 .17 33 .34
Incidence of D2 17 .85 16 .69 88 .92
Incidence of D1 11 .55 13 .56 84 .87
Incidence of rejection 2 .10 1 .04 0 0
Incidence of any re-
sponse containing M 3 .15 10 .43 76 .79
3 Detail One designates the lower red details of Behn Card No. Two; Detail Two, the
large black details ("dogs").
a very similar frequency pattern for the eight scoring categories. At this point we
are faced with two alternatives. Either the randomly and opportunistically selected
subjects did not in fact constitute two distinct populations, as had been assumed
(which is contrary to what the hypothesis predicted), or the measuring instrument
is inadequate to distinguish existing differences. The relative homogeneity of the
Attawapiskat Indians favors the first alternative. The junior author thought that
this point might be clarified by comparing the Indian samples with a population
obviously distinct from them. If non-significant differences appeared in that case
then it could be concluded that, although obvious differences were present, Card
Two of the Behn Rorschach set fails to pick them up.
In order to test the new hypothesis Card Two was given to 96 undergraduate
students at the University of North Carolina in four group administrations. The
results, compared to the two Indian samples, are included in Table 1. The Wil-
coxen Rank technique indicated the differences between the opportunistic and
undergraduate samples not to be significant at the .15 level. Since the differences
between the random subjects and undergraduates are obviously still smaller, no
statistical test was applied to them. The overall pattern of responses between the
two Indian categories on the one hand and the undergraduates on the other is again
We may safely conclude from the foregoing data that the second card of the
Behn set will not differentiate satisfactorily between two populations as disparate
as Attawapiskat Indians and North Carolina University students. It follows that
one cannot expect it to differentiate between two groups of Indians from the same
relatively homogeneous community. It should be pointed out, however, that the
Behn card did a much better job in distinguishing between students and Indians
than between the two categories of Indians. This would seem to indicate a certain
degree of discriminatory capacity and suggests the possibility of using the entire
series of Behn cards.
What began as a test of sampling reliability led to the unexpected conclusion
that care must be taken in choosing instruments of procedures for field work ex-
periments. It is not likely that the outcome of this particular experiment can be
used to question the assumption of modal personality differences between Atta-
wapiskat and North Carolina subjects. Familiarity with Rorschach work suggests
that the complete set of Rorschach cards does distinguish between Indian and
SOUTHWESTERN JOURNAL OF ANTHROPOLOGY
non-Indian subjects.4 The Behn card did not so discriminate. One reason for its
failure to discriminate may be suggested as a post hoc hypothesis to keep in mind:
Card Two of the set is highly structured and therefore does not invite free play of
projective fantasies such as might be expected to differ between two different cate-
gories of people.
UNIVERSrY OF NORTH CAROLINA
CHAPEL HILL, NORTH CAROLINA
4 We have Spindler's work in mind. He used a Euroamerican control group in his work
among the Menomini. See George Spindler, Sociocultural and Psychological Processes in
Menomini Acculturation (University of California Publications in Culture and Society, vol. 5,
1955). Analysis of Rorschach responses using statistical techniques to test the modal personality
assumption has recently been reported on by Bert Kaplan in A Study of Rorschach Responses in
Four Cultures (Papers, Peabody Museum of American Archaeology and Ethnology, Harvard
University, vol. 42, no. 2, 1954).