Title: Comparing village characteristics derived from rapid appraisals and household surveys : a tale from Northern Mali
Full Citation
Permanent Link: http://ufdc.ufl.edu/UF00085473/00002
 Material Information
Title: Comparing village characteristics derived from rapid appraisals and household surveys : a tale from Northern Mali
Physical Description: Book
Creator: Christiaensen, Luc
Publisher: Centro de Investigacion para la Accion Femenina
 Record Information
Bibliographic ID: UF00085473
Volume ID: VID00002
Source Institution: University of Florida
Rights Management: All rights reserved by the source institution and holding location.

Full Text


Discussion Paper 91


Luc Christiaensen, John Hoddinott, and Gilles Bergeron

Many empirical studies of household behavior in
developing countries rely on probability sample
surveys. Provided the sampling is random, such
surveys reduce costs of gathering data while allowing valid
inferences of the characteristics of the underlying population
to be made. A prerequisite for the drawing of a random
sample is a sampling frame, a list of the units in the
population from which the units that will be enumerated are
selected. In practice, this is often an actual list, a set of index
cards, a map, or data stored in a computer.
But unlike sampling issues such as the choice of sample
size or the mechanism for randomly selecting units, the
construction of the sample frame rarely receives much
attention. This is unfortunate. For example, a sample frame
that excludes the poorest households in a locality will lead to
biased inferences regarding the incidence and severity of
poverty in that community, irrespective of the quality of the
data collection or the sophistication of the subsequent
statistical analysis.
The starting point for constructing a sampling frame is
often an administrative list. These extant lists are regularly
flawed. They may include units that do not belong to the
population of interest (overcoverage), exclude a unit that
does belong to the population of interest (undercoverage), or
list the same unit several times. Although careful
crosschecking can rectify these flaws, doing so is not a trivial
exercise. Moreover, the need to validate these lists increases
the costs of undertaking iiousehuid surveys.
Even if the sample frame is carefully constructed, there
exists a view that the information collected subsequently will
be unreliable. Motivated by this and other concerns, the last
several years have seen the development of new methods for
obtaining information on the
socioeconomic characteristics of "In an environment
communities. One such ap-
proach falls under the very that most outsi
broad rubric of participatory financial resource
rural appraisal (PRA). PRA is data may be as mi
"a family of approaches and interactions as th
methods to enable rural people
to share, enhance, and analyze
their knowledge of life and conditions, to plan and to act."
The PRA approach is predicated on the notion that local
people have a wealth of knowledge that they can articulate.
Furthermore, the claim is made that a particular PRA
method-participatory village mapping-can be used to
obtain data on demographic characteristics and measures of

t whe
fers ar
s that
uch th
ey are

well-being more accurately than standardized household
surveys and at a fraction of the cost.

Purpose of the Study
This paper speaks to these issues. It investigates whether
inferences drawn about a population are sensitive to the
manner by which those data are obtained. Specifically, we
started with a common sample unit (the household) and a
common universe (five villages in northern Mali). In these
villages, we conducted two types of surveys. One was a
household survey based on the random selection of
respondents from a locally constructed administrative list
that had been carefully checked. The second was the
outcome of participatory activities-the construction of a
detailed village map-in these communities. We sought to
determine whether these two methods yielded comparable
characterizations of these villages.

Methodology and Results
We began by considering coverage error. We examined how
a sample frame, based on official census lists and revised in
discussion with local people, compared with one derived
from a participatory mapping approach. We found that the
revised official census suffered from a slightly higher level
of undercoverage than the participatory map. However, the
mapping exercise tended to lead to larger errors of
overcoverage. We then investigated if these errors led to
different conclusions with respect to certain characteristics of
the underlying population. We controlled for the survey
instrument used and found that households sampled from the
revised official census appear, on average, to be larger and
wealthier. If we characterized the villages in terms of total
size or total wealth, we
re everyone is aware obtained larger estimates from
the participatory village
'e associated with mapping because of the
are to be disbursed, overcoverage associated with
e outcome of social this technique. Finally, we
immutable 'facts.," examined if the characteri-
zation of these villages was
sensitive to the survey
technique used. In particular, we compared results obtained
from the same households, but drawn from different survey
instruments. We found that the participatory village map-
ping by which information on the households is obtained in
a public setting, produced higher estimates of household size
and lower estimates of household wealth than the

Discussion Paper BRIEFS

Food Consumption and Nutrition division of the International Food Policy Research Institute

household survey, in which households are surveyed in

Discussion and Conclusions
We must be careful not to over generalize from these
findings. They pertain to a particular region. Whether they
are replicable elsewhere is an empirical question. Second,
even if outsiders and participants agree on concepts and
definitions, there can still remain differences in interpretation
and application. Although we adopted local definitions of
households, there will always be borderline cases. In the
context of household surveys, our enumerators had criteria
by which they could adjudicate, for example, the definition
of a "migrant." By contrast, in the participatory mapping
exercise, respondents undertook this adjudication. This is not
to say that one set of criteria was more valid, but rather that
the same criteria can be used in different ways by different
actors. Similar considerations can be attributed to the notion
of a village.
Mindful of these caveats, these results can be read in a
number of ways. They can be used to support the claim that
participatory mapping is more accurate than a sample of
households randomly drawn from an ROC because it is less
likely to exclude poorer, smaller households. Conversely,
participatory mapping could be regarded as less accurate due
to the overcoverage we observe and the apparent
underreporting of assets.
Our interpretation is somewhat different. We surmise
that these results are principally driven by the particular
dynamics of these different activities. Despite our best efforts
to remain "invisible" during the participatory mapping
exercise, we suspect that even our minimal presence was
sufficient to induce households to alter their responses. In an
environment where everyone is aware that most outsiders are
associated with financial resources that are to be disbursed,
data may be as much the outcome of social interactions as
they are immutable "facts." Thus, for example, the "number
of people resident in a household" is not just a figure to be
measured, but also possibly part of a negotiation with a

respondent, who perceives that financial gain may come
from proposing a higher figure than is actually the case. A
different set of social interactions affected our household
survey. Here, there were repeated measurements of these
data conducted in a private, rather than public gathering, and
often interviewing was supplemented with direct observation
and triangulation with other information in the questionnaire.
If our supposition is correct-that different survey
techniques generate different social dynamics between
research teams and their respondents, then it is incorrect to
claim the "superiority" of one method over another. Instead,
it is important to carefully examine and acknowledge the
biases that may result from the particular method being used.
It also points to the importance of triangulating, or cross-
checking, information that is obtained. We further stress that
our use of both techniques was not driven so much by a
desire to determine the "right method," but rather by our
desire to enrich our understanding of these villages. The
participatory appraisal techniques allowed us to interact with
certain groups, such as women, in a way that was simply
infeasible when visiting individual households. They also
allowed us to observe the dynamics of these villages literally
"at work," and led to a more nuanced understanding of
dynamics within these villages (such as relations between
different ethnic groups) as well as their relationships with
outsiders such as ourselves. Our quantitative surveys enabled
us to complement these understandings with a more detailed,
in-depth look at a wide variety of measures of deprivation.

Keywords: survey methodology, household surveys,
participatory rural appraisal, Mali

Recent FCND Discussion Papers
Empirical Measurements of Household Access to Credit
and Credit Constrints in Developing Countries:
Methodological Issues and Evidence, Aliou Diagne,
Manfred Zeller, and Manohar Sharma, July 2000 DP90

The full text of this document and other FCND Discussion Papers are available on our Website
(www.cgiar.org/ifpri/divs/fcnd/dp.htm) or via B.McClafferty@cgiar.org


2033 K Street, N.W.
Washington, D.C. 20006 U.S.A.

S/( oe-no-o

U. ver- ity of Florida
P.O. Box 110240
Gainesville, FL 32611


I 0.

| I *K METER *

"In an environment where everyone is aware that most
outsiders are associated with financial resources that are to be
disbursed, data may be as much the outcome of social
interactions as they are immutable 'facts.'" DP91

i324 %% 4l0a


University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs