• TABLE OF CONTENTS
HIDE
 Front Cover
 Title Page
 Table of Contents
 List of tables and figures
 Foreword
 List of contributors
 Overview and synthesis
 Part 1: Food policy issues and...
 Part 2: Strengths and weaknesses...
 Part 3: Data quality and design...
 Improving data quality in household...
 Farm production data in rural household...
 Obtaining useful data on household...
 Collection of production and income...
 Some methodological issues in the...
 Collection of time use and labor...
 Food conusmption surveys: How random...
 On food consumption surveys: A...
 Open methodological questions related...
 On consumption and nutrition data:...
 Intrahousehold-related policy research:...
 Data on rural infrastructure and...
 Back Cover
 Reprint permisssion notice














Group Title: Data needs for food policy in developing countries : new directions for household surveys
Title: Data needs for food policy in developing countries
CITATION THUMBNAILS PAGE IMAGE ZOOMABLE
Full Citation
STANDARD VIEW MARC VIEW
Permanent Link: http://ufdc.ufl.edu/UF00085354/00001
 Material Information
Title: Data needs for food policy in developing countries new directions for household surveys
Physical Description: xvi, 268 p. : ill. ; 23 cm.
Language: English
Creator: Von Braun, Joachim, 1950-
Puetz, Detlev, 1957-
Publisher: International Food Policy Research Institute
Place of Publication: Washington D.C
Publication Date: 1993
 Subjects
Subject: Nutrition surveys -- Methodology -- Developing countries   ( lcsh )
Nutrition policy -- Developing countries   ( lcsh )
Abastecimento/Distribuicao De Alimentos (Aspectos Sociais)   ( larpcal )
Genre: bibliography   ( marcgt )
non-fiction   ( marcgt )
 Notes
Bibliography: Includes bibliographical references.
Statement of Responsibility: edited by Joachim von Braun and Detlev Puetz.
 Record Information
Bibliographic ID: UF00085354
Volume ID: VID00001
Source Institution: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
Resource Identifier: oclc - 29564093
lccn - 93048782
isbn - 0896293297

Table of Contents
    Front Cover
        Front Cover 1
        Front Cover 2
    Title Page
        Page i
        Page ii
        Page iii
        Page iv
    Table of Contents
        Page v
        Page vi
        Page vii
    List of tables and figures
        Page viii
        Page ix
        Page x
    Foreword
        Page xi
        Page xii
    List of contributors
        Page xiii
        Page xiv
        Page xv
        Page xvi
    Overview and synthesis
        Page 1
        Page 2
        Page 3
        Page 4
        Page 5
        Page 6
        Page 7
        Page 8
        Page 9
        Page 10
    Part 1: Food policy issues and new challenges for data
        Page 11
        Page 12
        Policy issues and problems of data collecton and analysis
            Page 13
            Page 14
            Page 15
            Page 16
            Page 17
        Linkages between food policymaking, policy analysis, and data collection
            Page 18
            Page 19
            Page 20
            Page 21
            Page 22
        New challenges for resource and environment data for policy research
            Page 23
            Page 24
            Page 25
            Page 26
    Part 2: Strengths and weaknesses of different survey approaches for food policy design
        Page 27
        Page 28
        Information needs for the food system: Conceptual frameworks and institutional problems
            Page 29
            Page 30
            Page 31
            Page 32
            Page 33
            Page 34
            Page 35
            Page 36
            Page 37
            Page 38
            Page 39
            Page 40
            Page 41
            Page 42
            Page 43
        Household data needs for food policy: Toward criteria for choice of approaches
            Page 44
            Page 45
            Page 46
            Page 47
            Page 48
            Page 49
            Page 50
            Page 51
            Page 52
            Page 53
            Page 54
            Page 55
            Page 56
            Page 57
            Page 58
            Page 59
            Page 60
            Page 61
            Page 62
            Page 63
            Page 64
            Page 65
            Page 66
            Page 67
            Page 68
            Page 69
            Page 70
            Page 71
            Page 72
            Page 73
            Page 74
            Page 75
            Page 76
            Page 77
            Page 78
            Page 79
        Using nationally representative household surveys for food policy analysis: An examination of the World Bank's living standards measurement study surveys
            Page 80
            Page 81
            Page 82
            Page 83
            Page 84
            Page 85
            Page 86
            Page 87
            Page 88
            Page 89
            Page 90
            Page 91
            Page 92
            Page 93
            Page 94
        Focusing small-sclae surveys on food specific policy issues: IFPRI's experiences
            Page 95
            Page 96
            Page 97
            Page 98
            Page 99
            Page 100
            Page 101
            Page 102
            Page 103
            Page 104
            Page 105
            Page 106
            Page 107
            Page 108
            Page 109
            Page 110
            Page 111
            Page 112
            Page 113
            Page 114
            Page 115
            Page 116
        Surveys at household level for monitoring and evaluation
            Page 117
            Page 118
            Page 119
            Page 120
            Page 121
            Page 122
        Survey design implications of household economics and farming systems approaches
            Page 123
            Page 124
            Page 125
            Page 126
            Page 127
            Page 128
            Page 129
            Page 130
            Page 131
            Page 132
            Page 133
            Page 134
            Page 135
            Page 136
            Page 137
        Rapid anthropological assessment procedures: Applications to measurement of maternal and child mortality, morbidity, and health care
            Page 138
            Page 139
            Page 140
            Page 141
            Page 142
            Page 143
            Page 144
            Page 145
            Page 146
            Page 147
            Page 148
            Page 149
            Page 150
            Page 151
            Page 152
            Page 153
            Page 154
            Page 155
        Innovations toward rapid and participatory rural appraisal
            Page 156
            Page 157
            Page 158
            Page 159
            Page 160
            Page 161
            Page 162
            Page 163
            Page 164
            Page 165
            Page 166
            Page 167
        Strengths and weaknesses of different household survey approaches for food policy design
            Page 168
            Page 169
            Page 170
    Part 3: Data quality and design of survey modules
        Page 171
        Page 172
    Improving data quality in household surveys
        Page 173
        Page 174
        Page 175
        Page 176
        Page 177
        Page 178
        Page 179
        Page 180
        Page 181
        Page 182
        Page 183
        Page 184
        Page 185
    Farm production data in rural household surveys
        Page 186
        Page 187
        Page 188
        Page 189
        Page 190
        Page 191
        Page 192
    Obtaining useful data on household incomes from surveys
        Page 193
        Page 194
        Page 195
        Page 196
        Page 197
        Page 198
        Page 199
        Page 200
        Page 201
    Collection of production and income data: A commentary
        Page 202
        Page 203
        Page 204
        Page 205
        Page 206
    Some methodological issues in the collection and implementation of time and labor use data
        Page 207
        Page 208
        Page 209
        Page 210
        Page 211
        Page 212
        Page 213
        Page 214
    Collection of time use and labor data: A commentary
        Page 215
        Page 216
        Page 217
        Page 218
    Food conusmption surveys: How random are measurement errors?
        Page 219
        Page 220
        Page 221
        Page 222
        Page 223
        Page 224
        Page 225
        Page 226
        Page 227
        Page 228
        Page 229
        Page 230
        Page 231
    On food consumption surveys: A commentary
        Page 232
        Page 233
        Page 234
        Page 235
    Open methodological questions related to nutrition data collection
        Page 236
        Page 237
        Page 238
        Page 239
        Page 240
        Page 241
        Page 242
    On consumption and nutrition data: A commentary
        Page 243
        Page 244
        Page 245
    Intrahousehold-related policy research: Implications for data collection
        Page 246
        Page 247
        Page 248
        Page 249
        Page 250
        Page 251
        Page 252
        Page 253
        Page 254
        Page 255
        Page 256
        Page 257
        Page 258
        Page 259
        Page 260
        Page 261
    Data on rural infrastructure and access to services
        Page 262
        Page 263
        Page 264
        Page 265
        Page 266
        Page 267
        Page 268
    Back Cover
        Page 269
        Page 270
    Reprint permisssion notice
        Page 271
Full Text

DATA NEEDS FOR FOOD POLICY
IN DEVELOPING COUNTRIES

NEW DIRECTIONS FOR HOUSEHOLD SURVEYS


EDITED BY
JOACHIM VON BRAUN
AND DETLEV PUETZ


o0 r Ir L crrIrmr

INTERNATIONAL FOO1111IID POLICY RESEARCH INSTITUTE


INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

























































As part of its publications program, the International Food Policy
Research Institute issues proceedings of workshops or conferences,
literature reviews, topical papers, and other special reports IFPRI believes
may be of interest to its audience. These occasional papers undergo peer
review prior to publication.









DATA NEEDS FOR FOOD POLICY
IN DEVELOPING COUNTRIES:
NEW DIRECTIONS FOR
HOUSEHOLD SURVEYS












DATA NEEDS FOR FOOD POLICY
IN DEVELOPING COUNTRIES:
NEW DIRECTIONS FOR
HOUSEHOLD SURVEYS

Edited by Joachim von Braun
and Detlev Puetz























International Food Policy Research Institute


Washington, D.C.
































Published in 1993 by the


International Food Policy Research Institute
1200 Seventeenth Street, N.W.
Washington, D.C. 20036
U.S.A.

Library of Congress Cataloging-in-Publication Data

Data needs for food policy in developing countries : new directions
for household surveys / edited by Joachim von Braun and Detlev
Puetz.
p. cm. -- (Occasional papers / International Food Policy
Research Institute)
Includes bibliographical references.
ISBN 0-89629-329-7
1. Nutrition surveys--Developing countries--Methodology.
2. Nutrition policy--Developing countries. I. Von Braun, Joachim,
1950- II. Puetz, Detlev, 1957- III. Series: Occasional
papers (International Food Policy Research Institute)
TX360.5.D38 1993
363.8'09172'4--dc20 93-48782
CIP











Contents



Foreword xi

List of Contributors xiii

1. Overview and Synthesis 1
Joachim von Braun

Part I. Food Policy Issues and New Challenges for Data

2. Policy Issues and Problems of Data Collection and
Analysis 13
Per Pinstrup-Andersen

3. Linkages Between Food Policymaking, Policy
Analysis, and Data Collection 18
Harris Mule

4. New Challenges for Resource and Environment Data
for Policy Research 23
Peter B. R. Hazell

Part II. Strengths and Weaknesses of Different Survey
Approaches for Food Policy Design

5. Information Needs for the Food System: Conceptual
Framework and Institutional Problems 29
Charles C. Mueller

6. Household Data Needs for Food Policy: Toward
Criteria for Choice of Approaches 44
Sara J. Scherr and Stephen A. Vosti









7. Using Nationally Representative Household Surveys for
Food Policy Analysis: An Examination of the World
Bank's Living Standards Measurement Study Surveys 80
Margaret E. Grosh and Paul Glewwe

8. Focusing Small-Scale Surveys on Specific Food Policy
Issues: IFPRI's Experiences 95
Detlev Puetz and Alison Slack

9. Surveys at Household Level for Monitoring and
Evaluation 117
Osvaldo N. Feinstein

10. Survey Design Implications of Household Economics
and Farming Systems Approaches 123
John M. Dixon

11. Rapid Anthropological Assessment Procedures:
Applications to Measurement of Maternal and
Child Mortality, Morbidity, and Health Care 138
Susan C. M. Scrimshaw

12. Innovations Toward Rapid and Participatory Rural
Appraisal 156
Margaret Buchanan-Smith

13. Strengths and Weaknesses of Different Household
Survey Approaches for Food Policy Design 168
Jan Low


Part III. Data Quality and Design of Survey Modules

14. Improving Data Quality in Household Surveys 173
Detlev Puetz

15. Farm Production Data in Rural Household Surveys 186
Sohail J. Malik









16. Obtaining Useful Data on Household Incomes from
Surveys 193
Harold Alderman

17. Collection of Production and Income Data:
A Commentary 202
Stephen Devereux

18. Some Methodological Issues in the Collection and
Implementation of Time- and Labor-Use Data 207
Tesfaye Teklu

19. Collection of Time-Use and Labor Data:
A Commentary 215
Michael Paolisso

20. Food Consumption Surveys: How Random Are
Measurement Errors? 219
Howarth E. Bouis

21. On Food Consumption Surveys: A Commentary 232
Carol Levin

22. Open Methodological Questions Related to Nutrition
Data Collection 236
Eileen Kennedy

23. On Consumption and Nutrition Data: A Commentary 243
Reynaldo Martorell

24. Intrahousehold-Related Policy Research: Implications
for Data Collection 246
Carol Levin, Katherine Ralston, and
Lawrence J. Haddad

25. Data on Rural Infrastructure and Access to Services 262
Sudhir Wanmali











Tables and Figures



Tables

6.1 Major methods for data collection 48

6.2 Factors to consider in selecting survey methods 54

6.3 Choosing data-collection approaches: scope of analysis 55

6.4 Choosing data-collection approaches: data
characteristics 60

6.5 Choosing data-collection approaches: logistical and
cost factors 65

6.6 Overall value of data-collection methods for different
uses 70

8.1 Overview of IFPRI surveys by policy issues 97

8.2 Research and policy questions of selected IFPRI
surveys 102

8.3 Sample characteristics for selected IFPRI surveys 107

10.1 Characteristics and potential usefulness of different
sources of farm-household information for food
policy analysis 129

11.1 Advantages and limitations of different RAP data-
collection techniques 143

11.2 Pluses and minuses of qualitative and quantitative
methods 146

11.3 Parameters involved in measurement of maternal
and child mortality, morbidity, and health care 148

12.1 Rapid rural appraisal and participatory rural appraisal
methods compared 158









20.1 Calorie expenditure elasticity estimates for Kenya and
the Philippines, by calorie variable 221

20.2 Family calorie availability and calorie intake per
capital, by food group and expenditure quartile for
Kenya 222

20.3 Family calorie availability and calorie intake per
capital, by food group and expenditure quartile for
the Philippines 223

20.4 Per capital consumption of foodgrains and estimated
income elasticities, by income quartile and urban
and rural populations in India 227

20.5 Per capital availability of foodgrains in India,
1951-84 228


Figures

1.1 Linkages and feedbacks desired in policy and data 2

5.1 Information needs for the food system 32

6.1 Considering priority data needs for food policy 72

8.1 Building capacity around surveys 111

10.1 Model of a farm household selling labor 125

18.1 A sketch layout of labor force disaggregation and
time use 209

21.1 An example of the decisionmaking process for
determining what data to collect and how to
collect them 235

25.1 Example of questions asked in a household survey
in Zimbabwe 266














Foreword



Information needs for food policy are expanding as changes take
place in different development and economic policy environments. A
reconsidered role of government-with reduced emphasis on direct
involvement in production and increased emphasis on infrastructure,
social services, and human resource investment-creates demands for
policy-relevant information on public goods and people. Also, the
concern for sustainable resource utilization and efficient targeted
interventions in support of the poor in a market-oriented policy
framework poses new challenges for information. Conceptual innova-
tions facilitate responding to these needs and challenges. Without
conceptual frameworks, data do not become relevant information.
Since the early 1980s, the International Food Policy Research
Institute (IFPRI) has been actively engaged in carefully conceptualized
household surveys for purposes of food policy analysis. Such involve-
ment by IFPRI and others is driven by the desire to simultaneously
address efficiency and equity issues in food policy. Isolated farm
production surveys, consumption/expenditure surveys, or nutrition
surveys cannot serve this policy analysis purpose. Such specific
surveys, as often undertaken by national institutions, do, of course,
play important roles in information provision, but it remains an
important task to further strengthen the linkages between such specific
surveys as well as between comprehensive (small) food policy-oriented
household surveys and larger-yet less comprehensive-national survey
activities. IFPRI's approach toward that end is to design and use food
policy-oriented surveys in close collaboration with national institutions.
This volume is an effort to enhance-directly and indirectly-the
benefits of information for poverty alleviation through more informed
food policy. However, it is selective and is compiled with the under-
standing that innovations in this area are progressing rapidly and,
therefore, the topic needs to be revisited continually in the future.
The volume resulted from a stimulating multidisciplinary workshop
on "Data Needs for Food Policy in Developing Countries: Directions
for Household Surveys," held at IFPRI in September 1992. It builds on
an IFPRI study funded by the Deutsche Gesellschaft fuir Technische
Zusammenarbeit (GTZ), which we gratefully acknowledge, and on a
number of original contributions by workshop participants from data-
using and data-producing institutions and by scholars in the field.









The contributors to this volume come from a great variety of
disciplines: economics, agriculture, anthropology, geography, nutrition,
and public health, among others. They bring with them experience with
food policy-oriented surveys from all parts of the developing world. To
have their insights represented in this volume may facilitate progress
toward improved data quality and sharing of information relevant to
food policy in low-income countries.
The panelists and commentators on the papers, with both their
participation at the workshop and their written contributions, helped
immensely to improve the volume. Rajul Pandya-Lorch, through her
editorial assistance and advice, greatly contributed to the volume.

Joachim von Braun
Director
Food Consumption and Nutrition Division











List of Contributors



Harold Alderman is senior economist in the Poverty and Human Resources
Division of the Policy Research Department in the World Bank. He
was formerly a research fellow in the Food Consumption and Nutrition
Division at the International Food Policy Research Institute, with field
survey experience in Egypt, West Africa, and India.

Howarth E. Bouis is a research fellow in the Food Consumption and
Nutrition Division at the International Food Policy Research Institute.
He has long-term field survey experience in the Philippines.

Joachim von Braun is a professor at the University of Kiel, Germany,
where he holds the Chair for Food Economics and Policy. He was
formerly director of the Food Consumption and Nutrition Division
at the International Food Policy Research Institute. His field survey
experiences extend over Egypt, The Gambia, Rwanda, Guatemala,
Sudan, and Ethiopia.

Margaret Buchanan-Smith is a research fellow in the Food Security
Unit of the Institute of Development Studies at Sussex University
in Brighton, U.K. Her field survey experience includes Chad,
northern Kenya, western Sudan, and Nepal.

Stephen Devereux is senior research fellow at the University of
Namibia. He recently completed his doctorate at Oxford University
on seasonality and household food insecurity in northern Ghana.

John M. Dixon is senior officer in the Farm Management and Produc-
tion Economics Service at the Food and Agriculture Organization
of the United Nations, Rome. He has field experience in Iran,
Ethiopia, and Nepal.









Osvaldo N. Feinstein is a senior monitoring and evaluation officer at
the International Fund for Agricultural Development. He has field
experience in most of Latin America and the Caribbean, India, the
Philippines, and Mozambique.

Paul Glewwe is an economist in the Poverty and Human Resources
Division of the Policy Research Department in the World Bank.

Margaret E. Grosh is an economist in the Poverty and Human
Resources Division of the Policy Research Department in the World
Bank.

Lawrence J. Haddad is a research fellow in the Food Consumption and
Nutrition Division at the International Food Policy Research
Institute. He has conducted surveys in the Philippines and India.

Peter B. R. Hazell is director of the Environment and Production
Technology Division at the International Food Policy Research
Institute. He was formerly a principal economist in the World
Bank's Agriculture and Rural Development Department. He has
survey experience in India, Malaysia, and several African countries.

Eileen Kennedy is a research fellow in the Food Consumption and
Nutrition Division at the International Food Policy Research
Institute, with field survey experience in Kenya and Ghana.

Carol Levin is an agricultural economist at the U. S. Department of
Agriculture's Economic Research Service. Based on two years of
field research in Indonesia, she has coauthored a manual on data
collection in developing countries.

Jan Low is a doctoral candidate in agricultural economics at Cornell
University. She was formerly project manager of a multidisciplinary
household survey, the Malawi Maternal and Child Nutrition Study.

Sohail J. Malik is a research fellow in the Food Consumption and
Nutrition Division at the International Food Policy Research
Institute and is based in Pakistan, where he guides a comprehensive
longitudinal household survey.

Reynaldo Martorell is Woodruff Professor of International Nutrition at
the Center for International Health, Emory University.









Charles C. Mueller is a full professor in the Department of Economics
at the Universidade de Brasilia and currently a visiting scholar at
the University of Illinois. During 1985-90 he was the director of
Agricultural Statistics and afterward, president of the Instituto
Brasileiro de Geografia e Estatistica (IBGE), which is Brazil's
central statistical office, census bureau, and geographical and
natural resources survey organization.

Harris Mule is a member of the Governing Board of the African
Capacity Building Foundation, Nairobi. During 1988-92 he was
assistant president of the International Fund for Agricultural
Development. Before that he was permanent secretary at the
Ministry of Finance in Kenya.

Michael Paolisso is director of research at the International Center for
Research on Women. His research focuses on microlevel and
household issues in agriculture, environment, and health as well as
on methodology. His survey experience includes Kenya.

Per Pinstrup-Andersen is director general of the International Food
Policy Research Institute. He was formerly director of the Cornell
Food and Nutrition Policy Program and professor of food econom-
ics at Cornell University. His field survey experience is in various
Latin American, African, and Southeast Asian countries.

Detlev Puetz is an independent consultant. He was formerly a
postdoctoral fellow at the International Food Policy Research
Institute. He has survey experience in The Gambia.

Katherine Ralston is an agricultural economist at the U.S. Department
of Agriculture's Economic Research Service. Her Ph.D dissertation
was on intrahousehold food allocation in West Java, Indonesia.

Sara J. Scherr is a research fellow in the Environment and Production
Technology Division at the International Food Policy Research
Institute, with survey experience in East Africa. She was formerly
a senior researcher at the International Tree Crops Institute and a
principal scientist at the International Centre for Research in
Agroforestry.









Susan C. M. Scrimshaw is professor of Public Health and Anthro-
pology at the University of California at Los Angeles (UCLA) and
the associate dean for Academic Programs at UCLA's School of
Public Health. She has done field research in Ecuador, Colombia,
Barbados, Haiti, Panama, Guatemala, and Mexico.

Alison Slack is a research analyst in the Food Consumption and
Nutrition Division at the International Food Policy Research
Institute and a Ph.D candidate at the Fletcher School of Law and
Diplomacy. She has collected survey data in West Africa.

Tesfaye Teklu is a research fellow in the Food Consumption and
Nutrition Division at the International Food Policy Research
Institute, with field survey experience in Indonesia, Zambia, Sudan,
and Botswana.

Stephen A. Vosti is a research fellow in the Environment and
Production Technology Division at the International Food Policy
Research Institute, with field survey experience in Brazil and
Ethiopia.

Sudhir Wanmali is director of the Outreach Division at the International
Food Policy Research Institute. He has field survey experience in
India and southern Africa.











Overview and Synthesis


Joachim von Braun



ORIENTATION OF DATA FOR FOOD POLICY NEEDS

Food policy reforms have dominated the development agenda in
many low-income countries in recent years. Governments in developing
countries as well as donors are becoming increasingly concerned about
the impact of policies on people's living standards, food security, and
nutrition. The World Summit for Children in 1990 and the International
Conference on Nutrition in 1992 as well as the country-level prepara-
tion and follow-up activities to these conferences are cases in point.
To develop effective policies and mitigate any potential negative
effects, it is necessary to (1) quantify the actual impact of food policy
changes on incomes, resources, expenditures, and nutrition; (2)
understand the response mechanisms of individuals, households, and
markets to changing socioeconomic environments; and (3) identify the
most vulnerable groups for targeting programs and projects. All these
activities require substantial amounts of data. Yet there are significant
deficiencies in data availability and quality, and gaps remain in
understanding the complex interrelations in food policy and nutrition.
Household surveys are indispensable for analyzing distributional
policy effects between and within households, for investigating
resource-allocation decisions at the household level, and for relating
welfare indicators such as income or nutritional status to policy and
program interventions as well as to macro, structural, socioeconomic,
behavioral, and environmental factors. Thus, they are complementary
to sectoral-, administrative-, and community-level data gathered by
using alternative survey methods (such as enterprise or market surveys)
or as part of administrative recording procedures (for example, land use
statistics). Where such data are not available, are not reliable, are
difficult to collect, or are not comprehensive, household surveys can
contribute significantly to improving basic socioeconomic statistics.
There is a broad consensus in the development community that
informed policy and program decisionmaking calls for improved data
from the household. It is critical to determine where to invest scarce










resources and to define criteria for appropriate information-collection
approaches, depending on specific food policy questions. Data needs
ought primarily to be a function of decision needs. Yet, past data
collection and related capacity-building efforts for information
utilization in developing countries have often been disappointing for
food policymakers, for instance, because relevant topics were not 1
addressed or results were not presented in a timely manner. Another
concern is that data quality issues are not being adequately addressed
in policy analysis.
Food policy researchers and policymakers prefer to focus on issues
and concepts. Data are taken for granted; if certain data are not
available for a study or cannot be easily collected, then, often, they are
assumed; for instance, intrahousehold equality is often assumed. Data,
however, can become information only when they are suitably analyzed
and placed in a context of food policy issues, as depicted in Figure 1.1.
Throughout this volume, it is stressed that "data collection" for food
policy analysis must be driven by issues and must be derived from clear
conceptualization and (household) theory. Yet the data-collection
component of the linkages is not one on which researchers are eager to
focus; they like to leave that up to others. In neither the research
communities nor the policymaking communities is it particularly
rewarding to focus on "data" and their quality.


Figure 1.1-Linkages and feedbacks desired in policy and data










OBJECTIVES AND OVERVIEW

To improve the contribution of future household-level data-
collection exercises for emerging food policy research issues, this
volume reviews experiences with household surveys focused on food
policy. Much can be learned from pooling the expertise of individuals
and institutions experienced with different types of data-collection
exercises oriented toward food policy in developing countries. While
this volume stresses that "data" must not be taken out of the conceptual
or issues linkage, it does focus on data-related issues.
There are three motivations for this volume. First, food policy
research needs theoretical and conceptual foundations, otherwise data
are just data and do not become information. Clarity in the emerging
food policy questions is needed so that today's empirical research can
address tomorrow's questions. As the food policy agenda broadens to
include sustainability issues, challenging new issues will be put on the
research agenda.
Second, policymakers, and the food policy researchers who serve
them, need information. But it is not obvious that optimal investment
in information collection is taking place. Information collection is costly
to both the institutions that collect data (fiscal costs) and the households
from whom the data are being collected (time and related opportunity
costs). These costs must be matched by benefits resulting from better
information and subsequent policy action, be it at the international,
national, or community level. These issues have not been addressed
with sufficient rigor in the past.
Third, the quality of household and intrahousehold information
needs to be improved. Researchers are paying more attention to proper
conceptualization of the household and its behavior (Singh, Squire, and
Strauss 1986; IFPRI 1992). Substantial changes occurred in the com-
plexity of household surveys in the 1980s, prompted by improved
concepts of the "household" and facilitated by technological changes in
data collection and processing. New insights have been stimulated as a
result. Yet the issue of data quality has not been given sufficient
attention, and in fact continues to be treated as a taboo topic. Question-
able quality of data could derail even the smartest empirical analysis.
But established criteria for checking on household data quality do not
exist. As rewards in the academic community are clearly for smart
analysis, researchers hardly question their own and their colleagues'
quality of data. It requires careful assessment (Zarkovich 1964; Hunt
1970; Collinson 1984; Kearl 1976).
While this volume provides considerable guidance on data collection










oriented toward food policy, it is not a "how to" manual on food- and
agriculture-related surveys. Much improved comprehensive literature
on this subject is currently emerging (for example, Poate and Daplyn
1993 and sources quoted therein); this volume is to be understood as
complementary to that important literature referenced in the various
chapters below.
The volume has three objectives: to derive data needs from a
discussion of emerging food policy questions (Part I); to critically
assess the role of different types of household surveys for policy and
project planning and implementation (Part II); and to draw attention to
operational aspects of survey management (Part III).
In Part I, Per Pinstrup-Andersen highlights emerging food policy
issues and stresses the need for food policy research and information
to be issue-oriented, relevant, and timely in order to prove useful to
policymakers. These points are underlined for the African context by
Harris Mule, who argues that quality and accuracy of data are defined j
by the function for which the data are being used. Mule also indicates
the problems of donor-driven data collection and contends that there is
a need to strengthen indigenous statistical data collection institutions.
The new challenges of resource and environment data for policy
research are addressed by Peter B. R. Hazell. Hazell notes that the data
and analytical demands related to management of common resources
are especially challenging.
A conceptual framework for information needs in the food system,
in the broader context of demand for statistical services, is developed
in Part II by Charles C. Mueller. This chapter highlights the problems
of the public goods nature of much of the needed food policy data and
related institutional problems, drawing on the case of Brazil. A
comprehensive overview of alternative household data collection
approaches is given by Sara J. Scherr and Stephen A. Vosti. Criteria
for systematically choosing reliable, relevant, and cost-effective
approaches are developed and presented in this chapter, which also
points out that the appropriateness of any approach depends on user
needs, scope of analysis, characteristics of data, and logistical factors.
The portfolio of investment in data has been growing very rapidly.
Various schools of thought have emerged that advocate large surveys,
small surveys, rapid appraisals, or not-so-rapid appraisals, among
others. A selection of "schools" is represented in this volume.
Margaret E. Grosh and Paul Glewwe review the World Bank
experience with large-scale, nationally representative household surveys
for food policy analysis, that is, the Living Standards Measurement
Study surveys. Small-scale surveys focused on specific food policy










issues are reviewed by Detlev Puetz and Alison Slack, who draw upon
the IFPRI experience. The scope for improvement of household surveys
for monitoring and evaluation of programs is discussed by Osvaldo N.
Feinstein. John M. Dixon describes the survey-design implications of
farming-systems approaches. The alternative and complementary
potentials of rapid anthropological-assessment procedures applied to
mortality, morbidity, and health care issues are demonstrated by Susan
C. M. Scrimshaw. How rapid and participatory rural appraisal methods
can be used to meet food policy needs is discussed by Margaret
Buchanan-Smith. Jan Low concludes Part II with a commentary on the
strengths and weaknesses of different household survey approaches.
Part III explores the state of the art in collecting data on important
food policy variables for different survey types and deals with
assurance of data quality and innovations for major areas of household
data collection. Collection of farm production data in rural household
surveys is addressed by Sohail J. Malik, who draws upon experience
from such surveys in Pakistan to highlight how biases can be intro-
duced into survey estimates through sample selection, questionnaire
design, type of information collected, timing of survey, and training of
enumerators. Harold Alderman identifies problems that can be
encountered when collecting data on household income and suggests
ways of getting around some of these problems. He notes that while
wage income, rental income, and remittances are comparatively easy
to collect, profits from family enterprises and income from livestock
and livestock by-products are particularly difficult to collect and to link
to a particular household member. Stephen Devereux discusses other
types of income that are difficult to measure, such as income from root
crops, tree crops, and leafy vegetables, and women's income.
Collection and implementation of time- and labor-use data are
considered by Tesfaye Teklu. He provides an overview of the most
common techniques used for collecting time-allocation data, drawing
heavily upon IFPRI research studies. Michael Paolisso discusses an
alternative approach, which is applied by anthropologists, for collecting
time-use data. This technique, random spot observation, has a randomly
determined observation period and an extremely short duration of
observation.
The two main techniques for collecting food consumption informa-
tion at the household level-intake technique and expenditure tech-
nique-are compared by Howarth E. Bouis. His analysis finds a
different pattern of food consumption between the two survey tech-
niques, but the precise sources of divergence between the two tech-
niques are not known. Carol Levin suggests using a decision tree











before designing a survey, to help define the best measure of food
security and the best method for collecting relevant data for a given
situation.
Eileen Kennedy reports on an inventory conducted by IFPRI in
which the most important concern expressed by implementers of food
security and nutrition monitoring systems was the failure to use the
information generated in decisionmaking and policy formulation. The
commentary by Ray Martorell highlights some of the difficulties in
using and interpreting anthropometric data.
How households and families allocate responsibilities and resources
has been neglected in policy research. Carol Levin, Katherine Ralston,
and Lawrence Haddad focus their chapter on intrahousehold aspects of
data collection and highlight some methodological, logistical, and
ethical issues that arise when collecting such data.
In the final chapter, Sudhir Wanmali shows how household- and
regional-level information on rural infrastructure and access to services
such as transport, marketing, and finance can be obtained. He identifies
variables for which data are easy to collect and others for which data
are difficult to collect, and stresses that difficulties in collecting certain
data must be considered before data collection proceeds.


TOWARD IMPROVING QUALITY AND FOOD POLICY IMPACT
OF SURVEYS

While each chapter in this volume has important conclusions or
recommendations for improving data quality, survey capacity, and food
policy impact, various generalizations can also be drawn from across
the chapters and emerging agreements can be highlighted. This section
presents some key points on which there appear to be general consensus
and a few instances where the jury is still out.

Letting Policy Issues Drive Surveys
Household surveys should be driven by country-specific issues and
policy needs. However, in many low-income countries, especially in
Africa, household data collection efforts are frequently driven by
donors. This need not be a problem. It can, however, be one if the
surveys reflect a specific policy agenda of donors) rather than of the
country, or if the surveys erode the existing capacities of statistical
services or prevent the building of such capacities.










Deregulating "Data Markets"
The public goods nature of statistical data collection at the house-
hold level and the monopolistic position of central national statistical
services in national "data markets" pose problems for efficient
utilization of resources. It is an unfortunate reality that policy analysts
in some countries do not get access to existing data and hardly can
influence design of data collection. "Data markets" need to be opened
up. There are two options for addressing this problem: strengthen the
central statistical services by working with and in them, and compete
with the central statistical services in order to improve the efficiency of
such systems in providing information for food policy. The first
approach is probably more relevant to the statistical services of small
countries, while the second approach could benefit food policy research
in countries with large but too restrictive and secretive statistical
services.

Overcoming Waste in Survey Investments
It is often not accepted that surveys are an essential element of
investment in information. Investment in information can have high,
low, or even negative payoffs, just like any other type of investment.
There are numerous examples of surveys that have not been used or
documented and therefore cannot be revisited for building longitudinal
data sets. Overcoming waste in surveys is essential.

Exposing Surveys to Economic Assessment
The costs and benefits of information must enter more explicitly
into considerations of food policy research, as data collection is often
a large share of the costs of food policy research. Survey economics
has hardly been developed conceptually. Also, the costs and benefits of
different household survey approaches vary. It is especially important
that costs that burden the poor, including time costs, enter cost-benefit
considerations for appropriate design of household survey data
collection.

Strengthening Training
Training in surveys oriented toward food policy in low-income
countries is needed to improve benefits and cut costs. Training courses
must emphasize the information and utilization aspects of surveys and
not limit themselves to technical aspects of data collection and
processing.










Scientifically Testing Survey Approaches
Undertaking complex surveys at the household level is often
considered an "art," that is, location-, researcher-, issue-, and time-
specific, rather than a "science," that is, the result of testable choices
among alternative approaches. There is a need to move from art to
science in data-collection approaches, that is, toward transparency in
choices among tested alternative survey and data-collection approaches.

Fitting Survey Approaches to Country and Issue Context
Survey approaches must not be considered in isolation; otherwise,
complementarities are overlooked. When determining optimal combina-
tions of survey instruments, the state of development of a country,
including its institutions, as well as the nature of the food policy
questions, must be considered. The toolbox for surveys has become
bigger and the "tools" in the box have become more sophisticated.
Choosing the optimal combination of survey tools-for example, rapid
appraisal and structured household survey-for the appropriate food
policy questions must utilize complementarities. For instance, participa-
tory rural appraisal methodology can be important for collecting
information and can also serve as an important front-runner for
structured policy-oriented surveys.

Emphasizing Capacity Building with Surveys
There are indications that data-collection approaches are converging.
In both structured sample surveys and participatory rural appraisals
there is a growing realization of the need to move toward more
replicability and more testability of information generation and-at
different levels-to pay increased attention to building institutional
capacity. In order to build such a capacity to serve food policymaking,
the survey units must be closely linked with policy decisionmaking
units. The policy-needs orientation of surveys as well as other
information generation requires institutional structure for articulation
and feedback.

Focusing on Policy Impact
There is considerable demand by policymakers for data that are
useful for evaluating the impacts of food policies. Impact evaluation of
policies with original data has great potential, but constraints and
limitations need to be kept in perspective (the frequent lack of baseline
information, time pressures, and so on). The critical issues here are not










just the comprehensiveness and detail of the data but also the timeliness
of the data in order to maintain policy relevance.

Focusing on Timely New Facts
Sometimes, modest approaches are called for in combination with
selective, in-depth impact studies. This has implications for the depth
of household data collection. The relevance of new factual information
for policy impact must be stressed. Focusing on patterns of problems,
rather than the more sophisticated details, is frequently what facilitates
more efficient policy. Policymakers are excited by specific new factual
information from household surveys that characterize a problem, thus
they are stimulated to take appropriate action. Even localized surveys
can be helpful in this respect.

Strengthening International Information Exchange
Reinforcing information exchange to facilitate the spread of
knowledge on surveys oriented toward food policy is overdue. Too
much "ad hocism" and "reinventing of the wheel" prevails in choosing
survey approaches. There is a need for some umbrella under which
related information exchange is strengthened and institutional memory
on surveys maintained. Such a mechanism should at least include the
relevant international agencies providing food and agricultural technical
and policy analysis assistance, such as the Food and Agriculture
Organization of the United Nations (FAO), Consultative Group on
International Agricultural Research (CGIAR), International Food Policy
Research Institute (IFPRI), World Bank, International Fund for
Agricultural Development (IFAD), and World Food Programme
(WFP).

Using Surveys as a Means of Articulation by the Poor
Household surveys are not just technical tools. Considerable
responsibility rests with survey planners and users vis-a-vis the
respondents. Surveys in developing countries, directly or indirectly, can
offer the poor an opportunity for articulation and participation that
should not be lost in stereotypical data-collection exercises.











REFERENCES

Collinson, M. P. 1984. Farm management in peasant agriculture.
Boulder, Colo., U.S.A.: Westview.

Hunt, K. E. 1970. Agricultural statistics for developing countries.
Oxford, U.K.: University of Oxford.

IFPRI (International Food Policy Research Institute). 1992. Under-
standing how resources are allocated within households. IFPRI
Policy Briefs 8. Washington, D.C.: International Food Policy
Research Institute/World Bank.

Kearl, B., ed. 1976. Field data collection in the social sciences:
Experiences in Africa and the Middle East. New York, N.Y.:
Agricultural Development Council.

Poate, C. 0., and P. F. Daplyn. 1993. Data for agrarian development.
In Wye studies in agricultural and rural development. Cambridge,
U.K.: Cambridge University Press.

Singh, I., L. Squire, and J. Strauss, eds. 1986. Agricultural household
models. Baltimore, Md., U.S.A.: Johns Hopkins University Press.

Zarkovich, S. S. 1964. The quality of sample statistics. Rome: Food
and Agriculture Organization of the United Nations.








Part I
Food Policy Issues and
New Challenges for Data















Policy Issues and Problems of
Data Collection and Analysis


Per Pinstrup-Andersen



For effective food policymaking in developing countries, it is
important that the volume of bad and unreliable data used in policy
analysis be significantly reduced. Although good data do not ensure
reliable research results, they do help. Bad data, on the other hand,
ensure unreliable research results.
Household surveys are very expensive, and a large amount of resources
have already been spent in less than optimal ways; in other words, wasted.
Yet it is also very expensive to be uninformed or misinformed. Thus it is
not just a matter of the cash outlays for household surveys but also a
matter of the opportunity cost of not doing them.
The International Food Policy Research Institute (IFPRI) relies
heavily on data from household surveys to answer aggregate and
macrolevel policy questions. IFPRI has been quite successful in doing
its research in this way. It took IFPRI a few years to realize the
importance of using household-level data for aggregate and macro
analysis. From its inception in 1975 to 1982, most of IFPRI's work
was done on the basis of existing and secondary data. But since the
early 1980s, the institute has rapidly expanded the use of household
survey data, an approach that has served it well.
Given that IFPRI has been extensively using household survey data
for more than 10 years now, it is appropriate to take stock of where it
is and where it wants to go, and to ask how household data collection
can be made more efficient and effective. Three issues are addressed
here: the direction IFPRI is taking in its policy research; a critical
assessment of ongoing household survey efforts in general; and ways
to successfully influence policy, using household data.
IFPRI is in the middle of developing its five-year plan. The policy
agenda for the next five years will be dominated by developing
countries trying to recuperate from the crises and adjustments of the
1980s. This is particularly the case for African and Latin American
countries but also for some Asian countries.










There is a great need for additional information to guide the market
liberalization and privatization efforts currently under way or planned
for a number of developing countries. IFPRI wants to look at both
agricultural input and output markets, since this is where many
mistakes have been made in the efforts to liberalize and privatize.
These mistakes have been made, in part, because of a lack of informa-
tion and partly because of factors such as pressure from political and
other interest groups. Additional research in this area would be
extremely useful, particularly as IFPRI focuses this work, and any
other work it does, on low-income people. IFPRI's concern is to assist
policymakers in implementing policy options that will be beneficial to
the poor.
Policy research on technological change in agriculture remains
important at IFPRI. In order to facilitate technical change and to benefit
everybody, including the poor, a credit system, infrastructure coverage,
and policies to modernize and transform the agricultural sector must be
in place. IFPRI exerts a good deal of research effort on that issue in
addition to the related output and income diversification issue.
Relative to the last five years, IFPRI will make its largest expansion
over the next five years in research on natural resource management
policies. The question of how poverty, technology, food production,
and natural resource management link together deserves a great deal of
research. Natural resource degradation has been a serious problem for
a number of years, but it is only in the last three to four years that
increased attention is being paid to it. IFPRI intends to take advantage
of the opportunity to do something about a problem that has been
neglected for too long.
IFPRI will also, on behalf of countries recovering from the crises
of the 1980s, look at macroeconomic policy reforms-how they affect
the poor and the agricultural sector. IFPRI will be selective in this
research, because quite a lot of work has already been done in this
area. In this context, it will also look at regional markets in developing-
country regions in terms of how they can benefit the agricultural sector
and low-income people.
Another topic that IFPRI will focus on is the demographic changes
expected to occur in the next five years and beyond. Attention will
primarily be paid to issues of urbanization. In the next 5-20 years,
governments will have to face up to the need to deal with rapid
urbanization and rapid population growth in urban areas to a much
greater extent than they have ever done in the past, particularly in
Africa. In all developing regions, urbanization will require major
attention from governments, but it is not clear that the information









needed to deal with the problem is available.
Attention must also be given to rural areas, of course, because food
must be brought to urban populations as they rapidly increase. This is
an additional reason for transforming the agricultural sector in rural
areas. Hence, there will be much emphasis on examining marketing
structures.
IFPRI will also continue to look at transfer schemes to alleviate
poverty and food insecurity. IFPRI has already done a great deal of
work in this area and will continue to do so on a selective basis.
Finally, underlying most of IFPRI's work is the desire to under-
stand the behavior of households, communities, and national govern-
ments. Within the household, for instance, allocation and decision-
making processes need to be examined and understood. Additional
information is urgently needed on household behavior.
Most of IFPRI's research will be undertaken in "integrated
multicountry research programs." For each of a number of sharply
focused projects, a small number of country studies will be undertaken
using a common methodological framework. After the country studies
are completed, a synthesis will be done in order to generalize the
results. This suggests two important points. First, to really understand
behavior, data must be collected where that behavior occurs; that is,
data must be collected from the agents who make that behavior happen,
which in many cases means from households. Second, for research
findings to be useful to policymakers, not only in a particular region
but also beyond, they must be generalizable.
There are some serious problems in data collection and a tremen-
dous amount of resources is wasted. Most data collection efforts
concentrate on the process of collecting the data, not on the uses of the
data collected. It is rare to find household survey programs that begin
by asking basic questions such as, For what purpose is the data being
collected? What is the information needed for? What are the decision-
making processes that the data are trying to influence? There is little
concern for the needs of information, so the effort is driven by data.
Similar problems are encountered in food policy analysis. Logic would
suggest that the process begin with the need for information, then move
to an analytical framework, and, only after that is achieved, identify the
data needs. Yet, frequently, that logic is not present, and the process
begins the other way around. It is not unusual to find household survey
questionnaires that have been developed from previous questionnaires.
Sometimes there is too little concern about whether the data collected
are those that are actually needed to answer the relevant policy
questions.










Does that mean that multipurpose questionnaires or multipurpose
surveys should never be used? Periodic surveys to monitor trends and
indicators and to provide some input into food policy analysis can be
extremely useful. Examples of such surveys include income and
expenditure surveys and the World Bank's Living Standards Measure-
ment Study (LSMS) surveys when they are administered on a continual
basis. But for the most part, multipurpose questionnaires will not be
sufficient, and sharply focused or tailor-made questionnaires will
frequently need to be designed to support periodic surveys. The
argument being made is not that governments should be told to stop
doing periodic income and expenditure surveys, but that the purposes
for which various kinds of surveys are being done should be made very
clear to them. One of the lessons learned is that comprehensive living
standard surveys are probably not feasible on a routine basis. As a
result, they have been greatly simplified.
Weak analysis is another issue. Frequently, it is found that surveys
have generated two-way tables, but there is often very little analysis
beyond these tables. Another issue is the failure to ensure a basic
understanding of the population of households where the sample is
drawn. Researchers sometimes go to a survey area with a structured
household survey but without paying attention to the important
preliminary ethnographic study phase. Often, this is because the time
needed to become familiar with the study population and the setting in
which a structured survey is to be done is lacking. If adequate
knowledge of the population is available beforehand, it is possible to
successfully go straight in with a structured questionnaire, but if that
information is not available, it is extremely risky because incorrect
conclusions may be drawn.
How can the information derived from household surveys be made
more useful to policymakers? There are four important considerations.
First, information provided must be reliable. That means there must be
good data and good analysis. Policymakers will frequently see what is
reliable and what is not, and if it is sensed that unreliable information
is being provided, important opportunities are lost.
Second, the information provided must be relevant. There must be
a focus on causation. Policymakers do not want to have described to
them what they already know or what they can read for themselves in
the newspaper. Policymakers want to be told what could happen if they
tinker with a certain policy intervention. So causation must be
examined, which has serious implications for the way analytical models
are developed. It also means that, in most cases, behavioral relation-
ships must be sought.









Third, the information provided must be timely. That means that
analysts must have foresight. Most economists tend to work on ex post
analysis, for example, telling the policymakers what they did wrong in
the last five years. While that is frequently important in order to learn
what should be avoided or pursued in the future, it is very important
that the information developed and passed on to policymakers be
timely, both in the sense of predicting what could happen if certain
policy interventions are changed and in the sense that this is what the
policymaker needs to know this week, month, or year.
Fourth, the information provided must be properly presented.
Policymakers are extremely busy and have only a few minutes to digest
the information; it is important to learn how to present information to
such people.
The time is past when policymakers believed that they could issue
decrees and things would happen. Major failures have resulted from
that type of attitude during the last few years, and many policymakers
now realize that they have to understand or be able to predict responses
to policy measures. Hence, it is important to look at behavioral issues
and for analytical models to have predictive power. Analysts need to
tell policymakers what they think will happen in the future if certain
things are changed.











3
Linkages Between Food Policymaking,
Policy Analysis, and Data Collection


Harris Mule



Examining the linkages between household survey data, policy
analysis, policymaking, and the ultimate use of that policy suggests that
in many cases data collectors tend to collect data as an end in itself. A
visit to any statistical bureau, especially in Africa, will find mountains
of data that have been collected over the years and that have never been
put to any use. Second in that chain is policy analysis. An incredible
amount of resources has been spent on policy analysis, work that may
be very exciting for the analysts, but all too often the results are never
or rarely put to use. Next, there is policymaking. Policymakers are
typically senior bureaucrats or politicians who take an incredible
number of policy initiatives. Again, these policymakers may find this
work exciting, but the results are rarely useful or relevant to producers
or consumers, who would be the ultimate beneficiaries from the
activities in that chain.
There is a need for demand-driven activities in policymaking, policy
analysis, household survey data collection, and data analysis. The
distinction made between the policy analyst or researcher and the
policymaker is forced. The objectives and motivations of both should
be identical. Research and policy analyses as well as the household data
that are derived from them should be driven by the needs of the
policymaker. The types of data that are collected and the analyses based
on those data should be perceived to be important by the policymaker.
This, in turn, should be relevant to the households, which will be the
beneficiary of the initiative.
An emerging issue in food policy is the link between macroeco-
nomic, mesoeconomic, and microeconomic analyses of household data.
Agricultural research carried out in the 1970s and 1980s has come up
with useful and interesting findings, such as that food production is a
key element of economic growth in developing countries, and that
technology is a very important factor in the development process. It is
also clear that the domestic terms of trade-the relative shares of










income between different sectors of the economy-are very important
for fostering agricultural development and food production. Macroeco-
nomic parameters, therefore, are important determinants of increased
food production. In that case, household data and other micro data
collected, as well as the analysis based on them, must be linked to
macroeconomic analysis. The International Food Policy Research
Institute (IFPRI) has made both technical and analytical breakthroughs
in this area. However, much more needs to be done.
One area where more needs to be done is food security. Food
security is now defined as being linked to income and as allowing
households and consumers to have access to income and available food.
That definition is being carried a step further by one school of thought,
which argues in terms of international trade that Third World countries
should emphasize production of those commodities in which they have
a comparative advantage and import those food products that they do
not grow well.
There is a second school of thought, especially prevalent in Africa, that
argues that the very process of increasing food production is an important
mechanism for introducing technology as a means of increasing general
economic growth. Even if this is not the case, for a country to rely on food
imports to meet a substantial portion of its food requirements is not
economically efficient. Furthermore, even if the first two cases do not
hold, food management is very important and food importation and food
procurement, even at the household level, involve a degree of risk that
needs to be minimized. The important policy determinant should not be
food security as defined by income, but food security as defined by food
self-sufficiency. This is an ongoing debate. Food policy researchers and
analysts should explore this issue further.
Policy analysis by researchers and, in many cases, actually by
policymakers, has been based on purely economic parameters by trying
to determine the supply or demand response, impact on incomes, or
impact on food production and consumption. In Africa, however, the
supply response is not all that important, but the research being carried
out in food policy and general economic management does not show
why supply responses are weak. Weak supply responses are observed
even when structural adjustment measures are implemented that try to
improve the way markets function and change relative prices, and
eventually have an impact on consumption and production. Food policy
research and analysis must go beyond simple description to tell
policymakers why things are not happening the way they should.
Therefore, research on food policy should touch upon behavioral
characteristics.










Food policy research and analysis must also look into institutional
issues because the behavioral characteristics of institutions and their
organizational arrangements can have a major impact on production and
nutrition. It must not be forgotten that the purpose of analysis, certainly
in the African and South Asian contexts, is to improve food production
and food consumption and, thereby, nutrition.
There are several implications of these aforementioned points
concerning data collection and analysis. Macroeconomic indicators,
national accounts of production and consumption, and demographic
information are all important data because any meaningful analysis of
household data in terms of an overall food policy must utilize sound
aggregate figures. When designing household surveys, it must be
ensured that besides having clear objectives for carrying out demand-
driven surveys, the collection of data must strengthen the existing
macroeconomic statistics. In many cases, collection of household
survey data has marginalized the existing statistics base.
The data must be timely. The researcher's agenda is, in many cases,
very different from that of the policymaker. Similarly, the researcher's
time perspective or time dimension is very different from that of the
policymaker. A good researcher wants to have a perfect product that can
be defended in an assembly of researchers. But a policymaker wants to
make a decision, and the decision must be made at a given time. Any
analysis, therefore, or data on which that analysis is based, that comes after
the horse has gone out of the stable is useless.
The quality and accuracy of data are also important. Inaccurate data are
likely to impair the researcher's credibility, but, even more seriously, they
are likely to lead to bad policy prescriptions, and if the policymaker takes
the research seriously, the consequences can be unfortunate.
The quality and accuracy of data are defined by the function for
which those data are being used. For example, when food relief must
be provided immediately in the case of famine, there is not too much
concern with the detailed accuracy of the data. For most policy
decisions, accuracy at the 95 percent level of confidence or even the 66
percent level may not be terribly important. It must be remembered that
policymakers must often make decisions with or without thorough
analysis and with or without accurate data. Any analysis or information
that can help them make better decisions will clearly be beneficial.
Therefore, any concerns about quality of data must be put into the
context of the purpose for which the data will be used.
Two sources of error must be taken care of: sampling errors, on
which much has been written, and nonsampling errors, which have
usually been the more serious errors. Questionnaires are often designed










by people who do not know the cultural and social environment for
which they are designing the questionnaires, and as a result, wrong
questions may be asked.
Enumerators are important people, but they are often overlooked.
After training enumerators, researchers typically send them out to the
field to complete the questionnaires and bring them back looking
beautiful. The type and accuracy of the information that the enumera-
tors collect, however, is questionable. Many questionnaires are based
on weak data collection. Scrimshaw (Chapter 11 of this volume) takes
the anthropological approach with a rapid rural survey; this type of
survey can provide useful insights into what should be done to
minimize the nonsampling errors that arise in questionnaires.
It is, of course, important that once household survey data are
collected, they are safely stored in a retrievable manner. The volume
of household data that has been collected and then lost or misplaced is
enormous.
It is important to understand the driving forces behind policy
analysis research and the implied household data collection and
analysis. In Africa, and to a lesser extent in other developing regions,
much of the policy analysis research and data collection is driven by
donors. So far, the discussion has been on demand-driven or supply-
driven research in household surveys. But, in many cases, data
collection and research initiatives are driven by donors (including
research institutions). Indigenous statistical data collection institutions
are actually being incapacitated and, in some cases, destroyed. It is in
the interest of everybody-donors, research institutions, and certainly
governments in the Third World-that efforts at research and data
collection and analysis be coordinated. One of the recommendations
that can emerge from this volume is that donors, national governments,
and research institutions such as IFPRI, should make a conscious effort
to coordinate their research and, especially, their data collection
systems and arrangements.
Institutional framework is another important issue. One of the
biggest problems experienced in data collection and analysis is
institutional limitation. Many institutions are not well established, they
are disjointed. In the last 30 years, these institutions, instead of getting
closer together, in fact, have tended to dissolve. One of the issues that
needs to be addressed is how to reinvigorate these institutions and make
them more coherent and cohesive.
Finally, it is intriguing to note that when economists and other
social scientists meet, they like to talk about the costs and benefits of
everybody else's work except theirs. It would be a good idea to find







22


out the cost-effectiveness and cost-benefit of the research that econo-
mists and policy analysts undertake. It may induce them to be much
more cost-conscious and to exert greater efforts to identify mechanisms
for introducing cost-effectiveness in data collection.











4
New Challenges for Resource and
Environment Data for Policy Research


Peter B. R. Hazell



Environmental issues are the "new boy on the block." They are not
new because they have suddenly become important, but because people
have suddenly become aware of them. There is a lot of catching up
going on in many organizations at both national and international levels.
Resource degradation raises some difficult issues for researchers and
policymakers. This is a relatively new field, where a lot of pioneering
work needs to be done on methodology and data collection. From the
researcher's point of view, there are basically two problems to worry
about. The first problem relates to the degradation of privately owned
resources. The question here is, why are farmers, who, we have
learned over the years, are really quite rational decisionmakers (one of
the early mistakes of the 1950s was to believe that farmers were
irrational or stupid peasants, and we subsequently learned that they
were a lot smarter than we were), degrading their own resources? This
is a very critical question that may reflect on our own perception of the
problem as much as it does on the reality of the situation.
The second problem relates to the degradation of what could be
called common property resources, primarily because of inappropriate
or inadequate property rights and externality problems. Examples are
the degradation of common grazing areas and woodlots or the pollution
of groundwater supplies.
Of these two issues, the first is relatively easy to handle, at least
conceptually if not empirically. The analysis requires various types of
data:
* time-series data on resource degradation at the very micro level,
that is, the field or the woodlot-measurements of soil or pasture
deterioration, tree loss, water pollution, and so on;
* matching socioeconomic data on households that manage those
resources, relating to variables like land use decisions, technology
choice, and investments in improving resources. To model house-
hold decisions so that predictions can be made about what would










happen if policies or technologies were changed, household
information on employment, labor use, income, consumption, and
other variables is also required; and
* some sort of data on personal discount rates and intergenerational
preferences.
This suggests something like the village studies data collected by the
International Crops Research Institute for the Semi-Arid Tropics
(ICRISAT) in India. These are time-series data collected over many
years and covering all aspects of a household's decision problem, but
backed up with a great deal more technical information on resource
monitoring at the field and woodlot levels. A fairly modest set of
requirements!
In some cases, a proxy to such time-series data would be cross-
sectional data. The International Food Policy Research Institute is
beginning a project in the forest margin areas of the Amazon, where
land is supposedly expected to degrade almost linearly with time once
deforestation begins. So, if a cross-section of farms is taken, starting
at the forest margin and working backwards so that farmers who have
been farming pieces of land for different numbers of years are
included, a very good cross-sectional variation on land degradation
could be obtained that would be a proxy for time-series information.
But such a case would be an exception rather than the general rule.
Turning to the management of common resources, data and
analytical demands become even more challenging. Not only is it
necessary to monitor resource degradation over time and to monitor and
model household decisionmaking, but it is also necessary to look at the
links between the different types of households that are making
decisions about those resources. The interactions have to be modeled
and the market-failure problems and externalities have to be captured
within models that simulate the effect of different technologies,
policies, and institutional organizations on the management of those
resources. This is a much more difficult task. It requires the same kind
of data mentioned earlier, but rather than just a random sample of
households in the community, a very structured sample is needed so
that all the different types of people who interact in the management of
a particular resource are captured. In both cases, there are difficult
challenges on the methodology and research sides. Some careful case
studies of selected sites are needed where these issues can be studied
and modeled in considerable detail.
Unlike many areas of food policy analysis, there is really very little
secondary data on natural resource management. There may be some
information on the loss of forests, although it is often unreliable, but










there is less information on the degradation of soils, pollution of water,
and so forth.
Clearly, one would like to see government statistics and forestry
departments and ministries of agriculture and irrigation collecting much
more systematic information on the resource base. Most countries do
not even have basic inventories of their natural resource base. It is
difficult to even get a decent soil map in many countries. So it will be
a tremendous step to go from the present state of knowledge to being
able to monitor changes in the quality of natural resources. It will be
very expensive, and it will require a great deal of institution building.
An important question to ask is, How many data really need to be
collected? Collecting data at public expense is like any other public
expenditure-it should be rationalized in terms of its social cost and
returns. There is a tendency for people who collect data, whether they
are researchers or employees of statistical departments, to view data
rather like the scientist's search for truth or the theologian's search for
divine wisdom. It is something that becomes a goal in itself and that
should be sought after at any cost. Clearly, this is not practical. Money
spent on collecting data, even valuable information like monitoring the
condition of the forests, could be spent on other things. Thus it is
necessary to think of doing social cost-benefit analyses of data-
collection activities.
With that thought in mind, several issues arise concerning the
collection of public information on the environment. The first is
whether collecting data on resource degradation of privately owned
resources is really a public good. If it is not a public good, then what
business does the government have in spending money to collect
information on those resources? But if such information is a private
good, should there not be more thought given to how governments can
encourage the private sector to set up data provision services, rather
than to how governments could collect such information? Involving the
private sector is not an entirely foreign concept; in many countries,
there are private services that do soil tests, for example. There is no
reason why, if information on privately owned resources was valuable,
people would not be willing to pay for wider data services, including,
perhaps, the provision of simple instruments, such as soil testing kits,
by which they could monitor the degradation of their own resources.
Turning to common property resources, there clearly is a greater
public goods element to information about these resources, particularly
resources that cut across a fairly large community, such as ground-
water. But here the problem may not be so much the lack of data on
resource degradation as the lack of effective ways of solving the










problem. There is increasing evidence to suggest that many of the
externality problems that arise in rural communities cannot be solved
by simple technology or public policy interventions. These problems
are going to require that communities organize to manage their own
resources in a way that internalizes those externalities. Traditionally,
many communities had indigenous institutions for managing common
property, but many of those institutions have become weak or have
broken down. They need to be rebuilt or replaced by other forms of
communal organization. But until communal action problems can be
solved, it may be that collecting a lot of information on the degradation
of communal resources does not have a social return, since it may not
be possible to implement solutions.
The last issue is that local people and farmers may have a lot more
knowledge about the state of their resources than outsiders think.
Outsiders may grossly exaggerate the state of degradation of resources.
Farmers work with individual fields, they live with those fields
throughout their lives, they inherit them from their fathers; there is a
tremendous amount of accumulated indigenous knowledge about the
condition of soils and other resources. Ways of tapping into that
knowledge bank must be found, as must ways of translating the
perceptions of farmers regarding the state of their resources into
concrete information that can be used by policymakers. That might be
a far more cost-effective solution than having lots of soil scientists and
other experts running around collecting vast amounts of scientific
measurements.








Part II
Strengths and Weaknesses of
Different Survey Approaches
For Food Policy Design













5
Information Needs for the Food System:
Conceptual Framework and Institutional
Problems


Charles C. Mueller



INTRODUCTION

This chapter takes an organic view of the system that collects,
processes, and disseminates data on the food system that are of
relevance to decisionmakers in developing countries. Such a view
considers the main activities and relationships taking place within the
subsector of food and nutrition. Following a discussion of the conceptu-
al framework of the evaluation of the role of statistics, the chapter
examines the main types of statistics on food from the point of view of
the agents involved in the food subsector. Three cases based on the
Brazilian experience are presented, and central institutional obstacles to
a more efficient statistical system are discussed.


THE BASIC CONCEPTUAL FRAMEWORK

Information is an important input in the economic process; it is almost
trivial to say that economic agents perform better when they are well
informed. Of course, there are important noneconomic roles for informa-
tion, but in the economic sphere it is vital for efficiency and development.
Statistics are systematic information required by society for several
purposes. Therefore, an evaluation of the role of statistics and of the
performance of the system that generates statistics is more meaningful
if considered in the context in which it operates, that is, making
statistics available as inputs. In evaluations that ignore this, there is a
tendency for concerns over aspects of how statistics are obtained to
prevail over issues of why, for what purposes, and with which
characteristics statistics are, or should be, obtained.
The System of National Accounts (SNA) provides a revealing
example of the importance of a conceptual framework for the produc-










tion of statistics. This comprehensive system is founded on a clearly
specified model based on Keynesian macroeconomics. The SNA not
only produces indicators that have a clear meaning, but it also demands
statistics on production, consumption, trade, and investment that have
to conform to the requirements of the basic model. Thus it provides
orientation for organizations that produce statistics.'
This chapter contends that an analysis of the generation and
dissemination of food statistics makes much more sense if based on a
conceptual framework. The production, processing, marketing, trade,
and consumption of food are recognized as being only a part of a much
wider socioeconomic system, but an effort is made, using the notion of
partial process, to show that it is possible to establish a meaningful
conceptual framework.
Several assumptions are made. First, a developing economy is
assumed in which markets perform allocative functions imperfectly, and
their distributive functions tend to weigh against the poor. There is,
therefore, scope for policies to improve the functioning of markets,
reduce disparities in income and wealth, and alleviate extreme poverty.
Second, it is assumed that policies do not aggravate market
distortions, although this may not be completely realistic given the
experience of development strategies in many developing countries
(Mueller 1992). Third, it is assumed that there are income or wealth
distribution policies, together with target-group food and nutrition
policies of the type reviewed by Timmer, Falcon, and Pearson (1983).
Fourth, it is assumed that the country has a statistical system managed
through a central statistical office. Finally, it is taken for granted that
information is a vital element, and that there is an inherent need to
improve the efficiency of the country's statistical system.
As Georgescu Roegen (1971) contends, the processes of nature and
society do not present the researcher with clear-cut dividing lines,
separating the various subprocesses for analysis by discipline. Conse-
quently, the analyst is forced, explicitly or implicitly, to establish a
partial process relevant for an analysis. Imaginary seams are estab-
lished, separating the elements that deserve central attention from those
considered less relevant, which are placed in the background; this is



'This is not meant to imply that the current SNA is flawless and perennial. There
have been several criticisms of the basic model behind SNA and about some of its main
procedures (see some of the papers in Ahmad, El Serafy, and Lutz 1989). However, it
provides an example of a functioning information system based on a consistent theoretical
background.










done because it is difficult to grasp simultaneously all elements of a
global process. However, the background elements should not be
ignored, and the artificial character of the boundaries of the partial
process should always be clear.
With this in consideration, a food and nutrition partial process is
devised and summarized in Figure 5.1. It is composed of three blocks:
food production, food processing and marketing, and food consump-
tion. The main features of each block are determined by a set of
structural factors. Their behavior relies on the operation of markets.
There is, in addition, the interference of policies under the above
assumptions.
The central structural elements of the food production block are
endowment of natural resources, land-tenure structure, characteristics
of the agricultural entrepreneurial class, qualification of the agricultural
work force, and technology. These structural elements influence the
behavior of food producers and the working of the food and agricultural
product and input markets, thus affecting the product mix. Also
important are elements and events taking place in partial processes
close to the food process, such as the agricultural nonfood process.
Finally, macroeconomic and specific policies influence the decisions of
agents involved in food production.
In the food processing and marketing block, the main structural
elements are basic infrastructure (such as the transportation network,
ports, and warehousing), composition of the processing and marketing
segments, market structures for food items, and systems for financing
food transactions. These elements influence the efficiency of food
marketing functions, with important effects on both food producers and
consumers. Of course, policies also affect the functioning of this block.
The central structural elements for the food consumption block
include the level and distribution of income, degree of urbanization,
tastes and preferences, education, availability of nutrition programs,
spatial disposition, and efficiency of the food retailing system. For the
medium- to high-income groups, the availability and market conditions
of goods and services that substitute for food are also important. Again,
several policies influence food consumption.
Two types of policies are relevant to the partial process: wide-
ranging policies that affect all blocks, and policies geared mainly to a
specific block. Among the first set of policies are macroeconomic ones
such as monetary and fiscal policies, foreign exchange policy, credit
policies, interest rate subsidies, and price and wage controls. In
different ways, they influence food production, marketing, and
consumption.










Figure 5.1-Information needs for the food system



MacroPolicies


Policy

O Food information needs
( Policy information and analysis


~--- L r'
Income I I SocialPolicies
and Wealth I I (education,
distribution i publichealth,
Policies sanitation)

i- ---.










The specific policies may have indirect effects on other elements of
the partial process, but they are designed to reach certain portions of
the subprocess. Agricultural support policies (for example, minimum
prices, production subsidies) and technical change policies are among
those aimed at the production block. Policies designed to improve
transportation and storage infrastructure can have effects at the farm
level as well as on marketing, processing, and trade. Policies geared to
the food consumption block include those affecting income and wealth
distribution (among the latter, a land reform policy would also affect
food production), social policies (education, public health, and
sanitation), and target-group nutrition policies.


STATISTICS FOR THE FOOD SYSTEM

The Main Users of Statistics
If the basic role of a higher-quality system of food statistics is to
improve the functioning of markets in the different stages of the food
process, the main users of these data would include economic agents
involved in the production, processing, and marketing of food; food
consumers; and governmental agencies responsible for food policies.
Ample access to better information would improve decisionmaking by
these agents, leading to a more efficient food system, with gains for
society as a whole.
Researchers and analysts on food matters and policies, especially
those in academic and research organizations, form a different category
of users. Their motivation differs from that of economic agents,
although they generate analyses that can be useful to the economic
agents.

Attributes of an Efficient System of Statistics:
Trade-off between Quality and Timeliness
Ideally, food statistics should be of excellent quality and should
reach the users exactly when they are most needed. However, as with
other goods and services, the production of statistics requires human,
financial, and capital resources. Since pursuing either attribute involves
the use of scarce resources, there is a trade-off between quality and
timeliness.
As a rule, economic agents particularly value the attribute of
timeliness. Of course, they require a minimum level of quality of their
information; otherwise, it is of little use to them. However, perfect but










dated statistics can be useless. This aspect is sometimes overlooked by
statistical organizations, especially organizations where dissemination
of information is not a priority and administration of information is
controlled by those who produce it.
Academic researchers, on the other hand, highly value the attribute
of quality. They generally do not have to make rapid decisions and, to
some extent, can wait for improved data.
A proper balance between quality and timeliness is a permanent
challenge for statistical systems. Usually, data come out in different
stages of perfection. In the first stage, when timeliness is essential, data
about a given phenomenon may have a lower degree of precision,
functioning more as indicators. With time, the quality of the data can
be improved, and they are transformed into a more precise measure-
ment of the phenomenon. In some cases, however, the requirement of
timeliness and the cost of employing a more sophisticated methodology
lead to data that are no more accurate than mere indicators.

Statistics Relevant to the Food Partial Process
Statistics and information relevant to the food partial process can be
grouped into the following categories, as evident from Figure 5.1:
agricultural production information; food processing and marketing
statistics; food trade statistics; food consumption information; social
policies and nutrition monitoring; and policy information and analysis.
This classification stresses the blocks where data are collected and
not the segments that benefit most from the information. The main
types of statistics obtained in each group are given below.
Agricultural production information: crop forecasting; monthly crop
production monitoring (area, output, yield); annual food and
agricultural production; farm sales of crop and animal output; fish
catches; prices received and paid by farmers; inputs employed by
producers; farm wastes.
Food processing and marketing statistics: animal and vegetable
food-processing statistics; prices at different stages of the processing
and marketing chain; costs, margins, and profits in marketing
channels; warehousing and food inventory statistics; market
structure and organization; market inefficiency.
Food trade information: volume and prices of food exports and
imports; demand conditions and prices in relevant world markets;
information on the main suppliers of food items; world inventories.
Food consumption information: food balance sheets; average
apparent consumption; consumption of food by income class; food










prices paid by consumers in different places and marketing outlets;
on-farm food consumption.
* Social and nutrition information: social monitoring information; data
on low-income groups; nutrition patterns; food consumption of
target groups.
* Policy information and analysis: information about policies that
affect the food system; analysis of the impacts of such policies.
An analysis of the roles, methodologies, peculiarities, and difficul-
ties of each type of statistics would be out of order here. Instead, the
next section presents short case studies highlighting important aspects
of the process of production and dissemination of statistics on food and
related subjects to the main users.


INSTITUTIONAL PROBLEMS

Obstacles can be encountered when an attempt is made to improve
food information systems in developing countries. A tendency to
overlook timeliness and difficulties in adequately sensing society's
requirements for statistics usually stem from these obstacles. The result
is an inefficient use of resources. How can these imperfections be
eliminated so as to make the system more responsive to the needs of
society?
The correct functioning of markets in a market economy leads to
efficiency. However, markets may fail. Information, especially
statistics, has the character of a public good. Among other things, this
means that markets cannot perform the role of transmitting to the
statistical system the wishes of society; it also means that, as a rule, the
costs of producing statistics cannot be fully recovered by sales of
information to the users. For these reasons, public organizations are in
charge of producing and disseminating statistics in almost every
country.
If markets fail to efficiently allocate resources in the production of
statistics, how can this be done? One of the problems with certain
statistical organizations is that the technical (production) unit is almost
solely responsible for decisions, not only on how statistics are to be
produced, but also on which statistics, for whom, and with what
characteristics. The unit that disseminates information, which should
maintain a close and active interaction with users, is often an append-
age to the technical unit. Sometimes it is little more than a sales office
that receives output readily and sells it to whoever comes by. It has no
influence on the allocation of resources in production.










In countries with an embryonic statistical system, the types of
statistics that are needed or that need to be improved are almost always
self-evident, and the main problem is that of generating a technical
capability to introduce the necessary changes. In such circumstances,
the technical or production unit ends up dominating the system's higher
administration. The main attribute pursued is technical excellence;
allocation decisions are performed by an enlightened technocracy with
reduced attention to the requirements of users.
As society develops and the economy is diversified, requirements
for information increase significantly. The reduced interaction between
the statistical system and the users of statistics, brought about by the
neglect of dissemination, leads to the affirmation of the technocratic
model of resource allocation in the production of statistics.
The production unit has some interest in dissemination of statistics,
but this tends to be incidental. It may want to assess the needs of
society, but this is not its main interest nor is it equipped to do this
adequately. The chief concern of the technocratic administration is to
obtain maximum resources from the public sector to implement its
work plan, basically drawn up on technical grounds. Any effort to
increase the efficiency of the statistical system would find this a very
difficult obstacle to overcome.
To improve the allocation of its scarce resources, the statistical
system has to devise ways to correctly detect society's information
needs. To do this well, it should closely interact with the segments
requiring statistics. It should have a well-functioning system for
dissemination, one that continually assesses the needs of actual and
potential users-decisionmakers in the private and public sectors and
researchers-and understands the role of various types of statistical
information as inputs in the economic development process. In fact, the
dissemination area should be placed in a dominant or central position
in the higher administration of the statistical system.
This is not to imply that the technical unit is not important or that
it should be relegated to an inferior status. This unit commands the
production functions and has the essential role of advancing efficiency
in the use of productive resources. However, a well-formed dissemina-
tion unit would know demand well; if imbued with a marketing
perspective and with a strong sense of the statistical system's institu-
tional mission, it would be able to bring into the allocation decision
process invaluable elements for the optimization of scarce resource use
in fulfilling society's needs for statistics.
The segment of food statistics would only optimize the use of
resources if a well-conceived dissemination system were made to










interact with the elements of all of the blocks sketched in Figure 5.1,
sensing the demands and making those a central feature in decisions on
resource allocation. Of course, this would have to be done within the
broader context of decisions about the production of all types of
statistics.


THREE CASE STUDIES

This section focuses on the problems of gathering meaningful
statistics on food production and consumption and the nutritional status
of low-income people. All case studies are drawn from the Brazilian
experience, bringing out some of the points discussed above.

Statistics on Vegetable Food Crops
There are considerable difficulties in making available statistics on
food production that are both reliable and timely. Resources at the
disposal of statistical organizations are scarce, agricultural production
is usually dispersed over a wide area, and it is difficult to correctly
follow food production through its various stages during the crop year.
This section discusses alternative means for producing statistics on
vegetable food crops.
Sophisticated methods for measuring production of vegetable food
crops are available, such as the U.S. Department of Agriculture
(USDA) methodology, which combines a survey based on a stratified
area sampling frame with the interpretation of satellite images, allowing
an estimation of the area cultivated under different crops and, under
certain conditions, of the situation of crops over time (Houseman
1975). However, not only is this an expensive method, but it is also
quite demanding of organization, management, and technical skills. If
operations are not executed with precision, imperfections and delays
can arise, and if the basic stratified area sampling panel is not carefully
built, estimates can have large coefficients of variation.
At the other extreme, nonprobabilistic information on food (and
agricultural) production can be obtained with the aid of a network of
informants from major agricultural areas. This method may ensure
timeliness, but the errors in the estimates cannot even be measured. For
a number of years, Brazil has employed such a method-the Systematic
Survey of Agricultural Production (SSAP), which is a series of monthly
estimates of production of the country's 35 main vegetable crops,
commencing with surveys of planting intentions at the beginning of the










crop year, followed by estimates of area planted and yields and, at the
end of the period, of amounts harvested (IBGE 1988). This is done for
each state and for the country as a whole.
SSAP obtains the information through monthly meetings of panels
of experts at local (municipal), state, and national levels. Participants
in the local monthly meetings would include, among others, extension
agents operating in the area, bank officials, staff members of coopera-
tives, input sales personnel, farm association leaders, and buyers of
agricultural products, under the leadership of an official from the
Institute Brasileiro de Geografia e Estatistica (IBGE), Directoria de
Pesquisa, the statistical central office. Sometimes, concrete information
from administrative records (such as the area of crops financed by the
rural credit system, the sales of certified seeds) are available, but
several of the members bring estimates based on experience and field
observations.
The data obtained at the local level are aggregated at IBGE's state
office, reviewed by a state panel of experts, and sent to the statistical
office's central headquarters, which undertakes the national aggrega-
tion. The national totals are reviewed by a panel of experts before
being released to the public.
IBGE is attempting to implement a crop information system
employing the USDA methodology. This attempt began in 1986, with
the implementation of a pilot panel for the state of Parani, an important
agricultural area (Mueller, Silva, and Villalobos 1988). The first survey
was conducted in 1987, followed by others. In recent years, three other
states have been included. The use of satellite images is still restricted
to the stratification process during the panel construction.
The estimates obtained by SSAP were not incompatible with those
obtained by IBGE. The commercial crops estimates tended to have low
coefficients of variations (cv's) in the new system, but the availability
of administrative records allowed estimates under SSAP that usually fell
within their confidence intervals. For the subsistence crops (some of
which are important food staples), the new system produced high cv's,
generating wide confidence intervals; frequently, the nonprobabilistic
estimates were within those intervals.
IBGE is continuing to improve the new system; it is reviewing the
sample panels in an attempt to reduce the cv's, which for many crops
are still uncomfortably large. However, there are enormous difficulties,
and problems of finance and management have created a sluggish
system. Quality is gradually improving but timeliness is not, which
represents a problem for crop statistics.










Food Consumption Statistics
Countries with serious problems of malnutrition would benefit from
reliable statistics on food consumption. Such statistics would play an
important role in orientating policies to increase staple food intake
among the very poor, either through special programs or through
improvements in the efficiency of food production and marketing
(Timmer, Falcon, and Pearson 1983). However, it is not simple to
obtain meaningful data on food consumption. Generally, two types of
food consumption information are generated: food balance sheets, with
data on availability of the main food items per person, and food
consumption by income class.
In Brazil, regarding food balance sheets, the Fundaqao Getdilio
Vargas (FGV), a reputable research organization, calculates the annual
total and per capital domestic availability of a group of food items by
employing annual data on food production and trade and estimating the
losses, from harvest to actual consumption, as well as the nonhuman
use (seeds and feed) (FGV 1991). However, the data thus produced
have to be used with care as there are problems of classification and of
compatibility between the production and trade statistics. Lack of
information makes for precarious estimates of losses and nonhuman
consumption. Moreover, the absence of adequate data on food
inventories precludes estimation of inventory variations, reducing the
reliability of the estimates. The data can only be used as indicators; a
moving average of the per capital availability of food items provides
clues on trends.
Even if perfect, estimates of per capital food consumption would be
of limited value, since averages hide disparities in distribution. Brazil
has reasonable average consumption levels for most food items, but its
skewed income distribution leads to extreme poverty and malnutrition
in low-income groups. For a clearer evaluation of such problems,
another type of consumption information should be generated-food
consumption by income class.
This information may be derived from household budget surveys
that most countries undertake when constructing their cost-of-living
indices. This assumes that some care is taken in the survey design.
However, if the survey design is left entirely to the unit constructing
the cost-of-living indices, the results may not be entirely adequate for
food consumption analysis, and the organization undertaking the survey
may not make the results available for other purposes.
Since the early 1970s, IBGE has implemented two major household
budget surveys for its cost-of-living indices: the Estudo Nacional de










Despesas Familiares (ENDEF) survey undertaken in 1974-75, and the
Household Budget Survey (HBS) of 1986-88. ENDEF was a huge,
multipurpose sample survey. One of its objectives was to obtain data
to calculate the weights of a national cost-of-living index. It had other
objectives as well, including the collection of data for nutrition and
food consumption analysis. The cost-of-living index objectives were
achieved, but the consumption and nutrition studies were downplayed.
In part, this can be attributed to political problems: the military regime
then in power did not want to have Brazil's social ailments exposed.
However, there were also operational problems that precluded a wider
use of the survey's data. ENDEF generated enormous amounts of data
but there was not enough political will and technical capacity to
meaningfully employ them.
The HBS survey was implemented in an open, democratic environ-
ment. Between September 1986 and February 1988, it gathered
information on the consumption habits of a sample of 17,000 house-
holds in 11 metropolitan areas. Its primary objective was to obtain data
for the weights for cost-of-living indices. Having collected information
on household characteristics, income, expenditures on specific goods,
and amounts purchased, the HBS has become an invaluable source of
data for food consumption studies. However, the price index objectives
have prevailed and access to HBS data is quite difficult, partly because
Brazil does not give much emphasis to food studies and partly because
of bureaucratic obstacles.

Nutrition Surveys
Sample surveys may be employed to understand society's nutrition
patterns and to determine the dimensions of its hunger and malnutrition
problems. With ample resources, time, and technical capability, a detailed
and painstaking survey may be implemented, collecting information on the
food consumption of each member of a household sample over a time
period, together with anthropometric measurements and other elements
considered important. However, at times, resources are limited and quick
results are needed; for this, there are simpler methodologies.
Again, Brazil's experience is revealing. Two large-scale household
nutrition surveys have been carried out: the already-mentioned ENDEF
survey of 1974-75, and the Pesquisa Nacional Sobre Satide e Nutriqao
(PNSN) survey of 1989. Both were national in scope, focused on rural
and urban areas, and based on a stratified sample, and employed
anthropometric measurement of members of the households in the
sample. But there were also differences, including timeliness.










An important objective of ENDEF, as already mentioned, was to
obtain elements for the cost-of-living index weights, but it also
furnished information for evaluating hunger and malnutrition. ENDEF
was a huge and expensive undertaking; around 55,000 households were
sampled at a cost of about US$20 million (INAN 1990a, 9). Moreover,
it was operationally complicated; enumerators were required to spend
seven days within each sample household, recording not only its
characteristics and the anthropometric measurements of its members,
but also registering the food consumption of the household throughout
the period, along with information for other purposes. As a result, the
field survey took 12 months to complete. A huge volume of data was
generated, creating problems of processing and tabulation. Results
emerged only very slowly. These problems and the previously
mentioned political problems rendered ENDEF inefficient and wasteful.
PNSN presents a contrast to ENDEF. It had different objectives;
while ENDEF attempted to examine the relationship between food
consumption and nutritional status, PNSN examined health and
nutrition. However, the two surveys generated comparable results
regarding the nutritional status of the Brazilian population (Coitinho et
al. 1991). Not only was PNSN much cheaper (its cost was around
US$1.2 million), but it was also considerably faster. It surveyed a
sample of 14,455 households (around 63,200 persons of all ages)
between June and September, 1989, and the first results were released
in March, 1990, followed by studies and comparative analyses (INAN
1990a, 1990b; Coitinho et al. 1991).
PNSN was a joint undertaking of organizations in the health and
planning ministries and of IBGE. It benefited from ENDEF's experi-
ence, retaining its positive elements but avoiding the mistakes of the
first survey. It also undertook anthropometric measurements, but the
food consumption recordings were eliminated. Instead, a careful
evaluation of the household and of its members was done, the elements
of which were later related to these measurements. Furthermore, an
advanced sampling technique was employed, allowing for a consider-
able reduction in sample size without loss in accuracy. Timeliness
became a central feature; PNSN was executed at the end of a presiden-
tial term and it was feared that a new administration would abandon the
undertaking. A more agile methodology was introduced for other
reasons, but it became invaluable, allowing the prompt conclusion of
the research, before the change in government. However, the quality
of its results are undisputed.







42


CONCLUDING COMMENTS

Improving the system of statistics would clearly raise the efficiency
of the food and nutrition system. By better designing the system to
meet the needs of private and public decisionmakers and of food and
nutrition researchers, farmers, processors, traders, and consumers
would be able to improve their performance. But this is not easy to
accomplish; there are instances when investing in equipment and in
developing the technical skills for production of food statistics would
not significantly ameliorate the situation. There may be serious
institutional and organizational obstacles that should be removed for the
system to become truly effective. If this is not done, even the best
technical aid and financing would not prevent the statistical system
from being transformed into a museum of gadgets, fancy methodolo-
gies, and mediocre results.










REFERENCES

Ahmad, Y., S. El Serafy, and E. Lutz. 1989. Environmental account-
ingfor sustainable development. Washington, D.C.: World Bank.

Coitinho, D. C., M. M. Leao, E. Recine, and R. Sichieri. 1991.
Condicoes Nutricionais da Populaao Brasileira: Adultos e Idosos.
Brasilia: Instituto Nacional de Alimentagqo e Nutriqio.

FGV (Fundaqio Getilio Vargas). 1991. Balanco e Disponibilidade
Interna de Generos Alimenticcios de Origem Vegetal-1986 a 1990.
Rio de Janeiro.

Georgescu Roegen, N. 1971. The entropy law and the economic
process. Cambridge, Mass., U.S.A.: Harvard University Press.

Houseman, E. E. 1975. Area sampling frame in agriculture. Washing-
ton, D.C.: United States Department of Agriculture, Statistical
Reporting Service.

IBGE (Instituto Brasileiro de Geografia e Estatistica, Directoria de
Pesquisa). 1988. Principals Caracteristicas das Pasquisas
Economicas, Sociais e Demogrdficas da DPE. Rio de Janeiro.

INAN (Instituto Nacional de Alimentaq~o e Nutriqao, Minist6rio da
Satide). 1990a. Pesquisa Nacional Sobre Sadde e Nutriao-
Resultados Prelimenares. Brasilia.

1990b. Pesquisa Nacional Sobre Sadde e Nutridao-Perfil
de Crescimento da Popula(ao Brasileira de 0 a 25 Anos. Brasilia.

Mueller, C. C. 1992. Agriculture, urban bias, and the environment:
The case of Brazil. Paper presented at the conference on resources
and environmental management in an interdependent world, 22-24
January, San Jose, Costa Rica.

Mueller, C. C., G. Silva, and A. G. Villalobos. 1988. Pesquisa
AgropecuAria do Parani-Safra 1986/87. Revista Brasileira de
Estatistica 49 (191).

Timmer, C. P., W. P. Falcon, and S. P. Pearson. 1983. Food policy
analysis. Baltimore, Md., U.S.A.: Johns Hopkins University Press
for the World Bank.












Household Data Needs for Food Policy:
Toward Criteria for Choice of Approaches


Sara J. Scherr and Stephen A. Vosti



DATA NEEDS FOR FOOD POLICY

Formulation of food policies requires accurate information about the
existing food system and about the likely effects of policy action on that
system. Because information gathering and evaluation are costly, in the
face of compelling alternative needs for investment of public resources,
"data experts" must explore ways to meet policy data needs most
efficiently and cost-effectively.
Major questions to be answered and decisions to be made in data
collection include
* What are the costs associated with inadequate information?
* What information needs to be collected for policy analysis?
* What is the appropriate unit for analysis?
* What method of data collection is most appropriate?
* How can it be ensured that data are representative of the popula-
tion?
How long and how frequently do data need to be collected?
This chapter will not address the problem of setting priorities in the
type of information needed. Rather, it will examine the decision to use
a particular approach to collecting that data. There is currently an
active debate among researchers about appropriate methods to inform
policy needs. Different groups make arguments for the superior quality,
relevance, or cost-effectiveness of methods such as national monitoring
systems, formal household surveys, "rapid appraisal," or, more
recently, remote-sensing techniques. Unfortunately, the polarization of
much of the debate and the narrow terms in which the debate is often
waged have delayed the evolution of a more positive and objective


The authors wish to acknowledge the valuable input they received from IFPRI colleagues
Lawrence Haddad and Detlev Puetz in the development of this paper, and the useful
comments of workshop participants.










synthesis of current knowledge about the costs and benefits of different
methods in meeting different information needs.
To take a step in that direction, some of the real costs of poorly
informed policy are identified in this chapter, the major types of data
collection methods are described, and factors to consider when selecting
a data collection method are discussed. The final section suggests ways
to further refine survey approaches, and identifies issues for data
collection arising from new policy initiatives for community organiza-
tion and natural resource management.


COSTS OF INADEQUATE POLICY RESEARCH

Policy research is not costless, and decisions about which issues to
research and which methods to use must be made thoughtfully.
Identification of key policy questions that require high-quality research
is one of the essentials of good policy planning.
In assessing the optimal level and type of investment in data
collection, the costs of not collecting the relevant data or of collecting
inadequate data need to be considered. The cost is the lost value of
improvements in policy design that would result from data collection.
Such costs are related to both lack of policy research and poorly
designed policy research.

Costs of Underinvestment in Policy Research
How many data are needed, or how precise they need to be,
depends on what the threshold for stimulating policy or program change
might be. For all data users except researchers, additional food system
data that will have no effect on the final design of policy are not worth
collecting, although information does itself sometimes change the terms
of policy debate. In development programs and community action,
monitoring or research programs should be set up only after clear
discussions of actions likely to result from new data.
For relief assistance, the costs of not collecting data include
* human and related economic costs from failure to provide assistance
to intended beneficiaries of the program;
* resource costs of assistance received by people who are not the
intended beneficiaries;
* resource costs associated with unidentified inefficiencies in relief
distribution systems; and
* overinvestment in insurance (for example, food stock) due to lack










of precise or timely information.
For development policy, the costs of not collecting data include
* economic and welfare losses for beneficiaries associated with
inefficiencies induced by misdirected policies;
* economic and welfare losses resulting from failure to provide
strategically important policy inputs;
* resource costs of public or induced private expenditure on unneces-
sary investments or programs; and
* resource costs to public agencies of unidentified inefficiencies in
policy implementation.
The willingness to invest in data collection depends on the probabili-
ty of significant costs of these types or the likelihood that the program
can be changed in any significant way to reduce them or both. The
costs of data collection thus include both the direct measurement costs
and the costs of acting upon the indicator information.

Costs of Poorly Designed Policy Research
There is always a risk associated with data collection that poor-
quality data will lead to a significant misdirection of policy. Poor-
quality data may result from a number of design flaws:
* use of inappropriate indicators or measurement techniques (for
example, area under forest cover may be used as a proxy for
fuelwood availability when, in fact, most fuelwood is collected from
farm-grown trees);
unrepresentative selected samples (for example, basing national
fertilizer policy on research undertaken only in high-potential areas);
provision of false, misleading, uninformed, or unreliable answers by
respondents (for example, drawing key informants about likely public
response to a change in policy only from the political party in power);
lack of inclusion of critical variables that affect response to policy
(for example, failing to disaggregate by farm type in farm supply
response research);
widespread and unsystematic errors in data collection; and
seriously flawed or misinterpreted analysis.
Major data-quality problems are not always recognized, and policies
are sometimes formulated on the basis of those poor-quality data. To
reduce this risk, an explicit analysis should be made during the data-
collection planning phase of areas where poor quality would be "fatal"
to any responsible use of the data for policy, so that special attention
can be paid to quality control. Another way of checking on data quality
is to share the results of preliminary analysis with members of the










group being studied (for example, farmers, consumers, or local
program staff), to see whether the analysis makes sense and whether
the evidence of levels, trends, and processes is being properly
interpreted.
What often happens is that major quality problems (which cannot be
corrected) are, in fact, identified, but policy advisers or policymakers
use the data anyway to guide policy decisions. The argument may be
made that, "Well, this is the best we have; data with problems is better
than no data at all."
Several hypothetical and empirical examples of food and nutrition
monitoring indicators that refute this argument are presented by
Haddad, Sullivan, and Kennedy (1992). One case evaluated the effect
of using no targeting or imperfect targeting ("land per capital ) to
identify calorie-deficient households in the Philippines for receipt of a
welfare transfer. The effectiveness of both in predicting nutritionally
at-risk households was compared with an existing data set that showed
actual households with household calorie adequacy of less than 80
percent (of intended beneficiaries of the program). This analysis found
that the levels of undernutrition resulting from an untargeted transfer
was lower than that for a targeted transfer. Haddad, Sullivan, and
Kennedy (1992) conclude that "the costs and benefits associated with
the collection of indicator data will be extremely sensitive to the design
and objectives of the intervention that the data serve."


BASIC APPROACHES TO COLLECTING POLICY DATA

A review of development literature and field experience suggests that
there are 14 basic data-collection approaches, which are listed in Table 6.1.
There are three basic types of approaches: (1) methods that collect data on
the entire population or land area; (2) methods that use a statistically
representative sample of the population (that is, with random sampling
from a formal sampling frame), such that distribution of characteristics
through the larger population is reliably represented; and (3) methods that
use purposive sampling, which characterize the sampled population but do
not provide statistical estimates for the larger population.

Complete Coverage of the Population Under Study
There are four methods that provide full coverage of the population
or land use: censuses, vital statistics records, remote-sensing tech-
niques, and resource inventories or mapping.











Table 6.1-Major methods for data collection


Method of Data Collection


Complete population
coverage



Statistically representative
sample of the population



Purposive sample
of the population


Census
Regular statistical records
Remote sensing
Resource inventories and mapping
Single-visit survey
Group survey
Panel or multiperiod cross-sectional survey
Physical monitoring survey
Key informants
Informal survey
Ethnographic methods
Case studies
Community or focus group interviews
Participatory rural appraisal


Censuses. Population, agricultural, and other censuses collect data on
the entire population or land area of a country, generally through
questionnaires at regular but long intervals. Most developing countries
census at least some variables. Research on policy questions at local or
regional levels may require censusing the relevant population (for
example, individuals for nutrition studies, farms for production studies,
market centers for marketing studies) to either quantify key variables
or provide a sampling frame for more in-depth studies with a sample
group. Comprehensive village studies are based on local censuses as
well, to facilitate the study of community-level and interhousehold
linkages (Connell and Lipton 1977; Lanjouw and Ster 1989).

Vital Statistics Records. For important events that occur infrequently,
countries may set up continual recording systems. Many developing
countries have established national systems for recording basic
demographic statistics such as births, marriages, and deaths. While
expensive to set up, such systems can typically provide timely data for
policy use. Other types of regular statistical records may include
changes in land or water rights.


Study Units










Remote-Sensing Techniques. Developing countries increasingly use
remote-sensing techniques such as low- or high-level aerial photography
or satellite images to obtain information about land use and land use
changes. Some important uses of these tools in policy are to estimate
changes in livestock numbers, forest cover, and area under crops, and
to predict the occurrence of drought. Computer digitization allows
quantitative estimates of land use features that can be linked to other
data generally delineated by administrative boundaries (for example,
census) or to other field data collected with geographic position. This
permits quantitative analysis of relationships between geographic,
demographic, and socioeconomic factors. Remote-sensing methods,
properly ground-truthed (that is, systematically checked for congruence
in the field), can provide a sampling frame for more in-depth research.

Resource Inventories or Mapping. Resource inventories and mapping
are used to assess the existence, composition, use, and access to land,
water, or vegetation resources. Spatial relationships are highlighted. A
range of field surveying techniques are used, depending upon the
resource under study. These methods may be used as a substitute or
complement to remote-sensing techniques. Participatory inventory and
mapping exercises have been successfully completed with local
communities, using local knowledge and classification systems.

Statistical Representation of the Population
The main methods that use statistical techniques to select representa-
tive study units include single-visit surveys, multiperiod panel or
cross-sectional surveys, physical monitoring surveys, and group
surveys. While there are some national sample surveys (often externally
financed), these techniques are usually used for regional, local, or
project studies. These methods may be used to collect both quantitative
and qualitative data. While statistical surveys are often considered to be
more "scientific" than the exploratory studies discussed below, their
claim to rigor depends entirely upon the appropriateness of the
underlying assumptions about the policy issue and the producer or
consumer response, sampling frame and sampling procedure, question-
naire design, and data-collection and analytical methods. For situations
and policy questions where these conditions cannot be met, alternative
methods may be appropriate.

Single-Visit Survey. Single-visit questionnaire surveys use a sample of
households (or other analytical units), drawn statistically. They rely on










recall and direct observations, and often focus on documenting current
attitudes, knowledge, practices, or conditions. They are most widely
used in studies of individual or household food consumption and farm
management but may also be used to sample wholesale or retail
enterprises in marketing studies. They may incorporate physical
measures of key variables, for example, nutritional status, farm area,
or area in crops (Casley and Kumar 1988; Rossi, Wright, and
Anderson 1983; Scherr, Roger, and Oduol 1990).

Multi-Visit Surveys. Multi-visit surveys involve repeated visits, and may
use some keeping of records by households as well as recall observa-
tion (Casley and Kumar 1988; Rossi, Wright, and Anderson 1983). The
two major types of multi-visit surveys are panel studies and multi-visit
cross-sectional studies (points on a continuum of surveys with some
overlapping households). Panel studies are used most often for intensive
data collection over a few years to document intertemporal variation in
consumption, production, input use, and so forth. For surveys at longer
intervals (for example, for impact assessment), the optimal choice of
overlap depends on the covariance between successive observations.
Panel studies theoretically provide better estimates of change, while
repeated, independently sampled cross-sectional studies provide better
estimates of means. Panel studies can deal with contamination of
econometric relationships by unobservable fixed effects, but measure-
ment error can compromise the quality of the estimates to the point
where it is unclear whether panel estimators are superior. For
sampling, measurement errors typically cannot reverse the superiority
of panel estimators (Ashenfelter, Deaton, and Solon 1986).

Physical Monitoring Surveys. Physical monitoring surveys involve
direct quantitative measurement of variables. Examples of such surveys
are health and nutrition assessments of individuals, measurement of
farm-field sizes or crop yields, recording of the grazing patterns of
livestock in communal fields, measurement of soil parameters, and
counting of truckloads of produce arriving at a major market. While
some single-visit studies incorporate selected physical measurements,
most monitoring studies attempt to document changes over time (Casley
and Kumar 1988; Kennedy and Payongayong 1992).

Group Surveys. Some types of information about group or collective
behavior are best obtained from direct interviews with the groups.
Examples of such surveys are interviews with a selected random sample










of village cooperatives to discover terms of informal credit, or of local
irrigation districts to document water allocation systems. A recent
major survey of farming systems in Africa used group surveys as the
primary data-collection technique (Lynam and Dvorak 1992).

Purposive Sampling of the Population
The third set of approaches to data collection involves exploratory
tools, which use purposive rather than random sampling to investigate
the population of interest. "Triangulation" (cross-checking of findings
from one source with several other sources) and purposive sampling of
individuals or groups to systematically explore variation substitute for
statistics in maintaining researcher objectivity. Methods include key
informant interviews, informal surveys, ethnographic studies, case
studies, community or focus-group interviews, and participatory rural
appraisals.
These methods may be used to directly address certain questions of
food-system diagnosis and policy assessment (Barlett 1980; Berry
1986). They may also be used as precursors to larger, statistically
representative surveys to identify key factors for study and criteria for
stratification, and to test indicators and measurement methods for
statistically representative studies.

Key Informant Interviews. Perhaps the most widely used method of
collecting data for policy formulation is key informant interviews. The
interviews typically take place as consultations between policymakers
and individuals identified as knowledgeable about the population or
policy issues. Most researchers use key informants to help in character-
izing local food systems, identifying important sources of variability,
and explaining how policies are implemented. Key informants can be
government officials, development program staff, leaders of farmers'
associations, and community elders, among others. The critical
challenges in the effective use of key informants are the selection of
informants representing a wide range of perspectives, assessment of the
limits of informant experience, and willingness of informants to share
crucial information.

Informal Surveys. Informal surveys are short-duration, loosely
structured questionnaires implemented by a joint team of local people
and technical experts to assess basic conditions, issues, and problems
of farmers, marketing agents, or other groups in a local area or region.
Informal survey methods are typically used to understand the general










character or dynamics of a food policy question. They are common as
a project or policy planning tool, and can be used in the predesign stage
for other data-collection tools. There are many survey approaches,
including random interviews and interviews obtained about farms or
farmers located along a topographic sequence or transect (Chambers
1992). While formal sampling frames are not used, care must be taken
to ensure that the sample includes representatives with different
characteristics of relevance to problem diagnosis or policy response, for
example, assets, market access, and cultural traditions.

Ethnographic Studies. Ethnographic methods are used to collect
information existing in local knowledge systems and to understand
farmers' decisionmaking patterns or principles of community or group
organization. While many ethnographic methods focus on qualitative
understanding, some methods have recently been developed that permit
significant quantification by researchers and informants. Methods
include participant observation, interviews, and structured exercises to
elicit preferences or decision sequences. Examples of application to
policy analysis include community wealth-ranking; farmer decision
trees (for example, for crop selection or use of veterinary services);
identification of local rules and procedures for access to land, tree, or
water resources; evaluation of household risk-management strategies;
and histories of changing land and natural resource use (Barlett 1980;
Plattner 1975).

Case Studies. The term "case studies" is used here for intensive studies
with a relatively small number of study units, usually combining
multi-visit interviews and physical monitoring. Depending upon the
type of study, the "cases" may be individuals, households, plots,
villages, small watershed areas, and so forth. Because sample sizes are
too small or samples are not randomly selected, results cannot be
extrapolated to characterize the entire population. Typically, case
studies focus on quantifying complex factors (for example, intra-
household decisionmaking or management of common property
resources), exploring processes (for example, the relationship between
resource use and land degradation), or studying small, unique popula-
tion groups. Case studies may be used where high data quality is
essential or topics are sensitive. The most serious challenge in
designing case studies is the careful selection of case-study individuals
or sites so that results can be usefully applied to other populations of
policy interest.










Group Interviews. Some policy information can be reliably or efficient-
ly obtained from community groups. Group interviews are also used to
assist in interpretation of research findings. There are two major
approaches to group interviews. One is managed as a dialogue, with the
group answering questions posed by the researcher. The other is a
"focus group interview," in which the researcher acts as a facilitator or
observer of group interaction and discussion on a selected theme. For
effective use of group interviews, it is critical to identify group
members representative of key sectors of the study population and to
manage meetings to ensure open exchange of views and broad
participation (Casley and Kumar 1988).

Participatory Rural Appraisals. The participatory rural appraisal
method offers a different model for data collection and analysis, one in
which the research is carried out by members of the community, not
policy analysts from outside the study area. The role of outside
"experts" is, at most, a facilitative or consultative one. Because
communities are typically neither organized for policy research nor
formally trained, participatory rural appraisal focuses on strengthening
group effectiveness and emphasizing informal methods (Chambers
1992).
Information is typically collected from informal surveys, focus-
group interviews, key informants in the community, and simple
physical monitoring studies, and by ethnographic techniques adapted for
implementation by community members themselves. Recording and
analysis of information (by the community group) emphasize visual
tools such as community maps, time lines, activity profiles, matrix
scoring, diagrams, simple graphs, and charts. Results are shared with
the community verbally and feedback is incorporated (Chambers 1992).


SELECTING A SURVEY APPROACH

It is interesting to note how often the choice of survey instrument
is made perfunctorily, based on disciplinary bias, ideology, or simply
familiarity to the researcher. In this chapter it is argued that a
systematic selection process is critical if reliable, relevant, and
cost-effective data are to be collected. The appropriateness of any of
the above-discussed methods of data collection depends on the scope of
the analysis, characteristics of the data, logistical factors, and user
needs, as summarized in Table 6.2.











Table 6.2-Factors to consider in selecting survey methods

Area Factor


Scope of analysis







Data characteristics







Logistical and cost factors


User needs


Type of analysis
Statistical representation
Unit of analysis
Geographic (spatial) relationships
Community involvement
Long-term potential for use of data

Level of aggregation
Required precision of population estimates
Required accuracy of measurement
Variability in the population
Frequency and period of data collection
Informant sensitivity

Need for prior information
Access to informants
Financial resources
Human resource requirements
Speed of implementation and analysis

Data quality versus speed of analysis
Data quality versus community participation
Short-term versus long-term data needs
Evidence versus proof


Scope of Analysis

A first step in policy research design is to determine the scope of
analysis. Six related factors to consider in selecting a data-collection
approach are statistical representation, type of analysis, unit of analysis,
importance of geographic (spatial) relationships, importance of
community involvement in research, and long-term value of data
collected. Table 6.3 presents the authors' view of the relative value of
different methods in these areas. Of course, the performance of any
particular instrument is dependent upon its specific design. The first
factor, statistical representation, was discussed earlier.










Table 6.3-Choosing data-collection approaches: scope of analysis


Potential for Capacity for Potential for Long-Term Value
Type of Multifactor Geographic Community of Data for
Approach Analysis' Unit of Analysis Analysis Linkages Involvement Multiple Users

Census Levels, Individual, household, Moderate Low Low Very high


Vital statistics

Remote sensing


changes
Levels,
changes
Levels,


changes
Resource inventory/ Levels,
mapping changes
Multi-visit survey Levels,
changes
Single-visit survey Levels,
changes
Physical monitoring Levels,
survey changes,
processes


Group survey

Key informants


census area
Individual, household

Geographic area

Geographic area,
community
Individual, household

Individual, household

Individual plot


Levels, Group, community,
changes geographic area
Changes, Community administrative
processes unit


Low/
moderate
Low/high

Low/
high
High/
very high
High/
very high
High/
very high

Low/
moderate
Low


Low/
moderate
Moderate

Low/
high
Low/
moderate
Low/
moderate
Low/
high

Moderate

Moderate


Very high

High


Low/ High
high
Low/ High
moderate
Low/ Moderate/
moderate high
Low/ High
moderate

Moderate/ Low/
high high
Low/ Low
high


(continued)










Table 6.3-Continued

Potential for Capacity for Potential for Long-Term Value
Type of Multifactor Geographic Community of Data for
Approach Analysis' Unit of Analysis Analysis Linkages Involvement Multiple Users

Informal survey Changes Individual, household plot, Low Moderate/ Low/ Low/
community, geographic area high high moderate
Ethnographic methods Processes Individual, household, Low/ Moderate/ Low/ Low/
community high high moderate high
Case studies Processes Individual, household plot High/ Moderate/ Moderate/ Low/
community very high very high high high
Community/focus-group Changes, Community, geographic area Low/ Moderate/ Moderate/ Low/
interviews processes moderate high high moderate
Participatory rural Levels, Individual, household, Low Moderate/ Very high High
appraisals changes, community high
processes


Note: High indicates that the method has a high potential to address the indicated use, whereas low indicates that the method is not well suited to the
indicated use. These terms reflect the authors' judgment only.
S"Changes" refers to changes in levels, and "processes" to processes of change.










Type of Analysis. All the methods providing complete population
coverage or statistical representation can be used to estimate "levels"
(of consumption, production, input use, market integration, and so
forth) in the population as a whole. All the general interviewing
methods can help to identify "trends" (changes in levels) in these
variables over time, although only multiperiod data collection methods,
properly implemented, can accurately quantify these. Only the
multiperiod surveys and some of the exploratory methods are good at
illuminating the "processes" of change, that is, the underlying factors
that explain changes in the food system or account for the nature of
response to policy.
A related consideration is the interest of the data user in quantita-
tive, multifactor analysis, that is, statistical testing of relationships
between variables from a given study unit. Agricultural census data, for
example, provide information on total production, total number of
farms, and average production per farm, but often do not permit
analysis of patterns of farm production, because of aggregation.
Surveys and case studies are the best tools for developing integrated
data sets for multifactor analysis, as are some types of inventories and
mapping where data can be stored quantitatively.

Unit of Analysis. The unit at which behavior or policy response will be
investigated is another factor in designing research methods. As Table
6.3 shows, some data-collection methods are more suitable than others
for characterizing different units, such as individuals, households, field
plots, groups, communities, or geographic areas.
The unit of analysis should be selected at the level required for
meaningful policy analysis, which is not necessarily the unit considered
in policy implementation. Decisions about appropriate aggregation of
data will follow. For example, input subsidies may be implemented at
the level of municipal farmers' associations, but while aggregate
municipal census data may be adequate to document levels of input use,
they cannot explain patterns of use or input response. The impact of
such subsidies on production must be studied at the level of individual
farms. Further disaggregated data collection at the field-plot level
would not be necessary so long as farmers can reliably report total
levels of input use and response.
Clear definitions of the unit of analysis are critical, as they will vary
depending upon the policy question. For example, in assessing nutrition
policy impact, a household might be defined as all individuals who eat
meals together, while in a study of farm management, a household










might be defined to include all individuals who live and participate in
production on a given farm, even though they reside and eat meals
separately. Complexities arise when the unit of analysis changes or is
dissolved, such as in studies of displaced people.

Geographic (Spatial) Relationships. Another factor is whether spatial
relationships between study units must be considered in answering
policy questions. Household surveys may be stratified according to
geographic characteristics (for example, a specified range of distance
from a road or from forest or water resources), or geographic sampling
procedures may allow for other types of analysis. In general, however,
they have not been used in this way. Obviously, remote-sensing and
mapping methods, as well as physical monitoring methods whose
sampling unit is the field or forest plot, are designed to describe
geographic units. To obtain policy-relevant information on land or
resource management, however, these methods need to be integrated
with (and interpreted through) data on households, communities, project
participants, or management entities.

Community Participation. The extent of community involvement in data
collection and analysis will influence the selection of data-collection
methods. Experience in development programs and community
development actions suggests the value of community input in data-
collection design, implementation, and analysis. Different data-
collection methods facilitate this involvement.
There is little potential for community input into census, vital
statistics, or remote-sensing activities. Statistical surveys can accommo-
date community involvement in identifying priority data to collect and
perform relevant stratification, designing user-friendly questionnaires,
collecting the data, interpreting analysis, and providing feedback.
However, they require formal technical expertise for sample selection,
resolution of technical issues in survey design, and data analysis.
Nonstatistical methods, particularly those developed in participatory
rural appraisal, are most amenable to community planning, implementa-
tion, and analysis.

Long-Term Potential for Data Use. A final consideration is the
expected long-term value of the data to be collected. While censuses
and vital statistics are very expensive, the data they provide can support
a wide range of food policy decisions. Similarly, representative and
empirically rich data from remote-sensing techniques, resource










inventories, and some multi-visit surveys can potentially be mined over
the long term by researchers, development planners, and policymakers.
Establishment of the necessary infrastructure for censuses, vital
statistics records, remote-sensing techniques, and permanent household
surveys or physical monitoring systems can also enhance national
capacity for data collection and analysis. Thus, while some of the more
expensive methods may not be needed to answer immediate policy
questions, they may be justified for their longer-term value as a
multipurpose statistical data base. On the other hand, this value should
not be assumed; in most countries, there exists a large body of data that
is neither analyzed nor even accessible to policy researchers with an
interest in their findings.

Data Characteristics
Specific characteristics of the data needed will also influence the
collection method used, as shown in Table 6.4. Five factors are
important to consider: need for quantitative population estimates,
required accuracy of measurement, variability in the population,
frequency and period of data collection, and informant sensitivity.

Importance of Quantitative Population Estimates. A fundamental factor
in selecting data-collection methods is the importance of obtaining
quantitative estimates of levels, trends, or processes in the whole
population of policy interest. Where a high level of confidence in
quantitative estimates is essential and deemed to be worth paying a high
price, then methods with complete population coverage, such as
censuses, vital statistics records, remote-sensing techniques, or
inventories that cover all relevant land area or resources, would be
preferable.
Where greater error in quantitative estimates is acceptable,
statistically representative surveys, such as multi-visit surveys,
single-visit surveys, physical monitoring surveys, or group surveys,
may be used. These methods are appropriate where the actual level of
policy response is considered critical (for example, the effects of a
change in price on level of crop production), or where the researcher
wishes to test statistically the response patterns of the larger population.
The precision of the estimates will depend upon the sample size, the
survey instrument, and underlying variation.
If approximate estimates of levels or indications of directions in
trends are sufficient for policy purposes, then interview methods with
carefully selected, but not statistically representative, individuals and










Table 6.4-Choosing data-collection approaches: data characteristics


Potential for Potential to Identify Frequency Potential Potential
Quantitative Variability in of Data to Capture Accuracy of
Approach Population Estimates the Population Collection Sensitive Data Measurement

rCensu Very high Moderate/high Low Low/moderate Low/moderate


Vital statistics


Remote sensing

Resource inventory/
mapping

Multi-visit survey


Single-visit survey

Physical monitoring
survey

Group survey

Key informants


Very high

High/very high

Low/moderate


High

High


High

Low/moderate


Very high

Low/moderate

High


High/very high


High/very high

High/very high


Moderate/high

Low/high


Low/moderate

Low

Low


Moderate/
very high

Low

Moderate/
very high

Low/moderate

Low


Low/moderate

Low/moderate

Moderate/high


Moderate/high


Low/moderate

Moderate/high


Moderate

Low/moderate


Low/high

High/very high

Low/very high


Low/high


Low/moderate

High/very high


Low/high

Low/moderate

(continued)










Table 6.4-Continued


Potential for Potential to Identify Frequency Potential Potential
Quantitative Variability in of Data to Capture Accuracy of
Approach Population Estimates the Population Collection Sensitive Data Measurement

Informal survey Low/moderate Low/high Low Low/high Low/moderate

Ethnographic methods Low/moderate Moderate/ Low/very high Very high Very high
very high

Case studies Low Low/moderate Low/high High Very high

Community/focus-group Low Moderate/high Low Moderate/high Low/high
interviews

Participatory rural Moderate/ Moderate/high Low/high Moderate/ Low/high
appraisals very high very high


Note: High indicates that the method has a high potential to address the indicated use, whereas low indicates that the method is not well suited to the
indicated use. These terms reflect the authors' judgment only.










groups may be sufficient. There are situations when statistical represen-
tativeness would be desirable, but because insufficient information
exists to select an appropriate statistical sample or design a focused
questionnaire, statistical surveys provide a spurious precision, and
nonstatistical methods may be preferable.
Where the primary interest is in understanding the nature of food
policy problems or the mechanisms by which policies take effect, rather
than in quantifying population estimates, exploratory methods (case
studies, ethnographic methods, and some interview methods) may be
most suitable and cost-effective.

Required Accuracy ofMeasurement. Research planning should consider
the degree of accuracy that is needed and relevant for key variables in
policy analysis. It is useful to recall the principles postulated by
Chambers (1992) of "optimal ignorance" (not trying to find out more
than is needed) and "appropriate imprecision" (not measuring more
accurately than is necessary for practical purposes). In the same vein,
an anonymous observer has noted that "it is better to be approximately
right than exactly wrong."
The potential of different methods in ensuring accuracy is deter-
mined by the innate characteristics of the method, the difficulties of
accurate measurement for particular data, and the potential for quality
control in data collection. Interview methods tend to be less accurate
than physical monitoring methods in estimating physical factors such as
farm size or crop yields, as long as the monitoring methods are field-
validated. There is usually a sharp trade-off, however, between
accuracy and cost. Poorly implemented physical methods are both
costly and unreliable, so large surveys must pay careful attention to
quality control. It may be necessary to sacrifice sample size in order to
reduce nonsampling errors. Data of the highest quality tend to come
from case studies, often because the principal researcher is most
intimately involved in the data collection.
Accuracy of qualitative information such as farmer attitudes and
practices or historical recall is determined more by the quality of
interviewing and by the use of terms, benchmarks, and analytical
categories that are locally familiar and amenable to recall or classifica-
tion. The highest-quality data tend to come from long-term work with
small samples. Populations with low literacy or numeracy may need
special types of data collection.

Variability in the Population. A third important data characteristic is









population variability. Where policy action is likely to be targeted, or
where there is concern with differential response to policy action by
different subgroups of the population (individuals, households, field
plots, communities, and so forth), then the data-collection tool should V
capture relevant variability. Only vital statistics, which are maintained
by geographic location, ethnicity, and so forth, intrinsically capture
population variability. Raw census data can be used to assess variabili-
ty, but published data provide cross-tabulations for only a few
variables, and much variability is disguised under preselected group-
ings. Other data-collection methods can be designed to assess variability
through careful stratification and selection of study sites. Identification
of the type and degree of variability may require considerable time and
systematic effort during the survey planning and design stage. Unfortu-
nately, it is common for this step to receive minimal attention.
The issue of variability is a critical one for sample size. Failure to
pre-assess variability commonly leads to a sample size for the strata
that is inadequate for making statistically valid subpopulation estimates.
If the cost of having large sample sizes for complex surveys is
excessive, an alternative is to implement a much simpler survey that
effectively documents the relative importance of different types of
variability, while using case studies to explore differential responses for
important subgroups.

Frequency and Period of Data Collection. Some types of data are
highly variable over time, so that collecting such data at a single point
in time may be inadequate or even seriously misleading for policy
purposes. Such cases require multi-visit household surveys, group
surveys, physical monitoring, or case studies. The frequency and time
period over which data are collected theoretically depend on the degree
of seasonal and annual variability, and on the ability of researchers or
policymakers to identify and correct for seasonal and other fluctuations
in generating point estimates.
Agricultural inputs, labor use, and crop yields may vary significant-
ly from season to season and from year to year, requiring a
several-year cycle of data collection to ascertain patterns. Understand-
ing changes in composition, quality, or use of natural resources (soil,
forests, and so forth) may require collection of data at infrequent
intervals during the year, but spread out over many years. Nutritional
status studies for famine relief may be one-shot efforts, while a risk-
monitoring program needs regular checks to predict undernutrition.










Informant Sensitivity. A fifth factor in method selection is the potential
of the instrument to handle culturally or politically sensitive informa-
tion. Some kinds of information are considered to be inappropriate for
sharing with strangers because of cultural norms of privacy, because of
fear of negative repercussions should the information be made public,
or because the information is considered embarrassing.
Actual physical monitoring or mapping may reveal some facts that
are not acknowledged publicly. The privacy and personal-interaction
elements of case studies can provide the trust needed to reveal sensitive
information as well as the opportunity for cross-checking answers to
sensitive questions. While focus-group interviews are a poor occasion
for eliciting sensitive personal information, they can provide an
opportunity for revealing general community conflicts. Selecting
culturally suitable enumerators and providing training in interviewing
techniques can substantially improve ability to capture sensitive
information from surveys. Effectiveness in obtaining sensitive informa-
tion can often be confirmed through complementary ethnographic study.

Logistical and Cost Factors
The above sections discussed theoretical factors that should guide
data collection. This section looks at five logistical factors that could
argue for choosing a "second-best" data-collection method (Table 6.5).
These factors are need for prior information, access to informants,
financial resources, human resource requirements, and need for speed
of implementation and analysis. As one workshop participant noted, a
"second best" method is the "first best" method under existing
restrictions.

Need for Prior Information. For remote-sensing and rapid appraisal
methods, little or no prior information is needed to begin data
collection, other than identification of priority areas to tackle. Census
data and vital statistics are collected uniformly for all areas, with little
local adaptation of questionnaires. For group interviews, group surveys,
case studies, or inventories, prior information is needed for selection
of interviewees or cases and for design of questionnaires.
However, substantial information must be available before study
design can begin for more detailed case studies and for physical
monitoring and household surveys. This includes an understanding of
priority data needs and the method to be used for analyzing the data,
relevant stratification, appropriate sampling frames, local terminology
and categories of classification, variances of key population and other










Table 6.5-Choosing data-collection approaches: logistical and cost factors


Approach

Census

Vital statistics

Remote sensing

Resource inventory/mapping

Multi-visit survey

Single-visit survey

Physical monitoring

Group survey

Key informants

Informal survey


Prior Infor-
mation Needed
for Design

Moderate

Moderate

Low

Moderate

Very high

High

High

Moderate/high

Low

Low


Problems Posed by
Difficult Access
to Informants

Very high

High

Moderate

High/very high

Very high

Very high

Very high

High

Low/moderate

Moderate/high


Financial
Resources
Required

Very high (start-up)

Very high (start-up)

High

Moderate/high

High

Moderate/high

High

Low/moderate

Low

Low/moderate


High-Skilled
Human Resources
Required

High

Moderate

High

High

High

High

High

Low/moderate

Moderate/high

Moderate/high


Timeliness

Moderate/low

High

Moderate

Low/moderate

Low

Moderate

Moderate/high

Moderate

Very high

Very high


(continued)











Table 6.5-Continued

Prior Infor- Problems Posed by Financial High-Skilled
mation Needed Difficult Access Resources Human Resources
Approach for Design to Informants Required Required Timeliness

Ethnographic methods Low/moderate Moderate Moderate/high High/very high Low/high

Case studies Moderate/high Low/high Moderate Moderate/high Low/moderate

Community/focus-group Low/moderate Low/high Low/moderate Low/moderate High
interviews

Participatory rural Low Low/high Low Low/moderate Low/moderate/
appraisals high


Note: High indicates that the method has a high potential to address the indicated use, whereas low indicates that the method is not well suited to the
indicated use. These terms reflect the authors' judgment only.










variables for sample-size selection, and requirements for effective
collection of specific types of data (for example, period of reliable
recall, frequency of events to be observed directly, the appropriate
informant for different types of data). Researchers must have a fairly
clear understanding of the focus of the study, for example, patterns of
land use or food consumption. In areas for which descriptive, empirical
data are limited and the local situation is poorly understood, it may be
best to substitute nonstatistical, exploratory methods for high-precision
methods, or establish a two-phase study with nonformal methods used
first (Scherr, Roger, and Oduol 1990).

Access to Informants. Difficulties of access to informants can also
constrain the selection of data-collection methods. Survey methods can
be especially biased where difficulties in reaching sampled units result
in incomplete data or failure to interview or monitor important strata.
In some countries there are restrictions on rural travel or on access by
enumerators to women. In such situations, it may be necessary to use
methods that minimize field movements (for example, remote sensing,
case studies, or ethnographic studies) or number of interviews (for
example, group surveys or interviews, single-visit surveys).

Financial Resources. A key factor in the selection of data-collection
methods is, of course, the financial resources required relative to the
resources of the agency collecting the data. Censuses and vital statistics
records are very expensive, particularly the start-up. Statistical survey
methods and remote sensing may also be quite expensive; as a rule of
thumb, they should be considered only where it has been determined that
cheaper methods would be inadequate. Rapid appraisals, community
interviews, and case studies tend to be cheaper, as do brief surveys. If
these latter methods are to be of high quality, however, more time may be
required for planning and organization than is commonly recognized.
Because of the costs associated with surveys, there is a common
urge to add to the survey questionnaire excessive questions of lower
priority or of interest to other agencies under the assumption that the
marginal cost of that additional information is very low. While this is
true to a limited extent, it poses several dangers. For instance, due to
fatigue or loss of attention, the quality of respondent information may
decline as the length of the survey increases. Or the sampling frame
and sample size needed for statistical analysis of added variables may
differ from the first, compromising the quality of the original survey,
without yielding reliable estimates of the new variables.










Human Resource Requirements. The availability of critical human
resources may also dictate choice of method. If the necessary skills are
not available from the early design stage, data-collection methods
should not be implemented by inexperienced or untrained people.
Census, remote sensing or mapping, and statistical survey methods
require highly specialized skills in design, data handling, and analysis.
While informal methods may require fewer computer or statistical
skills, they may require considerable judgment and experience in
prioritizing data-collection activities and sequences and in interpreting
findings (Scherr and Muller 1991).
High human resource requirements where these are limited do not
necessarily preclude use of a given method. Rather, the potential
long-term value and effectiveness of investment in training for human
resource development and institutional capacity-building should be
considered and factored into cost estimates.

Timeliness. In many cases, there is a trade-off between the desire for
data precision and the need for speed in implementation and analysis.
Rapid-appraisal surveys, community interviews, and some types of
physical monitoring surveys can have a fairly rapid turnaround time.
Multiperiod household surveys and case studies, like censuses and
detailed resource inventory work, can have a very long period between
data needs identification, data collection, and analysis. Single-visit
surveys can have rapid turnaround where sampling frames exist,
questionnaires are simple, and arrangements for on-site data entry and
analysis are made. Where speed is critical, such as in famine relief, it
may be advisable to accept less precision in estimates by using smaller
sample sizes, more limited data sets, informal data analysis methods,
or all.

User Needs
The final-yet one of the most important-criterion for selecting
data-collection methods involves the needs of the data users. Different
groups of policymakers, policy analysts, or policy lobbyists have
different information needs. Indeed, the locus of decisionmaking may
be in national or local governments, the private sector, or local
communities.
Five major groups of data users can be identified: those who organize
relief services, those who plan or evaluate development programs or
projects, those who plan or evaluate community development action, those
who formulate development policy, and those who carry out basic research










to understand the functioning of the food system and policy effects. Key
differences in their needs and priorities involve the relative trade-offs
between data quality and timeliness, data quality and community participa-
tion, short- and long-term data needs, and the need for rigor in testing
findings. Table 6.6 suggests the relative value of different data-collection
methods for different user groups.

Relief Services. Those who organize relief services place a premium on
accurate targeting of populations at risk and effectiveness of interven-
tion, and if the relief service is for famine or other emergencies, a
premium is placed on speed of data collection and analysis. Household
surveys and physical monitoring can be used to answer targeting
questions, with more streamlined designs for emergency work than for
policy planning. For example, nutritional status, which might be
measured for studies of development policy through detailed food
consumption records, might be measured for emergency relief work
simply by single-visit studies, using one or two key physical indicators.
Compromises need to be made in difficult survey circumstances, but
much lasting information can be derived, even in the short run (Teklu,
von Braun, and Zaki 1991; Webb, von Braun, and Yohannes 1992;
Riely 1992). Group surveys (for example, of holders of food stocks)
and community interviews can also be used to assess means and
effectiveness of delivery (Kennedy and Payongayong 1992). Figure 6.1
illustrates the importance of short-term factors and timeliness in relief
work relative to development work.

Development Programs and Projects. Those who plan and implement
development programs and projects, both staff members and participat-
ing communities, are interested in providing timely feedback to
improve project design or management. Quick turnaround time,
transparency of methods for the community and field staff, and capacity
for community participation in identifying indicators and interpreting
analysis are important. The need for impact data to satisfy agency or
donor sponsors of the project may require additional data collection,
but that audience should not drive the data-collection process. Almost
all of the local-level data-collection methods may be used, depending
upon information needs. Caution is required for use of formal
household surveys, however, as long turnaround times can severely y
limit their value as monitoring tools, and their high cost and time
requirements tend to crowd out use of interactive tools (see Scherr and
Muller 1991 and Feinstein in this volume).










Table 6.6-Overall value at data-collection methods for different uses


Approach

Census

Vital statistics

Remote sensing

Resource inventory/mapping

Multi-visit survey

Single-visit survey

Physical monitoring survey

Group survey

Key informants

Informal survey


Relief
Services

Moderate

Moderate

Low

Low

Very high

Very high

Very high

High

Moderate/high

High/
very high


Development
Programs

Low

Low

Moderate/high

Very high

Low/high

High

Very high

Very high

Moderate/high

High/
very high


Community
Action Programs

Low

Moderate

Low

Very high

Low/moderate

Moderate/high

Moderate/high

Moderate/high

High

Low/high


Development
Policy

Very high

Very high

Very high

Very high

Very high

Very high

Very high

Very high

Very high

Very high


Basic Policy
Research

Very high

Very high

Very high

Very high

Very high

Very high

Very high

Very high

Very high

Very high


(continued)










Table 6.6-Continued

Relief Development Community Development Basic Policy
Approach Services Programs Action Programs Policy Research

Ethnographic methods Moderate/high Moderate/high Very high Very high Very high

Case studies Moderate Very high Very high Very high Very high

Community/focus-group
interviews High Very high Very high Very high Very high

Participatory rural Moderate/high Moderate/ Very high Low/ Low/high
appraisals very high moderate


Note: High indicates that the method has a high potential to address the indicated use, whereas low indicates that the method is not well suited to the
indicated use. These terms reflect the authors' judgment only.











Figure 6.1-Considering priority data needs for food policy


/
/
/
/

Utility of For short-term For long-term
information policy action development
/ policy
















Nutrition Food Food Agricultural Natural
and health prices stocks production resource
status base for
agriculture
Time scale of change in key variables



Community Development. Local leaders and community participants in
community development action have different information needs.
Community interviews, community resource inventories, case studies,
and rapid appraisals are effectively used to generate useful information
and improve the capacity of local people to organize and assess
effective action. Simple physical monitoring and household surveys
have been widely implemented on a participatory basis, with only
modest training and support from outside technical experts.

Development Policy. Those who design food, agriculture, or resource-
development policy tend to be interested primarily in accurate assess-
ment of sectoral performance and problems, and in understanding the
likely response to changing incentives, policies, or direct interventions.
/ /
/ /
/ /
/
/
/
/
/
/
/
/
/





Nutrition Food Food Agricultural Natural
and health prices stocks production resource
status base for
agriculture
Time scale of change in key variables



Community Development. Local leaders and community participants in
community development action have different information needs.
Community interviews, community resource inventories, case studies,
and rapid appraisals are effectively used to generate useful information
and improve the capacity of local people to organize and assess
effective action. Simple physical monitoring and household surveys
have been widely implemented on a participatory basis, with only
modest training and support from outside technical experts.

Development Policy. Those who design food, agriculture, or resource-
development policy tend to be interested primarily in accurate assess-
ment of sectoral performance and problems, and in understanding the
likely response to changing incentives, policies, or direct interventions.










As indicated in Figure 6.1, there is a greater relative interest in factors
influencing long-term development trends rather than short-term
conditions.
Any data-collection method may be useful, depending upon the
policy question. Typically, policymakers are satisfied with "good
enough" information-a minimum that allows them to make specific
decisions with some confidence that serious negative repercussions will
not unexpectedly result. In practice, for lack of financial and technical
resources, policymakers in poorer countries depend most heavily on
key informants, single-visit and informal surveys, group surveys,
community interviews, and case studies. Strong efforts are needed to
strengthen national capacity to utilize these tools more rigorously and
systematically. Assuming that local-level responses do "sum up" to
sectoral response, intermittent microstudies aimed at improving and
updating the understanding of "response parameters" are needed. If
strategically selected, these more intensive studies can be highly
complementary with less-intensive data-collection methods used more
broadly to inform policy.

Policy Research Community. Professional policy researchers seek to
rigorously confirm hypotheses about the functioning of the food system
or the effectiveness of food policy. Again, all data-collection methods
may be suitable for different aspects of research, and all methods can
contribute to the rigor of analysis if properly used. There remains,
however, a strong inclination toward statistical survey methods to
permit reliance on statistical analysis to accept or reject hypotheses.
While this inclination is often justified, it may also lead to some
unfortunate results. There may be underutilization of more empirically
rich direct monitoring and in-depth case studies, which, due to cost of
data collection, can only be carried out on a smaller scale. Qualitative
and interactive methods such as focus-group interviews may be
neglected or poorly implemented because of the lack of publication
outlets. Only a few specialized institutions, such as national statistical
services or the International Food Policy Research Institute (IFPRI),
can command the resources to combine in-depth data collection and
qualitative studies with statistical coverage. At the same time, statistical
methods can create a false impression of rigor even when poorly
designed and implemented. Yet, peer reviews most often concentrate
on issues of data analysis rather than challenging basic assumptions,
definitions, sampling frame, selection of strata, or questionnaire
designs.










CONCLUDING THOUGHTS AND ISSUES FOR THE FUTURE

This review confirms that there are no simple rules to guide data
collection for food policy. It is difficult to identify "minimum data
needs" for relief, development policy, project or community monitor-
ing, or research, outside the context of particular policy decisions.
There is no real substitute for accurate conceptualization of the possible
policy linkages, grounded firmly in a practical understanding of the
local, regional, or national context. Given the costs of uninformed or
misinformed policymaking, it behooves the community of policy
researchers and analysts to make decisions about data-collection
methods more systematically and critically.

Refining Survey Approaches
The growing demand for policy research clearly calls for greater
attention to training of national researchers and policy analysts in use
of a wider range of data-collection methods. It may also require a
revision of professional guidelines for data collection and a willingness
to borrow and adapt methods across academic disciplines. Tentative
recommendations are drawn from this paper.

Appropriate Use of Statistical Methods. Policy analysts should be more
cautious about recommending use of complex statistical survey
methods, demanding that the necessary conditions to maintain quality
are met. Policy researchers should be encouraged to spend more time
in the field to develop appropriate designs for data-collection instru-
ments, and then be personally involved in extensive protesting and field
quality control. Where reliance is placed on nonstatistical methods to
inform policy decisionmaking, more systematic identification of data
sources and greater rigor in implementation and analysis are needed.
Multimethod data-collection approaches that strategically combine
statistical and nonstatistical data-collection methods may be most
efficient in the field, while providing opportunities to explicitly
cross-check findings from different sources.

Guidelines and Training for Data Collection. Researchers should be
encouraged to expand and improve upon the matrices presented in
Tables 6.3-6.6 to develop decision rules for selection of data-collection
methods in particular policy areas. Greater national and international
investment is needed in the training of policy researchers and analysts
from developing countries in a wide range of methods and in develop-
ing criteria for methodology selection and design.










National Data-Collection Strategies. The policy research community
should work to develop recommendations about the relative balance in
use of different data-collection methods to inform national policy (for
example, censuses, vital statistics, national household surveys, remote
sensing) that is appropriate to countries with different endowments of
research and financial resources. Ways by which international
household data-collection efforts, such as the World Bank's Living
Standards Measurement Study project, the nutritional monitoring
systems of the Food and Agricultural Organization of the United
Nations, or the International Institute of Tropical Agriculture-
Rockefeller Foundation study of African farming systems, can more
effectively complement and strengthen national data systems should be
found.

Research Design as an Art. Finally, it should be emphasized that
although research design is predicated on scientific method and
systematic planning, the process is, in many ways, still an art. The
selection of methods and design of data-collection instruments depend
heavily on judgment, drawn from experience in the complexities of
field reality as well as an understanding of the requirements, limits, and
potentials of different analytical methods. This fact does not argue
against continued progress in systematic assessment of the appropriate
use of different research methods. Rather, it highlights the importance
of treating research design not as a routine exercise, but with focused
attention, reflection, and imagination, always keeping in mind both the
data users and those whose lives will be affected by policy research
results.

Selected Issues for the Future
In considering these issues, their applicability to new policy
problems must also be explored. There are two elements of importance:
local social organization, and sustainable natural resource management.

Local Social Organization. There is growing recognition by policy-
makers of the importance of local social organization. Local organiza-
tion is seen as an important mediating factor for household decision-
making behavior, for example, in access to resources, local rules of
land use and consumption, or mobilization for new institutional forms
and technology. Where decentralization of policymaking to the regional
or local level is occurring, or where national policy seeks to achieve its
objectives through local nongovernmental institutions, new information










needs will develop. Policymakers will not only wish to have direct
information on the changes in institutional forms and action but also on
the resulting influences on and from household behavior.
Currently, most household surveys are aggregated for different
government administrative units (village, district, state, and national).
An important research question is how to build into survey methods
procedures and analyses that reflect the influence of social organization.

Sustainable Natural Resource Management. There is also growing
concern with devising policies that support sustainable natural resource
management. This poses a host of theoretical and practical challenges
for data collection. One such challenge is interpreting socioeconomic
data spatially, that is, assessing household decisionmaking, livelihood
strategies, nutritional status, and so forth, in light of geographic
features such as access to forest, grazing, and water resources (as well
as infrastructure and market access). Agricultural production decisions
and variability among farmers will need to be assessed in terms of
physical position in the local landscape and agroecological conditions
on individual farms.
Another challenge will be to have a much stronger temporal
focus, with monitoring over time not only of socioeconomic change,
but also of underlying changes in quality and composition of the
resource base. Separating the roles of biophysical, organizational,
and economic factors in inducing observed changes in production,
consumption, and livelihood strategies will pose complex problems
that may not be able to be resolved statistically. While eventual
formal modeling of these relationships may be a goal of the research
community, there may be mathematical constraints to developing
policy-relevant models.
Meanwhile, the research community is in the earliest stages of
identifying appropriate indicators of natural resource status or meaning-
ful categorizations of landscape for purposes of nutrition or food
production. Extensive theoretical and empirical work is needed to
identify practical indicators and sampling strategies. Furthermore, the
data intensity of these exercises (number of variables, length of data-
collection period, higher skill requirements at the field level) will
increase the cost of data collection significantly. Can new methods be
developed, or strategic combinations of existing methods be devised,
to provide useful guidance for policy decisions? As with current policy
issues, the requirements of data precision, scope, and logistics will
affect the optimal method for different data users.







77


National, regional, and local monitoring systems for measuring the
extent, quality, and use of natural resources, both on-farm and in
community or state lands, need to be developed. While this will mainly
be the province of natural resource scientists, it is critical that social
scientists participate in the design of data-collection instruments to
ensure that results will be usable in policy analysis and linkable to
household socioeconomic data. Key policy issues like the devolution of
resource management responsibility to local common property regimes
or comanagement between local groups and public agencies require
linking data on resource status to data on resource management
responsibility and decisions. These issues pose the newest challenges
to the research community.










REFERENCES

Ashenfelter, O., A. Deaton, and G. Solon. 1986. Collecting panel data
in developing countries: Does it make sense? Living Standards
Measurement Study Working Paper 23. Washington, D.C.: World
Bank.

Barlett, P. F., ed. 1980. Agricultural decisionmaking: Anthropological
contributions to rural development. New York: Academic Press.

Berry, S. 1986. Macro-policy implications of research on rural
households and farming systems. In Understanding Africans' rural
households and farming systems, ed. J. L. Moock, 199-216.
Boulder, Colo., U.S.A.: Westview Press.

Casley, D. J., and K. Kumar. 1988. The collection, analysis, and use
of monitoring and evaluation data. Baltimore, Md., U.S.A.: The
Johns Hopkins University Press.

Chambers, R. 1992. Rural appraisal: Rapid, relaxed, and participatory.
Institute of Development Studies, University of Sussex, Brighton,
United Kingdom. Draft.

Connell, J., and M. Lipton. 1977. Assessing village labor situations in
developing countries. Delhi: Oxford University Press.

Haddad, L., J. Sullivan, and E. Kennedy. 1992. Identification and
evaluation of alternative indicators of food and nutrition security:
Some conceptual issues and an analysis of extant data. Final report,
Food and Nutrition Monitoring Project. International Food Policy
Research Institute, Washington, D.C. Mimeo.

Kennedy, E., and E. Payongayong. 1992. Inventory of food and
nutrition monitoring systems. Final report, Food and Nutrition
Monitoring Project. International Food Policy Research Institute,
Washington, D.C. Mimeo.

Lanjouw, P., and N. H. Stern. 1989. Agricultural changes and
inequality in Palanpur 1957-1984. Discussion Paper 24. London:
London School of Economics, Development Economics Research
Program.










Lynam, J., and K. Dvorak. 1992. Theory and construction of a
socioeconomic data base for African agriculture. Paper presented to
CGIAR social scientists, 17-20 August, The Hague, Netherlands.

Plattner, S., ed. 1975. Formal methods in economic anthropology.
Washington, D.C.: American Anthropological Association.

Riely, F. Z. 1992. Household responses to recurrent drought: A case
study of the Kababish pastoralists in Northern Kordofan, Sudan.
Famine and Food Policy Discussion Paper 6. Washington, D.C.:
International Food Policy Research Institute, Food Consumption and
Nutrition Division.

Rossi, P. H., J. D. Wright, and A. B. Anderson. 1983. Handbook of
survey research. Quantitative Studies in Social Relations Series. San
Diego, Calif., U.S.A.: Academic Press.

Scherr, S., and E. Muller. 1991. Technology impact evaluation in
agroforestry projects. Agroforestry Systems 13: 235-257.

Scherr, S., J. Roger, and P. Oduol. 1990. Surveying farmers'
agroforestry plots: Experiences in evaluating alley-cropping and tree
border technologies in western Kenya. Agroforestry Systems 11:
141-173.

Teklu, T., von Braun, J., and E. Zaki. 1991. Drought and famine
relationships in Sudan: Policy implications. Research Report 88.
Washington, D.C.: International Food Policy Research Institute.

Webb, P., J. von Braun, and Y. Yohannes. 1992. Famine in Ethiopia:
Policy implications of coping failure at national and household
levels. Research Report 92. Washington, D.C.: International Food
Policy Research Institute.












Using Nationally Representative
Household Surveys for Food Policy
Analysis: An Examination of the World
Bank's Living Standards Measurement
Study Surveys


Margaret E. Grosh and Paul Glewwe



In this chapter the scope for large-scale, nationally representative
surveys in food policy analysis is considered, with special reference to
the Living Standards Measurement Study (LSMS) surveys developed
at the World Bank. The essential features of LSMS surveys and their
history are given, a set of strategic questions related to aspects of
utilization and institutionality is discussed, and future directions for
LSMS-type surveys are suggested.


ESSENTIAL FEATURES OF AN LSMS SURVEY

LSMS surveys were established with the objective of collecting
household-level data that could be used to evaluate the effects of
various government policies on living standards in developing coun-
tries.' Because living standards can be measured in many dimensions,
and because these dimensions are related to each other, LSMS surveys
collect data on all major aspects of household well-being. The surveys
are designed to support a wide variety of analyses of the causes and


The findings, interpretations, and conclusions expressed in this paper are entirely those
of the authors and should not be attributed in any manner to the World Bank, to its
affiliated organizations, or to its Board of Executive Directors or the countries they
represent.

For a more detailed description of the LSMS, see Glewwe 1990. For a description
of the prototype questionnaire, see Grootaert 1986. For a description of the prototype
field work plan, see Ainsworth and Munoz 1986.










consequences of household behavior, as opposed to simply measuring
levels of certain variables.
The typical LSMS survey consists of (1) a multipurpose household
questionnaire (collecting data on demographics, housing, education, health,
anthropometrics, employment, migration, agricultural activities, nonagricul-
tural self-employment activities, food and nonfood consumption, fertility,
income and savings); (2) a community (village) questionnaire for rural
areas (collecting data on the community's economic structure, availability
of schools and health clinics, and local agroclimatic characteristics); and (3)
a price questionnaire. Thus, the surveys collect a large amount of data
from each household. The prototypes have relatively small sample sizes
(1,600-4,800 households per country), compared with large sectoral
surveys. Samples are designed to minimize sampling error in estimates of
a few key variables.
The emphasis in data collection is on high quality to ensure
minimum nonsampling error. There is a lot of supervision used to
ensure the high quality of the data collected; usually, there is one
supervisor for every two interviewers. The year-round fieldwork
schedule allows work to be conducted by only a few well-trained,
carefully selected, professional interviewers. Almost all responses on
the questionnaires are preceded, which eliminates coding or transcrip-
tion errors and time delays. Customized computer programs are used
to check data for inconsistencies. Data entry is done in the field, and
interviewers return to households and clarify problems detected by the
computer program.
The simultaneous entry and checking of data during the fieldwork
allows for quick turnaround between fieldwork and analysis. First
descriptive reports have been completed within two months of
completion of the last interview. The turnaround between the identifica-
tion of the need for information and the analysis can, however, be
much longer. The prototype fieldwork organization extends data
collection over a full year, and, in some countries, several months of
preparation are needed prior to the fieldwork for preparing the
sampling frame, designing the questionnaire, training the field teams,
and procuring equipment.


HISTORY OF THE LSMS

LSMS surveys have been conducted in 11 countries thus far: C6te
d'Ivoire, Peru, Ghana, Mauritania, Bolivia, Jamaica, Morocco,











Pakistan, Venezuela, Tanzania, and Vietnam.2 Arrangements for new
surveys are well advanced in Nicaragua, Guyana, and Ecuador, with
fieldwork scheduled for 1993. Possibilities for new surveys are being
explored in Colombia, Nepal, and Bangladesh. Although the first few
LSMS surveys followed a very similar format, as time passed and as
countries with different circumstances were added, substantial variety
has arisen in the surveys across the different countries.

Diversity in Implementation
The Living Standards Measurement Study was initiated in 1980. Its
purpose was to promote the use of household survey data as a policy tool
in developing countries. In the early 1980s, the work focused on two areas:
(1) assessing the experiences of developing countries in implementing
household surveys and using the results for policy purposes; and (2)
investigating the information needed to assess the impact of various
government policies on households' living standards and the best way of
collecting that information by means of a household survey. The end result
of these efforts was a draft questionnaire in 1984.
The first two LSMS surveys were fielded in C6te d'Ivoire in 1985
and in Peru in 1985-86. In both cases, financing was obtained from the
World Bank's Research Committee, and the main actor in implementing
these surveys was in the Bank's Development Research Department.
The main motivation of the surveys was to test whether such a complex
survey design could, in fact, be successfully carried out in a developing
country. Overall, the C6te d'Ivoire and Peru surveys were successfully
implemented. The data were used immediately (and are still being used)
for a wide variety of research projects and, less quickly and to a lesser



2 The Social Dimensions of Adjustment Project (SDA) in the World Bank's Africa
Technical Department has assumed responsibility for the surveys in C6te d'Ivoire, Ghana,
and Mauritania. SDA "Integrated" surveys, which are very similar to LSMS surveys, are
in the field in Uganda and Mauritania, and are planned for Madagascar, Senegal, and
Guinea. The SDA also has a much shorter questionnaire to be used with larger samples
for monitoring purposes. These "Priority" surveys have produced data in seven countries
(Chad, The Gambia, Guinea, Guinea-Bissau, Senegal, Zaire, and Zambia). The SDA is
also supporting survey programs where data collection is now or will shortly be under
way in the following countries: Kenya, Central African Republic, Burkina Faso, Rwanda,
Malawi, Mali, Tanzania, Zimbabwe, Cameroon, Mozambique, Niger, and Togo. This
paper focuses on the LSMS surveys themselves, but to the extent that the Integrated
surveys are similar, the same critique is broadly relevant. The Priority surveys differ
enough from the LSMS surveys that they have a somewhat different set of advantages
and disadvantages.




University of Florida Home Page
© 2004 - 2010 University of Florida George A. Smathers Libraries.
All rights reserved.

Acceptable Use, Copyright, and Disclaimer Statement
Last updated October 10, 2010 - - mvs