PLANNING AND CONDUCTING
Chris 0. Andrew, Peter E. Hildebrand
PLANNING AND CONDUCTING
Chris 0. Andrew Peter E. Hildebrand
Department of Food and Resource Economics Institute of Food and Agricultural Sciences University of Florida
MSS Information Corporation 655 Madison Avenue, New York, N.Y. 10021
Library of Congress Cataloging in Publication Data
Andrew, Chris 0
Planning and conducting applied research.
1. Research Methodology. I. Hildebrand, Peter E., joint author. II. Title. Q180.55.M4A5 001.4'2 75-45131
Copyright 1976 MSS Information Corporation
All Rights Reserved
TABLE OF CONTENTS
Preface .. v
Applied Research 3
The Book 5
PART ONE: PLANNING APPLIED RESEARCH
EFFECTS OF RESOURCE AVAILABILITY ON APPLIED
Information Resources .. 11
Secondary and Primary Information . . 12 Time Series and Cross-Section Data . . 13 Experimental and Non-Experimental Data. 14
Human Resources 15
Physical Resources 16
Financial Resources 17
Time Constraints 17
ORIENTATION AND FOCUS OF PROJECTS: RESEARCHABLE
PROBLEMS, HYPOTHESES, AND OBJECTIVES . . . 19
A Conceptual Model 19
Specification of a Researchable Problem . . 22
Problems Reflect Felt Needs . . . 23 Problems are Non-hypothetical . . . 24
Problems Suggest Meaningful, Testable
Problems are Relevant and Aanageable . 25
Researchable Problems vs. Problematic
Situations .. ...............26
Examples of Problem Statements. .. .....27
Formulation of the Hypotheses. ..........31
Characteristics of Hypotheses .........32 Some Examples of Hypotheses ..........33
Delineation of the Objectives. ..........38
Summary ...................... .4
PART TWO: CONDUCTING APPLIED RESEARCH
EXPERIMENTAL DATA COLLECTION .. ............
Experimental Design .... .............47
Relationship to the Problem. .........48 Relationship to Resources ..........52
Secondary Experimental Data. ...........56
Multi-purpose Experimentation. ...........58
NON-EXPERIMENTAL DATA COLLECTION .. ..........64
Selecting Respondents. ..............65
Designing the Questionnaire. ...........67
Difficulties in Interpretation and
Designing for Data Retrieval. .........70
Pretesting the Questionnaire ...........72
Size of Pretest ...............73
Information Checking. ............73
Time Difficulties ..............76
Selecting and Training Interviewers ..........77 Verifying Primary Data. ................78
Verifying and Using Secondary Data .. ......... ..79
DATA UTILIZATION--WHAT DOES IT ALL MEAN?. .. .....83
Flexibility of Interpretation. ...........86
Meaning of the Results ... ........... ..86
Reliability of Results ... ........... ..90
Presentation of the Results ... ........... ..95
Summary ......... ..................... .97
ReferenCes ........... ........................ 100
Appendix .............................. 108
This book is the culmination of a group effort to eliminate a deficiency made evident during the organization of a graduate course in research methodology at the UN-ICA Graduate School in Agricultural Sciences in Bogota, Colombia. 1The deficiency revolved around the difficulties in organizing research projects oriented toward real world problems and formulated so that 1) the research can be completed within the available period of time, ahd 2) the results will be useful in helping to resolve the problem toward which the study is directed. It became obvious
to a group of agricultural economists working with the graduate program that the various standard approaches to the presentation of research methodology are not successful in helping students become efficient researchers consistently able to make meaningful contributions to the resolution of agricultural and related problems of their country.
Initially the efforts of Michael Steiner who was responsible for the methodology class and of James Driscoll,Chris Andrew, and Peter Hildebrand, who were helping in the
1 The graduate school is jointly administered by the National University of Colombia (UN) and the Colombian Agricultural Institute (ICA). Beside the graduate school, ICA has responsibilities in research and extension as well as service activities such as control of agricultural chemicals and port sanitation.
2 This group consisted of several Colombian agricultural economists with ICA including those mentioned in the acknowledgements and agricultural economists with the University of Nebraska Mission in Colombia. This technical assistance team worked with ICA, National University and the Graduate School from 1967 to 1972.
development of the material, focused upon new means of presenting the requisites for successful research in a manner that would have real meaning and utility for the students.
Thus, the core idea now presented in Chapter III germinated. The topics are not new; most research methodology material
discusses research problems, hypotheses, and objectives. It is the means of fitting these parts together into a more useful form which is new. During several courses and while counselling students and fellow staff members of ICA in research, we were able to revise and improve the concepts. We found at the same time that this approach to planning and executing applied research could be easily understood and used. As the success of the approach became clearer, we decided that it was worth the time required to present it in book form for a wider audience.
In developing the book, to which all four of us initially contributed, we found that, although it was relatively easy to use the approach i-n training students and counselling researchers, it was difficult to present the approach in a form understandable and usable by persons with whom we would have no direct contact. Because of our various commitments and spatial separations it became increasingly difficult for all four of us to coordinate our efforts, so finally Andrew and Hildebrand assumed the responsibility for
the long process required to convert the vague ideas and concepts into a form which could be readily conveyed to students, researchers, and others.
The basic theme of this book is that of applied research as a service to a client with a problem for which the information obtained by research can help resolve. Because applied research has a definite purpose, there is usually a time constraint or deadline within which the work must be completed as well as a limit on the other resources the client has available or is willing to use in the resolution of the particular problem. Consequently, the researcher must be cognizant of the efficient use of the research resources while at the same time functioning so as to maximize the likelihood of providing a useful product to the client.
Besides the approach to applied research (covered in the book), another important factor affecting the success a researcher will experience in serving his clients is the research environment within which he labors. An applied researcher cannot be effective in satisfying clients when he is isolated from them by a system that reduces or pre1
vents effective communication among them. This can happen, for example, when an extension service with direct client contact has little communication with research personnel even though they may be in the same organization. It can also happen in research organizations in which projects are dictated by administrators who have little contact with the clients and hence have no appreciation of their real problems. This may be an argument for maintaining only
1See [2,30,38,60,61,65,69,75,80,82,86,87,881 for readings on the role and impact of institutions in the research process.
small research organizations, but more realistically, it is an argument for an organization in which the researcher maintains close personal contact withthe clients and where he in turn shares in determining the research priorities of the organization. We suggest that a better coordinated working relationship between research administrators, researchers, and clients will develop if all three groups understand the approach to problem identification which is presented in this book.
Although a majority of the examples are authentic, and are drawn from Colombia where most of the writing was done, the research difficulties presented are not unique to that country;.they are encountered throughout the world including the United States. Our desire is to provide the researcher, wherever he may be,with an approach to research under various time and resource limitations which will help him be of greater service to his clients. This is particularly true in developing countries where applied research is so needed.
To acknowledge the assistance of each individual for comments presented during numerous discussions and on the seemingly endless series of drafts leading to this book is impossible. Without the early dedication and ideas and comments by Mike Steiner (presently with the Armor Food Company) and Jim Driscoll (presently with the Economic Research Service of the United States Department of Agriculture) this book would not have been initiated nor, possibly, completed. Two Colombians, Juan Acosta and Ramiro Orosco, at the Colombian Agricultural Institute (ICA) deserve special recognition for reviews of the material and constructive criticism, and for using concepts and early drafts in the classroom. Likewise to Rafael Samper, Department Head in Agricultural Economics at ICA, and his staff, gratitude is due for sustained interest and encouragement.
Ideas and most of the drafts for the book were forthcoming while the authors were under contract with the UniversityofNebraska at the Colombian Agricultural Institute and the National University of Colombia. This contract, funded by the Ford Foundation and the USAID, helped develop the agricultural economics profession in Colombia including three undergraduate programs, a graduate program, and both research and extension programs. To our colleagues, and to students at these institutions, we offer sincere acknowledgments and hope that the book will be useful to them.
We express our appreciation for assistance received from the Food and Resource Economics Department at the University of Florida. To Fred Prochaskaandhis students who
have used the text for two years in the research methodology course, we are grateful for constructive criticism. Special recognition is due to Leo Polopolus, Chairman of the Food and Resource Economics Department, and W.W. Mc Pherson, Graduate Research Professor, for reviews and consultation. Special appreciation is extended to Beth Davis for supervising preparation of the final manuscript for printing.
Also, we extend our appreciation to theMinistryof Agriculture of El Salvador, where the second author was stationed on a technical assistance contract, for translation and preliminary publication of the manuscript in Spanish. For final reviews of the Spanish translation and preparation ofthe final manuscript for printing, gratitude is extended to the Guatamalan Institute of Sciences and Agricultural Technology where the second author is presently employed.
Lastly, we want to express our sincere gratitude to our families who shared their time for numerous weekends and nights with an idea which may have seemed vague and boring.
About midmorning, the Minister of Agriculture is just completing a phone conversation as the door to his office opens to admit one of the young men in the research group from the Planning Department of the Ministry. At about the same time the Vice Minister and the head of planning enter and the four men seat themselves around a small conference table. The Minister opens the meeting, directing his comments to his Vice Minister: "I'm told that wehave a final report on that project onimpbrting fertilizer, that's great. It's just in time. Now we ought to be able to convince the import-export people that they can't reduce our request for fertilizer and still expect us to meet our new trade commitments. "
Turning to the young researcher, the Minister says encouragingly, "OK, young man, they tell me you've made an excellent study. Now tell us exactly how much fertilizer is it going to take over the next five years to meet our production targets?"
"Sir?" replies the researcher, apparently a little confused.
"Come, come, now, don't be nervous. Just tell us about the research you've been doing. What are the results?" The Minister realizes the researcher may be a bit timid.
"Oh, yes sir," smiles the researcher, "thank you."
'Well, as soon as we got your request for information on the importance of fertilizer, we checked to see what data we had on Japan and some other countries Where there have been recent increases in production. We thought this ought to give us some good ideas about the relationship
between fertilizer use and crop production. Here, sir, we
have a series of graphs showing the correlation between these two variables for a number of countries. "
"Yes," replies the Minister, "it's quite evident that
fertilizer is important in increasing crop output. Now, how much are we going to need?"
The researcher continues, "According to the latest census, which unfortunately is several years old as you know, only about 40 percent of our farmers are using any fertilizer. This is considerably below the rate in the other countries I mentioned. And inthose countries income
per farm family has been increasing rapidly, again demonstrating the importance of fertilizer." The minister nearly interrupts but lets the researcher continue. 'N\ow if we want to double the number of farmers using fertilizer, we might be able toassume that we need twice as much fertilizer as now.1
"Yes, Isuppose"l replies the Minister, "but what about the land area involved and what about the requirements for the different crops?"
The iesearcher, thumbing back through his report answers, "We don't have any information on area of each crop
that is fertilized but cotton and sugarcane consume about 80 percent of the fertilizer used and..."1
He is interrupted by the Minister who says, 'ut don't you have any estimates of the quantities required to get the production we need over the next five years? Our problem is that they want to cut back on fertilizer imports to
help domestic fertilizer production just at the time we have to try to increase crop production in a big hurry. We need to-know how that could affect our program and how much importation we need to ask for." Turning to the head of the planning group he says, "I thought we discussed this
pretty thoroughly that day it came up. What happened?"
"Well, that's right, and I knew we were going to have
a hard time convincing them how important fertilizer was in
our program and I know we talked about that in my office, didn't we?" the head of planning asks the young researcher.
"Yes, sir," replies the young man, "we knew we had to get you some good information on the importance of fertilizer and that's why I have this information on Japan and those other countries. "
'Vell," replies the Minister, "'hat's not going to be much help in solving our problem. We have the meeting with them tomorrow. But maybe we could hold off a decision for a couple of days if you think you can get me the information we need by that time. Why don't you call someone at the experiment station? Maybe they canhelpyou. But you better get going. Don't forget how important fertilizer is to us."
There are several important points in this dialogue, which though fictional, represents a real-life situation encountered much too frequently. The most important point is that after waiting right up to the deadline, the Minister, who is the client, did not obtain the information he needed for ameeting of great importance. As a result, the research costs incurred by the Planning Department for this project yielded little of value. The most serious consequence, of course, is the cost associated with not having the relevant information for the meeting. Although some of the reasons for the unfortunate situation are evident in the dialogue, others are more subtle. Regardless of the reasons, we hope that with this book we can contribute to the more effective use of research resources and help prevent the kind of unhappy discussion as that between the Minister and the researcher.
Research i. the orderly pocedure by which man incrLea.ses his knowledge and 46 contrasted to accidental
discovery because it follow a series o6 steps designed precisely 6o& the putpose o6 developing information.
Knowledge gained by research may be used by man to produce a greater abundance of food and fiber, to lighten the burdens of his labor, or in any number of ways to generally improve his well being. Or new knowledge may simply be added to man's store of concepts about the universe to await application at some future point in time. Research undertaken specifically for the purpose of obtaining information to help resolve a particular problem is applied research. Foraresearch undertaking to be applied research it is not necessary that the results (the new knowledge) in fact resolve or help resolve the problem which initiated the project (though hopefully they will), but it is necessary that the research have a specific problem orientation. It is this kind of research -- that oriented toward resolution of specific problems -- toward which this book is directed.
The development of Mexican or dwarf wheat was the result of an applied research process oriented toward the resolution of a specific problem [72). Afertilizerexperiment oriented toward making recommendations to farmers is another example, as is the work of a government planner trying to estimate the likely supply response to fertilizer for a particular commodity under a proposed new program, or the total fertilizer requirements necessary to reach specified production goals, as in our dialogue. Determining acceptability of anewly developed feed concentrate for fattening hogs in tropical areas on the development of a hand seeder for steep terrain in primitive areas also would be classified as applied research. In general, the research referred to in this book is oriented toward providing useful information to decision makers such as farmers and public administrators.
Applied research, such as that just described, is carried out in all parts of the world -- it is a much more
1For thoughts concerning the meaning of research and the scientific method see [9,10,11,12,14,15,19,22,23,24,25, 28,34,39,52,71,73,74,81,83]
widespread activity than basic research which isa necessity but one that only the wealthiest countries can afford. Most applied research is conducted under moderate to severe resource limitations which necessitate efficiency in the research process. An effective applied research methodology iz directed toward the.efficient use o6 available %eueatch resoources to maximize the ptobability o6 achieving meaning6ut results to help %esoZve problem6. Disappointment in the results of applied research--a "So what?" response --in most instances can be traced directly to the use of an inadequate and/or ineffective applied research methodology which failed to correctly identify the problem.
Perhaps the most critical deficiency in methodology is the failure to adequately identify the specific problem toward which the research is to be oriented (as happened to the researcher in the Ministry). This may result when the researcher uncritically accepts the problem as stated by the client or by his spokesman (the "importance of fertilizert was not the Minister's problem). Another serious deficiency may occur even after properly identifying the problem. This is 'the failure to formulate hypotheses and objectives correctly oriented toward the resolution of the problem and to use appropriate analytical techniques (what hypotheses did the Ministry researcher use?) The most critical concepts and the interrelationships among problem identification, hypotheses, objectives, analytical techniques, and resource restraints.
The role of theory, though not developed within the main text, is also critical to applied research. Without discounting the value of practice and experience, the greater the command of theory possessed by the researcher, the broader will be his capabilities and the more efficient he will be in planning and conducting the applied research project. This istrue because theory envelopes and supports
the entire research process. 2Without a good command of stress theory an engineer cannot properly design nor efficiently build a safe bridge. A plant breeder must understand the theory of genetics before he can hope to efficiently develop a strain resistant to a certain disease. An agricultural economist cannot determine an optimum farm
organization without knowledge of production economics theory.
The researcher's foundation in theory provides the orientation for defining a problem that is researchable within the discipline or disciplines involved in the research and with the resources available. Theory also provides the basis for the formulation of hypotheses and in the selection of the analytical techniques to be used. And it shbuld be obvious that the interpretation of the results depends heavily on the theoretical orientation of the researcher.
Although theory permeates theentire research process,
in applied research, frequently conducted under sub-optimum conditions, the researcher's practical experience is equally important. Institutional and budgetary restraints, less than ideal field conditions, poorly trained personnel, inadequate background information and other similar factors have a very significant effect on the research process and
therefore must be recognized and dealt with accordingly. Practical experience is invaluable in helping the researcher overcome the obstacles which are so often encountered in applied research. In all phases of the material to be presented, the difficulties associated with sub-optimum research conditions under which the individual research is apt to be working are considered.
In this book, we have divided the topics into separate chapters and the chapters into two parts derived from
2 For literature concerning the role of theory in research see [20,35,40,53,62363970].
the book title; a convention towhich we adhere, though not
without some reservation. N either section. nor the material in any chapter is independent.
Plannin g activities are discussed in Part I of the book. In planning the research project, one must always take cognizance of the means available for conducting the research, and during the research process it maybenecessary to modify portions of the original plan. Each of the activities is affected by the others and by the research resource restraints under which the researcher is toiling. The kinds
and sources of data which will be used and the methods of analysis will be dictated by the hypotheses and objectives,. but they, in turn, must be finalized only after having taken into account the effect of resource conditions on availability of data and/or analytical competence.
In Part TI, Conducting Applied Research, we discuss experimental and non-experimental data collection, verification and interpretation of data, and presentation of the results to the client.
Because of the deliberate and intense emphasis placed
upon problem identification, the book does not discuss in detail each basic element of a research project. One should be cognizant, however, of the interrelated components of a
research project and employ them when planning and conducting applied research. These essential elements are: 1) a problem statement accompaniedby sufficient information to justify the need for research; 2) hypotheses; 3) objectives; 4) budget; 5) the appropriate theoretical and analytical approach and procedures; 6) data requirements including sources and procedures for obtaining data; 7) a detailed work plan showing jobs to, be done and time sequences; and 8) the reports to be issued for each audience.
EFFECTS OF RESOURCE AVAILABILITY ON APPLIED RESEARCH
The relationship of research activities to the availability of research resources is an important difference between applied and basic research. In much of what is commonly considered basic research, aproposal is prepared, and if funding is granted the project is initiated. It normally continues so long as financing is available and then whatever can be concluded is presented as the results of the project. Seldom are the results periodically scrutinized to determine their relationship to the proposal because the urgency associated with solving a pressing problem is missing. Nor are results frequently weighed against the use of resources to estimate the productiveness of the project.
Applied research is more often (though not entirely)
carried out under other circumstances. Because the research v is oriented toward the resolution of a specific problem, there is usually a time restraint fixed by the need to make a decision. Such research is also carried out under varying degrees of financiat restrictions and usually under rather severe shortages of trained manpower and modern data processing resources. Another research resource which is seldom abundant under many conditions of applied research is published data, other forms of secondary data or reliable information in general. Basic physical facilities such as means of transportation or land area for research can also limit the scope of research activities.
In the process of fitting the project to the resource restrictions, three alternatives are possible; 1) the resources may be expanded to fit the project, 2) the project
may be narrowed to fit within the restriction, or 3) both may occur within limits. The first alternative is appropriate if the level of precision or degree of confidence desired by the client prohibits a reduction in the scope of the project. In this case, the client must be prepared
to provide additional resources where the limitations are critical whether it be in physical facilities, manpower, funds, additional time for completion, or some combination
___of these provisions.
If resources cannot be expanded for aparticular proj.ect and the deadline for conclusion is firm, the researcher has four further possibilities open to him. First, he may
study fewer variables, ignoring some relationships, which affect the analysis, but those which he hopes are less important than the ones included. Second, he may also aggregate variables into groups. In this manner, it ispossible
to include those relationships which may otherwise have been omitted, but the nature of the relationships, because of the aggregation, becomes less clear. A third alternative is to
modify or change the nature of the analysis to'be carried out. Less complex analyses can usually be conducted more rapidly and with fewer facilities, but the precision of the results is reduced accordingly. Finally, the researcher may choose to make fewer observations either in the form Aof fewer experiments or a reduced sample.It is clear that resource availability has an important effect on the nature of the research product derived and the level of precision which can be achieved -or the level of confidence which can be placed in the results.
Even in very limited resource situations, however, decisions are necessary and researchers are expected to provide useful information. The ultimate decision as to the quantity of resources to be made available for any particular project and the time limit for its completion rests with the client, or with the person responsible for making decisions related to the resolution of the problem toward which the research is oriented. At the same time, the client de10
pends on the researcher to provide him with accurate measures of resource requirements and the scope and precision which he can expect from devoting different amounts of resources to any particular project.
The following sections will describe the principal research resources and include a general discussion of how each resource, when restricted, can affect a project. .A single chapter does not adequately cover the range of alternatives open to the researcher and his client, but we hope that it will provide sufficient stimulus so that the researcher, with imagination will be flexible in adapting his research efforts to any resource situation. 1
Informat ion Resources
Information is the foundation upon which research is based; hence, one of the major tasks of the researcher is collection of information for use in the research process.
Information in general and data specifically are as critical to the problem identification phase of the project as they are to analysis. Their availability profoundly affects both the quantity and quality of research which can be produced within a given period of time. There are few limits to the quantity of data that can be accumulated given sufficient time and resources, however, requirements
are narrowed and brought into focus by careful research planning.
In general, published information provides the basis for problem formulation while either published data or data generated in the research process (or both) may be the source of information for analysis. Published data,
or any information not generated or accumulated under the
1 See  for a discussion concerning allocation of scarce resources to alternative research programs and projects.
control of the researcher but utilized in the particular project are considered to be zecondaty data. In contrast, any data generated by the researcher and directly associated with the research project are prLimaty data. Another classification of data which has an importance to research is that which differentiates time sries data, or observations made at specific intervals over a period of time, from cto -4ection data which are taken at one point in time. A third comparison is that between experimentat and non-expetimentat data. Each of these kinds of data or data sources has different costs associated with availability and analysis, and each has different implications with respect to confidence in conclusions based upon it.
Secondary and Primary Information
When secondary information related to the project is available, its cost to the researcher, both in terms of time and money,is usually less than that required to obtain the same kind of information first hand. However, the usefulness of secondary data as a research resource is not always as great as that of primary information. The researcher must always select the appropriate primary secondary data mix and the techniques used to combine the two in order to obtain the most useful research product within the limits of his other resources.
For practical purposes, some data are only available from secondary sources. Price series, crop production series and census information are examples. It is simply not appropriate to consider obtaining this kind of information first hand. But it does not follow that available secondary data of this nature are always adequate, representative, or even relevant to the particular project. The researcher must satisfy himself that any particular series really measures something that is relevant to the project or something which can be made relevant through acceptable modifications or manipulation. If care is not
taken to verify secondary information in this manner, the researcher may well draw false conclusions from his analysis of the data.
Primary data usually will be more closely related to a particular project than secondary data which are collected for a multitude of purposes or for projects with other objectives. But primary data collection almost always requires more time than isnecessary when using secondary data, and may require more of the other resources. Hence, although primary and secondary data are not necessarily substitutes for each other, the researcher should be aware of the availability of secondary information and assess its
relevance as an alternative to the collection of primary data.
Time Series and Cross-Section Data 2
Time series data, as the name implies, are data obtained in a series over a period of time. Examples are price series, production and acreage data, and indices of costs of living or wages, most of which must be accumulated over long periods of time to be useful. Cross-section data are those taken at a fixed point in time (or over a relatively short period of time) and include observations of several different strata or levels of a population. In many cases, time series data are essential to a project, but occasionally, cross- 'section information can be substituted. Time series data of consumption and income may be used to study the same relationship at a given point in time. Because, as in this case, cross-section data can substitute for a time series, the researcher should not despair if atime series is not available. Nor,
by the same token, should he blindly use the time series without considering the advantages of obtaining and using the cross section data. Or in some cases it might be desirable to use a combination of the two.
2 For a brief discussion of how these two types of data influence demand analysis as one example, see [64~].
Experimental and Non-Experimental *Data
Experimentation is a means of obtaining data with relatively high precision in measurement of the variables.
In many instances this precision is associated with a longer time requirement than that needed for obtaining nonexperimental data. In crops a minimum of one seas-on is required and much longer periods may be needed if, for example, effects of weather are to be determined. A year or
more may be common for some animal experiments. When, as discussed in Chapter I, the Minister requested that the researcher check with someone at the experiment station to help resolve the fertilizer demand question, he was hoping
that secondary experimental data might provide specific crop requirement guides to be used in preparation of a demand estimate. He knew time would not permit the design and analysis of crop experiments.
In some cases, however, experimentation can reduce the time and resources required to resolve a particular problem when compared with non-experimental data collection. For example, an experiment to determine potential consumer acceptance of a new product before it is marketed can be less time consuming, require fewer resources and involve less financial risk for an industry than consumer response research following the full scale production and marketing of the product.
A possible alternative to experimentation is the collection of non-experimental data through a survey of a number of people who have knowledge or experience with the phenomenon in question. This procedure usually requires less time, but may be less precise and more costly than experimental data collection. An approximation of a fertilizer experiment for example, can be made by surveying a
group of farmers, each of whom uses different quantities of fertilizer. Obviously, results will be less precise than experimental results, but at the same ti me, estimates
of variance will be more realistic than from a controlled
experiment so that farmers will have a better idea of the range of response toexpect from the use of fertilizer.
Another non-experimental source of information in the context of applied research is the simulation of results through the use of "best guesstimates" of the most knowledgable people available to the researcher. In some ways, this is similar to an informal survey. For instance, sufficiently detailed input requirements, yields, and resource restrictions can be generated in this manner for use in preparing budgets, and ultimately a linear program or simulation model for the agricultural sector of aregion. Such a model, while less valid than could be possible under more optimal conditions, can be used successfully in project planning where time limits prohibit experimentation and current conditions in the area will not provide the detailed information needed from a survey.3
As with most resources, the human element must beconsidered from the points of view of quantity and quality. Sheer availability is not sufficient for most research undertakings; the training and capabilities of personnel must be considered when planning the project. Except in rare instances, the time factor in applied research prohibits the training of professional personnel though there may be time to train some non-professionals such as interviewers. A field hand who cannot read or write may be willing to do the physical labor of an experiment, but he cannot be relied upon to maintain records of the results. Nor can professionals with aminimum of training in statistics be expected to carry out complicated statistical analyses. In the first case, other arrangements will have tobe made, and in
3Examples are research projects currently underway in Viet Nam and El Salvador  by the Food and Resource Economics Department, Institute of Food and Agricultural Sciences, The University of Florida.
the second, less sophisticated techniques will have to be employed.
A common misuse of research resources exists where elegant data collection techniques are employed but the data are not fully utilized because appropriately trained personnel are not available to make proper or correct analyses. In the short run, simpler experiments and surveys which can be analyzed readily by available personnel are more appropriate. Money saved by not conducting elegant data collection programsmay be used to train personnel to conduct and analyze more complicated and sophisticated experiments and surveys in the future.
Non-technical physical resources include transportation facilities, land, office space, machinery, typewriters, and other items of a similar nature. Like all other research resources, their availability must be considered when planning the project.
Technical physical resources include scientific instruments, calculators, electronic computers, etc. Certain kinds of instruments may be indispensable for particular aspects of aproject --- some hypotheses may need to be eliminated if the correct instrument is not available or is too costly compared to expected results to justify its use. On the other hand, an electronic computer, while not indispensable, may substitute for other resources such as money or time. The use of a computer,when available, sometimes can shorten the time required to achieve useful results. If a researcher must wait long periods of time, however, for cards to be punched and programs to be de-bugged, he may be better off to undertake appropriate analyses onadesk calculator. When computers and calculators are unavailable a diligent researcher might still provide rough but meanihgful recommendations based upon experience and imaginative use of the most basic of physical resources -- a pencil and paper.
This resource to a certain extent can be substituted for all the other resources. At the same time it may not be appropriate, feasible, or even possible to make this substitution because of other considerations. Nevertheless, funds are required for almost all research projects and their availability is an important consideration in research planning.
Time is usually not thought of as a resource in the same terms as physical facilities, information and human resources, but its effect on the planning and execution of
applied research is similar. Considered as a resource, time can interact with other resources in that substitution of one for another can be made. If a decision on a particular
problem is critical and time is limited, a greater number of other resources will be required to achieve a given level of confidence than would be necessary if more time could be taken. Hence, the use of more time is an effective substitute for quantities of other resources. But also, consuming more time on one project reduces the amount of that limited resource which is available to help resolve other problems.
In some situations, time can be overwhelmingly limited. This is usually the case when one is involved in so-called "brush fire" research of the-type frequently faced by planning groups within the various ministeries of government. In such instances the researcher must always maximize the efficiency with which he uses this resource. Only data
readily available can be used and lengthy methods of analysis cannot be considered. Many times "best guesstimates" are the only means available to the researcher under these conditions.
Approaching the other extreme are theses at any of the levels at which they are written in various education
systems of the world. Often they are not oriented toward any particular problem so the time factor is not relevant,
but among those that are problem oriented, too often time is relegated to a secondary role in resource utilization. As a result, many theses are not. written in time to be of any great use -- their value reduced by failure to account for the time factor. Those who excuse this fault by emphasizing that theses are only meant to be training tools often deprive the student of an opportunity to perform and benefit from meaningful research.
Research undertaken specifically for the purpose of obtaining information to hblp resolve a particular problem is applied research. The purpose of this chapter has been to examine the relationship between the scope of an applied
research project and the quantity and quality of research resources which canbe devoted to the solution of the problem at hand. The urgency of problem resolution makes time
an important resource or constraint which interacts with the other financial, human, physical, and information re-sources.
The researcher must be aware of the effect that certain resource limitations can have on his research. This cognizance will improve his research effort by increasing the probability that the proposed project will produce useful results. Projects designed in the absence of this consideration can and frequently do run into difficulties such that the productive potential of the resources utilized is not attained. The result is that less effective information is made available for decision making and problem resolution. Careful consideration of resource availability can help pre-vent situations where the deadline arrives and the researcher, still engrossed in gathering data or making analyses, has little of value to report to his client.
ORIENTATION AND FOCUS OF PROJECTS:
RESEARCHABLE PROBLEMS, HYPOTHESES, AND OBJECTIVES
Proper management of applied research requires clear definitions of the goals to be achieved through the project. 16 one does not know Jor what he is striving he cannot hope to effectively accomplish the task. Orientation and focus of the project include the specification of the problem in terms which make it amenable to research, the formulation of hypotheses which are subject to being tested, and the delineation of the specific objectives which the project should accomplish. The interaction of these three phases and their clear exposition serve as a plan or guide for determining the procedures to be followed by the researcher in conducting the research. Adequate orientation and focus of the project will help assure that it can be completed satisfactorily within the resource limitations facing the researcher and will also serve to explain the nature of the undertaking to the client or the administrators for whom the research is being undertaken.
A Conceptual Model
We have found it useful to consider the process of plan-ning the applied research project as being equivalent to a funnel with a series of filters (Figure 1.) Such a funnel is used to reduce a large volume of liquid to manageable proportions. The orientation and focus of aresearch project serves the same purpose -- it reduces alarge volume of information to manageable proportions. Extraneous infor-mation and ideas are eliminated as foreign matter might be filtered in the funnel. Each part of the' project statement-19
Pbazin a xct)u~o
Figre .--ittng hereseacht poetoheroues 20ohe6.
problem, hypotheses, objectives -- serves to narrow down the proposal, to bring it into sharper focus, and to filter out surplus or unrelated information to make the orientation more precise. One can consider the size of the lower opening of the funnel as beingdeterminedby available research resources (the bottleneck). Hence, the proposal as it finally emerges must fit within these resource restrictions.
The top of the funnel will be the general subject matter orientation of the researcher. The research will be within the area of interest of the individual researcher and usually related to his. individual talents. The general problematic situation will fall within these limits, but any problematic situation suggested by a client may contain several researchable problems. Hence, the selection of a researchable problem based upon the clients needs is equivalent to sharpening the focus ona particular aspect of the more general problematic situation. Hypothesis formulation narrows the problem to tentative relationships which will be tested in the research process. Finally, the objectives specify the limits within which the project will be conducted and describe the useful product which will result.
Obviously, information, ideas and relationships do not flow through the funnel like water but are filtered time and again. The process ofmoving from the top to the bottom of the research project funnel requires push and pull, pare and adjust, specify and redefine until the proper proportions are achieved. The problem, the hypotheses and the objectives may each have to be changed and adjusted many times before a satisfactory product results.
At this point, aword of caution is in order. The result of this funnelling process should be a plan of execution that has ahigh probability of accomplishing research which will be helpful in resolving the problem toward which the research is to be directed. It is much too common, and seemingly easier, to embark on the next steps of the research process -- data collection, analysis, and interpretation-with a poorly specified project statement. The consequence
is usually that budget and time restrictions cannot be met and one of the following results:
1) Conclusions must be drawn on the basis of inadequate evidenbe,
2) More resources and more time must be devoted
to the project to allow completion, or
3) The project withers and dies and is relegated
to a drawer in the file, never to be heard from
No matter what fate the research meets, the return on the investment in the project will be lower than necessary. To avoid these consequences, careful orientation and focus
are essential and can well be the most productive time spent on the entire project.
Specification of a Researchable Problem
Problem specification or elaboration of a problematic situation in such a way that it presents a researchable problem, is a vital step in the process of applied research [142, 84]. Seldom is aclient's problem defined for the researcher so that the requirements of the research process are obvious. Even in cases where it may at first appear that it is so defined, it usually is not the case. A common and deceptively simple appearing example is the problem of determining crop production costs. An economist cannot uncritically accept a "cost of production" project without understanding which specific cost components are of interest to the client. The researcher, because of his training, will usually have a greater appreciation for the technical
characteristics of the problem than will the client and should therefore consider it part of his responsibility to
identify symptoms and diagnose the problem. He should perceive the identification of the problem as amajor task. 1
1 For a discussion of various attitudes toward problem identification in research see [68,77,79].
On the other hand, the researcher cannot assume that he automatically understands the problem better than does the client. The researcher must work with the client until they have jointly defined an acceptable and researchable problem.
Problem specification is nota simple process. Hildreth and Castle summarized a discussion concerning problem identification as follows:
"The start of research is the most important and difficult stage of research. It requires far more than logic; it includes procedures which cannot be neatly categorized and communicated." 
But there are several characterisitcs which a project statement will possess if it defines a researchable problem within the context of applied research. These characteristics of a problem statement are that problems reflect felt needs, problems are non-hypothetical, problems suggest meaningful and testable hypotheses, problems are relevant and manageable, and aresearchable problem differs from a problematic situation.
Problems Reflect Felt Needs
A problem exists when there is a need felt by a client. This client may be an individual, a group, or a society. The need must be "felt" in the sense that the originating party believes that change can be realized, and it may arise from social tensions, doubts, conflicts, failure torealize goals, concern for an anticipated occurance which might be preventable, or the simple lack of knowledge if this knowl-edge is necessary to contribute directly to the resolution of another felt need. To be appropriate as a researchable problem, the need must be amenable to change as a result of the information supplied by the research process. Hence, all felt needs are not necessarily functional so far as permitting the formulation of a researchable problem statement.
Problems Are Non-hypothetical
A researchable problem statement must bebased on factual evidence. The relationships expressed can be neither hypothetical nor subject to doubt or question in the mind of the researcher. The researcher must use his judgement as he sifts through available information to determine to his satisfaction what are and what are not acceptable facts and factual relationships. In a probability sense,of course, few facts can be accepted with complete confidence. However, the facts or factual relationships accepted by the client and the researcher in the problem statement must be such that testing of their validity is unnecessary. If a particular proposition cannot beaccepted as fact, then if it is relevant to the case, it must be relegated to the status of a hypothesis.
All researchers and their clients will not accept the same information as facts of factual relationships because individual judgements, knowledge and experience affect this choice [51. In defining the researchable problem related to a shortage of manufactured dairy products, one researcher, because ofhis experience and general knowledge, maybe willing toaccept the proposition that there is no shortage of milk processing equipment in a particular area. For him this forms part of the problem statement. Another researcher may feel that the validity of this proposition is not clear, so in the second case, it must become one ofthe hypotheses (if it is relevant to the problem). Obviously, the nature of the research project will be different in each case.
This is the stage of the research 'process in which a review of literature is most productive. By means of this review the researcher seeks to utilize the experience of others to help in the orientation of the project. It should be apparent that this kind of orientation can change the focus of the research effort and can have asignificant impact on the productivity of the research resources.
Problems Suggest Meaningful, TestableHypotheses
Because the statement of the problem serves to orient the entire research process, it must suggest testable hypothetical relationships. Hypotheses are formulated as partial explanations ofthe unknown relationships which create
the problem, and those which cannot be tested will be of little assistance in the resolution of the problem. Hypotheses are testable when information about their validity may be collected and analyzed.
The hypotheses must also be developed from the problem statement in a manner which does not result in trivial solutions. Triviality indicates a tautology, an obvious solution, or an infeasible solution. The hypothesis that Vie capita consumption of food products is low because
there are too many people," derived from a problem statement referring to the existence of hunger in a country is such a hypothesis. Even if the hypothesis is substantiated, it results in an answer which is of little or no use in the alleviation of the immediate felt need.
If the problem statement does not suggest testable, hypotheses for resolution of the problem under investigation,
the researcher has not adequately formulated the problem for research.
Problems are Relevant and Manageable
Agricultural scientists tend to work at extremes. "They tend to work either on problems where the outcome is highly predictable but which has little impact on problems or on problems so large as to be unmanageable" [56, p. 38].
This comment was made with reference specifically to agricultural economists but it applies to other disciplines as well. An agronomist designing an experiment to determine yield response on a certain soil, even though information about similar soils is readily available, is an example.
The researcher knows with a high degree of certainty whether or not a response will be observed.
Over ambition, lack of adequate forethought, and inexperience are the principal causes of unmanageable projects. An over-ambitious researcher may be motivated by a desire to study all the problems in a given sector so that he can answer any question that arises. Or anunmanageable research project can result from suggestions by clients or administrators unfamiliar with the discipline of the research. In this case the researcher should not accept the project without first more precisely defining the problem and reducing the proposed project tomanageable proportions within the time requirement and within the human, financial and other resources available for the project.
An unmanageable research project yields few benefits to anyone. The most likely result of such a project is superficial treatment of part of the problem and the neglect of other parts. Little detail will be achieved, and much that is presented may be found to be inaccurate or insufficient. The contribution of this type of study will almost always be less than that of a more, specific study which examines fewer phenomena, but does soin more detail.
Researchable Problems vs. Problematic Situations
Confusion is likely to exist relative to the differences between a problematic situation and a researchable problem. In our context, there is a real and functional difference between the two. First, aproblematic situation is a phenomenon which exists; a researchable problemmust be identified and defined. A problematic situation represents a generalized situation but a researchable problem, expressed in the above terms, must be specific. "An in-creasing rate of crime in the cities," assuming the statements is true, is an example of a problematic situation-it exists and it pertains to felt needs. But in no way can the statement be construed as astatement of a researchable
problem. The expression of a problematic situation can serve as general orientation to the broad area of interest of a research project, but it is not sufficient to serve as a guide to a research project.
A problematic situation can be the source of a variety of researchable problems. Different clients, different research administrators and different researchers are likely to arrive at a variety of researchable problems from the same problematic situation. That is to say that there is no single right, or correct, researchable problem -- there is only a correct framework within which the researchable problem must be expressed to be useful to the researcher. The selection of the researchable problem which will actually be undertaken will depend on the situation within which the research is being conducted.
Examples of Problem Statements
In evaluating statements which are considered by their author to delineate researchable problems, it is difficult, if not impossible, to determine when deficiences are due to lack of care in problem formulation on the part of the researcher or are due to an unclear concept of the nature of the problem. An example of such a statement is the following:
1) "Rural unemployment created by an increase
in the use of agricultural machinery."
There is no doubt that the author was focusing on a problematic situation and this interest had to do with unemployment. But it is not clear what specific problem existed in his mind. The statement does not provide the basis for a research project though it might spark a lively debate. As it stands, it does not meet the basic requisites for the. specification of a researchable problem. Other examples of
2The examples used in the remainder of this chapter are taken from class papers and research proposals submitted by graduate students in the Department of Agricultural Economics, Instituto Colombiano Agropocuario, Bogota, Colombia.
incomplete or incorrectly formulated problem statements are the following:
2) "Few cattlemen vaccinate against hoof
and mouth disease."
3) "Exportation of fresh meat from Colombia."
Let us analyze the above three statements from the point of view of the requisites necessary for a correctly specified problem statement. Do they concern a felt need? Probably all do, but what exactly, is that felt need? The felt nae.ds suggested by those statements are numerous. Are the rural unemployed aproblem from the standpoint of crime or poverty in the cities, or is the author perhaps considering those unemployed as a source of inexpensive labor for rural industry? Is the author (or client) of the second statement the president of adrug company or is he the head of a meat export company? Is the third statement aproblem from the point of view of Colombia, which may be looking for increased foreign exchange, or, perhaps of Argentina which may be considering Colombian competition in the world market? Each of these points of view suggests a different problem, and hence, adifferent research project. It would be folly to initiate aresearch project on any of the above topics without more complete specification of the client's felt need.
Are the statements hypothetical orsubject to question in the mind of the author? This is the most difficult test for a third person to make because he is not aware of the precise knowledge held by the researcher or client. For this reason, proper background information in the research project statement is critical. Not everyone will automatically be convinced that the increased use of agricultural machinery leads to rural unemployment. However, if such a relationship is acceptable to both the client and the researcher, and is considered firm and non-hypothetical, it can be acceptable in the problem statement as a non-hypothetical relationship. The second and third statements do not express causal relationships and hence are not subject to this criterion. They are simply incomplete.
Do the statements suggest testable hypotheses? The first statement could be formulated as a testable hypothesis, but if it is to be accepted as a factual relationship for aproblem statement, then it cannot be-a hypothesis for the purposes of the research. Although testable hypotheses associated with the other two statements could be contrived, any firm relationship between them and the problem in the mind of the reader would be purely coincidental.
Another problem statement will be analyzed to determine if, or how, it might be improved.
4) "Deficient milk production reduces domestic
consumption and makes imports necessary in
this agricultural subsector,"
On the surface this appears to be a more complete problem statement than any of the preceding. Certainly it represents a felt need; in fact more than one. Is the problem to which the author is directing himself a production prob.lem, one of poor nutrition because of inadequate quantities of milk, or is he concerned with the expenditure of scarce foreign exchange? From the present statement, it is impossible to be sure.
The relationships expressed probably meet the non-hypothetical requisite, though not necessarily. The first relationship is really a tautology if "deficient" is defined in terms of domestic consumption. The second relationship could be hypothetical unless government policy, for example, has provided for the imports ofmilk to make up deficits in domestic consumption.
The statement does not legitimately suggest any hypotheses except the trivial ones that increased production would increase consumption and/or reduce the necessity for imports. And these are not really testable in the usual research framework. Hence, even though the problem would be amenable to one or more meaningful solutions, it is probably not researchable within the usual limits of time and funds. Several years would be required to determine experimentally if increased domestic production would, in fact, result in increased consumption.
In order to improve this statement, it is first necessary to focus more precisely on the orientation of the author with respect to his felt need. It turns out that the author of the statement was concerned with problems of production, principally with high costs, low productivity, and the small profit margins of the producers of milk. His research interest then focuses on the dairy herd and the dairy farm, and possibly on the farm-to-market process. Al though it will never be possible to determine the exact nature of a proposed research project from the problem statement alone, amori precise statement is necessary before one can proceedto formulate hypotheses and specify objectives. A better formulation of the above statement resulted from more reflection by the author on his orientation:
5) "The low level of technology on dairy farms
contributes to high costs of production, low, average productivity, and a deficient marketing system for milk. These factors cause low profits to the producer, price fluctuations for the consumer, and adeterioration in the balance of payments because of the necessity
to import milk."
In this form, the statement implies, in accordance with the requisites, that the author accepts as fact that a low level of technoiogy is a causal factor in low productivity, high costs of production and a deficient marketing system for the product. Furthermore, he accepts that these factors cause low profits, price fluctuations and the necessity of importing milk. By accepting these relationships as fact, they cannot appear as hypotheses to be tested by the research process. If the researcher or his client can not accept any of these relationships as factual, then the problem statement should be modified and the doubtful statements submitted ashypothesestobe tested (assuming they can be tested within the limits of the available research resources).
Formulation of the Hypotheses
It is clear that the logical sequence of events in the process of applied scientific inquiry begins with the observation of phenomena in the empirical world in common sense terms. Determination and classification of these events into problematic situations and specific researchable problems set the stage for postulating various potential means of problem resolution. This part of the process involves the formulation of hypotheses which formalize the premises to be tested by the research effort.
Hypotheses are tentative propositions that must relate to the problem so as to be helpful in providing means of resolution. Any suggested or indicated means of resolving a problem must be formulated so as to be testable and the relationship to the problem must be evident. To be logically complete and functional a hypothesis must involve a relationship. Such a formulation constitutes, implicitly if not explicitly, a hypothesis in the form of ii-then propositions. The "if" clause describes the relationsnip between the postulated condition and the proposedresult. For example, "If Colombia can increase its beef production by 15 percent and reduce the costofproductionby 10 percent, then it can successfully export beef andmeet domestic requirements." The first clause "If Colombia can increase its beef production by 15 percent and reduce the cost of production by 10 percent" sets the conditions that must be met and the remaining clause relates the proposed results.
Hypotheses are derived from the observations and relationships accepted as, or assumed to be, fact in the problem statement. They provide the guidelines for the type of data and techniques necessary for analysis. This implies that hypotheses are formulated before the data collection activity of the research project has started. In this sense hypotheses indicate the direction for data collection; hypotheses that are formulated to explain observations aftertheyare collected may not be useful for prob31
lem resolution. The hypotheses, then, are a necessary link between the problem and the data collection and analytical stages of the research.
The basis for correct formulation of hypotheses is the knowledge of the researcher, this knowledge being founded primarily in theory.3 The broader the experience of the researcher in relating theory to applied problems, the more efficient he will be in formulating appropriate hypotheses. Whereas the researcher and the client, jointly, must share the responsibility for problem specification, it is primarily the responsibility of the researcher, as the expert in his field,.to formulate the hyptheses.
Characteristics of Hypotheses
Hypotheses appropriate to applied research have the following characteristics:
1) They must be formed as if-then relationships
and stated in such a manner that their implications and relationships to the problem can be shown logically. The explicit use of the words if-then is not necessarily required; the relationship, however, is critical and often an explicit if-then statement will
assure an accurate relationship.
2) They should be stated as simply as possible
both interms of theoretical complexities and implications and in terms of number of variables.
3) They must be capable of verification or rejection within the limits of the research resources.
4) They must be stated in a manner which provides
direction for the research. The hypotheses,
3For this argument and others relating theory to the research process see [35,45,53,63,73.
when well formulated, will suggest the appropriate data and analytical techniques for testing that should be employed in the research process. Thus, a set of hypotheses
can be thought of as a plan of action.
5) Taken together, they must be adequate and
efficient in suggesting one or more meaningful solutions to the problem. They must
provide for an acceptable level of confidence in the results, but at the same time economize
in the use of scarce research resources.
Some Examples of Hypotheses
To continue with the proposed dairy study, the fllowing are the hypotheses which were submitted inthe second draft of the proposal:
1) "Increasing the level of technology and of
physical production and the economy per unit of exploitation, in association with a reduction in the costs of production, would result
in stability between supply and demand."
2) "Providing more financial resources for increasing production and restructuring the market channels would allow simpler price
3) "Establishing milk regulations would provide
optimum quality anda price warranted by that
Although all the hypotheses are generally related to the problems as stated, it is quite obvious that they encompass a much broader front than we were imagining from the reformulation of the problem statement. This revelation leads one immediately to the suspicion that the hypotheses violate the second requisite of hypotheses -- simplicity. The first hypothesis encompasses a fair proportion of all economic theory and certainly does not clearly specify how one moves from supply to demand with ease. Any reasonably low-cost means of testing this hypothesis is hard to imagine. The second and third hypotheses are apparently related to
the problem through the concern with price fluctuations and, perhaps with the low profits of the producer. However, the relationship to the problem is only tenuous -at best, and actually introduces new concepts that were not in the problem statement. Rather than helping to clarify the research proposal, as they should, these hypotheses tend to add confusion. Clearly, they do not provide direction or
guidance for the research which is one of the primary functions of hypotheses.
Because the author of the hypotheses in the example did not progress beyond this point, we must now begin to act as if we were the researcher and formulate the hypotheses according to our understanding of the problem. In-.doing so, it may be necessary to better identify the problem itself (normally done in consultation with the client) so that the proposal can be improved, but because this occurs many times in the formulation of a good research proposal, we will not be wasting effort. Few people are capable of writing an acceptable research proposal on the first attempt.
Several modifications usually are necessary as the orientation and focus become clearer. The final version of the problem statement in the previous section read as follows:
ttThe low level of technology on dairy farms contributes to high costs. of production, low average productivity, and a deficient marketing
system for milk. These factors cause low profits to the producer, price fluctuations f or the consumer, and adeterioration in the balance of payments because of the necessity to import milk."
Let us assume that we were correct in deciding that thia orientation of the proposal was more toward production than toward marketing. Let us also accept the relationship-s expressed inthe problem statement although many of us may
not be comfortable with some of them. Then, presumably, the focus of the proposed research project is the low level of
technology on dairy farms, this being the cause of the unfavorable relationships expressed in the problem statement.
Because the hypotheses must be testable within the resource limitations, care must be exercised in comparing
resource requirements and limitations before continuing with the formulation of the proposal. Assume that the client in this case is the Planning Department of the Ministry of Agriculture. In preparing their plans for the following year, they are considering the long range necessity of importing milk. They have requested a series of studies to
help them clarify the situation so they can make firmer predictions and to aid them inestablishing domestic policies. They consider that the country is capable of producing more milk but are not sure why it is not doing so. One factor that is evident is that there is a low level of technology used in the dairy industry. In this particular study their desire is to determine why technology has not
improved in recent years, because general knowledge far exceeds implementation. Professionals will be provided to work full time on the project, and the Ministry wants a preliminary report in four months and a final report in six.
Generally, calculator and computer facilities are available and it will be possible to hire the services of some interviewers for a short period of time if necessary.
The first hypothesis might well treat the profitability of the new technology because, if it is not profitable, most assuredly the producers will not adopt it. For a beginning, let us use:
1) "There exist in the country improved methods
of dairy production which, if used by producers, would increase their profits."
Stated in this fashion the hypothesis does indicate the possible direction which the research might take. In veri f ying or rej ect ing it one met hod might b e a s imp le review of literature to determine if we can satisfy ourselves as to the existence of profitable new technologies. Perhaps the technologies which are available have never been subjected to tests of profitability, which would indicate
the need f or some partial budgeting or possibly linear programming for a series of typical farm resource situations. Of course it might also be true that the client is satisfied that there are profitable new technologies and there35
fore does not desire or require verification of this hy pothesis. Assuming that we will need to include this hypothesis in the project, further classification will be made in the statement of the objectives and the procedures.
The hypothesis can be stated in an i4-then form, "If farmers adopt presently known technology then their profits will increase,"1 and the relationship to the problem statement is clear. Hence, it satisfies the first requisite of hypotheses.
Depending on the other hypotheses which are derived,
all of the previously discussed means of verification or rejection of the hypothesis could be accomplished under the limitations of the budget which we have described.
The theoretical implications of the hypothesis are about as simple. !as it is possible to make them and still retain meaningful relationships to the problem. To be prof itable, the new technologies must fit within the resource
restrictions of the farmer and must be profitable given present or projected price situations. Notice that it is
not adequate, even though it is simpler, to hypothesize only that the new technologies could increase farmers Iproduction.
The last criterion of hypotheses applies to all of them taken together, so it cannot be applied individually
except in the sense that it would not be adequate to consider only an increase in production without considering, at the same time, the profitability of the practice.
Assuming that in-the course of the research we will be able t~o accept the first hypothesis, we must then consider additional hypotheses because the first, alone, is not adequate to answer the question put to us by the client. The second hypothesis might be that,
2) "Farmers are not aware of the new technologies."
As stated this is aknown fact, but the i-t~-hen implication is that if farmers are not aware of profitable alternatives then they will not (or cannot) adopt them. The direction of the research in this case is also indicated. A sample
survey of farmers should be able to provide the evidence necessary to accept or reject the hypothesis.
Another hypothesis could treat credit, or the related financial situation of the farmers.
3) "If special credit sources are not made
available, farmers will not be able to
adopt the new technologies."
This can be considered a broad hypothesis with two subhypotheses which will be tested,
a) "Farmers are unable to adopt new technologies
because internal financial resources are limited."
b) "Farmers are unable to obtain credit, which
limits their ability to finance changes. in
their production techniques."
An additional hypothesis could treat the problem of price instability from the point of view of the producer.
14)~ "A milk price stabilization program could induce
farmers to adopt improved methods of production,." This hypothesis is not quite as straightforward as the others. To be able to verify or reject it empirically, a great deal of time and money would be involved -- certainly more of both than we have available within our research budget. However, attitudes of farmers toward a price program could be ascertained and certain conclusions drawn regarding the p'tobabte outcome of such a program. Again, if thi s hypothesis is included, the precise nature of the research process will need to be specified in the objectives and the procedures.
Other possible hypotheses could be suggested, but those proposed above are adequate for present purposes. Probablvi, all can be accomplished within the research budget, but it will be necessary to clarify them further (and perhaps modify them) as we develop our statement of objectives and procedures.
In summary, our hypotheses are the following: 1) "There exist in the country improved methods
of dairy production which, if used by the producers, would increase their profits.
2) Farmers have not adopted the new methods because they are unaware of their existence.
3) Special credit sources are necessary if dairy
farmers are to adopt improved methods of milk
a) Dairy farmers are unable to adopt new
technologies due to internal financial
b) Dairy farmers are unable to obtain credit,
which limits their ability to finance
changes in production methods.
4) A milk price stabilization program could induce
farmers to adopt improved methods of production."
Are these hypotheses, taken together, adequate sand efficient in suggesting a means to one or more meaningful solutions to the problem? If they cover the range of possibilities open to the government (extension programs, credit programs, and/or price programs, for example) they should be adequate in suggesting guidelines to one or more meaningful solutions. They are efficient if we cannot contrive other hypotheses which could provide solutions to the problem with the use of fewer of our research resources, do so in less time, or result in more precise information within the research budget.
Delineation of the Objectives
Objectives usually are expressed in lay terminology and are directed as much to the client as to the researcher. The primary objective of applied research will be either 1) to'suggest or recommendtothe client practical means of problem resolution, or 2) to provide information to clarify an unknown situation. Generally, the objectives taken as a group will 1) define the limits of the research project for the researcher, 2) clarify the means of conducting the research, 3) identify the client or clients, and 4) -describe the expected product of the research for the client.
The objectives link the theoretical relationships presented in the hypotheses to the analytical and methodolgical
orientation necessary for conducting the research. An objective specifies what the researcher intends to do or find in the project and suggests one or more research procedures to be used. Later in the research proposal the specific procedures must be defined. These procedures ofcourse, must be appropriate, not only to the stated objectives but also to the resource availability as discussed in Chapter II. It is necessary, therefore, that the objectives be suffi-ciently broad to satisfy the needs of the client but also sufficiently specific to conform to budgetary restrictions.
An erroneous idea of the nature of the objectives of a research projectshould be clarified. Research objectives are neither political objectives nor are they objectives of an action program of the government. Objectives of a research project suggest what information will be obtained for the client to help resolve the problem which initiated the research. This information, in turn, can be used by the client to establish an action program. For example, "to redistribute land in Mexico" suggests a government action program.This cannot be an objective of a research project. But, "to study the past and present agrarian reform programs in Mexico and their effects on land redistribution" is a legitimate research objective related to a felt need concerning land use and productivity. In the terms of our dairy problem, "to establish a credit program for dairy farmers" is not a legitimate research objective. though it may beafeasible solution inthe mind of the researcher and the client. "To determine if dairy farmers are able to obtain credit for improving methods of production." or "to recommend methods of resolving credit deficiencies if identified,"1 on the other hand, are acceptable research objec-tives.
Given these conditions, the primary objective of the dairy study which we are proposing could be, 1vto determine the obstacles to the adoption and optimal use of new technology by dairy farmers." This objective precisely describe the purposesof the project. Accomplishing this objective
will provide information to clarify an unknown situation for the client, and, if we are successful, should provide him with the information he needs tomake policy decisions and predictions concerning future milk supplies.
Secondary objectives can include the following, listed in order of their relationship to the hypotheses:
1) Determine if presently known modern technology
is profitable to the dairy farmers given their present resource situation and market outlet.
2) Determine if the Extension Service is effectively providing necessary information to farmers concerning possible alternatives for
3) Determine if the required changes to adopt
new technologie on farms are outside the
financial means of the farmers.
4~) Ascertain whether present credit sources are
adequate in providing for the farmers'I needs
related to new practices.
5) Obtain farmers' opinion about a price stabilizat ion program and their possible reaction to it with respect to changes in use of technology and concurrent production practices.
Notice that each of these objectives is directly related to ahypothesis and help to clarify the direction of the re-search. If the client does not understand the terminology used in the hypotheses, he should be able to understand the nature of the research from reading the objectives. Also,. the objectives clarify the means of conducting the research. It is clear that interviews with dairy farmers willbe necessary and, in general, what the content of these interviews will be. Specific questions must be written from the guidelines given in the objectives. These questions either singly or as groups will provide the information to test the hypotheses.
Of special interest is the fifth objective. The related hypothesis (a milk price stabilization program could induce farmers to adopt improved methods of product ion )stat es a positive relationship without giving any indication of the nature of the test which the researcher has in mind. The
client in such a case could well formulate an erroneous impression of the nature of the results which the researcher
proposes topresent him. The fifth objective specifically states that the researcher expects no more than to obtain the farmers? opinions and possible reactions. This is clearly different from empirical evidence which the client might otherwise expect if he only had the hypothesis for information.
We have been talking about a special or explicit client but other clientele could also benefit from the research in question. If the researcher is hired specifically by a client, and the client considers the researchprivate,then,, of course, he is the only one who will directly benefit from the work. However, as is more often the case, this type of research is undertaken byapublic or semi-public entity so the research becomes public property and the identification of a broader audience is useful. In the context of the pressent problem, the full clientele could be identified in.a sixth objective:
6) Provide information to farmers about the
profitability of new methods of production, to bankers and other credit institutions of possible sources of new business, and to government planners to aid them in making decisions concerning the
future of the dairy industry.
Three of the most important and critical aspectsof the planning of a research proposal --problems,hypotheses,and objectives -- have been presented and discussed. These three parts are not independent from each other, nor are they independent from other portions of the proposal and aspects of conducting the research to be discussed in following chapters. They serve as a framework for developing the data collection and analytical procedures, the budget including time sequences and the publication plans for the research results.
Time spent in careful development of the problem statement, the hyptheses, and the objectives is the key to efficient research and can well be the most productive use of
time by the researcher. Even in cases where the researcher may have only one month, one week, or perhaps just one day to provide an answer, the time spent in this phase of the research is critical to the success of the undertaking. On many occasions whenaperson is givenarush task, the tendency is to "come up with something." Little time is devoted to analyzing the situation to determine precisely what the client wants, what is really needed, and what resources are available to accomplish the necessary task. Too often the results have no value because the "something"which the researcher "comes up with" is not really related to the problem of the client.
Problems appropriately specified for applied research have the following characteristics:
1) They are based on felt needs of individuals,
groups, and societies;
2) The causal relationships expressed in aproblem statement are not hypothetical and are
relevant to the problem;
3) Problem statements must suggest testable hypothetical relationships that, when analyzed,
yield relevant and non-trivial results;
4) The problem and the research to resolve the
problem must be relevant and manageable with
Researchable problems can be distinguished from problematic situations in that numerous researchable problems can be formulated from a problematic situation.
The hypotheses serve as guides to executing the research. Hypotheses must:
1) Be stated to provide direction for the research.
2) Be formulated as causal relationships with
3) Be capable of tests within the limits of the
4) Be stated as simply as possible; and
5) As a group be adequate and efficient in
suggesting means to one or more meaningful
solutions to the problem.
Objectives in general describe what is expected to be achieved by the project. Specifically, objectives:
1) Define the limits of the research project; 2) Suggest or clarify the means of conducting
3) Describe the nature of the potential research
product to the client; and
4I) Identify the client or clients.
The following problem statement, hypotheses and obj ec.tives are those of 'the example discussed in the chapter)
The low level of technology on dairy farms contributes to high costs of production, low average productivity, and a deficient marketing system for milk. These factors cause
low profits to the producer, price f luctuations for the consumer, and a deterioration in the balance of payments because of the necessity to import milk.
1. There exist in the country improved methods of dairy production which, if used by the producers, would increase their profits.
2. Farmers have not adopted new methods because they are unaware of their existence.
3. Special credit sources are necessary if farmers are to adopt improved methods of production.
a. Farmers are unable to adopt new technologies
due to internal financial resource Limitations.
b. Farmers are unable to obtain credit, which
limits their ability to finance changes in production methods.
14. A milk price stabilization program could induce farmers to adopt improved methods of-production.
4 See the appendix for an example of a complete project statement.
1. To determine the obstacles to the adoption of new technology by dairy farmers.
2. Determine if presently known modern technology is profitable to the dairy farmers given their present resource situation and market outlook.
3. Determine if the Extension Service is effectively providing necessary information to farmers concerning possible alternatives for production.
4~. Determine if the required changes to adopt new technologies on farms are outside the financial means of the farmers.
5. Ascertain whether present credit sources are adequate in providing for the farmers needs related to new ,practices. 6. Obtain farmers opinions about a price stabilization program and their possible reaction to it with respect to changes in use of technology and concurrent production.
The chapters to follow will continue with discussions of data sources and collection, and analysis and presentation of research results, all of which depend upon well specified problem statements, hypotheses and obJectives.
EXPERIMENTAL DATA COLLECTION
The execution of research involves the collection of data which pertain to the project, the utilization of the
data to test the hypotheses (usually called analysis of the data), reaching conclusions which are useful in the resolution of the problem which initiated the research, and making appropriate recommendations to the client. If the project has been properly planned, the type of data which are needed have been determined prior to conducting the research. The researcher will know if only secondary information will be used or if primary data collection will be necessary. The planning phase of the project will also determine if
experimentation is necessary or if the source of data will be non-experimental. If experimentation is to. be used, a great deal of care must, be exercised in experimental design.
Experimentation has been the principle basis for obtaining scientific information and will continue. to remain of paramount importance. The use of non-experimental data, however, is becomingmore prominent, due to improvements in measurement techniques and development of more adequate dat~a series. The advantage of experimental data over non-experimental data is basically the degree of control which the
researcher is able to exert over the variables included in the study. In most experiments, with the use of the proper design the researcher can select the factors which will vary, the levels at which they will be included in the experiment, and the pattern in which they will be used'. By using appropriate equipment he can also usually obtain quite accurate measures of the input variables as well as of the results
of the experiment. With non-experimental data, the levels
and combinations of variables are predetermined by nature or society, so the researcher must measure and use them as they exist. Because these variables are very difficult to identify and measure, non-experimental data are usually more subject to inaccuracies than are experimental data.
Experimental design, the form in which the experiment is to be set up, is a critical factor in the generation of experimental data.1 There are any number of designs which can serve a particular need, but there will usually be one that is better suited to the conditions under which one experiment is going to be conducted and which is the most efficient in the use of the research resources. Selection of the most efficient experimental design will provide more information directly related to problem resolution for the given set of research resources than any other alternative design.
Selection of the number of treatments, the number of controlled and measured factors, the levels at which the variables are to be included' and the number of replications to be used in the experiment is not a simple process. The final choice can be complicated in applied research because the conditions under which the researcher is working are often poor in relation to what he would like for improved precision. Good research planning is extremely helpful to the researcher in choosing an experimental design. This is so because the design must be related to theproblLmand the methods of analysis to be used and these should have been carefully considered in the planning phase of the project.
1Selected references on experimental design include [3,4,7,261.
In the choice of a design.$ then, the researcher will be able to anticipate the type of information which willbe forthcoming from any particular design. He will be able
to predetermine the applicability of the design to the problem and to the methods of analysis which are appropriate. The researcher will also ascertain whether or not the experiment can be conducted within his resource limitations.
Relationship to the Problem
Although to achieve measurement accuracy with experimentation is possible,. it does not necessarily follow that experimental data as a source of information for a research project will assure research precision. If the design of the experiment is not properly related to the problem orientation of the project it will not be possible to achieve precision in determining relationships useful to problem resolution. An example may best serve to illustrate the point.
A common type of design in fertilizer experimentation is the following, with each treatment being replicated a number of times (frequently four).
Treatment Number N P K
10 0 0
2 0 0 1
3 1 0 1
4i 2 0 1
5 0 1 1
6 1 1 1
7 2 1 1
8 1 1 0
In the project porposal an objective might read something like, "to determine the effect of nitrogen (N),phosphorus (P) and potassium (K) on the production of pangola grass in the Cauca Valley." As we now know, of course,part of the difficulty here is that we do not know what theidentified research problem is. But that aside, the objective as stated is vague and is not an adequate guide for designing the experiment. Toward what end is the experiment oriented? Does the researcher want to know simply,"is there a response to N, P, and K?" -- or is he more interested in the magnitude of the response, or even the kind or form of response over a range of N, P, andKinvarioumombinations?
Each of these questions may well require the use of a different experimental design, number of treatments or number of replications in order to achieve precision and efficiency in arriving at an answer. In some casestwo or more questions can be answered efficiently from one experiment, but in others, the attempt to answer too many questions from a single design may render the experiment useless for answering any question.
Without discussing the theoretical logic for the statements, the following can be said about the relationship of Design I to some of the questions which could be asked. When the researcher is interested in knowing only ij there is a response to each of the three nutrients, it is not necessary to include all the treatments.and levels of N in the design. A simpler design such as the following could be used:
Treatment Number N P K
10 0 0
2 1 0 0
3 0 1 0
14 0 0 1
Here, instead of eight treatments there are only four. Using four replications of each treatment, only 16 separate parcels are required rather than 32 as would be necessary in the first design if four replications were also used. The choice of the level of each nutrient (1= 100 kg/ha or
1 = 200 kg/ha, etc. ) is important because this design provides information only for the one level chosen.
If the researcher wanted to know the magnitude of the response for two different positive levels of N, it-would
be necessary to include the three different treatments (0, 1, 2) of this nutrient. Apparently in the design shown it was desired to know something about the magnitude of response to Nfor two different levels of P, because the N treatments are repeated for 0 and for 1 P. This question can be answered with the use of treatments 2 through 7 of Design I.
The last treatment of Design I can only answer two questions, 1) "Is there an effect from K when N = 1 and P = l?" This requires treatments 6 and 8. No other information will be available on the ef fect of K. The last treatment along with the control (treatment 1) can show the response of N and P together when no K is applied.
Because we are interested in applied research, let's
suppose that the ultimate use of the data from the proposed experiment will be to make fertilizer recommendations to farmers. To what extent does Design I permit us to make the appropriate analyses? One alternative is to compare the cost and return of each treatment with the control (0-0-0) and determine the net return from each. Assuming that there
were significant differences between treatments, we could then recommend the treatment with the highest net return.
Another alternative would be to determine the functional relationship between N and production (production function for N) for each of the two levels of P. Assuming the curves thus generated were of proper form (satisfyingtheconditions for optimality), we could determine the economic optimum quantity of N for each level of P, make a cost-return estimate for each, and choose the one with the higher net return as that which we would recommend.
The second procedure would probably result in a somewhat different recommendation than the first method. There is obviously less precision with respect to N in the first method than in the second but theremay be even less precision with respect to P because we have only two treatments to choose from. Apparently, the design is not well suited to answer the question the farmer might ask, "What is the best combination of N, P, and K to use?" even though it does appear to be at least reasonably suitable for answering some other questions.
To answer the farmer's question may require a more complicated design and could require more resources than are available, but much more precision is possible if there are sufficient resources. Asan example, a complete factorial with 3 or 4 levels of each nutrient would require 27 or 64 parcels respectively for eachreplication, but could provide rather precise answers for the farmer (the use of the 43 factorial, of course, would provide more precision than would the 33 factorialbut it also requires more resources). Another, more efficient design is the rotatable central composite which is amodifiedfactorial that requires only 15 parcels for each replication when three nutrients are considered. For this last design, fewer replications are necessary so that with only about 40 parcels a complete experiment can be conducted,2 providing a wide range of information.
2All Treatments are not replicated the same number of times .
In summary, the problem toward which the research is directed has a strong bearing on the type of experimental
design which should be chosen if the researcher is going to use experimentation in the execution of the project. Should the magnitude of the problem require a very detailed experimental design, the researcher should be encouraged to consult with a competent statistician when one is available.
Also, if the researcher is going to use secondary experimental data, the design which was used in the experiment should be considered when choosing between various sources. More will be said on this aspect in a later section of this chapter.
Relationship to Resources
The ultimate size and complexity of an experiment depends on many factors including number of independent factors to be controlled or measured, analytical techniques to be
used, statistical precision required, quantity and quality of prior information available, the objectives of the research project, the time within which results must be obtained, and the resource constraints encompassing the research project.
If it is necessary to determine the best combination of N, P, and K, a rather complex design with a relatively
large number of experimental observations (parcels) is required. But if it is simply not possible to conduct such
a complex experiment, the research project must be modified For example, if some information is available which indicates little or no response to K, then the number of independent variables could be reduced to two (N and P) and K could be excluded or held constant at some predetermined level in the experiment. The best combination of N and P can then
be determined with a rotatable central composite design requiring only about 26 experimental units rather than the 40 units required for three independent factors (N, P, and K).
The design can be simplified further by considering only one independent factor. This factor could be the one considered most important such as perhaps N or it could be some fixed combination of several factors such as a complete fertilizer containing N, P, and K with an analysis of 1030-10, for example.
An experiment requiring only about 15 experimental units will yield as much statistical precision for one factor as 26. for two factors. But it is important to realize that the total amount of information is reduced accordingly as the
number of independent factors and resource requirements are reduced. With three factors and 40 parcels, we should be
able to tell the farmer, who is on similar soil as that used for the experiment, how much N, how much P, and how much K he should use to achieve the greatest net returns from fertilizer use. With two independent or variable factors, we have the option of telling the farmer the best combination of only two nutrients and can do that only for some predetermined quantity of the third nutrient. With only one variable factor it is not possible to recommend best combinations but only the best quantity of that single factor. We may be able to recommend 300 kilos per hectare of 10-30-10 as the best quantity of that complete fertilizer but we would not be at all confident that the resulting 30 kilos of N, 90 of P, and 30 of.-K would be the best'combination of the three nutrients to use.
The effect on resource requirements of the analytical technique to be used is more complex, but~ an example will
serve to illustrate the point. Suppose that it is desired to determine if there are significant differences in the response of three different levels of new feed additive on fattening steers. Aprobable design would have three treatments with the additive (one for each level to be tested) and a control. This results in four treatments per replication. Under most conditions three to four replications
have been found to be required to obtain reliable statistical
estimates and provide enough degrees of freedom for this type experiment and the analysis of variance which would be used. A total of 12 to 1 6 experimental units (pens of cattle) would be required to execute the research.
An alternative to analysis of variance is-regression analysis. If this technique is envisioned for the feed additive experiment the same four treatments could be used and if the interval between treatments was not uniform, they could be adjusted to make them equal (a desirable though not necessary attribute of a design for regression analysis). In analysis of variance, the observations from only two treatments are compared at anyone time. In regressional analysis the observations from all the treatments are considered simultaneously. For this reason roughly the same amount of
statistical precision can be obtainedwith two replications of the four treatments for regression analysis as with four replications of the four treatments for analysis of variance:. Hence, regression analysis may require fewer experimental
resources than analysis of variance. Similar relationships hold for other types of analysis which may be required in
the project and should be considered when choosing the design to be used.
The statistical precision required in experimental results is associated closely with the nature of the problem for which the research is being undertaken. Lower levels of confidence are acceptable in research dealing with animal health than with human health, for example, and lower confidence levels may be satisfactory in some social research than in some biological research. But regardless of the
nature of the research, an increase in the precision required is almost always associated with the need for more experimental resources. For any given research objective or experimental design, more replications and/or more treatments will usually result in more precise statistical estimates than would fewer treatments or replications.
In addition to increasing the number of treatments or
repicaion, precision can also be increased by closer supervision and better control during the experimental process. Spotty application of fertilizer by hand broadcasting on
grass plots can result in large experimental errors as. can carelessness and lack of thoroughness during harvest. More time, workers with more kill, or specialized machines or
equipment can reduce this source of experimental error, but all will add to the requirements for experimental resources. From a practical point of view, the least expensive mean s of reducing experimental error from this source is closer
supervision by the researcher during all phases of the experimental process. Time spent at the experimental site by the researcher can be highly productive particularly when it is possible to prevent the complete loss of data through carelessness.
The use of prior information can reduce the need for
resources by reducing the range and number of treatments otherwise required, by taking advantage of known estimates of variance to minimize the number of replications, and perhaps even by providing all the data needed making further experimentation unnecessary. In the following section we
will discuss in more *detail the use and utility of secondary experimental data, whichis one sourceof prior information available to the researcher.
The time limit within which adecisionmust be made is also an important factor in determining the size and complexity of an experiment. If time is not a critical factor, an exploratory experiment of relatively simple design can
be conducted and this can be followed by an experiment which is more exact in its focus. The range over which the experiment is conducted can be successively narrowed until the
required accuracy is achieved. None of the experiments in this sequence need be large nor complex. But if timing is critical and decisions must be made rapidly, it maybe necessary to increase the size and complexity of the experiment in order to include several links of the chain all at the same time. That is, to conserve on time (in this case
a very limited resource) other available resources may need to be substituted in order to achievea usefulresearch product within the time periodallowed for making a decision.
Secondary Experimental Data
When experimentation is,undertakenin the execution of the researchproject, theexperimentaldesigncan and should be tailored to the needs of the project. The data obtained will then be the best available and will be well suited to the needs of the researcher. But it is not always necessary to conduct an experiment to have suitable or adaptable experimental data for analysis, because in most places where applied research is being conducted, at least some prior experimental data are available. These data, though generated for other research purposes, frequently provide an insight into the nature of the relationships to be studied in the current project, and may provide sufficient data so that additional experimentation need not be undertaken.
In situations where time is a limiting factor, the use of secondary experimental data may be advantageous or even essential, but data generated by another person or from one or more other experiments must be usedwith caution to assure that they are relevant and comparable. Some adjustment or selection may well be necessary to adapt the data to the current study.
Several means can be used to adapt, select, or adjust data of this nature. Onemeans is to select only the relevant portions of the data and exclude those parts not related to the current analysis. An example would be to select data in a fertilizer experiment from those parcels where potash was at a constant level, eliminate plots where trace elements were included, and make the analysis for the portion of the data in which only nitrogen and phosphorus were variable.
A second method of using secondary datais to analyze
the results of several experiments and search for con; istencies which may indicate relationships not otherwise evident. In the Cauca Valley, Colombia, production functions for a series of regional fertilizer trials in corn resulted, individually, in very poor statistical estimates. For any one trial, little confidence could be placed in the conclusions.. But after dividing the trials into soil types, it was noted that there was a great deal of consistency in the functions within a soil type. The form of the curves was similar with each reaching a maximum within a short range, andwith calculated optimum applications at about the same levels. Hence, by using informationfrom all the curves, together, general recommendations could be made even though the individual analyses yielded little information.
A third useful method of analysis is to consider the
possibility of combinations of datafrom two or more different experiments which were conducted under similar conditions. This method was used to estimate the optimum stocking 'rate for fattening steers on grass from three different experiments E50]. Each experiment was conducted to determine the ef fect of hormones, and all were conducted on similar grasses, under similar conditions and with comparable cattle. Because the rate of stocking was different in each case, it was possible to use a mathematical response relationship from which the optimum stocking rate could be calculated.
One additional point with respect to secondary data should be mentioned. When one has spent time attempting to use secondary experimental data, he appreciates the productivity of any efforts made by the first researcher to preserve the data for other users. Nothing is more frustrating than to discover a descriptionof an experiment that should have provided usable inf ormat ion and then to f ind the data in such poor shape that they are impossible to use. Simple notations such as units of measurement are often not even included.
The thoughtful researcher will leave a clear record of his data so that users who follow will have no trouble interpreting them.
As a practical matter, a great deal of experimentation is carried out with an orientationthat is only partly research centered. An important example in agriculture is the demonstration trialusually conducted by, or in cooperation with, the extension service. One of the purposes of this type of research is to demonstrate the results of research under realconditions, i.e., under conditions which will be applied by the client toward whom it is focused. For this reason rather poor experimental control is to be expected, and accordingly, the experimental design is nearly always quite simple. In many cases, as few as two treatments, with and without a particular input or package of inputs, are included. At times, a complete experiment is conducted at one location; in other trials, different locations are considered to be different replications of the same experiment with all treatments being repeated at each; and in some cases only one or a few treatments are conducted at each location.
It should be obvious that as more and more locations are included in a demonstration trial, exposure to the clients is increased, but experimental control is decreased. Also, with more treatments in a project, more information is possible, but also, the supervision of the project becomes more difficult. Hence, the persons responsible for the project must determine,based on the orientationof the project and the available resources, what the size and scope should be.
Presenting an example of a fairly successful demonstration trialmay be the best means of discussing some of the more important aspects to be consideredwhen initiating a project of this nature. This project was conducted in southwestern Colombia in an area not far from the city of Cali. The area comprised that of a large number of descendants of former slaves who had dividedtheir holding into an average of about 2 hectares per family. The farms were nearly all
in old stands of cocoa, coffee, and plantain which were in such poor condition that monthly income per farm barely reached fifteen dollars. General nutritional deficiency was eviden as was the nearly complete absence of any source of animal protein. Physical conditions for any type of livestock were very bad, and, of course, little or no money was available for supplemental feed.
As part of a development program for this area, it was thought that Khaki-Campbell ducks (an egg laying breed)could prove to be a potential source of animalprotein to supplement the diets of these poor rural families. Indesigning the project, several alternatives were possible. The simplest of those considered reasonable was to select a small number of families (those most apt to be conscientious in maintaining the necessary records), to give each about 10 ducks andprovide themwith a recommended concentrate ration. The results couldprobably be establishedwith a fairlyhigh level of confidence and would show whether or not the ducks could survive under the conditions of the area. It would also be possible to determine the cost of the eggs to the families and whether or not they would eat the eggs.
A second approach that was considered was to use more families and experiment with several rations for the ducks. At first, three rations were considered adequate, with three families providedwith each ration. In this manner, it was hoped to be able to determine not only if the ducks would survive and lay under the severe conditions prevalent in the project area, but also what the lowest cost ration would be. In other words, there would be three different rations to choose from. Although more information could be obtained, more families would be needed and more supervision required to assure that the rations were correctly administered. The extension personnel who were cooperating in the project felt they could find nine families and had sufficient resources to do the majority of the supervision. Over 100 young female ducks were going to be available, so the extension people
began to locate the cooperating~families.
As it turned out, 16 families were eager to cooperate in the project, and the extension people felt this number should not be too great a supervisory burden. The use of this many families would reduce the number of ducks available per family to seven. It was decided that this would be satisfactory, but that it would be more efficient in the use of the families to include four rations and have four replications of each. With four rations it was hoped that there would be sufficient information to estimate a production function for eggs and with this information be able to determine the lowest cost ration even if it were not one included in the experiment.
The final design included one group with a complete concentrate ration, one with 2/3 of this amount, onewith 1/3of a ration, and a control group with no concentrate. Except for those receiving a full ration, all could receive whatever scraps were available and be allowed to graze (or be fed chopped grass). Material for constructing adequate housing for the ducks was available locally at no cost and all families were required to be ready to receive the ducks by a fixed date.
A one day short course was held with all families participating sufficiently far enough in advance of the delivery date to allow time for construction of the housing. The care of the ducks was discussed and also the design of the experiment and its purpose. The families were all given very simple record forms and instructed in their use (to be able to use the forms it was not necessary to be able to read or write or even count). Cups were provided to measure the exact amount of feed for each duck for each ration so that in the event of losses, the adjustment of the ration would be simple. At the close of the course a meal including duck eggs was served to assure the families that the eggs were good to eat since there was some local bias against eating the duck eggs.
With the encouragement of the extension agent, all par60
ticipating families were ready on the date that the twelve week old ducks were delivered. Initially, extension field. men and the researchers checked frequently with the families to ascertain if everything was being done properly. The
records were collected every two weeks during the whole project to assure they were kept current.
To be sure, there were rather large dif ferences in the results for each ration. One family, for example, never was rewarded with even one egg. Although they were a bit embarrassed and the neighbors were sure they were not taking good care of their ducks, they took it good naturedly because they were assured that this was not unexpected and was a necessary part of the experiment. Nevertheless, by having four replications and using the average for each ration, adequate information was obtained. It was possible to determine the
best level of concentrate to use if the eggs were going to be sold, as well as the ration which resulted in the lowest cost for the eggs if they were to be used to supplement the family diet. The second use, of course, was the primary purpose of the project.
Upon completing the analysis of the results, a field
day was held for the families at which time the results were presented. After the general results were shown, those of each family were discussed in an attempt to determine why some had better results (hence lower costs) than others with the same ration. Following this discussion, all the families participated in the selection of the final recommendations which were published [551.
In summary, this project was tailored to fit within the ,resources available and yet to provide the greatest exposure in the project area. Although experimental control was relatively low, sufficient treatments and replications were
included to maintain adequate statistical reliability inpart due to the presence of a high degree of supervision by the researchers.
Multidisciplinary experimentation is another? means of
conserving scarce research resources. When researchers from two or more disciplines cooperate in a project it is frequently possible to obtain answers for each with little change in the basic experimental design. 3 Too often a researcher in one disc ipline minimizes the effects of those factors commonly included by others, so that in the absence of cooperation, the product is of low value to other researchers. An example occurs in beef or dairy projects where the researchers raise their own corn for silage without the cooperation of the corn researchers who may be at the same experiment station. The animal researchers may be able to document accurately
the effect of the silage on the animals but be unable to estimate animal production perhectareof corn because they had little interest in the corn production. Even more diff iculties will arise if it is later desired to make an economic evaluation of feeding silage to animals .
Of course, it is not always possible to obtain answers for more than one discipline without increasing the size and the complexity of the experiment beyond manageability
and available resourc-es. But it is precisely in those cases when reso urces are most scarce that it is important to consider the advantages of multidisciplinary research. Most experiments are costly both in terms of time and of other
resources so it is obvious that if the results can serve two or three researchers, the additional effort required in planning the experiment can be extremely valuable .
Experimentation and experimental design usually are
associated with objectivity, precision, and scientific purity
--concepts that imply rigidity and inflexibility in thought
3Fra brief and excellent discussion of multidisciplinary cooperation in extending experimental results to practice see .
and procedure. In basic research this is mostly true, but in applied research, considerations other than pure scientif ic objectivity can become more important in determining the type of design to be used for any given experiment. Resource limitations will usually force a 'reduction in the accuracy obtainable, and factors such as demonstration uses will modify the nature of the design ultimately selected. It is important that the applied researcher maintain a flexible attitude with respect to experimentation and experimental
design in order to increase his effectiveness and make him more efficient in his work.
NON-EXPERIMENTAL DATA COLLECTION
A major difference between experimental and non-experimental research is the degree of control the researcher exercises over the variables being studied or measured. In
an experiment, the researcher controls the design and levels of certain variables and the measurement of phenomena resulting from the experiment. In non-experimental research, the researcher in most instances cannot determine the design and level of the variables nor directly measure the phenomena, but controls only the technique used in measurement (primarily a sample survey and questionnaire) For nonexperimental observations sample survey design plays a role comparable to that Of experimental design when experiments provide the observations to be analyzed. Again, a competent statistician, if available, can be a helpful consultant.
The non-experimental. researcher relies upon interviews and questionnaires to communicate his measurement need to a respondent. Often it is the respondent, not the interviewer, who performs the measurement based upon his experience. The respondent's experience may be compiled, for example, in farm records or consumer budgets, but frequently his responses are based upon asubjective evaluation of the phenomenon as he best remembers it.
Thus, in non-experimental research the researcher usually controls only the general levels of variables, through stratification and sample selection of respondents, questionnaire development and interview training. Successful non- experimental measurement rests primarily upon all of these activities. But even with a good sample, a tested questionnaire, and a well trained interviewer, the re64
searcher cannot completely control interviewer-respondent communication, part of which may be misleading [161.
This chapter will first focus on the selection of respondents and the design and implementation of questionnaires and then discuss verification and preparation of both primary and secondary data for analysis. No specific emphasis is given to unstructured interviews and case studies because
most of the approaches included are applicable in these situations. Time and resource restrictions along with the needs expressed by the hypotheses and objectives will dictate whether the measurement process will be a large survey or a case study .
The selection of the respondents, or sample design, is an extremely important part of the process of non-experimental data collection. Because the researcher has no control over the distribution of the factors he is studying (these exist as aresult of nature and social organization), the proper
selection of the respondents is necessary to assure that the information is being obtained from an appropriate population. Once respondents have been selected and the interview process is completed, a faulty or ill-designed sampling procedure
may generate data that do not describe the target population. To resolve~this difficulty more time and resources, if available, must be expended than would have been necessary with a more carefully designed sample.
Two commonly employed sampling approaches are the random sample and the stratified sample [8,18,20]. For information of a broad census nature, a completely random sample gives each person in the total population (all farmers, high school students, married women, etc. ) an equal chance of being chosen. In this manner, results can be associated with the characterisitics of that population. Stratified samples where particular sub-groups are chosen are used to assure that each sub-group is equally represented, or to assure uniformity or representation over specific ranges of
certain group traits (different farm size or different income groups for example). Such a strat if ied sample can approximate experimental measurement but with less precision than is normally associated with experimental research. A strat if ied sample is often more efficient than a random sample in use of scarce research resources because the sample size required may be samller thus reducing both interview and data processing expenses.
In drawing a sample, one hopes to have available a reasonably good listing of the general population including some descriptive characteristics to aid in stratification.
To help determine the sample size measures of variance within the population may then be calculated and used along with resource limitations and levels of confidence desired by the researcher and his client.
Many times in appplied research, these general characteristics of the population are unknown, making sample selection more difficult. One researcher studying farms in
a jungle area was forced to utilize somewhat unorthodox but otherwise relatively functional techniques . In one of the three zones of the region to be studied maps were available showing farm locations. A simple random sample of farms was easily obtained from these maps. In another zone "trail maps" were available but the farms were not included. The approximate number of farms in total was known and the
trails and farms were, in general, uniformly distributed throughout the region. A complete enumeration to develop the sample frame would have been difficult and would have
required more time than was available. Theresearcher chose to send out interviewers by mule with instructions to obtain an interview at every f if th f arm. The sampling problem was thus handled as the interviews were taken. In the third zone, maps were extremely poor and little was available to guide the sampling process. To assure that the entire zone would be represented, the interviewers were instructed to ride their mules for one hour and then taken an interview at the nearest farm, again proceeding along the trail. As
one might expect, errors did result, but adjustments in the
analysis were based upon general observations made by the researcher while in the zone.
Had logistical problems not been so extreme, the above researcher could have used asequential sampling technique. That is, based upon a predetermined level of confidence, he could have measured the variance in selected population characteristics by using the interivew data taken daily While in the field. From this calculation he could then determine
when the sample had reached the size needed by reducing the variance to meet the desired confidence level. When that level had been achieved he could have moved to another zone.
A further need for flexibility in the sampling process to allow for obtaining needed information is illustrated by a research project recently completed in the East Uttar Pradesh of India . Initial interview attempts with Indian farmers. ooncerningtheir landholdings were met with considerable suspicion and apprehension, possibly, because of speculation about land ceiling legislation. Interviews
conducted in villages as planned encountered farmers unwilling to.provide the desired information. Yet when the farmers were selected at random while traveling along the roads in the district they responded readily to questions about village and other matters related to land owernship. Since personal id entity and specific location of the respondents were not imperative the sampling approach was changed. substantially but to the benefit of the entire research effort.
In summary, a flexible researcher who places majorem.phasis on problems and views his sampling techniques as tools and not ends in themselves will be able to perform applied
non-experimental research under much less than optimum condit ions.
Designing the Questionnaire
At the risk of seeming elementary, it is advisable to
emphasize that questionnaire development must be closely
tied to the problem, hypotheses, obj ectives, analytical techniques, and available resources for the proposed research. Just as the research proposal or plan must reflect a relevant problem, and the hypotheses and objectives must pertain directly thereto, the respondents to be interviewed and the questions to be asked must be relevant tothe research proposal. Beyond relevancy, a questionnaire must include only those questions that are highest in priority for analysis and testing. A multitude of questions could be "relevant' or "interesting" but only a limited number can be used fully in the project because of research resource limitations. Interview time, for example, represents a costly resource allocation not only for the research project, but also for the respondent. The researcher must always remember that excessive and/or irrelevant questions try the respondent's patience, reduce interviewer-respondent rapport, threaten credibility of the responses, and waste scarce resources of the research project.
Difficulties in Interpretation and Communication
The method of obtaining information in a questionnaire is thespoken word, so language problems must be given special consideration. Obvious problems arise when questions are translated from one language to another, but even within the same language, word and expression usage vary with different cultural, social, educational, and economic classes. Differences of interpretation may exist between the researcher and the interviewer as well as between the interviewer and the respondent. There will also be differences between respondents in the interpretation of some questions. The magnitude of this difference will increase as the heterogeneity of the sample increases.
The complete elimination of language problems is nearly impossible, but careful consideration of cross-cultural communication difficulties will minimize problems of interpretation in the questionnaire. Two major cross-cultural commu68
nication problems in questionnaire formulation are differences in terminology and cultural differences in beliefs and values.
The names by which things are known or the terms which are used to describe something vary widely even within a language and culture group [171. These differences in terminology must be considered in the construction of the questions. Generally, the lower the general level of education and the greater the degree of isolation, the more important it is to take local usage into account. When groups are isolated, indigenous objects and phenomena frequently have names which are known only within the groups. Objectsor phenomena which are not indigenous will probably have similar names, but if they are unknown within the group, they may be difficult to explain. Terms used by professionals maynotbe understood at all by the respondent, yet if the interviewers are allowed freedom in explaining these unknown terms, an interpretational bias may arise. In some cases,professionals may be able to communicate with each other and with the interviewers with one question, but several questions maybe necessary to obtain the information from the respondents of a survey.
A common example of varied usage in Colombia is the problem with land measures. Four measures, the hectae, the plaza, the cuadta, and the fanegada, are used in different parts of the country. When a survey is conducted, one must construct the questionnaire to account for this variation of risk interpretational errors in measuring farm size, crop yields, and other factors associated with land area.
A different problem arose in Colombia with respect to
a question concerning tapeworms in swine. In each of three different departments (states), adifferent local term had to be used in the question to avoid erroneous conclusions
1The question is from a questionnaire which provided information for a Masters thesis .
which could have been drawn when determining the respondents knowledge of the incidence of this parasite in the survey area.
Unknown or misunderstood differences in beliefs and values are the other important cause of interpretational problems. An example can be drawn from interviews with housewives in a Colombian village where the drinking water was badly contaminated . Many of these housewives realized the importance of boiling their drinking water and understood the technique involved but still did not do it. Informal interviews determined how many housewives did boil
water, how many realized they should,. and how many understood the technique, but the survey failed to determine why the
majority did not actually boil water.The difficulty. developed because appropriate questions were not asked due to inadequate cultural knowledge. The professionals assumed that. boiling water was "good", and based upon survey results and their
own normative considerations, they determined the housewives "'should" have been boiling water. Only after further questioning did the researcher discover that the villagers believed that water was boiled only for the very ill. Those who drank boiled water were considered to be ill, so to avoid this stigma, many who otherwise might have boiled their water failed to do so.
Designing for Data Retrieval
A questionnaire must lend itself to efficient and useful data collection, processing and analysis. The allocation of time and resources 'between fieldeollection and data retrieval and verification from completed questionnaires needs careful consideration when designing a questionnaire. Long and detailed qualitative responses to open-ended questions
may be more informative than quick semi-quantitative responses involving selection of alternatives, but the time required
to correctly obtain, retrieve and analyze one long response may be the same as for twenty short selective response questions. One must determine which of these two techniques (or
combinations of both) most efficiently provide information while minimizing interpretation and response errors.
Both collection and verification are facilitated by logic and consistency and by organizing the questions to lead the. respondent with ease through the questionnaire in a manner that stimulates spontaneity inhis responses. If one question asks about his marketing problems for rice, and is followed by the age of each member of his family, and the next returns to the price of rice, the interview becomes burdensome and less effective. One exception tothis rule is important. If one desires to cross-check or verify a particular response because of interpretational problems or a reluctance to respond accurately, it may, be desirable to ask the same question twice, each time in a different manner and at different points in the interview.
Dividing the questions into major groups will help reduce respondent fatigue while maintaining interest. Groups of questions that involve sensitive issues;such as income or profit that may be subject to taxes, might follow several relatively less sensitive groups of questions in order to establish rapport and confidence. Sensitive issues covered prematurely in.a questionnaire may destroy the entire interview and thus should be asked toward the close of the interview.
Coding, or preparing the data for computer analysis, is necessary if the data are to be punched oncards for electronic data processing. Precoding which refers to specifying on the questionnaire the column requirements for computer cards, can eliminate the need to transfer the data to code sheets by permitting the data cards to be punched directly from the questionnaire. This method requires more careful editing of the questionnaire but usually reduces retrieval costs and the chance of human error by removing one transfer of data. To be more efficient, however, the researcher must be certain beforehand of how he wants to organize and process the data. The editing and checking process, on the other hand, helps assure that all necessary data will evolve from
the questionnaire. Precoding is less advisable for highly
qualitative responses or when the researcher has no indication of the range of responses he might expect for each question.
Pretesting the Questionnaire
Pretesting the questionnaire or checking to see if it will obtain the information sought, must be accomplished under actual field conditions before beginning the general survey. No amount of intellectual exercise in the office can substitute for properly testing a questionnaire among the respondents in the area to be surveyed. This phase will determine the weaknesses in the questionnaire, establish whether the information sought can be obtained in a useful form, and also may provide additional information which can help improve the relevancy of the questionnaire.
Whenever possible, the interviewers who will be conducting the survey and the director of the research project should be involved in the pretest to assess problems with
the questionnaire and with respo ndents. Prior to pretesting, interviewers should become familiar with the research project and the questionnaire. Each interviewer should be carefully briefed and sent to several of the different areas which have been selected to assure experience with a variety of respondent types. When each interviewer returns, a debriefing with the researcher should be performed immediately. This debrief ing should include a review of the.-completed questionnaire
to identify points of misunderstanding, unnecessary repetition, time difficulties and so on. General reactions by both the respondents and the interviewers concerning the subject matter and length of the questionnaire should be discussed.
For some interviews care should be taken in selecting the areas where the pretest will be performed to avoid the danger of contaminating amajor target area by pre-testing the questionnaire within-its bounds. Especially in close72
knit rural areas, advance knowledge of attitudinal responses inthefinal survey couldbeaffected by word of mouth discussion among potential respondents and those participating in the pretest.
Size of Pretest
The required number ofpretest questionnaires depends upon the research problem, the homogeniety of the survey population, the data collection and analytical techniques to be employed, the total number and complexity of the questions tobeasked, andtheresearch resources available fortheproject. The range maybe from five to one-hundred or more questionnaires. If the population is very heterogeneous with respect to information required, more interviews willbenecessary for an adequate pretest than fora questionnaire tobe administeredto amore homogeneous population.
If the pretest is alsotoprovide desired information about variance to determine the population homogeneity for sampling purposes, more pretest interviews may be needed than when only a test of the questionnaire is desired.For potentially large studies where general population parameters are unavailable, arather extensive pretest can aid in designing the sample. The marginal costs associated with these additional pretest questionnaires must be measured against costs associated with sampling errors due to having too few respondents. Extra costs resulting from obtaining excessive questionnaires in the main interview effort may be greater than the costs for a few additional questionnaires in the pretest.
Howapretest improved a particular questionina survey of settlers in the colonization project in Caqueta, Colombia is illustrated as follows .
Where did you live before you moved here?
In which Department were you born?
Where did you live before moving to Caqueta?
Where did you live before moving to this farm?
The researcher wanted to know from which departments the settlers had migrated to Caqueta, but the original question only revealed information about therespondent's last move which may have been within Caqueta and not from another department. Even though questions were added, the results were easier to interpret. Based on this researcher's information needs, the first question had no value so he was forced to choose between no question and three questions.
A pretest will assist the researcher to develop an efficient but completequestionnaire. A poorly pretested set of questions designed to obtain information about farm management practices and production costs for potato producers in Colombia illustrates this point.3 The respondents were asked the number of hectares devoted to potato production for the most recent year at three different points in the questionnaire. At the beginning, producers were asked to give total farm size and land use by crops, pasture, and other classifications. Later in the interview they reported on potato plantings for each semester. This information often did not agree with the first question because of double counting. It was difficult to verify the response because plantings in the first semester were usually much larger than in the second, so the extent of the double counting was impossible to determine. And finally, when queried about hectares harvested the area given was usually slightly different from that previously used. Could it be assumed that those hectares not harvested were
2From a portion of a producer questionnaire used in Colombia and summarized in .
lost to blight or drought? Perhaps, but it would be better to be certain that the difference was not in part, due to either intentional or unintentional reporting error. Not
only was the duplication unnecessary, but the questions were ambiguous. A better pretest could have minimized this discrepancy by guiding the preparation of one comprehensive and easily understood set of questions related to hectares of potatoes seeded and harvested for the past year.
After completing the pretest, responses should be* tabulated and critically reviewed to determine whether the questions were understood clearly and whether the information
which resulted would help to resolve the problem. This step frequently uncovers gaps in information, or responses which are not in the best form for the analyses which are to be
undertaken. While the pretest may identify needed modifications in the questionnaire, it also provides an opportunity to determine whether or not the responses are in a form such that the hypotheses can be tested. The researcher should carefully consider what is being collected in the pretest and how it can be used in the hypothesis testing and analysis phases of the research.
Some research advisors insist that their students use pretest data on a "practice run" through the analysis which slows the research process, but only momentarily. Analyzing pretest data can aid greatly in refining and even specifying the, exact analytical tools which typically are not given too much thought until all of the data are collected. This may also provide some ideas asto the format for the final publication. If these benefits are gained from analyzing pretest data then the saving of time at a later stage in the
research process will more than compensate for the early time loss.
Often a pretest indicates that either too little or too much information is being collected. At times, the pretest may show that a certain type of information will not be avail-able, and therefore, some modification will need to be made in analyses, hypotheses, or objectives. More commonly, the questionnaire may be gathering more information than is ne75
cessary which suggests that costs of getting the excess should.be carefully considered relative to potential uses of the information.
Only when pretests have shown that interpretational problems are minimal and that the information can be obtained in the form required, should the questionnaire be finalized and the complete survey undertaken.
Because timing is so important, the pretest should be used to help specify interview time problems; time of year, month, week, and day must be carefully considered to fit the respondents' work schedule.3 If, for example, one is proposing to interview truckers at check stations or consumers at a retail market the time element may be more critical than in interviews with producers. Retailers usually prefer granting interviews when the fewest customers are in the store. For small retailers, adaytime interview can be three or four times as long as an uninterrupted nightime interview; but if an appointment cannot be obtained for nonworking hours, as is often true, the more lengthy daytime interview may be necessary. Agricultural producers probably will find harvest time to be inconvenient.
Numerous examples canbe cited but few guidelines can be specified for timing interviews because each survey and often each interview represents adifferent situation. In any case, should the respondent feel that the questionnaire is completely irrelevant and awaste of time, the responses will lack credibility evenwhen the interviewer is fortunate enough to complete the interview.
3Mail questionnaires help resolve time problems but often limit response quality and, due to low response rates, require large samples. A bibliography concerning mailquestionnaire research is found in .
Selecting and Training Interviewers
A well-designed and pretested questionnaire will not
yield good results if administered by an interviewer who is poorly trained or who has a disagreeable, non-cooperative arnd culturally or class biased attitude toward the respondent. The contact made with the respondent will not only
influence the interview but also influence the respondent Is evaluation of the entity for whom the interviewer is working, and his acceptance of action programs that might evolve from the research recommendations. Requisites ofa good interviewer include: 1) an interest in and an understanding of the research project, 2) an interest in people and the ability to communicate that interest and sincerity to a respondent, 3) a willingness and ability to follow instructions and definitions without regard for personal beliefs or convictions, and 4) in general, a good public relations attitude.
Trainihg is necessary to familiarize both experienced
and inexperienced interviewers with the problem to be studied, the abject ives of the research proj ect and the organization sponsoring the work, the questionnaires, the respondent selection procedures and the interviewing techniques. An interviewer's manner of asking questions must be objective and he must be neutral and honestin recording responses. The most important goal, of the survey is to get an unbiased opinion or bit of data for each question. Whatever the interviewer thinks of the respondent and his opinions should not influence interviewer objectivity.
Interviewers should be instructed carefully on methods for introducing the research project to the respondent. A most dif ficult question to answer is arespondent'Is inquiry about the legality of the study. Letters of introduction
and any other documents from official and credible sources should always be in the possession of interviewers. To estab lish respondent confidence, the introduction used by the interviewer and the letter itself should explain that the
interview is confidential, and that the information sought will be used to determine attitudes and characteristics of groups for comparative purposes, and not to illustrate char.acteristics of specific respondents.
Although the interview must be objective, the interviewer must be given flexibility to obtain a useful response. He should record specific comments and not vague or meaningless generalities such as "because it is interesting" or "I like it because it is good," Why is it interesting? Why is it good? The "I don't know" response is one of the most difficult tomanage. One is not sure whether the respondent really did not know, whether he did not understand the question, or whether he did not want to respond for various reasons. The respondent should be encouraged by the interviewer to believe that an "I don't know" response is not an admission of ignorance. A simple guess or an estimate calculated to suit what the respondent feels the interviewer wants is unacceptable. The respondent, however, should not be encouraged to give an "I don't know"
response when he most likely can give a meaningful and honest reply.
Verifying Primary Data
Verification of the data after it is collected is necessary before data processing and analysis begin. The researcher must become fully aware of the limits and potential uses of his data based upon the objectives of his research. A careful review of data with well performed revisions at this stage of the research process will reduce the chance of costly errors.
Numerous and detailed verification activities are necessary so the researcher can be confident that the data best measure the phenomena needed to fulfill the research
objectives, given the resource limitations placed upon the project. The verification and preparation process must consider techniques for handling missing observations, falsi78
fied data, completely inaccurate measures, and so on, as well as the necessary conversions where measuring units have dif fered. Often the required transformations corrections, and general manipulations performed on data are dictated by the type of facilities available for processing. Computer processing is not necessary for all research, particularly when projects are small and calculators are readily available.
Both time and monetary costs for each alternative should be considered along with the type of analysis desired and the degree of accuracy needed. But regardless of the processing facilities and techniques to be employed, well specified methods for verifying and coding datawill improve efficiency in research resource use.
During field data collection and while verifying and
coding results, the researcher should accurately record his collection and verification techniques. This record, commonly called a code book, is also oriented to describing data collection and verification techniques and it may also include some data refinement such as frequency distributions, means, standard errors and so on. It not only serves as
a guide to his present work, but also serves to indicate in detail what the data represent for use in further research efforts.
Verifying and Using Secondary Data
The researcher usually has no control over secondary
data measurement but he has the responsibility for checking very carefully to determine exactly how it was derived, the nature of the aggregations made and if the data will meet the needs of his research [651. Secondary sources can be classified into two major types: 1) regular and long-run measures of phenomena such as prices, income, production,
rainfall, and temperature; and 2) less regular and, in general, short-run measures of phenomena more often associated with another person's research such as crop response to improved inputs, consumer attitudes, and management characteristics.
Time series data can often be difficult to use and verify because they are secondary and not necessarily designed for a specific research project. Because time series frequently cover extended time periods, collection is performed by an agency which is responsible for maintaining continuity in the collection process. No particular person is responsible for collecting these data over the years, but certain procedures are established and revised occasionally to accurately and adequately measure the particular phenomena under study. Nevertheless, definitions should be checked carefully so that the researcher can decide what the data in question are supposed to measure, and since he has no control over the collection of these data, he must
satisfy himself that they in fact do measure the phenomena in which he is interested.
Comparability may be the most common difficulty in using times series data. This refers to 1) comparability of different time series which purport to measure the same phenomena and 2) comparability of different periods within the same series. For example, due to plant breeding and food processing programs, the corn produced and marketed fifty years ago and the corn produced and marketed today are virtually two different products. A price series for corn covering this period of time is not comparable during the ent-ire period. Coupled with product changes in a price series is the equally difficult problem of non-comparability in currency values caused by price inflation or deflation in the national economy.
A serious comparability problem, and one that is dif ficult to detect within a time series, is a change in definit ion. For example, a change in quality standards will af fect a price series, but may not necessarily be specified. A change in the point of measurement such as the location of price determination from the farmer-assembler level to the wholesaler-retailer level will influence the value of the economic variable in question. In census data one may find a change in the definition of a farm. Similarly, changes
in the population of acity may in part be due to changes in the boundary lines.
Non-comparability between series attempting to measure the same phenomena is common particularly in developing countries because time series measurement systems have often not been standardized under the authority of one agency. Unfortunately, in many cases these entities neither clearly define the phenomena to be measured nor the precise points and times of measurement. And when the points and times are defined, a change indefinition may not be spelled out after it is made.
As an example, consider the separate series of potato production data compiled by the Caja Agtatia (a public agricultural credit agency) in Colombia and IDEMA (a public agricultural marketing agency) [36, p. 230]. These series are vastly different, with the production data of IDEMA averaging about 40 percent lower than the data of the Caja Agraria. This difference is probably due to differences in methods of measurement of the series. The Caja Agraria primarily develops their series from producer loan data, while that-of IDEMA is based on measures taken in rural markets. Thus, Caja Agratia more closely estimates total production and IDEMA measures the production entering commercial channels. If the researcher is interested in the volume of commercial sales of potatoes, the IDEMA data probably are more accurate. On the other hand, if the researcher wished to estimate per capita potato consumption, including both rural and urban consumers, the IDEMA series would underestimate this variable by about 40 percent less losses and seed requirements.
Where atime series is needed but unavailable or possibly inappropriate for a specific problem, cross-section data sometimes can be substituted. Cross-section data (experimental and non-experimental) represent phenomena measured only at one point in time or a limited number of times. One means of using cross-section data to simulate longevity is in questionnaire research where respondents recall information concerning an event at present, last year, five years ago, and soon. Care must be observed in interpreting these
measures because of what has been termed a telescoping bias. That is, the tendency to completely overemphasize or underemphasize certain phenomena by projecting present conditions to previous points in time while forgetting others that may have influenced these observations.
The success of non-experimental data collection rests
upon the ability of the researcher to accurately sample the defined population and, once the sample is drawn, communicate with the selected respondents. In designing the questionnaire which is the means of obtaining information from
the respondents, two important communication problems must be considered: 1) differences in terminology between various groups, and 2) cultural differences in beliefs and values.
Pretesting the questionnaire under actual field conditions can provide information both about its effectiveness as a
data gathering tool and information on the population which can help in establishing sample size.
As in the case of experimentation, guidelines for optimum, sample selection procedures and questionnaire design in non-experimental research can be stated, but in applied research it is important that the researcher remain flexible in his attitude. Scientific perfection can serve as a norm, but the researcher must remember-that for the client, it is almost always better to have some information to help him make his decisions than to have no. information except that
the researcher is still designing a better questionnaire or trying to decide on the best means of choosing the respondents.
DATA UTILIZATION -- WHAT DOES IT ALL MEAN?
The real skill of the applied researcher comes into play after the collection of the data has been completed. Experience and imagination have a particularly high payoff in the analysis and the interpretation of the data and can make a difference betweenausefulproject and one which ends up in a file drawer. It is in this process that the researcher finally comes down to the point of determining what the data entail; data do not "speak for themselves" but must be interpreted and analyzed. Theresearcher must draw conclusions from the analysis and in the end make recommendations to his client to help in resolving the problem that originated the project. This, of course, is the reason for undertaking applied research in the first place. No amount of planning, no elegant data collection procedures, and no sophisticated analyses are goingto help the researcherwho is too timid when the moment of truth arrives to utilize all his information, draw meaningful conclusions and make appropriate recommendations to the client.1
When this moment arrives, the client is expecting a useful product and the researcher is the most knowledgeable person available to him. At the conclusion of this project, the researcher should know more about the subject being studied than anyone else with whom the client has contact. If this is not so, the client should have gone elsewhere for his information.And if the client had not needed the information
1For references particularly concerned with interpretation and communication of research results see [21,44,57,781.
he would not have contacted the researcher nor utilized the other research resources. Hence, the researcher must assume that his knowledge is vital to the client and that the client desires the fullest utilization of the resources which have been expended by the project. Also, because of the nature of applied research, the researcher is usually facing a deadline, so additional data collection is seldom possible. Conclusions and recommendations must be made on the basis of the data at hand because that is the best information that is or will be available within the allowable time and resource restrictions.
Rather than cover the myriad of analytical procedures available to the researcher, which are presented in detail in a variety of good sources, this chapter treats the more personal aspects involved in the inter~pretat ion of research results. 2 These are the aspects whiich might be called in part, the art of research, and which also might be called
subjective analysis or, by the purist, personal viewpoints and judgements. We prefer to think of subjective analysis as flexibility in ones attitude toward the scientific procedure and the approach discussed in this book. This is the attitude that allows the researcher to look beyond the numbers which result from the analysis. It is this attitude that
allows the researcher to completely milk the data and draw. out all the information which might be of help to the client. Furthermore, this attitude encourages the good applied researcher to insist on a role in the interpretation of the
statistical or other analysis which have been used (either by him, by another person, or by a computer) rather than accept these impersonal results without question. 3
sampling of basic texts covering analytical techniques includes [13, 20, 26, 6, 32].
3For a lengthy debate concerning the role of social science research in prediction and prescription see L45, 591.
Applied research is not useful to the client when the researcher reports that he had to go back for more data so has no conclusions, orthat based on such and such a level
of confidence there is no significant relationship between the variables. It is more useful to report that, although a relationship is not strong, and high confidence cannot be placed in the conclusion, there is a tendency toward a particular relationship and that because this response illogical, the relationship can be used in resolving the problem. Or elsewhere, just as strong a statement could be made for there not being a relationship between certain variables. The
client is depending on the researcher to draw a conclusion and make a recommendation. The client, then, armed with
this best estimate of the researcher, ultimately makes the decisions to be taken to alleviate the problem.
Because the client must make a decision, it is also necessary that he understand the information which the researcher presents as the results of the research. Too often the investigator writes his report as if he were communicating only with other professionals and thereby ignores the needs of the client toward whom the presentation must be directed.
In this chapter, two factors associated with the utilization of data in applied research are discussed. First, an attitude of flexibility in the analysis of data prevents the researcher from becoming boxed in by tradition to the
point that he is unable to understand what his data are trying to tell him. A lack of flexibility on the part of the researcher can impede the complete interpretation and full utilization of the data so that the client does not achieve maximum benefit from his investment in the research undertaking. The second aspect of data utilization to be covered is the presentation of the results in a form such that the client can adequately understand the implications of the project
and use the results accordingly in his decision-making process.
Flexibility in Interpretation
Flexibility in the interpretation of the data and the
analyses of the research project does not mean their manipulation to achieve Predetermined results. This defeats the purpose of a project undertaken to resolve aproblem. Flexibility refers to the capability to really comprehend what the data and the analyses mean and how the relationships which they express can be used to advantage by the client in making a decision.
In close conformity to the needs and desires of the
client, the applied researcher should utilize all his training and experience as well as the knowledge gained from the current project in order to provide useful information. This includes a complete examination of the results to determine their meaning as well as their reliability. Over-emphasis on measures of reliability and insistence on rigid standards frequently set for more optimum conditions or different problems, reduce the ability of the researcher to explore the data in more detail and to fully understand the meaning of the results.
Meaning of the Results
A common fault in the research process is to accept results of the analyses as something sacred, even if they do not appear logical. An important aspect in the research process is the selection of choice criteria, the measures
of performance, efficiency or success which serve as guides in the theoretic construction of the problem statement. The performance criterion which should have been of concern to the researcher in our opening dialogue of this book was the increase in the total production of certain crops in his country for which trade agreements had just been made. An agronomist working on the development of a new variety may have as a performance criterion the resistance of the crop
to a certain disease. The same agronomist also, of course, has a secondary criterion, that of increasing production per unit of land area in which the crop is grown. A farm economist usually considers the maximization of profit to
some resource base as the most relevant performance criterion to use.
Blind adherence to a predetermined set of performance
criteria and the lack of flexibility in considering alternatives can frequently obscure the real nature of the problem which is being treated. Three examples of errors in
interpretation owing to the misuse or the misunderstanding of the performance criteria will be discussed. These examples should provide the researcher with some ideas of the kind of flexibility that is needed in the interpretation of his research results.
Example 1.4 The performance of potato producers in
Colombia, as in most places, has been measured by research and extension specialists in tons per hectare simply because of professional tradition. Experimental yields per hectare have improved annually since the early 1950's to the point where they now at least triple and often quadruple average producer yields. Yet yields on both commercial and subsistence farms have risen only slightly over the same period. The research and extension programs are often under criticism for not stimulating at least part of the yield
increases which are known to be. possible. At the same time farm research reveals that both large and small producers are applying fertilizer and pesticides at levels near to those recommended, and many are also using improved seed. Total potato production has increased to keep pace with
population growth mainly through the dedication of more land to the crop, but yields per hectare remain embarrassingly low. Should producers, agronomists, and extension specialists be embarrassed? Maybe not.
Except f or the f ew potato f armers who rent land f or potatoes, land represents a relatively low cost input to even
4 Taken from .