TropSoils Program in Indonesia
REPORT OF THE EXTERNAL EVALUATION PANEL
BY JOHN K. COULTER, PETER E. HILDEBRAND AND STEPHEN L. RAWLINS
TropSoils Program in Indonesia
REPORT OF THE EXTERNAL EVALUATION PANEL
BY JOHN K. COULTER, PETER E. HILDEBRAND AND STEPHEN L. RAWLINS
1. The Panel membership was: John K. Coulter (Chairman), Peter E. Hildebrand and
Stephen L. Rawlins. Present for the field visits and discussions were: Charles B. McCants
(Management Entity), Ray Smith, Goro Uehara and Russell Yost (University of Hawaii),
Carol Colfer (formerly with the University of Hawaii), I.P.G. Wyaja-Adhi and H. Suwardjo
(Center for Soil Research) and Duane Acker (AID, Washington).
2. The Panel arrived in Jakarta Feb. 14. It visited the field program in Sitiung, West Suma-
tra Feb 15 Feb 18. It met with the Director, Center for Soil Research, and his staff in
Bogor and with resident staff of the USAID mission. The Panel completed its review and
reported the major aspects of its findings before dispersing on Feb 21.
3. A number of documents were presented to the Panel; these included budgets for 1986-
87, a work plan for 1986-87, a strategic plan for the Indonesian TropSoils program, reports
on the social scientist's contribution to the program (including annual reports for the period
1.1.86 1.10.86), reports on the expert system and an annual report for FY 85.
4. The Panel examined the research work in the field with a view to evaluating the quality
of the work and of the staff involved in it. The latter was of particular concern to the Panel
because the experienced staff members in the previous team had been replaced by two other
scientists, both new to Indonesia, and only one with experience in the developing world.
Previous visits to the Sitiung site by members of the Panel made it keenly aware of the need
to have a strong team, with good management, on site because of the isolation of Sitiung
and also the need for a close relationship with, and good leadership from, the Program
5. The major cuts in overall funding of the TropSoils program made it imperative for the
Panel to evaluate also the University of Hawaii's proposals for next year's annual work
plan and the longer term strategy for research in Indonesia. Such an evaluation was consid-
ered necessary in order to advise the Management Entity on whether the continuation of the
Indonesian program would be advisable in the present and projected era of financial strin-
BACKGROUND OF THE PROGRAM
6. This project on the acid upland soils of Indonesia arose from the need to evaluate some
of the findings of research done under similar conditions at the Yurimaguas site in the
Upper Amazon area of Peru and to expand on these findings in another location, and from
the need to find better soil management practices for the large developments for transmi-
grants from Java to the Outer Islands. Many thousands of these have been moved under a
government program which resulted in clearing large areas of tropical rainforest, often
leading to destruction of the topsoil by mechanical clearing and serious degradation by
erosion. Apart from lowering population pressure in overcrowded Java, the program was
designed to increase food production, particularly rice, at a time when Indonesia was a large
importer of this commodity. Now that Indonesia is self-sufficient in rice, there is no longer
the pressure to concentrate on this crop, except to make the settlers self-sufficient. On the
other hand, the Government of Indonesia is now emphasizing domestic production of soya
bean, to cut down imports, and is giving strong support to a program for growing this crop
on the acid upland soils. The nature of agricultural development is thus strongly influenced
by government policy, and research is needed to help the government formulate sound
policies as well as to help farmers develop more efficient and productive farming systems.
7. Until July 1986, the program at Sitiung was a joint activity of the University of Hawaii,
responsible for providing the Principal Investigator (now called Program Coordinator), and
North Carolina State University as the support organization. The University of Hawaii
provided two senior scientists on site and North Carolina State one. An innovative feature
of the program, due to the perception and insight of the Program Coordinator, was the
participation of both socioeconomic and biological scientists. The original multi-discipli-
nary team, with training before arrival in Indonesia and with a thorough knowledge of the
Indonesian language, were able to establish a good working relationship with local counter-
parts and farmers of the area.
PROGRESS OF THE PROJECT TO DATE
8. The work of the TropSoils project in Indonesia has been widely recognized for its
quality and its potential contribution to agricultural development on upland acid soils.
Much of this is due to the dedication of the senior TropSoils researchers who were resident
in Sitiung, their GOI colleagues and the graduate students from the University of Hawaii,
North Carolina State University and the University of Florida. Annex I gives a list of
reports on the progress of the work.
9. There have been no major scientific surprises with respect to soil management in the
findings of the project to date. It has been verified that bulldozing is the worst possible way
to clear these soils and that crops respond to even low levels of lime, nitrogen, phosphorus
and potassium. However, it was also found that organic matter, in the form of green ma-
nure, could substitute for low levels of lime, phosphorus and potassium, and that low levels
of these nutrients produced adequate, if not outstanding, levels of production. Early studies
showed that the income from annual crops alone was small and the return per man day was
much less than that from rubber growing. Addition of tree-crop and livestock enterprises to
the farming system could increase farmers' income substantially. Work on alley cropping,
green manure from cover crops, and the introduction of fruit and other trees into the farm-
ing system have been incorporated in the research program. Many of the findings of the
work in Sitiung have been incorporated by the Center for Soils Research into the work
being initiated at the Kuamang Kuning station in Jambi Province. Hence, research findings
are already being used to improve the technologies available to transmigrants in Sumatra.
10. The socioeconomic studies have provided important information on the transmigrants
and the Minankabau (the local people). The importance of forage was discovered in a time
allocation study because of the amount of time required to gather it. Also the importance of
the home garden, relative to the upland fields, has resulted in more emphasis being put on
this enterprise in the research program. The surveys showed that the transmigrants are
aware of the usefulness of fertilizers and pesticides, but, even where they have some cash
income with which they could purchase these, they have alternative uses that could preclude
them from using more than minimum levels of purchased inputs on their annual crops.
These alternative uses for cash include education expenses for their children and the pur-
chase of livestock, which, along with gold, serve as a form of savings.
11. Many of the biophysical and socioeconomic findings have been incorporated into an
"expert system" for lime use, which can be utilized not only for the Sitiung area but is also
expandable to areas outside the study area and can serve to help extrapolate the information
developed from this project. The model is in a form to accept new information as it be-
comes available, and is kept up to date at the University of Hawaii.
12. Individual members of the EEP have visited Sitiung on several occasions during the
course of the project. They have confirmed the technical quality of the on-site research but
were worried about the on-site overall management. The Chief of Party in Sitiung con-
cerned himself mainly with the logistics of the program (an important and time-consuming
task, considering the start-up problems and difficult logistics of the Sitiung site), but he
provided little scientific leadership or coordination for the program. Members of the Panel
were also made well aware of budgeting and financial management problems both during
visits and in the annual discussions of the program. These concerns were not about wastage
of money on-site indeed the team members lived and worked in a very frugal fashion but
rather about the laissez-faire attitude of the Program Coordinator about programs, budget-
ing and monitoring of costs.
13. On the technical aspects it has always been evident that there were really two research
thrusts included in the Indonesian program. Under Wade's and Colfer's leadership the on-
site priorities were primarily determined by them, based on their observations of the condi-
tions at Sitiung and their decisions about what was most needed to meet the needs of the
local farmers. These priorities took into account differences in values among ethnic groups
as well as serious technical limitations of farming systems based solely on field crops in
these highly weathered tropical soils. The focus was primarily on meeting needs in Indone-
sia. The Program Coordinator, on the other hand, placed primary emphasis on the construc-
tion of semi-variograms using geostatistical techniques to map soil variability, modeling
phosphorus-lime interactions, and developing expert systems to improve and transfer soil
management research information.
14. The dichotomy of these approaches really came to a head during the current review.
The 11 written project statements provided to the EEP prior to the review included projects
from both areas, but there appeared to be little linkage between them. At the end of the field
review the Panel expected the discussions to provide an opportunity to evaluate these
written project statements. The Panel needed to determine how these individual activities
could be more effectively integrated to deliver a product that would consist of something
more than the sum of its parts. This discussion never took place. Instead, on the next to last
day of the review, in an oral presentation, the Program Coordinator introduced an entirely
new research program. He stated that the project now had different people and that they
could not accomplish the same things as the previous team. His new program, to be jointly
developed with AARD and the USAID local mission, was to include the following projects:
1. Economic analysis using the expert system for lime recommendations.
This expert system was to incorporate new data from low-input systems
and be used as a new way to extend research results to farmers, using
lime as an example.
2. Develop a decision-support expert system to deal with environmental
degradation. This was to lead to practices to reduce soil erosion and
pollution from pesticides and agricultural chemicals.
3. Develop databases based on geostatistical techniques to drive the
4. Develop an expert system to incorporate the results of Colfer's socio-
economic research into a decision-support system.
5. Incorporate the potential payoffs from agroforestry into local farming
6. Soil physics. (Here the Program Coordinator stated that this program,
to be carried out by the new soil physicist, as well as the program of the
new agronomist on site, were wide open, and up to them to develop).
15. The Panel members were frustrated by this turn of events, for the program they came
to review had disappeared. The Panel was left with verbal statements of what the Program
Coordinator hoped would be in the next research outline, but obviously these were impos-
sible to review. The Panel got the impression that these new thrusts had not been fully
discussed or agreed upon by the Hawaiian team. As indicated earlier, the Panel considered
that one of its major roles would be the review of the future program. Without a well-
thought-through program to review, it looked seriously at the possibility of recommending
the termination of the work in Indonesia, but decided not to do so, because it considered
that much of the progress made by Colfer, Wade and Yost would be lost. The Panel there-
fore recommended that a planning workshop be held, and that it should include these
scientists. This was not a vote of confidence in the management, but rather in the hope that
cooperative planning by those who successfully led the program in the past could utilize
their experience to develop a sound program for the future.
16. The Panel worked with the Hawaiian management in an attempt to help them lay some
groundwork for this planning conference, but it is questionable that this is an appropriate
activity for an external evaluation Panel.
APPROPRIATE USE OF MODELS AND EXPERT SYSTEMS
17. Soils research by its very nature needs to be carried out by an interdisciplinary team.
Soil is not a commodity but a medium. It has no inherent agricultural value outside of the
products grown on it As a minimum, the research team must include experts in growing
and protecting the test crop. Economics and other important disciplines must be represented
if the team is to make recommendations that will lead to profitable crop production in the
short term and sustainable soil productivity in the long term.
18. Frequently it is impossible to locate all of the necessary disciplines at a single site.
Expert systems provide some hope for substituting for these vacancies on research teams. A
good example emanating from this research is the lime recommendation expert system. The
Panel can visualize this excellent contribution providing very valuable input to a team
without a soil chemist experienced with tropical soils. The expert system not only would
help the team ask the right questions, but from its specifically designed database it could
supply much needed knowledge from past research. Without the expert system and its
supporting database this knowledge would be hopelessly lost in the sea of information that
threatens to inundate everyone. An expert system developed by social scientists on the team
could also structure questions and give answers to help similar teams lacking the expertise
of a social scientist.
19. Models and expert systems also represent focal points around which the efforts of
teams can be coordinated. They are tangible products that require specific steps to be
followed in producing them. First, the team must define the problem the product is going to
help solve, and who the users will be. It must then determine from the users what the
specific requirements of the model or expert system are, including economic inputs. Next,
clear specifications for the product must be drawn up so that team members can independ-
ently develop parts that fit together into a functioning product. Some will structure the
database. Others will determine what is already known from the literature to fill the data-
base. Some will run experiments to fill gaps that remain in the database. Others will de-
velop the structure for the expert system or model, delineating the separate modules and
specifying how they interact. Someone else may code the computer implementation of the
model. All in all, a closely knit, well planned effort is required to lead to a successful
20. But with these and many other very good reasons for incorporating expert systems and
simulation models in research programs, there is a need to look at some serious limitations.
One limitation is high cost. The USDA/ARS is currently involved in an intensive effort to
develop a simulation model to replace the universal soil loss equation as a predictor of soil
erosion. The effort will require the help of at least 25 scientists working for several years at
an estimated cost of more than $10,000,000. Costs are similar for examples of expert
systems developed in industry, such as X-CON, for making certain the correct components
are included in orders for VAX computers. Even some of the smallest commercial expert
systems cost at least $1,000,000. The six projects outlined by the Project Coordinator in
paragraph 14 are all good ideas. Ultimately it is necessary to capture scientific knowledge
in expert systems where it can not only be used to directly support the decisions made by
end users, but also permit better communication among scientists. But it is impossible to
envision the successful achievement of these six expert systems within the time-frame and
budget of this project. Just because expert systems and models represent an important hope
for the future does not mean that everything else should be abandoned in their pursuit.
21. The Panel therefore recommends that the team continue to develop expert systems
similar to that developed for lime recommendations. But these should be viewed as means
to ends, not ends in themselves. The team would profit from a coordinated effort to build
the many parts that are still needed to provide the missing parts for this specific expert
system. This would provide the newly arrived scientists at Sitiung with a framework within
which they could develop an agreed research program, with well defined objectives and
clear lines of research. Such a program should include research on maximizing the impact
of limited quantities of lime, fertilizers and organic matter, including the recycling of
(FOR POLICY MAKERS AS WELL AS FOR FARMERS)
22. The Panel concurs with the desire expressed by all parties, including the USAID
mission in Jakarta, for economic analysis to help policy-makers as well as extension person-
nel and farmers understand the implications of the research being undertaken in Sitiung. At
the same time, the Panel recognizes the work recently completed by the graduate student
from the University of Florida. The preliminary analysis from this work has already been
incorporated into the expert system at the University of Hawaii and more complete analysis
is under way at Florida. However, it is also recognized that this work, in itself, is not
sufficient and that additional economic analysis, both of what farmers are actually doing
and of research results, is needed.
SUMMARY OF CONCLUSIONS AND RECOMMENDATIONS
STAFFING AND MANAGEMENT
23. The Panel cannot make categorical statements about the quality of the resident senior
scientists on the research team after such a short visit. Nevertheless, it has concerns about
the length of time that these scientists are taking to understand the problems of the area and
to familiarize themselves with the past research. This is reflected by the lack of specific
research proposals. A lack of understanding of the language and culture can obviously
contribute to this delay, but overall the Panel feels that these scientists may not contribute
the same broad understanding and expertise that the previous team members were able to
do. If this perception is correct then clearly the Program Coordinator will need to be much
more active in planning and managing the agronomic and socioeconomic program at the
site. However, no amount of guidance from Hawaii can compensate for inadequate leader-
ship on-site, and the Program Coordinator is clearly responsible for changing the personnel
if they do not measure up to the task. The Panel members are only too well aware of the
difficulties of finding good staff for such an isolated area, but must emphasize the need for
high-quality on-site staff.
24. The Panel was clearly concerned about the level and quality of management from the
University of Hawaii and with the problems outlined in paragraphs 12-16. It refrained from
recommending the most drastic step the closing down of the program because this would
be a major setback for the project, but it would certainly do so in the future if there were
not substantial improvements in the management. Such a responsibility falls squarely on the
Program Coordinator, but if he is unable or unwilling to undertake the additional work that
would be necessary, then the Panel would suggest that the responsibility be split, with the
present Program Coordinator remaining responsible for the technical quality of the program
and another staff member responsible for the management. The present Program Coordina-
tor would thus provide the University of Hawaii's leadership for determining the strategy of
the research and the development of the annual work program, while the manager would be
responsible for day-to-day management and implementation and for monitoring progress in
both the technical and financial aspects.
DESIGN OF THE FUTURE PROGRAM AT SITTING
25. The Panel recommends that Drs. Colfer and Wade, the former senior scientists who
were involved in most of the research, be brought into the program as consultants to work
for a period of two weeks in the design of the future research program and its detailed
elements. It expects that this could be done by about mid-May for presentation to the
Management Entity and to the EEP. The purpose of this exercise is to assure that the future
program be built upon the basis created in the past, thus avoiding abrupt changes, reflecting
only the personal interests of the new scientists. Research in which the former team mem-
bers have confidence need not be repeated except for farmer testing and evaluation. Further,
specific gaps in knowledge can be determined and can be incorporated into the new pro-
gram. Bearing in mind the discussion in paragraphs 17-21, it will be essential to achieve a
sensible balance between the field work at Sitiung and the work on models and expert
systems. The Panel emphasizes the need for urgency in designing the new work plan, so
that a year is not lost in the process.
FUTURE FUNDING OF THE PROGRAM
26. As indicated in paragraph 5 there have been substantial cuts in the overall funding of
the TropSoils CRSP, but the Panel believes that it will still be possible to carry out a mean-
ingful program in Indonesia with the projected level of funding. Additional funding would
not compensate for a weak research program or poor management of that program. During
the discussions held in the course of the review, the Panel expressed its ideas on the kind of
program considered appropriate and the elements of this have been outlined in preceding
paragraphs. It is, of course, necessary for the University of Hawaii, using the process
described in paragraph 25, to design the next year's work plan and the longer term strategy
for the program around these ideas and carry out the essential program budgeting work,
taking into account the potential contributions from the local USAID mission.
CONTRIBUTIONS BY GOI
27. The Panel considers that participation of GOI scientists in all aspects of the research is
essential. The involvement of scientists from several directorates of AARD is a step in the
right direction. Drastic cuts in the GOI development budget make it virtually impossible for
AARD to fund anything beyond salary costs, so that additional funding will be necessary.
On the other hand, shortages of operational funds will probably mean that GOI staff will be
more readily available for Sitiung.
28. The Panel would urge the GOI especially to provide a full-time economist for the
Sitiung site whose principal objective would be to work with the TropSoils team to collect
necessary data and to conduct appropriate economic analyses. The University of Hawaii
should provide backstopping support for this position if and as necessary.
29. There is no current consensus from the present team on which are the most promising
soil management technologies for the transmigrants to evaluate in an expanded on-farm
testing program. The two former senior scientists could use their experience to identify such
technologies and to advise on an on-farm testing program.
30. The Panel recognizes the efforts of the scientists who have been to Sitiung to conduct
on-farm research in order to make their results realistic. The Panel is aware of the increased
efforts required to conduct this type of research and would like to commend the scientists
for undertaking it and the Program Coordinator for encouraging it. However, there are
some ways in which the on-farm research could be improved, and the Panel would encour-
age the new team to incorporate such methods into their program.
31. In order to improve the on-farm work at the Sitiung site, two main aspects should be
considered. First, the work to date has used fairly substantial base treatments of non-studied
variables in order to reduce variability of results. This is accepted research procedure, but
when the research is being conducted with farmer adoption in mind, it leads to results
which are hard or impossible to interpret for farmer application. Farmers are not usually
able to apply the base levels of other inputs, so the production functions or other results
obtained from experimental work are different from those the farmers will encounter if they
use the same technology. In other words, the researchers cannot predict what response
farmers might expect. It is necessary, therefore, to conduct at least some of the on-farm
research under the same conditions as the farmers who will be using the technology.
32. Secondly, some of the technologies deemed most appropriate for farmer adoption
should be placed in a wide-ranging on-farm test program organized for the purpose of
farmer, as well as researcher, evaluation. In this effort it may well be possible to gain the
support of the extension service so diffusion can begin at the same time as farmer-accepta-
bility testing is conducted.
33. Finally, the Panel reemphasizes the need for urgency in completing and implementing
a new work plan for Sitiung so a year of meaningful research is not lost.
LIST OF PAPERS IN TROPSOILS FILES
1. Survey of small ruminant farmers in Sumatra.
2. Sistim tanaman industry di Koto Padang.
3. Faktor-faktor yang membatasi produksi usaha tani di daerah Sitiung
4. Peranan wanita dalam proses pengambilan keputusan dan kaitannya dengan pertanian.
5. Tree crop farming system pattern in Koto Padang.
6. Incorporating farmer conditions and preferences into farming system research.
7. Transmigrants' garden, a neglected research opportunity.
8. People factor in farming system; an anthropologist's view.
9. Preliminary analysis of Sitiung diet survey.
10. Tingkat pendidikan dan hubungan dengan proses pengambilan keputusan serta kaitannya
dengan pertanian di Koto Padang.
11. Time allocation study; a methodology food production system.
12. Diet and production survey of Sitiung.
13. On-Farm research in Sitiung with emphasis on maize.
14. Effects of phosphorous fertilizer rate.
15. Time efficiency and prediction for soybean.
16. The role of fertilizer in Indonesia food crop production.
17. Liming in transmigration areas.
18. The contribution of farmer managed research in technology development
19. Constraint to crop production a comparison of research.
20. Phosphorous fertilization management
21. Potassium dynamics.
22. Green manure applications at Sitiung.
23. Field experiments, 1985-1986 planting season.
24. Penelitian secara kerja sama dengan petani.
25. Sistim pengetahuan dengan tempat asal sehubungan dengan management tanah.
26. Survey tentang gizi dan pendapatan.
27. Studi pengelolaan tanah dengan petani di daerah Tampung Mini.
28. Pengambilan keputusan dalam keluarga.
29. Studi tentang pertanian tanaman keras Minang.
30. Penelitian Alokasi waktu.
31. People's perceptions of soil, Sitiung.
32. Pengaruh berbagai cara pasca penbukaan hutan terhadap sifat tanah dan
33. Response of rice and soybean to potassium fertilization, residue management
and green manuring in Sitiung.
34. The effect of liming on soybean yield and soil acidity parameters of three
red-yellow podzolic soils of west Sumatra.
35. Utilization of on-farm experimentation for agronomic technology develop-
36. A survey of small fishponds in Sitiung.
37. Social science farming system.
38. Field status report
a. November, 1984
b. February, 1985
c. March, 1985
d. October, 1985
e. March, 1986
f. May, 1986
g. June/July/August, 1986
FIELD RESEARCH BRIEFS
1. P fertilization and maintenance.
2. Effect of green manuring on food crop response to lime and p fertilization.
3. 1502. Phosphorus rates and methods of Application.
4. 1201. Tillage and soil amendments for land reclamation of abulldozed area in Sitiung.
5. Response of corn and soil base levels to applied at three lime rates in Sitiung 1.
6. Response of upland rice and soil base levels to applied K at three lime levels in Sitiung 2.
7. Effects of clearing methods on soil properties and crop production.
8. Response of upland rice and soil K levels to K fertilization and green manure applications at Sitiung 5.
9. P fertilization and maintenance Sitiung I A.
11. Response of soybeans and soil K levels to K applied at three lime levels in Sitiung 2.
12. Response of peanuts and soil base levels to applied K at three lime levels in Sitiung 1.
13. Response of soybean and solid K levels to K fertilization and green manure applications at Sitiung 5.
14. Phosphorous rates and methods of application.
15. Residual and maintenance for lime.
16. Source and management of green manure preliminary uniformity trials.
17. Sulfur responses and reactions in soils of Sumatera.
18. Effect of post clearing methods on soil properties and crop production.
19. CIAT regional trial B establishment phase.
20. Alley cropping trial tree establishment period.
21. Mungbean and cowpea response to application of K at three lime rates.
22. P fertilization and maintenance, Sitiung I A.
23. Lime reaction rate and effectiveness, 1002, 1103, 1404.
24. Effect of post clearing methods on soil properties and crop production.
25. Rice response to K fertilization, residue management and green manure application in Sitiung V.
26. Mg Fertilization for annual food crops.
27. Residual and maintenance rates for lime.
28. Lime reaction rate and effectiveness, 1002, 1103, 1404.
29. Continuous function (many mini) trial: soybean response to lime, P, K, Mg and green manure.
30. Lime residual and maintenance; with special "blocking" design.
31. Mg fertilization for annual food crops.
32. Farmer practice and production study characterization of home gardens in Aur Jaya (Sitiung VC).
33. Alley cropping experiment 1985/1986 growing season.
34. Farmer and crop responses to different sources of fertilizers: A farmer-managed study on home gardens.
35. Farmer practice and production study soil and crop management practices in Aur Jaya.
36. Source and management of green manure 1985/86 season.
37. Tree legume special assessment