The Relevance and Challenges of an Economic Evaluation of Florida's 4-H Program
http://edis.ifas.ufl.edu/ ( Publisher's URL )
Full Citation
Permanent Link: http://ufdc.ufl.edu/IR00000972/00001
 Material Information
Title: The Relevance and Challenges of an Economic Evaluation of Florida's 4-H Program
Physical Description: Fact Sheet
Creator: Flores-Lagunes, Alfonso
Timko, Troy Thomas
Publisher: University of Florida Cooperative Extension Service, Institute of Food and Agriculture Sciences, EDIS
Place of Publication: Gainesville, Fla.
Publication Date: 2008
Abstract: The 4-H Program -- The Importance of 4-H Economic Evaluation -- Economic Evaluation of Programs: A Primer -- Research Roadmap -- Conclusion
Acquisition: Collected for University of Florida's Institutional Repository by the UFIR Self-Submittal tool. Submitted by Diana Hagan.
Publication Status: Published
General Note: "FE753."
 Record Information
Source Institution: University of Florida Institutional Repository
Holding Location: University of Florida
Rights Management: All rights reserved by the submitter.
System ID: IR00000972:00001


This item is only available as the following downloads:

The Relevance and Challenges of an Economic Evaluation of Florida's 4-H Program ( PDF )

Full Text


FE753 The Relevance and Challenges of an Economic Evaluation of Florida's 4-H Program1 Alfonso Flores-Lagunes and Troy Timko2 1. This is EDIS document FE753, a publication of the Food and Resource Economics Department, Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences, University of Florida, Gainesville, FL. Published October 2008. Please visit the EDIS website at http://edis.ifas.ufl.edu. 2. Alfonso Flores-Lagunes, assistant professor, and Troy Timko, Ph.D. student, Food and Resource Economics Department, Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences, University of Florida, Gainesville, FL. The Institute of Food and Agricultural Sciences (IFAS) is an Equal Opportunity Institution authorized to provide research, educational information and other services only to individuals and institutions that function with non-discrimination with respect to race, creed, color, religion, age, disability, sex, sexual orientation, marital status, national origin, political opinions or affiliations. U.S. Department of Agriculture, Cooperative Extension Service, University of Florida, IFAS, Florida A. & M. University Cooperative Extension Program, and Boards of County Commissioners Cooperating. Larry Arrington, Dean The 4-H Program 4-H is America's largest youth development organization, serving over 6.5 million young people worldwide. The 4-H organization started in the early 1900's when educators began to emphasize the needs of young people and to introduce nature study as a basis for a better agricultural education. As a result of 4-H's roots in agriculture, it is most widely thought of as an agriculturally focused organization. A more accurate picture of 4-H today, however, is one of an organization that focuses on the development of life skills through a diverse range of subjects balanced around citizenship, leadership, healthy living, and science and technology programs. The 4-H program is operated and supported through a combination of public and private partners, including the National 4-H Headquarters; the United States Department of Agriculture (USDA); State Land-Grant University Extension; county-based governments; and national, state, and county 4-H foundations. Educators at 107 land-grant universities operate 4-H programs in every state and U.S. territory, including military installations around the world. Given the scope of the 4-H program, it is safe to say that few youth development organizations worldwide have the reach and potential of 4-H to make a positive impact on youth. The state 4-H programs are operated by and associated with the land-grant universities in each state. In Florida, 4-H is part of the Florida Cooperative Extension Service within the Institute of Food and Agricultural Sciences at the University of Florida (UF/IFAS), and receives support from combined efforts of the county, state, and federal governments. In 2007, over 234,000 youth were involved in Florida 4-H at a cost of over $21 million (UF/IFAS Extension 2007 Annual Report). Of these members, approximately 43 percent live in suburbs and cities with populations greater than 50,000 (Florida 4-H 2007). The scale and reputation of 4-H in Florida renders it an essential network through which the lives of Floridian youth are positively influenced. The Importance of 4-H Economic Evaluation There is ample evidence that 4-H positively impacts the lives of youth (Guion 2002;


The Relevance and Challenges of an Economic Evaluation of Florida's 4-H Program 2 http://florida4h.org/news/ impact_on_young_people.shtml). The evidence comes from studies that are particularly concerned with learning the types of attitudinal and behavioral impacts that the different activities and projects within 4-H have on participants. To accomplish their goals, these studies typically rely on surveys of opinions and perceptions provided by youth participating in 4-H programs. While these types of studies are extremely helpful in guiding 4-H in the design and development of its programs, they are less useful for the purposes of estimating the economic impacts of involvement in 4-H. Economic evaluations of programs such as 4-H are crucial in helping administrators and stakeholders undertake the responsibility of making informed decisions about the program. Their main purpose is to provide objective cost-benefit analyses of the program based on comparing its costs with the economic benefits over a particular period. Ultimately, cost-benefit analyses constitute an input into the allocation decision of resources, making them important tools for administrators and policy makers. Examples of recent large-scale economic evaluations of government programsusually mandated and funded by policy makers include the National Job Training Partnership Act Study (Bloom et al. 1997) and the National Job Corps Study (Schochet, Burghardt, and Glazerman 2001). These studies measured the costs and benefits of training programs for disadvantaged adults and youth, respectively. As a simple illustration of how an economic evaluation of a program leads to meaningful estimates of benefits to participants, consider a hypothetical example in which involvement in a youth development program such as 4-H is found to cause a significant increase in the probability of graduation from high school. Given this causal effect of the program, one can estimate an average dollar-amount impact of 4-H using information from the population about the average difference in wages between individuals who graduate from high school and those who do not. This estimate can be regarded as part of the long-term effects of 4-H on participating youth; a benefit that needs to be taken into account (along with other benefits and costs) in a comprehensive economic evaluation of the program. Economic Evaluation of Programs: A Primer The central problem in evaluating a program is similar to establishing cause-and-effect links. Holland (1986) calls this problem the fundamental problem of causal inference. This problem arises because outcomes for each individual can only be observed under one of two situations: the individual participated in the program or the individual did not participate in the program. One illustration of this problem is how participation in 4-H affects the grades of an individual student. For a specific range of time, the student would either participate or not participate in 4-H. After that period, an observation would be made to see how participation in the 4-H program affected the student's grades. Unfortunately, it would be impossible to compare participating and non-participating grades since the student cannot participate and not participate in 4-Ha comparison necessary to obtain the causal effect of 4-H on the student's grades. This is the fundamental problem of causal inference. Because the grades the student would have gotten had the other situation occurred are impossible to observe due to the established facts of the situation, they are often called counterfactual. Notice that, if the outcomes under both situations were observable, then the difference in the two outcomes could be causally attributed to the program. Given the impossibility to simultaneously observe outcomes under both states of the world for the same individual, the outcome under the state that is not observed (i.e., the counterfactual) has to be estimated to solve the problem and obtain a causal effect. Within this paradigm for estimation of causal effects of programs, the difficulty is that counterfactuals for each individual are missing, making it impossible to estimate causal effects for each person. As a result, estimation is centered around average causal effects in which the average missing counterfactual is statistically imputed to estimate the average causal effect of the program on a particular outcome of interest. Another important implication in estimating causal effects is that information is required on both program participants


The Relevance and Challenges of an Economic Evaluation of Florida's 4-H Program 3 and non-participants, such that there is enough information to be used in the imputation of the average missing counterfactual that allows the estimation of average causal effects. Finally, an important conceptual tool in the estimation of causal effects is to think about the process through which individuals are assigned to the program, as will be illustrated in the following example. This conceptual tool is useful in clarifying the assumptions needed to estimate the average missing counterfactual and average causal effects. A classic example in which the estimation of causal effects is crucial and can be used to illustrate the ensuing discussion is a medical trial. Suppose that a researcher is interested in analyzing the causal effect of a new cancer drug (a treatment) on a given outcome, such as the survival probability within the next year. Naturally, for each patient, only one state of the world can be observedreceiving treatment or notso that the causal effect to be estimated is the average treatment effect. Also, medical trials involve two sets of patients: those who are given the drug (i.e., the treatment group) and those who are not (i.e., the control group). Finally, a common method in medical trials to assign patients into treatment and control groups is to assign them at random. In this case, the random nature of the assignment mechanism guarantees that treatment and control groups are similar to each other, on average. As a result, the average counterfactual outcome for each group can easily be estimated with the average outcome from the other group. In this fundamental case of random assignment, an average causal effect is easily established as the difference in average outcomes between the treatment and control group members. Economic program evaluation can be seen as conceptually mimicking this example of a medical trial. Given the simplicity and uncontroversial nature of the results that follow randomized trials, the evaluation of several U.S. government programs has been based on social experiments that randomize individuals within a program (Bloom at al. 1997; Schochet, Burghardt, and Glazerman 2001). Random assignment provides a clear benchmark to gauge the alternative assumptions that are necessary to estimate the causal effects using data that do not come from randomized experiments. In the absence of randomized experiments, social scientists estimate average causal effects by utilizing assumptions that allow them to conceptually "restore" a random assignment mechanism (Heckman, LaLonde, and Smith 1999; Morgan and Winship 2007). These methods employ data on the outcomes of interest and on observed characteristics of both program participants and non-participants. Importantly, given that the estimation of average causal effects in the absence of randomization relies on assumptions, it is crucial to evaluate their plausibility to determine whether the estimates are indeed causal effects. Finally, the estimated economic causal effects of the program are combined with the related costs of providing the program to perform a cost-benefit analysis. Research Roadmap An economic evaluation of the 4-H program that emphasizes the estimation of average causal effects has never been performed in Florida or nationally. This void needs to be filled. A first step in the research agenda would be to conduct a survey of 4-H participants and non-participants in a sample of Florida counties. The survey would collect data on the demographic characteristics of both groups and the extent of participation in 4-H and other extracurricular programs such as school clubs. In addition, data on outcomes that are plausibly impacted by a youth development program such as 4-H would need to be collected. These data include information such as school grades, school attendance, standardized test scores (e.g., FCAT), and indicators of individual behavior while at school and outside of school. These data would be used in a pilot study for causal estimates on the impact of 4-H on Florida's youth. These causal estimates would offer preliminary cost-benefit analyses of the Florida 4-H program. An evaluation of this type would provide evidence on the economic value of the 4-H program, and could serve as a benchmark for its implementation on a larger scale.


The Relevance and Challenges of an Economic Evaluation of Florida's 4-H Program 4 Conclusion 4-H is a significant youth development program at the state and national levels. Considering the size, scope, and cost of the program, there needs to be an accurate picture of the economic impacts of 4-H. It is difficult, however, to estimate the economic impacts of any program without having reliable estimates of its causal effects. Obtaining such estimates is therefore a prerequisite to providing solid cost-benefit analyses of 4-H. The lack of studies that provide causal impacts of 4-H on youth at the state and national levels represents a void in the research on 4-H that needs to be filled. Filling this gap will require cooperation among the many individuals involved in the participation, delivery, and funding of this important youth development program. References Bloom, H., L. Orr, S. Bell, G. Cave, F. Doolittle, W. Lin and J. Bos. 1997. The benefits and costs of JTPA Title II-A Programs: findings from the National Job Training Partnership Act study. Journal of Human Resources 32(3): 549-576. Guion, L. 2002. Communicating the outcomes and impacts of 4-H. PE059. Gainesville: University of Florida Institute of Food and Agricultural Sciences. http://edis.ifas.ufl.edu/PE059 (accessed September 26, 2008). Florida 4-H. 2007. 2006-2007 Florida 4-H statistical snapshot. http://florida4h.org/news/files/ 07Statistical%20Snapshot.pdf. Heckman, J., R. LaLonde, and J. Smith. 1999. The economics and econometrics of active labor market programs. In Handbook of Labor Economics, edited by O. Ashenfelter and D. Card, pp. 1865-2097. North Holland: Elsevier Science Publishers. Holland, P. 1986. Statistics and causal inference. Journal of the American Statistical Association 81: 945-970. Morgan, S. and Winship, C. 2007. Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge, UK: Cambridge University Press. Schochet, P., J. Burghardt, and S. Glazerman. 2001. National Job Corps Study:The Impacts of Job Corps on Participants' Employment and Related Outcomes. Princeton, NJ: Mathematica Policy Research, Inc.