This item is only available as the following downloads:
Bryan D. Terry, Rick Godke, Bill Heltemes, and Lori Wiggins2 1. This document is 4-H 6.10, one of a series of the Florida 4-H Program, Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences, University of Florida. First published October 2010 on http://edis.ifas.ufl.edu. Please visit the Florda 4-H website at http://florida4h.org. 2. Bryan D. Terry, assistant professor; Department of Family, Youth and Community Sciences; Rick Godke, 4-H extension agent, Duval County Cooperative Extension; William Heltemes, 4-H specialist, Northeast Region, Florida 4-H; Lori Wiggins, 4-H extension agent, Taylor County Cooperative Extension; Institute of Food and Agricultural Sciences; University of Florida; Gainesville 32611. Volunteers are an important component of 4-H youth development programs and provide direct and indirect service to support youth. At the same time, the ability to utilize volunteers to meet 4-H's mission, goals, and objectives depends upon the effectiveness of the county 4-H program to manage, lead, and educate volunteers. Volunteer administration requires a commitment of time, energy, and financial resources, putting to rest the idea that volunteers are free. Given this commitment, program efforts to support volunteers should be evaluated similarly to other outreach initiatives. This fact sheet addresses the following questions: Why should I evaluate volunteer programs? What should I evaluate? What should I consider when evaluating volunteers? What should I consider when evaluating volunteer support by 4-H staff? How do I measure the volunteer program? What tools are available? What resources are available to assist in evaluation? Osborne and Gaebler (1992) recognized the importance of evaluation in the public sector. From their work these observations were identified as still relevant today: What gets measured gets done. If 4-H agents don't measure results, 4-H agents can't tell success from failure. If 4-H agents can't see success, 4-H agents can't reward it. If 4-H agents can't reward success, 4-H agents are probably rewarding failure. If 4-H agents can't see success, 4-H agents can't learn from it. If 4-H agents can't recognize failure, 4-H agents can't correct it.
Engaging Volunteers through ISOTURES: Evaluating Volunteer Programs 2 If 4-H agents can demonstrate results, 4-H can win public support. Evaluation is not the decision itself, but rather the preparation for the decision. Evaluation is an ongoing process that should be built into the planning from the very beginning. The result of evaluating volunteer programs in 4-H should be organizational learning. Measuring volunteer program effectiveness is necessary for success. A comprehensive evaluation system for a volunteer program might include any of the following: Mission-based evaluation Client-based evaluation Output-based evaluation Outcome-based evaluation Standards-based evaluation Each of these types of evaluation helps communicate the value of volunteer programs. A description of each is provided below. A mission-based evaluation examines the extent to which 4-H volunteers assist the county 4-H program in achieving its mission and purpose. In a real sense, a mission-based evaluation looks at whether the volunteers connected to the county 4-H program are directly involved in work that ought to be done. The 4-H mission is...: ...to create supportive environments for diverse youth and adults to reach their fullest potential. Therefore, a mission-based evaluation system would measure how well the volunteer program supports youth and adults in achieving their goals. Earlier in this series, a puzzle square was used to illustrate one of the roles of the 4-H agent (Figure 1). That is, to identify opportunities for volunteers that helps accomplish the county 4-H mission. Volunteer opportunities may include program support, volunteer support, education delivery, or service in an advisory capacity. A mission-based evaluation will help assess the extent to which the county 4-H program has accomplished its goal of volunteer involvement and where improvement is needed. Ask the following: Does the county 4-H program have a philosophy related to volunteerism? Is it developed by all members of the 4-H program including the County Extension Director, paid staff, volunteers and youth? Is the county 4-H program mission and vision written? Does it include volunteers? Is it part of promotional and educational materials? Has the advisory committee identified county needs? Is one of those needs a leadership team devoted to volunteerism? Have roles and responsibilities been identified and communicated to paid staff and volunteers based on county youth development needs? Are episodic, short-term, long-term and virtual roles included? Have role descriptions been developed for each position identified? Are they used in recruitment efforts?
Engaging Volunteers through ISOTURES: Evaluating Volunteer Programs 3 Does the county 4-H program have resources and infrastructure to support volunteer involvement? Is the volunteer leadership team involved in evaluating the 4-H program and communicating these results with others? A second area that should be evaluated relates to those people who come into contact with the 4-H program. This includes volunteers, youth, teachers, parents, funders, other county office faculty and staff and the general public. Each of these groups can be surveyed periodically to determine the extent of their relationship with the 4-H program and their levels of satisfaction with its operation. Recall that one of the responsibilities of a volunteer administrator is to match the interests, skills, and abilities of a volunteer with an appropriate role (Figure 2). Matching program needs and volunteer roles requires evaluation of potential volunteer interests, skills, and abilities as well as maintenance of a plan for volunteer involvement. Feedback is often useful for improving the operation of the program and shows all members connected to the program that their concerns will be addressed. These results can help increase volunteer retention and youth participation. Here are some examples of feedback questions that could apply to each of the client groups: How well does the 4-H staff support 4-H efforts? How well do other volunteer leaders support 4-H efforts? How would 4-H agents describe the relationship with 4-H staff? How would the 4-H agent describe the relationship with other 4-H volunteers? How useful are resources in helping volunteers accomplish the volunteer role? How useful are available resources in helping accomplish goals? Overall, what was the level of satisfaction with the 4-H experience? Overall, what was the level of satisfaction with the level of 4-H training? Overall, what was the level of satisfaction with the volunteer role in 4-H? Overall, what was the level of satisfaction with 4-H volunteer orientation? Overall, what was the level of satisfaction with communication with 4-H staff? Overall, what was your level of satisfaction with communication with other 4-H volunteer leaders? Output-based evaluation in volunteer programs generally refers to a numerical counting of activities associated with volunteerism. There are several numerical measures that will help report volunteer productivity. Some of these are: Number of volunteer roles identified. Number of hours contributed by volunteers. Number of volunteers involved in direct service to youth. Number of volunteers involved in indirect service (middle management) to the 4-H program.
Engaging Volunteers through ISOTURES: Evaluating Volunteer Programs 4 Number of youth served by volunteers. Number of youth initiatives involving volunteers. Keep in mind; outputs are not program outcomes, they are merely a reflection of productivity. The key to successfully reporting outputs is to plan for the collection at the beginning. Have a plan developed and resources ready from the start to collect output information. Outcome-based evaluation asks a seemingly simple question: "To what extent do the activities conducted in the volunteer program actually achieve their goals." Logically, inputs lead to activities that create outputs that should generate client outcomes. In 4-H, that client is youth and the outcomes are the essential elements of youth development: Belonging 1. Positive Relationship with a caring adult 2. An inclusive environment 3. A safe environment Mastery 4. Engagement in Learning 5. Opportunity for Mastery Independence 6. Opportunity to see oneself as an active participant in the future 7. Opportunity for self-determination Generosity 8. Opportunity to value and practice service for others Volunteer programs should increase the number and quality of experiences for youth to develop these essential elements. In Florida, a survey has been developed to assist in this process. It is provided in the resources at the end of this section. Instructions for its use can be obtained from the 4-H RSA or state volunteer specialist. Standards-based evaluation is a comparison technique used to compare various aspects of the volunteer program with similar volunteer programs. In Florida 4-H, there are three components that determine the success of a volunteer program. These are paid faculty and staff, volunteers, and the processes that are used to manage the 4-H volunteer program. Each of these aspects requires periodic standards-based evaluation. If volunteers are the key to 4-H success, understanding the competencies that will provide a successful 4-H volunteer experience is important. In their national study Culp et al. (2004) described six domains that volunteers will need to effective in the 4-H youth development program. Although not every volunteer will achieve each competency, these domains serve as a standard for all volunteers and are described in more detail below. 1. Communications Volunteers demonstrate the ability to create, deliver, and interpret information effectively through formal and non-formal means. a. Speaking Skills b. Listening Skills c. Writing Skills d. Non-Verbal Skills e. Information Delivery & Dissemination f. Marketing & Public Relations g. Use of Technology 2. Organization Volunteers demonstrate the ability to engage others in planning, providing and delivering positive 4-H youth development programming in a community. a. Planning & Organizing b. Time Management c. Parent Recruitment & Involvement d. Delegating Tasks to parents, members and other volunteers
Engaging Volunteers through ISOTURES: Evaluating Volunteer Programs 5 e. Service to the Community f. Marketing & Publicity 3. 4-H Program Management Volunteers must understand and follow appropriate policies, procedures and safety guidelines when acting on behalf of Extension. a. Organization & Structure of Extension and 4-H b. Upholding the 4-H Mission c. Risk Management/Risk Reduction d. Liability Awareness and Reduction e. Club Management f. Behavior Management g. Record Keeping h. Financial Management i. Computer Skills 4. Educational Design & Delivery Volunteers demonstrate the ability to plan, implement, and evaluate research-based learning opportunities that effectively promote positive personal development. a. Use of Age-Appropriate Activities b. Utilization of Multiple Teaching Strategies c. Understanding of Differences in Learning Styles d. Knowledge of Subject Matter e. Team Building Skills f. Application of Experiential Learning g. Program Evaluation Methods h. Use of Approved and Appropriate Curriculum 5. Positive Youth Development Volunteers demonstrate the ability to intentionally and appropriately apply the principles and best practices that result in the positive development of youth. a. Developing Life Skills b. Leadership Skills c. Understanding Ages & Stages of Youth Development d. Empowering and Enabling of Others e. Practicing Youth Adult Partnerships f. Ability to Motivate & Encourage Youth g. Appreciating Diversity 6. Interpersonal Characteristics Volunteers demonstrate the ability to develop effective relationships, work competently with individuals and groups, and express empathy and understanding for others. a. Caring Attitude b. A Compassionate Nature c. Acceptance of Others d. Honesty, Ethics, Morality e. Patience f. Ability to Develop & Strengthen Relationships g. Flexibility h. Positive Outlook The third aspect of the volunteer program that should be measured against a standard is the volunteer program itself. Program evaluations have been conducted to improve programs, provide accountability, and to obtain evidence related to the effectiveness (Rossi et al., 1999). 4-H programs derive benefits when volunteer resources have been used successfully. These organizational benefits from volunteers have included: organizational cost savings; ability to provide youth development opportunities that are not provided otherwise; access to volunteers with specialized skills; increased public support; and detailed attention to 4-H members and volunteers. In addition to the benefits accrued by deploying volunteer resources, 4-H programs also face challenges when involving volunteers. There are financial costs such as employee salaries and benefits, legal fees and direct financial support for volunteer initiatives. In addition, there are non-financial costs such as risk to clientele and damage to organizational reputation. Some of these costs manifest themselves in terms of time. This includes timeinvested in: recruiting volunteers; providing orientation and training to volunteers;
Engaging Volunteers through ISOTURES: Evaluating Volunteer Programs 6 supervising volunteer involvement; evaluating volunteer and organizational performance; and, volunteer retention. Increased time increases the economic cost to the organization. For other non-financial costs, such as poor service quality to clientele, there is a risk to clientele and damage to the organizational reputation. The larger the challenge the more cost to the organization. The key concept for a 4-H agent is the understanding that volunteers are not "free." In 2005, Hager and Brudney stressed the importance of program evaluation and outcome measurement of programs designed to utilize volunteers. Their national study measured the net benefits of volunteer programs in over 2,000 public charities. The net benefits framework is easy to use and can be used to compare individual volunteer programs with other counties and organizations. Net benefits refer to the difference between the benefits that volunteers contribute to an organization and the challenges associated with volunteer administration. For more information related to measuring the net benefits of volunteer programs, contact the state volunteer specialist. Annually, Florida Cooperative Extension requires a Plan of Work (POW) and Report of Accomplishment (ROA). This is true for the volunteer initiatives as well. A well-developed POW and ROA should include: (1) situation statement, (2) measurable objectives, (3) outcomes, (4) publications, and (5) impact. The goal is to have a comprehensive volunteer program evaluation at the time of permanent status/promotion. A comprehensive evaluation is one that addresses mission-based evaluation, client-based evaluation, output-based evaluation, outcome-based evaluation, and standards-based evaluation. On the 4-H website (florida4h.org) in the faculty and staff section are examples of an annual POW and ROA that include all elements of a well-designed volunteer program. Evaluating volunteer programs is an important function and responsibility of the 4-H agent. Evaluation is not the decision itself, but rather the preparation for the decision. Evaluation is an ongoing process that should be built into the planning from the beginning. The result of evaluating volunteer programs in 4-H should be organizational learning. A comprehensive evaluation is one that addresses mission-based evaluation, client-based evaluation, output-based evaluation, outcome-based evaluation, and standards-based evaluation. A mission-based evaluation examines the extent to which 4-H volunteers assist the county 4-H program in achieving its mission and purpose. A client-based evaluation determines the extent of a volunteer's relationship with the 4-H program and their levels of satisfaction with its operation. Output-based evaluation in volunteer programs refers to a numerical counting of activities associated with volunteerism. Outcome-based evaluation measures the extent activities conducted in the volunteer programs actually achieve established goals. In 4-H, outcomes refer to the essential elements of youth development. Standards-based evaluation is a comparison technique used to compare various aspects of the volunteer program. In Florida 4-H, there are three components that determine the success of a volunteer program. These are paid faculty and staff, volunteers, and the processes that are used to manage the 4-H volunteer program. Each of these aspects require periodic standards-based evaluation. Boyd, B. L. (2003). Identifying Competencies for Volunteer Administrators for the Coming Decade: A National Delphi Study. Journal of Agricultural Education, 44(4). Bussell, H. & Forbes, D. (2003). The Volunteer Life Cycle: a marketing model for volunteering. Voluntary Action, 5(3), pp. 61. Culp, K., McKee, R.K., and Nestor, P. (2004). Volunteer Research Knowledge & Competency: Taxonomy for 4-H Youth Development.
Engaging Volunteers through ISOTURES: Evaluating Volunteer Programs 7 Fogarty, K., Terry, B.D., Pracht, D.W., and Jordan, J.C. (2009). Organizational supports and youth life skill development: Adult volunteers as mentors, managers, and "mediators." Journal of Youth Development: Bridging Research and Practice, 4, 21. Graff, L.L. (2006). Declining profit margin: When volunteers cost more than they return. International Journal of Volunteer Administration, 24(1). Retrieved Mar. 11, 2008, from http://www.ijova.org/PDF/VOL24_NO1/ IJOVA_VOL24_NO1_Profit_Margin_ Linda_ Graff.pdf. Hager, M.A. and Brudney, J.L. (2005) Net Benefits: Weighing the Challenges and Benefits of Volunteers. The Journal of Volunteer Administration, 21(1), 26. Lerner, R.M., Lerner, J.V., and Phelps, E. (2007). The Positive Development of Youth. Boston: Tufts University, Institute of Applied Research. McCurley, S. and Lynch, R. (2006). Volunteer Management (2nd ed.). Ontario: Johnstone Training and Consultation, Inc. Osborne, D. and Taylor, T. (1992). Reinventing Government: How the Entrepreneurial Spirit is Transforming the Public Sector. New York: Addison-Wesley. Rossi, P.H., Freeman, H.E., and Lipsey, M.W. (1999). Evaluation: A Systematic Approach (6th ed.). Thousand Oaks, CA: Sage Publications. University of Wisconsin Extension. Available at http://4h.uwex.edu/ncrvd/vcomp/index.cfm If you need guidance, ask the Regional Specialized 4-H Agent (RSA) or State Volunteer Specialist for assistance. They are prepared to help you plan for and report accomplishments. In addition, the following resources are available to help increase your understanding and implementation of a comprehensive evaluation system. Everyone Ready Training 3-Year Webinar Series Self-Instruction Guide Volunteer Performance Assessment (Feb 2011) Seminar Demonstrating the Impact of Volunteer Involvement (Mar 2011) Self-Instruction Guide Reports with Clout: Showing the Impact of Volunteers (Feb 2012) Self-Instruction Guide Methods of Volunteer Program Evaluation (Apr 2012) About Everyone Ready Everyone Ready is available to all Extension staff regardless of title or program area. Online Seminars include a Flash audio-visual trainer presentation and downloadable handouts. Both types of modules take about 1 hour per month to complete. Self-Instruction Guides provide participants with a way to combine written information and exercises that they can do at their desk or with colleagues. Available at http://nextgeneration.4-h.org/volunteerism/. Articles Measuring Up: Assessment Tools for Volunteer Programs. Cost: $10.00 Available at http://www.energizeinc.com/. The Dollar Value of Volunteer Time, by Susan J. Ellis. Cost: Free. Available at http://www.energizeinc.com/art/subj/prog.html. Inputs Matter, 68-page PDF by the Charity Finance Directors Group in the UK, with an Appendix on "The Development of Recommendations on Valuing Volunteers." Cost: Free. Available at: http://www.cfdg.org.uk/cfdg/includes/downloadfiles/ Policy_CFDG_Inputs_Matter.pdf. Measuring Volunteering: A Practical Toolkit, by Independent Sector and UNV, designed as a toolkit to assist countries in conducting surveys assessing their national, regional, and local volunteer levels (2001,
Engaging Volunteers through ISOTURES: Evaluating Volunteer Programs 8 40-pg. PDF). Cost: Free. Available at http://www.independentsector.org/ programs/research/toolkit/IYVToolkit.PDF. A Note On Assessing Value, from BEST OF ALL: The Quick Reference Guide to Effective Volunteer Involvement. Cost: Free. Available at http://www.energizeinc.com/art/abeso.html. The Impact of Investing in Volunteers by Ben Cairns and Romayne Hutchison from the Centre for Voluntary Action Research at Aston Business School (UK, 2006 40 pg. PDF). Cost: Free. Available at http://iiv.investinginvolunteers.org.uk/NR/rdonlyres/ CFD0C332-DDA6-4F0B-AC27-B290F8B4003B/0/ ImpactAndBenefitsreport2006.pdf. Measuring the Difference Volunteers Make: A Guide to Outcome Evaluation for Volunteer Program Manager. Cost: Free. Available at http://www.energizeinc.com/download/ Measuring_the_Difference2003.pdf. Volunteer Competencies: VRKC Model. The Volunteer Research Knowledge and Competency Taxonomy Model (VRKC) is based on a study that identified the competencies that volunteers will need in order to effectively deliver 4-H Youth Development programs and activities in the next decade. This taxonomy provides 4-H professionals with a national focus and direction related to the levels of competency and the perceived needs of 4-H volunteers. It provides insights into priorities for volunteer development. See http://4h.uwex.edu/ncrvd/vcomp/index.cfm.