|UFDC Home||myUFDC Home | Help ||
STANDARD VIEW MARC VIEW
This item is only available as the following downloads:
1.This document, PE054, is adapted from Fact Sheet PE-53 and PE31, a series of Program Evalua tion and Organizational Developme nt, Florida Cooperative Extension Service, Institute of Food and Ag ricultural Sciences, University of Florid a. First published: October 1993 and Octobe r 1990. Reviewed: October 2001. Please visit the EDIS Web site at http://edis.ifas.ufl.eduThe Institute of Food and Agricultural Sc iences is an equal opportunity/affirmative ac tion employer authorized to provide resea rch, educational information and other services only to individuals and institutions that function without regard to race, color, se x, age, handicap, or national or igin. For information on obtaining other extension publications, contact your county C ooperative Extension Servic e office. Florida Cooperative Extension Service / Institute of Food and Agricultural Sciences / Unive rsity of Florida / Christine Taylor Waddill, Dean 2.Marilyn N. Norman, associate professor and district extension director, and Nayda I. Torres, professor and chair, Department of Family, Youth and Community Sciences, Cooperative Extension Se rvice, Institute of Food and Agricultural Sc iences, University of Florida, Gainesvi lle, 32611. PE054Design Team Basics1Marilyn N. Norman and Nayda I. Torres2What Is a State Major Program?A State Major Program (SMP) is an interdisciplinary programming effort that focuses on a major issue or situation. Further, it requires the expenditure of human and financial resources on a problem of importance to a relatively large audience. An SMP is based on county programs and state, national, or international issues usually beginning with an awareness or knowledge of impending changes in legislation; social, economic, or environmental conditions; or grassroots concerns. State Major Programs are identified by state Program Leaders in collaboration with faculty, appropriate Department Chairs, and Extension Administration. Once a program has been labeled a "State Major Program," a Design Team is established to provide leadership for addressing the issue through educational programs. An SMP is designed to bring about behavior and/or practice change by participants. Inherent in the Design Team responsibilities is the establishment of priorities and a refined focus on problems and issues of concern to people. This involves the comprehensive planning, development and evaluation of a set of educational subject matter and appropriate methods of delivery. Purpose of a Design TeamThe purposes of a Design Team are to:#help County Faculty with the planning of county major programs that make up SMPs;#identify, develop, and/or help obtain educational materials needed to support county major programs;#give leadership to the implementation of SMP activities and support the implementation of the county major program throughout the program term;#give leadership to SMP evaluation efforts; facilitate and support individual county major program evaluation efforts. Initiating Design Team WorkMost issue-based Design Teams begin with a growing awareness of an unmet need. The awareness may come through knowledge of impending changes in legislation; social, economic, or environmental conditions; or grassroots concerns. One key indicator is a commonly identified concern expressed by faculty in several counties. In addition, local advisory groups, stakeholder groups, and others may bring these issues to the attention of county or state level faculty.
Design Team Basics Page 2 February 2002In creating a new Design Team, faculty may wish to conduct literature reviews, read county needs assessments, interview citizen leaders, or sponsor seminars about the issue. A comprehensive descriptive narrative of the situation (e.g., a white paper) should be introduced for discussion among peers, Program Leaders and Extension Administration. Of particular importance for inclusion in the white paper is the role of research and extension in addressing the issue. It is critical that a careful needs assessment has been conducted to identify specific educational needs and to ensure that the proposed program is focused on appropriate solutions. Discussions will be held about the establishment of a new Design Team, assigning the issue to an existing Team, or holding the request until more information can be gathered. A Program Leader has the final approval in granting Design Team status. When a new program is initiated, the Design Team should develop a plan or blueprint which is subsequently peer reviewed. The review should be conducted by reviewers selected from subject-matter experts in Florida and other states. If a new Design Team is established, then committee membership recruitment and the writing of an SMP Plan of Work will begin. Writing an SMP Plan of WorkEach SMP is expected to have a Plan of Work. Sources of information may come from current research, citizen leaders, focus groups, county Plans of Work, etc. A Plan of Work for a SMP consists of several major parts including: #Mission or Purpose #Overall Objectives#Situation Analysis (quantitative indicators of existing condition)#Target Audience#Impact and Evaluation which include the projected changes (outcomes) to be measured by the evaluation study. Impact includes the value of change to a community#Measures of change (outcomes) and the Impact on the community#Professional Development/In-Service programs for County FacultyAdditional Internal Elements:#Committee Structure#Planned Supporting Activities and Events, including collaborations#Evaluation Strategies to be used including Output, Outcomes and Impact Indicators#Curriculum Materials Proposed or Available (graphical presentations, video, news releases) Planned Publications Major portions of the SMP Plan of Work will be included in the State Plan of Work called for in AREERA (Agricultural Research, Extension, and Education Reform Act of 1998). This larger Plan of Work includes all major work efforts for Extension in Florida and is aligned with Federal core programs and priorities. The AREERA state report is due annually with major revisions due at the beginning of the next four-year planning cycle.Reporting on a SMP Plan of WorkAt the close of each program year, County and State Faculty are required to finalize their Faculty Activity System (FAS) reports on professional activities during the prior 12 months. Specifically, faculty describe their accomplishments through description of educational activities (inputs and outputs) and outcomes/impacts related to program objectives. Department Chairs, Center Directors, District Extension Directors, Program Evaluation staff, and State Specialists review these reports for key information for performance appraisal and the aggregating of outcome and/or impact data for numerous state and federal reports. Those Design Teams which have provided strong programs and easy to use evaluation tools that contain indicators are able to draw conclusive statements about the total impact of the SMP effort. But the primary purpose of these evaluation tools and the data collected is not to provide information for state reports. State Major Programs can be improved and sharpened, bringing about the even greater behavior and practice change among clientele.
Design Team Basics Page 3 February 2002The major purpose of evaluation is to strengthen the SMP and the educational programs conducted through the Design Team. Design Teams and Inputs/Outputs and Outcomes/ImpactsIn providing direction for Extension efforts to measure outcomes and impacts of an SMP, Design Teams should identify the specific indicators that measure the effectiveness of their efforts. Indicators should be tied closely to the gap or difference between the current situation and the preferred situation as stated in the Plan of Work. Most often, these are expressed in measurable objectives that call for:Sinput of materials, human effort, and collaborations, Soutput of resources, activities, targeted audiences reached,Soutcomes of knowledge gained, practices changed or adopted, and Simpacts of social, economic or environmental change on the community. These indicators can and should take multiple measurements of different dimensions of the total effort ranging from simply counting the resources expended in support of the program to an analysis of the social, economic, or environmental impact to Florida, the nation, or the world. In essence, team members need to start with the end in mind, what should clientele do as a result of this educational program. The evaluation instrument, as well as the educational objectives, need to be focused upon what is identified as a result of asking this question. Table (see pages 6 and 7). Design Teams should lead the way in providing both the indicators and the actual measurement tools for County Faculty and others to use during program implementation phases. The evaluation tools should be an integral part of the curriculum. When standardized evaluation tools are tied to the SMP objectives with both short and long term perspectives, county staff will be more consistent and effective with routine evaluations of localized programs. The process will also better reflect statewide impact through the aggregated reporting of similar data from multiple county reports.Design Team FormationA Design Team may be comprised of the entire faculty interested and dedicated to the resolution of a specific issue, or it may be a core of representatives who have a vested interest in the larger group/issue. In either of these situations, attention to group process, team building, and extensive communication is necessary in order for the team to function appropriately. For large groups and for groups with numerous sub-issues, smaller task forces, ad hoc committees, or action teams may be formed to provide focused effort. For example, SMP FL114 has subcommittees that provide specialized leadership for targeted audiences such as Professional Horticultural Services and Florida Yards and Neighborhoods. SMP FL107 instituted Action Groups to carry out functions necessary for the group to achieve its goals. Each Design Team is periodically reviewed. Specifically, this may happen during the Extension Long Range planning cycle, which is every four years. At that time teams are assessed for impact, effectiveness, and continued ability to address the identified issue(s). Teams may be continued, combined or dropped at the four year interval. Adjustments in focus, scope, and membership of the team can be made at intermediate times in between, but these adjustments should be made in consultation with the Program Leaders. When a new program is initiated, the Design Team should develop a plan or blueprint, focused on educational outcomes or impacts, which is subsequently peer reviewed. The review should be conducted by reviewers selected from subject matter experts in Florida and other states. Additional peer reviews should be conducted at four to eight year intervals.Design Team CompositionThe core of a Design Team consists of Extension Specialists within the primary commodity/program area and main supporting disciplines; a Design Team is usually interdisciplinary. County Faculty are also active members of Design Teams. In some situations, leaders from non-faculty sectors are also invited to
Design Team Basics Page 4 February 2002participate. Non-faculty representation would be seen as consultants, not full Design Team members.* The team members are identified by appropriate state Program Leaders in collaboration with Design Team Leaders, Department Chairs, Center Directors, and Extension Administration. In most cases, a Specialist from the primary commodity/program area is designated as team leader or the team is co-chaired by both State and County Faculty. County Faculty representation on Design Teams should be made in consultation with the Program Leaders and the District Extension Director. Representation from County Faculty is desirable for Design Teams. These appointments are to enhance the Design Teams perspective of county level programmatic concerns and to strengthen communications between counties and Design Teams. County Faculty are encouraged to serve on one or two teams, particularly for those at Agent II and above. County Faculty often express a desire to serve on multiple Design Teams, but are encouraged to limit their involvement to one or two that align with their primary program expertise. County Faculty are encouraged to be active members of the Design Team. Membership and leadership on Design Teams should be rotated to facilitate the addition of new ideas, diversity, and an enhanced analysis of the problem. Two liaison positions are established for each Design Team. These include a representative from the Program Development and Evaluation Center and the District Extension Directors. Although these individuals may not attend every meeting, they are supportive and available to work closely with the team. They serve to represent Extension administration and help to ensure that the SMP is focused upon educational outcomes. *Consultants are not Extension employees and therefore are not held responsible for conducing program activities and generating desired outcomes in the same way that team members are. Consultants provide grassroots input that serves to increase relevance.Providing Leadership to a Design TeamTeam leadership, whether held individually or as a co-chair, requires that a faculty member understand the issues inherent in the SMP and that he or she creates momentum to bring the resources of the Florida Extension Service (including research, adult education principles, faculty skills, and group process) together to adequately address the problem. Most issues are extremely complex and interrelated. Any effort to respond to legislation or to impact social, economic, or environmental change requires multiple approaches and methodologies. Through Design Team meetings, these needs should be discussed thoroughly. An SMP should be written, including the situational statement, preferred situation (including projected changes to be made by participants and value of changes to community), priorities, planned activities and publications, and evaluation tools. Plans from other states may be reviewed if there is a possibility of multi-state cooperative programming. For over-lapping issues within other SMPs, contact should be made with Design Team leaders to discuss possible cross-membership and activities. A multi-year calendar of events and activities (including publications) should be established with responsibilities allocated to team members. Some Design Teams may require sub-committee work (ad hoc committees or action team), each with its own assignments. Communicating with Program Leaders/Extension Administrators, particularly about the Design Team mission, dates, and activities, will help them keep the needs of the Design Team in mind for possible resource allocation or linkages with appropriate governmental or funding groups. Staying in contact will also help the Design Team leadership be aware of internal resources available to advertise County Faculty In-Service Training, essential to the work effort of the Design Team. In addition, the Design Team Chair(s) will play important roles in the development of the State Plan of Work and the aggregating of County and State Faculty Reports of Accomplishment for the State Report of Accomplishments.
Design Team Basics Page 5 February 2002State Major Program Numbering SystemFor ease of reference and cataloging, each SMP is designated a numeric category. 100-199 Agriculture 200-299 Florida A & M programs 300-399 Sea Grant 400-499 Natural Resources 500-599 Family and Consumer Sciences 700-799 4-H/Youth Development 800+ Cross-discipline or EmergingReferencesFact Sheet PE-31 Role of a Design Team, October 1990 Fact Sheet PE-56 Concept of State Major Programs and Design Teams, December 1994 Fact Sheet PE-49 Concept of a Major Program, January 1994 Circular PE-63 Extension Specialist's Role and Responsibility Statement, July 1996Recommended Web Sites for Current SMP Plan of Work and AREERAPOW: http://www.ifas.ufl.edu/~smpweb/ AREERA: http://pdec.ifas.ufl.edu/ Faculty Web Modules: http://pdec.ifas.ufl.edu/newfaculty.htm AcknowledgmentsThe authors wish to thank the following individuals who have reviewed this document: Larry R. Arrington, associate dean, Florida Extension; Joan Dusky, assistant program dean, Florida Extension; Howard Ladewig, professor, Agricultural Education and Communication; Glenn Israel, professor; Agricultural Education and Communication; and Nick Place, assistant professor, Agricultural Education and Communication..
Design Team Basics Page 6 February 2002 TABLE. Evaluation of Program Performance.Outcome Indicators and ImpactsSEECEvidence of impact on individuals and communities Have targeted and/or other social, economic and environmental conditions been affected through targeted changes in practices? How has the public (including non-program participants) been affected by the program? Have program participantsindividuals, familie s, and communitiesbeen helped and/or hindered by the results of program induced changes in tar geted practices? In what way? To what degree? Terms for stating outcomes Economic value indicated, social change, intervention avoided, cost / benefit ratio, reduction / increase in usage, safetyPracticesHave participants changed targeted patterns of behavior as a result of program induced knowledge opinions, skills, or aspirations? (KO SA) In what way? To what degree? At what aggregate level? Demonstrates, adopts, produces, solves, uses, changed, selects....KOSAKnowledge, opinions, skill, and aspirations KnowledgeDid participation increase awareness, understanding, and/or pr oblem solving ability as targeted? Relative to what practices? Defines, describes, identifies, labels, lists, matches, names, outlines, reproduces, selects, states, es timates, summarizes OpinionsDid participation change outlooks, perspectives, or viewpoints as intended? Relative to which practices? Expresses, reports, shares, studies, synthesizes, analyzes, SkillsDid participation increase verbal or phy sical abilities as targeted? Did participation develop new skills or improve perfo rmance as targeted? Regarding which practices? Assembles, builds, calibrates, changes, cleans, compiles, connects, constructs, corrects, creates, designs, dismantles, follows, makes, organizes, prepares, AspirationsDid participation alter ambitions, hopes or selected courses of action as intended? Regarding which practices? In what areas? Describes, differentiates, explains, initiates, indicates, desires, proposes, justifies, Participant reactionsEvidence of participant satisfaction Did participants react to the program as intended? Di d they rate the activities as informative, interesting, and applicable? Do they perceive any immediate benefits? Recalls, asks questions, sits erect, replies to questions, interested, takes notes, liked the experience, was satisfiedtable concludes on next page
Design Team Basics Page 7 February 2002 Output IndicatorsParticipantsEvidence of program implementation How many of the targeted participants became in volved in the program activities and events? Which targeted customers participated (descrip tive characteristics)? How extensive and intensive was their involvement? Number of participants, reached targeted audience (how do you know), staying power of presentation, repeats, consultationsActivities implementedWere the targeted activities implemented as par t of the program? Was the targeted content or subject matter used? What promotional strat egies worked or failed? Did the delivery and implementation methods work or fail? Di d the participatory methods work or fail? Activities held, information presented, people attended, appearance of interest, engaged learnersInputsResources usedWere targeted resources actually expended on the program (tim e, money, staff)?Who involved, number of days, planning completedFrom Targeting Outcomes of Programs (TOP), Bennett and Rockwell, March 1995.