<%BANNER%>

Use of the Delphi technique to derive a common definition for work-related education

University of Florida Institutional Repository

PAGE 1

USE OF THE DELPHI TEC HNIQUE TO DERIVE A COMMON DEFINITION FOR WO RK-RELATED EDUCATION By MICHAEL LEE DROLL A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLOR IDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2005

PAGE 2

Copyright 2005 by Michael Lee Droll

PAGE 3

iii ACKNOWLEDGMENTS I could not have completed this major lif es project without the genuine concern and support of many people. I thank my wife, Molly, for her stick-to-itiveness by example and undiminished motivation mixed with lots of love that spurred me on to mission accomplished. To my parents, Dot and Chuck, my sister, Kim, my sons, Karl and Mark, and all my other family members--the sky has no limits when one is the recipient of unconditional love such as I have r eceived from my family. Dr. Larry W. Tyree is my true mentor and colleague, and I will be forever grateful for his wisdom, calming demeanor, and mutu al admiration. Dr. Dave Honeyman was, without a doubt, my man of the hour. My sinc ere appreciation goes to him as I would not have achieved so much so soon without his en couragement. I especially thank Drs. Lynn Leverty, Dale Campbell, Jim Doud, and, in particular, my suite mates, Drs. Art Sandeen and Phil Clark, for all their support during my doctoral program j ourney and for making me a part of the colleges extended family. Finally, I could not have completed this long journey without financial support beyond what the G.I. Bill provided. I am esp ecially grateful to Dr. Tyree for funding my graduate assistantship. I give heartfelt thanks to Dr. Ha rry T. Albertson for the once-in-alifetime, paid internship as recorder to the Council of Pr esidents by way of the Florida Association of Community Colleges. La stly, I would like to acknowledge Dr. Doug Olson, my dear colleague in institutional res earch, who reciprocated with paid consulting. I plan to stay in contact with everyone!

PAGE 4

iv TABLE OF CONTENTS page ACKNOWLEDGMENTS.................................................................................................iii LIST OF TABLES............................................................................................................vii ABSTRACT....................................................................................................................... ix CHAPTER 1 INTRODUCTION........................................................................................................1 Background for the Study.............................................................................................1 Statement of the Problem..............................................................................................5 Theoretical Framework.................................................................................................7 Purpose of the Study.....................................................................................................8 Primary Questions.................................................................................................8 Secondary Questions.............................................................................................9 Definition of Terms......................................................................................................9 Significance of the Study............................................................................................12 Overview of the Research Process.............................................................................13 Assumptions, Delimitations, and Limitations............................................................15 Assumptions........................................................................................................15 Delimitations.......................................................................................................15 Limitations...........................................................................................................16 Organization of the Remainder of the Study..............................................................16 2 LITERATURE REVIEW...........................................................................................18 Introduction and Scope...............................................................................................18 The Concept and Identity of Community Colleges....................................................19 Work-related Education..............................................................................................23 A Purpose of the Community College.................................................................24 Terminology of Work-related Education............................................................29 Learning College Theory and Work-related Education..............................................31 Recent Attempts at Educational Reform.............................................................31 Learning College Theory.....................................................................................32 Six Principles of the Learning College................................................................35 Components of Work-related Education....................................................................38

PAGE 5

v Mission and Organization....................................................................................39 Funding................................................................................................................39 Needs Assessment and Documenting College Success.......................................40 Instruction, Programs, and Delivery Systems.....................................................42 Staffing................................................................................................................43 Coordination and Planning..................................................................................44 National Proclamation and National Database....................................................45 Summary.....................................................................................................................46 3 METHODS.................................................................................................................48 The Setting..................................................................................................................48 Primary Questions...............................................................................................49 Secondary Questions...........................................................................................49 First Stage............................................................................................................50 Second Stage.......................................................................................................50 Third Stage..........................................................................................................51 Fourth Stage.........................................................................................................52 The Participants..........................................................................................................52 Tasks and Materials....................................................................................................54 General Operational Design.......................................................................................56 Data Collection...........................................................................................................57 Round One...........................................................................................................57 Round Two..........................................................................................................60 Round Three........................................................................................................63 Communication Process.............................................................................................65 Data Management and Statistical Procedures.............................................................65 Reliability of the Instrument................................................................................66 Validity of the Instrument...................................................................................67 Analytical Procedures..........................................................................................69 Descriptive Statistics...........................................................................................70 Parametric Statistical Tests..................................................................................71 Summary.....................................................................................................................72 4 PRESENTATION AND ANALYSIS OF DATA......................................................74 Introduction.................................................................................................................74 Primary Questions...............................................................................................75 Secondary Questions...........................................................................................75 Results of the Delphi Technique.................................................................................75 Selection and Confirma tion of Participants................................................................77 Response Rates to Delphi Surveys.............................................................................79 Verification of the Accuracy of the Delphi Technique..............................................80 Round One Results.....................................................................................................81 Round Two Results.....................................................................................................83 Round Three Results...................................................................................................84 Differences in Responses by Subgroups and Rounds.................................................85

PAGE 6

vi Confidence in the Accuracy of the Delphi Technique..............................................110 Data Relationships to the Research Questions.........................................................111 Primary Questions.............................................................................................112 Secondary Questions.........................................................................................112 Research Question Pertaining to Pr inciples of the Learning College.......................113 Principle I..........................................................................................................114 Principle II.........................................................................................................115 Principle III........................................................................................................116 Principle IV.......................................................................................................117 Principle V.........................................................................................................118 Principle VI.......................................................................................................120 Research Question Pertaining to Com ponents of Work-related Education..............121 Mission and Organization..................................................................................123 Funding..............................................................................................................124 Needs Assessment and Documenting Success..................................................125 Instruction, programs, and delivery systems.....................................................128 Staffing..............................................................................................................128 Coordination and Planning................................................................................130 National Proclamation and National Data base for Work-related Education.....131 Research Question Pertaining to Strongest Advocated Principles and Components131 Strongest Advocated Principles.........................................................................132 Strongest Advocated Components....................................................................134 Consensus Reached by the Panel of Experts............................................................136 Relationships between the Pr inciples and Components...........................................136 Relationship of Compon ents to Principles........................................................137 Summary...................................................................................................................139 5 CONCLUSIONS AND RECOMMENDATIONS...................................................141 Introduction...............................................................................................................141 Primary Questions.............................................................................................142 Secondary Questions.........................................................................................142 Model of Work-related Education............................................................................144 Commonality of Componen ts across Principles.......................................................157 Suggestions for Further Research.............................................................................160 Implications for Community College Leadership....................................................161 Implications for Policymakers..................................................................................162 APPENDIX QUALITATIVE RESPONS ES FOR ALL THREE ROUNDS.................164 LIST OF REFERENCES.................................................................................................184 BIOGRAPHICAL SKETCH...........................................................................................189

PAGE 7

vii LIST OF TABLES Table page 1-1 Percent Breakdown of General Op erating Funds for 1998-1999 for States Represented on the League for Innovation Board of Directors...............................5 1-2 The 20 Colleges Represented on the Bo ard of Directors to the League for Innovation in the Community College...................................................................10 4-1 T-test of Average Total, Principles and Components Scores for CEOs and Administrators during Round One.........................................................................86 4-2 T-test of Average Total, Principles and Components Scores for CEOs and Administrators during Round Two........................................................................86 4-3 T-test of Average Total, Principles and Components Scores for CEOs and Administrators during Round Three......................................................................86 4-4 Duncans Multiple Range Test of Scores between All Three Rounds..................87 4-5 Descriptive Statistics of Statem ents Compared Across All Three Rounds...........87 4-6 Principle I Statements in Round Three................................................................114 4-7 Principle II Statements by Exception in Round Three.........................................115 4-8 Principle III Statements by Exception in Round Three.......................................116 4-9 Principle IV Statements in Round Three.............................................................117 4-10 Principle V Statements by Exception in Round Three........................................118 4-11 Principle VI Statements by Exception in Round Three.......................................120 4-12 Mission and Organization Statem ents by Exception in Round Three.................123 4-13 Funding Statements in Round Three....................................................................124 4-14 Needs Assessment Subcomponent Statements in Round Three..........................125 4-15 Documenting Success Subcomponent Statements in Round Three....................127

PAGE 8

viii 4-16 Instruction, Programmin g, and Delivery Systems Statements in Round Three..128 4-17 Staffing Statements in Round Three....................................................................129 4-18 Coordination and Planning Statements in Round Three......................................130 4-19 National Proclamation and National Da tabase Statements in Round Three........131 4-20 Strongest Advocated Principles in Round Three.................................................132 4-21 Strongest Advocated Components in Round Three.............................................134 4-22 Correlation Matrix of Principl es and Components in Round Three....................137 4-23 Significance between Principles and Components in Round Three....................138

PAGE 9

ix Abstract of Dissertation Pres ented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy USE OF THE DELPHI TEC HNIQUE TO DERIVE A COMMON DEFINITION FOR WO RK-RELATED EDUCATION By Michael Lee Droll May 2005 Chair: David S. Honeyman Major Department: Educational Leadership, Policy and Foundations This study was designed to (a) test if work-related education conformed to OBanions six principles for the learning co llege and (b) if the principles could be supplemented with other components of work-re lated education. The central point was to further the knowledge of these relationships, which in total co uld be modeled to derive a common definition for work-related e ducation at community colleges. A three-round Delphi technique was conduc ted to seek levels of agreement and consensus on OBanions princi ples and seven components a ssociated with work-related education. The Round 1 sample (n=20) attrite d by Round 3 (n=15) to six CEOs and nine administrators from community colleges w hose CEOs were represented on the League for Innovations board of directors. Content validity was essentially built-in by the development of the content of the scale matching the content domain, as conveye d by the participants responses and what they considered to be the constructs of inte rest. Internal validity claims were met by

PAGE 10

x following the established procedures for the Delphi technique to answer inferential questions about the scores and to develop well-founded conclusions from the data. A Duncans multiple-range test confirme d significance between Round 1 and Round 3, which combined with the research procedur es and study attributes validated reaching a superior group view. Review of the quantitative and qualitative da ta over the three rounds revealed levels of agreement and consensus on the princi ples and the components. Correlation coefficients were organized to facilitate comparisons between the principles and the components for significance. Significant corre lations were found between the principles and six of the seven components. Ther e was specific commonality found among the funding component and the coordination and pl anning component across the majority of the principles. The correlations were mode led to derive a comm on definition for workrelated education. Results of this study suggest that such a prototype model and further research could facilitate a consolidated position and common definition for work-related education. The studys conclusions have implications for why community college leaders and policymakers should pursue a common definitio n for work-related education within a national context to uphold the community college s role as preparers of the nations workforce.

PAGE 11

1 CHAPTER 1 INTRODUCTION Background for the Study The central and comprehensive role that community colleges fulfilled in preparing Americas workforce this past century a nd continue to accomplish into the new millennium has been most apparent. Indeed, work-related education was a key aspect of the community college mission from its beginnings. In the early 1900s, Witt, Wattenbarger, Gollattscheck, and Suppiger (1994) pointed out in Americas Community Colleges: The First Century : Whereas universities molded students to fit their classical curricula, two-year colleges adapted to meet the needs of a changing nation. They created vocational, technical, and preprofessional programs to train skilled workers, from nurses to keep people healthy, to mechanics to keep people mobile. (p. 3) In recent years, the work-related educa tion mission of community colleges evolved, as substantiated by Bragg (2001): Mode rn community colleges have a major responsibility for preparing the nations curr ent and future midskilled workforce, which accounts for three-fourths of all employees in the United States ( p. 5). As reported by the Department of Labor, occupations requi ring a postsecondary vocational award or an academic degree accounted for 29 percent of all job growth in 2000 and will account for 42 percent of total job growth from 2 000 to 2010 (Bohlen, 2004, p. 4). The National Center for Education Statistics (NCES), a nd the Education Commission of the States (ESC), and the Center for Community College Policy (CCCP) reported that about half of the 565,000 associate degrees (Na tional Center for Education Statistics [NCES], 2001)

PAGE 12

2 and the greater part of the 165,000 advanced certificates (C enter for Community College Policy [CCCP], 2002) awarded annually were in work-related fields, including nursing, business, and engineering technology. However, defining work-related education as a national model has been difficult because of the lack of agreement about what was salient to all or most of work-related education, including how policy was made and funding was channeled. In addressing the Annual Economic Forum in Scranton, Pennsylva nia, Dr. Patricia D onohue, president of Luzerne Community College, advised panel me mbers that this coun try doesnt have a definition for what workforce development is and we dont have a workforce policy. We dont have it nationally. We don t have it at the state [level ]. We dont have it locally (Donohue, 2004, p. 9). Contributing to the lack of a common definition was the wide range of individuals who sought work-relate d education from diverse backgrounds with varying purposes and outcomes. This range included high school and technical center graduates, current and tran sitioning employees, immigrants as well as adults in continuing education. Donohue emphasized the difficulty in trying to define the broad mission of work-related education noting that [it] could mean anything . from how do we help a welfare person get basic skills to how do we help a double Ph.D. get a new skill for the highest tech thing and everythi ng in between (p. 9). Another contributing factor during the past 20 year s has been the attempt of legi slators to address the issue by funding work-related education th rough 382 pieces of federal legi slation, not to add on all the state legislati on (Donohue, p. 9). Community colleges, individually and co llectively, have ne ver been static institutions, and they have demonstrated gr eat flexibility and creativity in response to

PAGE 13

3 student needs, work-related education expect ations, and governmental requirements. At the same time, the varying and vast descrip tions and labels, as well as the categorical funding and the multiplicity of funding stream s, have made it difficult for community colleges to maintain consistency in their work-related education programs and to establish new programs based on emerging workforce needs. Part of the disconnection between comm unity colleges and their stakeholders perpetuates itself with the seemingly ev er-changing terminology used for work-related education. In the case of vocational edu cation, it would seem the terminology for work-related education has come full circle from the early 1900s to the new millennium. In 1918, Floyd McDowell published the first kn own dissertation of the first national study of junior colleges, and it was noted t hat 17 percent of the work offered by public junior colleges was vocational (Witt et al., 1994, p. 41). The phrase career studies was the in vogue terminology for work-relate d education in Cohen and Brawers (1996) The American Community College (third edition). By th e fourth edition, however, vocation-technical or simply vocati onal education was again the preferred terminology (Cohen & Brawer, 2003, pp. 222). In fact, during the past century many different terms have been used as labe ls for work-related education, such as Occupational, career, techni cal, or technological, semiprofessional, subbaccalaureate, and terminal with each of these labels having a slightly different but admittedly related intent (Bragg, 2001, p. 6). Consistency in describing, labeling, and defining workrelated education had yet to be achieved during the 100-plus y ears that community colleges have existed. Bragg referred to anot her set of terminology associated with work-

PAGE 14

4 related education in the 1990s, includ ing workforce preparation, workforce development, human resource development, and economic development (p. 6). Communicating a common definition for work -related education to the highest levels has been one of the more complex pr oblems for community college leadership and government officials. However well-meani ng and practical, the community colleges natural tendency to incorporate the local co mmunitys situation and state-specific funding in its delivery of work-related education has resulted in many forms of organization, multiple sources of funding, and varied percep tions on the part of governing agencies. In fact, there was considerable variation in the support patterns and organization of work-related education among the states. According to the Education Commission for the States, state policy was a key factor to how effectively a community college supported work-related education in its area. In a survey conducted by the Center for Community College Policy (2002) at the Edu cation Commission of the States, 17 of the 45 state agencies, which were responsible for oversight of community colleges, indicated the level of funding for workforce development was a policy issue that had been debated during the past two years (p. 5). In additi on, 25 respondents stated that coordination of workforce funding was also a policy issue which was debated. The Center for Community College Policy stated that t o support workforce-training programs, community colleges often need to cobble together funding from a variety of sources (p. 7). Moreover, according to a survey conducted by the Center for Community College Policy: Too often, community colleges are seen as junior colleges or remedial institutions. This widespread lack of appreciation for their contributions to local economic development is surely one r eason that community colleges are often poorly funded, particularly in comparis on to four-year institutions. (p. 9)

PAGE 15

5 In a 50-state survey conducted by the E ducation Commission of the States (ECS, 2000), significant variations were noted about how community colleges are funded across the country, as shown by example in Table 1-1 fo r the states that comprise the League for Innovation in the Community College Board of Directors. For example, Illinois community colleges receive only 0.08 percen t of their funding from federal sources whereas Texas community colleges receive 14.4 percent of their overall general operating funds from the federal government. From a state-to-state perspective, Arizonas community colleges collect only 21 percent of their operating funds from the state, compared to North Carolinas community co lleges, which collect 75 percent of their operating funds from the state (ECS, 2000). Table 1-1 Percent Breakdown of General Operating Funds for 1998-1999 for States Represented on the League for Innovation Board of Directors Tuition & State Federala State Local Fees Otherb AZ 1.00 21.00 57.00 20.00 1.00 CA 3.80 50.90 44.50 0.80 0.00 FL 0.25 68.51 0.02 23.06 8.00 HI 2.70 61.80 0.00 16.80 18.70 IA 3.21 45.66 5.89 38.97 6.27 IL 0.08 25.77 43.24 26.93 3.97 KS 2.00 24.00 40.00 16.00 18.00 MI 0.30 26.50 25.00 23.20 25.00 MO 2.00 41.00 26.00 24.00 7.00 NC 3.20 75.20 12.90 8.20 0.50 NY 5.70 29.00 31.30 34.00 0.00 OH 2.71 45.29 16.73 32.21 3.05 OR 11.50 39.90 19.90 16.20 12.50 TX 14.40 37.90 17.90 19.90 9.80 WA 5.00 59.00 0.00 17.00 19.00 aIncludes all Perkins funds. bIncludes federal financial aid and restricted funds other than Perkins. Statement of the Problem Numerous authors and practitioners ha ve described and discussed how workrelated education has been symptomatically de veloped, established in policy, and funded.

PAGE 16

6 However, the root cause of many work-related education issues that community colleges face underscores policy and funding challeng es, specifically the lack of a common definition for work-related education beyond th e local community, state, and regional perspectives to a national level. While community college leaders collectively possessed an appreciation and understanding of work-related e ducation, local governments, state legislatures, and the federal government did not necessarily acknow ledge or hold in the same regard the comprehensive nature of work-related educati on in the role of edu cating and training the nations workforce. It was not apparent in the literature that a common definition existed for work-related education, which was univers ally recognized by community colleges let alone all those governmental agencies. On th e other hand, it also was not apparent that community college leaders have collectivel y conveyed the vision and mission of workrelated education beyond the communities that they serve. This lack of emphasis in vision and mission may have subsequently im pacted the ability to present a common front for policymaking and funding at higher levels of government. Throughout the past century, work-relate d education has had its opponents and advocates who have argued on political and ideological premises. Bragg (2001), however, states that rarely have they been based on empirical result s (p. 13). A large part of the issues surrounding work-related education deals with its changing focus and evolving goals and how research may addre ss questions about new ideas, models, and approaches (Bragg, pp. 13-14). The situation was summarized as unfortunate because it suggests that community colleges have missed opportunities to steer vocational education in directions that would provide the greatest benefi t (Bragg, p. 13). Therefore

PAGE 17

7 the challenge ahead would be to derive a common definition for work-related education and thus add to the body of knowledge pe rtaining to work-rela ted education at community colleges. Theoretical Framework Many community college practitioners agre ed about the concepts of humanistic education, learning communities, and, in particular, the learning college. However, there were no direct references in the literature that focused specifically on the applicability of the six principles outlined in OBanions model of the learning college to work-related education. T he learning college was a term used by OBanion (1997) as a generic reference for all educational in stitutions (p. 47). The theoretical framework of this study was based on the applicability of the learning colleges six principles to the processes and structure of lear ning in the work-related educa tion setting. This theoretical application and the research study method of fered community college practitioners an opportunity to carry out a di alogue about reframing work-related education in the theoretical context of the learning college on an equal footing with credit-based, degreegranting programs of study. However, the si x principles could not be considered allinclusive to the research by OBanion s own admission: Content, funding, and governance are examples of key issues that mu st be addressed and for which principles must be designed (p. 61). For the purpos es of this research study, OBanions theory was augmented by additional components specif ic to work-related education, as were identified in the literature. The learning colle ge and its six principles thus served as the theoretical framework to develop, evaluate, and reach consensus on how student learning should take place in work-related education settings as part of deriving a common definition for work-related education.

PAGE 18

8 Purpose of the Study The primary purpose of this study was to te st if work-related education conformed to the six principles of the learning college (OBanion, 1997, p. 47). Furthermore, this study would supplement OBanion s principles by examini ng additional complementary components of work-related education. The ove rall purpose would be to determine if in total the principles and components could be identified, categorized, and ranked to derive a common definition for work-related education at community colleges. In addition, this study tested the application of a Delphi technique to dete rmine if a Delphi technique could be effectively applied to an educational forum for leaders of community colleges. The point would be to achieve consensus and levels of agreement to support the rationale for establishing a consolidated position on and a holistic approach to what constitutes work-related education. Such a position coul d facilitate clarity and consistency in policymaking, particularly funding decisions, at the federal, state, and local levels. By participating in this study, the community co llege leaders acted as a panel of experts assisting in the research to derive a common definition fo r work-related education at community colleges. This researcher anticipa ted that through the co llective focus of the panel of experts, useful insights might be gained into how work-re lated education could pursue an increased presence and improved le vels of support for community colleges in the United States. The following research quest ions were developed to guide this study: Primary Questions 1. Which, if not all, of OBanions six princi ples of the learni ng college could be associated with work-related education? 2. What other components could be identifie d for the work-related education function at community colleges?

PAGE 19

9 3. What were the most strongly advocated principles and components supporting work-related education? Secondary Questions Additionally, secondary questions were id entified that could be answered as a result of this study. These questions were addressed through a comp ilation of answers to the primary research questions. 1. Could a selected group of community colle ge leaders reach consensus, using a Delphi technique, on what principles and co mponents could be identified to derive a common definition for work-related education? 2. Could meaningful relationships be confir med between the six principles and the identified components to derive a comm on definition of work-related education? The research questions were first addressed through an exte nsive search of relevant literature on work-related education in the field of higher education with a focus on community colleges. This literature review examined classical and current literature in the field of vocational and occupational educ ation, as it related to the postsecondary educational organization. The need for this research was to establish a basis for testing the theoretical framework of OBanions six principles of the l earning college and to identify components of work-related education. The principles and identified components were subsequently used in th e development of the initial survey instrument. The purpose of the survey was to collectively present a basis of potential principles and components which could generate expert feedback. Th is feedback would contribute to deriving a common definition for work-related education. Definition of Terms Community colleges were defined as any pub lic, regionally accredited, comprehensive, two-year institution. Th is study was limited to those 20 community colleges and districts listed in Table 1-2 whose chief executive officers (CEOs) served on

PAGE 20

10 the League for Innovation in the Community College Board of Dir ectors. Community colleges were not necessarily defined as a sing le college. They may have been part of a consortium or multi-campus facilities. Table 1-2 The 20 Colleges Represented on the Board of Directors to the League for Innovation in the Community College State/ College City Province Anne Arundel Community College Arnold MD Central Piedmont Community College Charlotte NC Cuyahoga Community College Cleveland OH Dallas County Community College District Dallas TX Delta College University Center MI Foothill-De Anza Community College District Los Altos Hills CA Humber Institute of Technology & A dvanced Learning Toronto Ontario Johnson County Community College Overland Park KS Kirkwood Community College Cedar Rapids IA Lane Community College Eugene OR Maricopa Community College District Phoenix AZ Miami-Dade College Miami FL Monroe Community College Rochester NY Moraine Valley Community College Palos Hills IL San Diego Community College District San Diego CA Santa Fe Community College Gainesville FL Seattle Community College District Seattle WA Sinclair Community College Dayton OH St. Louis Community College St. Louis MO University of Hawaii Commun ity Colleges Honolulu HI Community college leaders were defined as the expe rts on work-related education within public community colleges. These lead ers were identified as the colleges chief executive officers (CEOs), academic affairs o fficers, business/industry liaison officers, continuing education officers, or occupationa l education officers (American Association of Community Colleges [AACC], 2004, p. 3) at the 20 colleges listed in Table 1-2 whose CEOS comprised the League for Innovati on in the Community College Board of Directors. The League for Innovation is an international orga nization dedicated to catalyzing the community college movement (League for Innovation in the Community

PAGE 21

11 College [League], 2004, About the League, pref ace). Twenty (19 in the United States and 1 in Canada) CEOs from some of the most influential, resourceful, and dynamic community colleges and districts in the world comprise the Leagues board of directors (League, para. 3) The League has more than 700 member institutions from 10 different countries and has partnerships with more th an 100 corporations. With this innovative core of directors, members, partners, and co llaborators, the League leads projects and initiatives including the expa nsion and improvement of workforce training programs in the United States and Canada. Components were developed from a self-study guide published in a National Council for Occupational Education (NCOE) monograph (Hamm & Mundhenk, 1995, pp. 4-9). The components were defined as: (1) mission and organization; (2) funding; (3) needs assessment and documenting college success; (4) instruction, programs, and delivery systems; (5) staffing; (6) coor dination and planning ; and (7) national proclamation and national database. The Delphi technique was defined as a methodology th at utilized the expertise of current community college and district CEOs a nd their designated representatives. This methodology was used to reach levels of ag reement and consensus on principles and components. The purpose was to derive a co mmon definition of work-related education at community colleges within the realm of higher education in th e 21st century. The methodology is based on a series of questionnaires or surveys with each being more structured and requiring more focused reflectio n on the part of the participating experts. This technique is a preferred methodology in the measurement of subjective judgments when the problem or study does not lend itself to other precise anal ytical methodologies.

PAGE 22

12 The Delphi technique is an ite rative process that is recogn ized as an inductive-based approach to examining multiple issues and ex tracting specific answers to questions in a variety of disciplines. Higher education institutions were defined as all inst itutions of higher education, inclusive of two-year, four-yea r, private, and pub lic institutions that grant undergraduate or graduate degrees. A panel of experts was defined as those indivi duals who were selected to participate in the Delphi technique study. Th e panel of experts was chosen based on the participants knowledge, familiarity with the problem, and skill with written communication. Significance of the Study The point of this study was to depict th e ongoing plight for work-related education at community colleges, as well as the potenti al in pursuing a comm on definition model. To illustrate the dilemma, Donohue (2004) framed her remarks to the 2004 Economic Forum by stating: this country doesnt have a definition for what workforce development is, and we dont have a workforce policy (p. 6). This study was significant to community colleges and community colleg e leadership to present a consolidated position on how work-related education could be defined for students, policymakers, business and industry--all stakeholders in a comprehensive sense. If the research demonstrates that a common definition fo r work-related education is desirable, community college leaders could more eff ectively present a unified position for workrelated education for policymaking and funding d ecisions at the federa l, state, and local government levels. Communication of a comm on definition to bridge the gaps among

PAGE 23

13 practice, policy, and funding could be critic al for work-related education to achieve a level of parity with other programs. This study sought to confirm levels of agreement and consensus on OBanions six principles of the learning college and ot her identified components of work-related education, which in total supported a comm on definition. Although the results of the study could not be generalized to anothe r group or nationally, the concepts and framework were readily transferable for us e by other researchers. Policymakers and community college associations may choose to use this study as a guide for further research and policy development. They can gi ve due consideration to the complexity of work-related education and how identifying the principles and the components significant to a common definition may simplify the con cept. Developers of future work-related education programs may find the research helpful in determining the priority, focus, and applicability of new programs, which are base d on the levels of agreement and consensus reached in this study. Overview of the Research Process This investigation was based on a constructionist epistemology and a phenomenological perspective. It used a De lphi technique methodology for the research and for any generalization of the results (C rotty, 1998, p. 42). The research method used to gather and analyze data was based on a mixed survey of the Likert scale and openended items. Research participants had c onsiderable influence over the development of the survey instrument related to the research questions or statemen ts. This methodology was identified as the Delphi technique, which uses a series of questionnaires or surveys to aggregate the knowledge, judgments, or opinions of experts in order to address complex questions (Moore, 1987, pp. 50-51). This De lphi technique was conducted in three

PAGE 24

14 rounds of surveys, in which the instrument was adapted with each subsequent round. The Delphi technique was recognized as an appr opriate study design and assessment to make important decisions about educa tional policy (Clayton, 1997, p. 386). In addition to the Delphi technique, the theoretical framework offered by this research provided an evaluation frame to be tter compare, interp ret, and support workrelated education with other programs. Th e panel evaluated work -related education in the context of OBanions six principles of th e learning college. Th e panel also identified other components that could contribute to a common definition for work-related education to establish a cons olidated position. The panel of experts was surveyed to assess their perspectives regarding work-relate d education at their college, in their state and region, as well as nationally. The philo sophical stance of this study was based on research of numerous works published duri ng the past century. These works defined work-related education only to the extent that others addressed the problem to satisfy self-interests or the focus of a certain period in its history. Even mo re recent studies and research in the 1990s did not offer a solid context for today's policymakers to better determine the who, what, why, and how of wo rk-related education programs. Several community college leaders and researchers ha d alluded to the problem and its impact on policymaking, funding, and delivering quality work-related educati on, but the dialogue fell short of deriving a common definition. The use of qualitative data in educational re search is recognized as important to the study and the understanding of educational phenomena, as well as providing a natural basis for interpretation with explanations emerging from in tensive examination of the data (Tuckman, 1999, p. 400). Validity was esse ntially built-in after each phase by

PAGE 25

15 virtue of the Delphi techniques developmen t of the content of the scale matching the content domain. This was conveyed by the resp onses from the panel of experts and what they considered to be the constructs of inte rest. Internal validity claims were met by following established procedures for the Delphi technique to develop well-founded conclusions. External validity was dependent on the selection of the pa nel of experts as a representative body. Their scores or ratings may or may not be generalized for all community colleges in a particular sample, group, or population. Assumptions, Delimitations, and Limitations Assumptions This researcher made the assumption that participants would answer the survey honestly and they would return each of the three phases in a timely manner. This researcher also assumed that the information collected would be usable as part of this research and that the participants were represen tative of the opinions of all their peers. Finally, this researcher assumed that those participants who chose to respond electronically would read, understand, and resp ond to each phase of the survey within a specified time frame without delegating th is responsibility to a subordinate. Delimitations The research was delimited to public community colleges and districts whose CEOs served on the Leagues Board of Dir ectors. Twenty CEOs were contacted to participate in the research study. In addi tion, each CEO was asked to identify another participant for the research study who carri ed out duties pertaining to work-related education at their institution.

PAGE 26

16 Limitations The limitations of this study included the use of a sample group from a specific organization. Only current chief executive officers and their designated representatives who were actively employed at community colleges were asked to take part in the study. While the theoretical focus was on OBanions si x principles of the learning college, it could not be construed that th e participants understood the paradigm shift offered by this framework, nor directly endorsed the six pr inciples, nor implemented the learning college vision in its totality at their re spective institutions. Although there is no guarantee, this study may provide valuable info rmation to other orga nizations or regions of the United States which were not dire ctly represented by the sample group. Only two-year public community college l eaders were included in this study. Universities and private colleges were not in cluded. This study did not attempt to compare two-year community colleges to four-year colleges a nd universities. Additionally, these data were self-reported rather than observed by an impartial third party. It was vital that these data were the direct reflec tion of the community college leaders and others who were chosen to participate--and not the opinions of subordinates. This limitation was discussed in in itial contacts with th e participants. Each participant received and submitted materials electronically through the use of electronic mail systems and web-based systems at their respective community colleges. Organization of the Remainder of the Study Chapter 2 represents a review of literature pertinent to work-related education in the community college, in particular, the con cept of the learning college as a theoretical framework for the study, and also the identi fication of other com ponents pertinent to work-related education. Chapter 3 descri bes the research methodology, including the

PAGE 27

17 development of the initial survey instrument the three consecutive rounds of the Delphi technique, and the statistical procedures and tests employed for data analyses. Chapter 4 identifies the stratified sa mple of the population group w ithin the public community college arena as chief executive officers and administrative leaders who comprised the panel of experts. Chapter 4 also contains detailed reporting of the data results and analyses from the three-round Delphi tech nique. The Delphi technique focused on evaluating the applicability of OBanions six principles of the learning college to workrelated education, as well as other com ponents identified from the literature and developed through the three-round survey pro cess. Additionally, qualitative comments, opinions, and responses were encouraged from the participants to enrich the study, and their inputs were reported in the appendices. The use of qualitative data in educational research was recognized as important to the study, as well as and understanding of educational phenomena and providing a natural basis for interpretation with explanations emerging from intensive examination of the data (Tuckman, 1999). Chapter 5 presents conclusions based on the results from the data that were compiled and the relationships which were identified to expand the existi ng body of knowledge pertaining to workrelated education. A prototype model of a common definition for work-related education is outlined, the commonality of certain com ponents is identified, recommendations for further research are offered, and implicat ions for community college leaders and policymakers are presented.

PAGE 28

18 CHAPTER 2 LITERATURE REVIEW Introduction and Scope This chapter consists of a review of th e relevant literature for the study. The research focused on the concept of work-rela ted education in community colleges and on the principles and components that could be identified to derive a common definition for work-related education beyond community, state, and regional perspectives to a national level. Models of higher education were rese arched to seek a lear ner-centered innovation as a theoretical basis for framing a comm on definition for work-related education. Specific theory principles identified in the literature established the framework of the study and the three-round Delphi technique to which participants could respond to the application of learner-centeredne ss in work-related education. To complement the theoretical framewor k, components of work-related education identified in the literature were included as touch points. These touch points were used for an expanded agenda to discuss the resear ch topic in an effort to seek potential agreement of what constitutes work-related education. The research considered the evolution of work-related education in higher education and primarily focused on community colleges. The results of the litera ture review were arra nged in this chapter by topics that evolved from the beginnings of community colleges and work-related education to the theoretical framew ork and component identification.

PAGE 29

19 The Concept and Identity of Community Colleges The comprehensive literat ure on community college evolution (Koos, 1924, 1925; Witt et al., 1994; Baker, Dudziak, & Tyle r, 1994; Cohen & Brawer, 1996, 2003) begins with historical, yet perceptive, analyses of how community colleges evolved and then culminated with trends, challenges, and for ecasts for the future. From a historical perspective, the initial reactions of many to what the function of the community (junior) college was could be summed up as: to l ook upon this new unit in the school system solely as a sort of isthmus connecting the mainland of elementary and secondary education with the peninsula of professiona l and advanced academic training (Koos, 1925, p. 16). To the contrary, Koos (1925) purpo rted that he and ot hers like him had higher expectations for the junior college as an institution affecting much larger proportions of the population and influenci ng profoundly the organi zation of education on levels above and below (p. 16). Witt et al offered the following concept of two-year colleges: Whereas universities fought to remain excl usive, junior colleg es measured their success by inclusion. Whereas universities molded students to fit their classical curricula, two-year colleges adapted to m eet the needs of a changing nation. They created vocational, technical, and preprofe ssional programs to train skilled workers, from nurses to keep people healthy, to mechanics to keep people mobile . truly becomes the university of the common man. (p. 3) Gleazer in his foreword to Americas Community Colleges: The First Century emphasized that community colleges have cont inually pursued a search for institutional identity for recognition and public understanding in terms of a mission different from and yet in some respects sim ilar to the missions of both its progenitors, the secondary school and the college (Witt et al., 1994, p. vi ). By building on Kooss (1925) views, Gleazer made the point that a longstanding complaint of community colleges was the

PAGE 30

20 lack of understanding and mis understandings that evolved from the mixed parentage of the earlier days. Gleazer noted in those days that the junior college could be another two years of secondary school--an extension of the high school . . or it could be the first two years of college (p. vii). Lucas (1994) shared the perspe ctive that early twoyear schools viewed themselves as a preparatory step to university life and a professional career. . by the late 1920s a nd early 1930s the trend was . as terminal institutions where students of limited means mi ght prepare themselves for skilled trades and semiprofessions (p. 221). Cohen and Brawer (2003) stated: The easily accessible, publicly supported school became an ar ticle of American faith, first in the nineteenth century, when responsibility for educating the individual began shifting to the school, then in the twenti eth, when the schools were unwarrantedly expected to relieve societys ills. (p. 3) Campbell, Leverty, and Sayles (1996) found that community colleges assumed these responsibilities readily as a new concept in higher educ ation based on their demonstrated flexibility in adapting to social and economic challenges facing communities, states, regions, and the nati on (p. 172). Lucas (1994) found that the somewhat ambiguous and paradoxical role of tw o-year institutions was to satisfy the precept that in a democracy everyone is entitle d to access to higher education (p. 221). Cohen and Brawer described the beginning evolu tion of the concept and identity as based on community colleges setting a ne w precedence in higher education: The community colleges thrived on the ne w responsibilities because they had no traditions to defend, no alumni to question their role, no autonomous professional staff to be moved aside, no statements of philosophy that would militate against their taking on responsibilitie s for everything. (p. 3) However, along with flexibility, ad aptation, and educational innovation, community colleges have experienced an evolutionary mix, which contributed to

PAGE 31

21 confusion about the community college missi on, definition, and nomenclature (Baker et al., 1994, p. 4). The Morrill Act (Land Gr ant College Act of 1862) provided basic expectations for higher education within the states. However, the manner in which states carried out implementation pr oduced many differences in organization, structure, and control of these institutions (Witt et al., 1994, p. 226). Ratcliff (1994) in A Handbook on the Community College in America (Baker et al., 1994) provide d a historical context of this dilemma as: Many things are meant by the terms co mmunity college, junior college, technical college, and technical institute. The lack of definition of these terms is attributable in part to the wide va riation in mission, governance, finance, and structure of two-year colleges in the United States. (p. 4) The use and definition of such terms as secondary education, vocational education, colleges, universities, and even higher education were peculiar to each state. These terms compounded the identity predicament with essentially 50 different bureaucratic implementations by states of the federal vocatio nal acts that funded the states (Witt et al., 1994, p. 226). Gleazer (Witt et al.) found that while legislative language and accrediting manuals define and describe community college s for their particular needs, there still persists, most notably at the federal and st ate levels, less than full appreciation of the community college as an institution with an id entity of its own (p. viii). At the federal level, the community colleges functionality was driven by the wars in the 20th century and, particularly, in the 1960s as educational legislation grew. Gleazer (Witt, et al.) also noted that it was often necessary to examine congressional intent in order to determine whether community colleges were included in le gislative measures affecting colleges. . . (p. viii). Campbell et al. (1996) found that state funding for highe r education reflects

PAGE 32

22 each states preference for higher education among other services funded by the states (p. 174). Cohen and Brawer (2003) stated: Nevertheless, other writers in education, and certainly the majo rity of those who comment on the role of the community co lleges, suggest that education is an essential expenditure for economic grow th, a common good, and is not merely a nonproductive sector of the economy, a form of consumption. (p. 242) Witt et al. (1994) found that the Carnegie Commission on Higher Education recommended that more students be channeled into two-year colleges, particularly into vocational programs, which in turn help ed mold Americas emerging statewide community and junior college systems (p. 216). Honeyman and Bruh (1996) noted an observation in the preface of the Seventh Annual Yearbook of the American Education Finance Association made by Mary P. McKeown in the 1980s. She stated that higher education funding issues were tied to sources of funding, levels of funding, and the very existence of the institutions with educati on being equated to economic growth in the United States and the world (p. vii). Transitioning to the 1990s, Honeyman and Bruh (1996) endorsed the public expectation that colleges and universities contribute to th e economic well-being of our nation by producing a highly trained and skil led workforce while noting the change from McKeowns reference point to two th emes of increased attention in American higher education: institutional accountability and educational quality . with new performance-based accountability demands (pp. 9-10). Again, the impetus of many aspects of the community college movement can be tied to national policy and significant pieces of legislation, which formed the con cept and identity of community colleges over time. Witt et al. (1994) identified the following two examples of major national

PAGE 33

23 legislation, which expanded American hi gher education. These examples were significant to the establishment and expansion of community colleges as well: The Land Grant College Act of 1862 and the GI Bill of 1945 have represented great steps in moving American higher educa tion toward the universal educational opportunity envisioned by Thomas Jeffe rson and by the Ordinance of 1787. The agricultural and mechanical colleges were often referred to as peoples colleges as were early junior colleges. (p. 275) Cohen and Brawer (2003) stated that comm unity colleges would continue to appeal to recent high school graduates becaus e of easy access, low cost, and part-time attendance possibilities, as well as to job seekers because of the high demand for people in occupations for which some postseco ndary training but not a bachelors degree is expected (p. 407). Witt et al. (1994) f ound from the beginning of the community and junior college movement that these colleges a dvocated a mission with three basic tenants: preparing students for transfer to a four-yea r college, providing vocational training, and serving as a source of con tinuing education for the comm unity (p. 235). Although all three tenants have been stressed throughout th e history of community colleges, the workrelated education tenant in particular has not always re ceived equal attention. Work-related Education In between the origins and the outlook for the future, community college researchers and authors of th e past century often identifi ed and addressed work-related education as one of the purposes of the twoyear colleges, as well as making an early connection of community and college within their works. Cohen and Brawer (2003) described the early days of vocational programming as a force where the community colleges grew in part because some of their earlier proponents recognized the coming need for semiprofessionals and despaired of the universities adjus ting rapidly enough to provide this less-than-baccalaureate educati on (p. 220). In fact, at the American

PAGE 34

24 Association of Junior Colleges (AAJC) org anizational meeting in 1920 and at nearly every meeting throughout the 1920s and 1930s, occupational education was on the agenda (p. 221). The discussions at th ese early AAJC meetings centered around arguments on behalf of work-related education. Cohen and Brawer found that the thesis of Brint and Karabels book The Diverted Dream (1989) is that the AAJC was the prime fo rce in effecting a change in community college emphasis from prebaccal aureate to terminal-occupation al education (p. 221). In 1922, the AAJC revised its statement of purpose in its constitution to better reflect the ties to community and work-related education: The junior college ma y, and is likely to, develop a different type of curriculum, suite d to the larger and ever-changing civic, social, and vocational needs of the entire co mmunity in which the college is located (Witt, et al., 1994, p. 40). Parnell (1985) stated that work-related education at community colleges brought vocational and technical e ducation into the ha lls of ivy-covered institutions and noted that community colle ges communicated that there was dignity and worth in all honest labor (p. 87). A Purpose of the Community College Cohen and Brawer (2003) found that voca tional-technical education was one of the curricular functions written in the plan s for public colleges from the beginning and identified in most states le gislatures. In 1900, William Ra iney Harper suggested that many students were likely to terminate their education after completi ng junior college in order to seek positions as teachers or to go into business (Cohen & Brawer, 2003, p. 220). In 1915, Angell noted the interest of some junior colleges to specialize particularly in industrial, engineering, and vo cational directions, with its main interest centered on young people who will not go beyond the instruction it offers (Witt et al., 1994, p. 39).

PAGE 35

25 The 1920s found junior college leaders, in particular, Leonard V. Koos, focusing more and more on vocational education. In 1924, Koos was referred to as the greatest booster of vocational programs by some colle ge researchers and authors (Witt et al. 1994, p. 48). He sought out university deans to confirm which professions could be taught at two-year colleges. Koos (1924) confir med that in the field of engineering, as an example, the university deans he contacted id entified 43 occupations that could be moved to a junior college level (p. 155). These s emiprofessions, as Koos referred to the occupations, became touch points on a national effort to expand work-related programs at junior colleges. Koos (1925) found an appreciable beginnin g and partial awareness of the large need represented, as he sought out work-related education (semiprofessional training) at 300 institu tions or organizations (p. 133). Hi s overall investigation identified 20 purposes for the junior college, which were organized into five groupings. While the first purpose pertained to transferability, it was more conscionable when Koos (1925) made the point that: Purposes (2 and 3) are among those whic h would make it possible for the junior college to serve the interests of those w ho are not going on. The former urges for such students the provision of opportunities for rounding out their general education, opportunities which are not given if the work offered is only that regarded as preliminary to some fo rm of advanced training. (pp. 19-20) Koos (1925) was seeking specific status fo r work-related education when he coined the phrase semiprofessional training, a descript or for the third purpose. This referred to preparation for occupations, such as teachi ng as a sole occupation for which states granted certificates to teach upon the completion of some or all of the work of the two years if the candidate includes courses in education (p. 20). Gleazer (1968) made a point in the 1960s regarding national studies:

PAGE 36

26 But actually two-thirds of those enrolling will not transfer to a four-year college . . They will require organized educational experiences other than those leading to the bachelors degree. . A substantial part of the two-thirds will prepare for employment. (pp. 66-67) Parnell (1985) advocated a concept for improving work-related education in the mid-1980s to address the same concern about the education system as the lack of a rigorous, constructive, and focused program of study to prepare the sixty to seventy percent of our high school students who will not likely be pursuing a baccalaureatedegree program (p. xi). Bryant (1996) noted that ten years after Parnells concept was introduced, the initia l tech prep steam, however, ha s begun to slow down (p. 414). He attributed the slowdown, not to any viabi lity issue with the concept, but rather with the inability to adequately define the concept --a system which was as fluid as the wind. Bryant advocated that several of the problems disappear with the proper definition of the articulation system and offered a mode l based on required, suggested, and postsecondary components (pp. 418-421). Ratcliff (1994) presented a different perspe ctive of the community college (Baker et al., 1994) and sought to cast a social cont ext by describing the community college evolution as seven streams of educati onal innovation of which one stream was identified as the vocational education movement (p. 4). Ra tcliff also referred back to the influences of William Rainey Harper who brought about the es tablishment of twoyear colleges, including the Lewis Institute of Chicago in 1896 and the Bradley Polytechnical Institute (now Bradley University ) in Peoria, Illinois, in 1897 (Baker et al., 1994, pp. 11-12). Witt et al. (1994) documented th at following the classic works of Koos in the 1920s, work-related education took on a central role as a so lution to the Great Depression era. In 1934, Doak Campbell, the secretary of the Amer ican Association of

PAGE 37

27 Junior Colleges declared that education wa s the strongest and cheap est social insurance that could be employed, and the nation that ne glected it was inviting disaster (p. 104). Lucas (1994) noted that two-year institutions we re lauded as instruments of social utility and efficiency . [and that ] junior colleges continued to flourish throughout the Depression years, even when larger public un iversities languished fo r lack of adequate funding from state legislatures (p. 221). Witt et al. noted that Campbell foresaw a time after the current disaster when three-quarters of all junior college graduates would be in vocational and terminal programs (p. 104). Cohen and Brawer (2003) found that vo cational education enrollments began growing at a rate greater than liberal arts en rollments in the 1960s and continued to do so for 20 years. They attributed the increases in enrollment for work-related education to several causes: This rise is attributable to many causes: the legacy left by early leaders of the junior college movement and the importuni ties, goadings and sometimes barbs of later leaders; the Vocational Educati on Act of 1963 and later amendments; the increase in the size of public two-year colleges; the increase in part-time, women, disadvantaged, disabled, and older students; the community colleges absorption of adult education programs and postseconda ry occupational programs formerly operated by the secondary schools; and the changing shape of the labor market. (pp. 226-227) Indeed, as found by Witt et al. (1994), V ocational programs were a boon to local industry, and junior coll eges could quickly adapt to the needs of employers by retrofitting existing programs and developing new ones which satisfied industries needs (p. 49). Consequently, it was those towns with a job training function that attracted local business, and industry reciprocated by becomi ng leading supporters of their local junior college.

PAGE 38

28 Witt et al. (1994) found that community colleges had traditionally been shortchanged in federal funding whereas Congr ess directed vocational funds to local high schools and technical centers. Community coll ege needs often fell through the cracks of federal funding, as well when higher educat ion programs, such as those funded by the National Defense Education Act, went primarily to universities (p. 209). Witt et al. found that the American Associati on of Junior Colleges (AAJ C) Commission on Legislation under chair Kenneth Skaggs was the leader in developing and publishi ng principles for legislative action that gained wide use (p. 226). The commissions model for state legislation drew from the research carried out by subsequent comm ission chair Dr. James L. Wattenbarger and others, and the model wa s used as a guide by a number of states (Witt et al., p. 227). Two major successes for work-related educat ion came in the form of two landmark education spending bills signed by Presiden t Lyndon Johnson in 1963. Witt et al. (1994) noted that the Higher Education Facilit ies Act of 1963 provided $1.2 billion for postsecondary construction projects, of wh ich $690 million was authorized for matching grants for undergraduate inst itutions (p. 209). With community colleges guaranteed 22 percent of the Facilities Act funds, $151 million brought premier attention to the significance of community colleges. Congr ess soon became aware that there was a community college or junior college in almo st every congressional district (Witt et al., p. 209). The Vocational Education Act of 1963 provided $450 million in new funds for construction and operation of vocational educat ion schools. The act created departments or divisions of junior colleges that were associated with work-related education entities and thus eligible for the funds. Witt et al. noted that this act was amended in 1968 to

PAGE 39

29 fund equipment grants, exemplary programs, consumer and homemaking education, and curriculum development (p. 210). Witt et al. (1994) found that the Education Amendment Act of 1972 provided $707 million for postsecondary vocational programs. Those funds were supplemented with Congress appropriating an additional $981 million. Two years later, they found that the number of vocational graduates doubled, and by the end of the decade, 62.5 percent of all two-year college graduates had received o ccupational degrees (pp. 251-252). Besides federal appropriations and legi slation, states were also co mmitted and, in some cases, more supportive of work-related education. The state legislators noted that while workforce reform was traditionally cent ered on federally funded programs, state expenditures exceeded those of the federal government (Cohen & Brawer, 2003, p. 228). Terminology of Work-related Education Cohen and Brawer (2003) found that in its earliest beginnings, work-related education in community colleges was based on teaching skills that were beyond the level of high schools. These skills were originally conceived as an essential component of terminal study--education for st udents who would not go on to further studies. . . (p. 22). Bragg (2001) details the beginnings of work-related education labeling, such as occupational, career, technica l or technological, semiprofe ssional, subbaccalaureate, and terminal. This education labeling was in addition to the newer terminology of the 1990s, which included workforce preparation, workforce development, human resource development, and economic development. The different usages were based on the historical significance, inte nt, or focus, for example, technology, and how broadly defined the activities for the re ferenced label (p. 6). Koos (1925) again coined the term semiprofessional for that final training, which required more than secondary school

PAGE 40

30 years and was classified as trades and cont rasted to professions, which required four or more years of work beyond high school (p. 20). Wattenbarger (1950) noted that terminal education may be thought of as being general and vocational (p. 60). Wattenbarger was first referring to the ties to high schools and the liberal arts college as genera l education. He referred to vocational education, as defined by Arthur B. Mays, as the unprecedented developments in the physical sciences, in medicine, and in the so cial science since the turn of the century [that] have greatly expanded the vocational area for which careful training is required (p. 62). Cohen and Brawer (1996) found that sem iprofessionals typically referred to engineering technicians, gene ral assistants, laboratory tec hnicians, and other people in manufacturing, business, and service occupa tions (p. 224). Cohen and Brawer (1996) referred throughout their third edition of The American Community College to the term career education and chose this term to represent a collective term for all occupational, career, and technical studies . Career Education was the term popularized by the U.S. Office of Education in the 1970s but was co ined in the 1950s to connote lower-school efforts at orienting young people towards the workplace (pp. 216). Lombardi (1992) stated that t he term career education has no more potency than the old names--occupational, semiprofessional, t echnical, vocational, tr ade (p.79). By their fourth edition, Cohen and Brawer (2003) chose to change their point of reference back to vocational education as the descriptor for work-related educa tion, suggesting that career education never quite caught on (pp. 224).

PAGE 41

31 Bragg (2001) noted the histor ical significance of the many terms that have been used to describe work-related education, but she chose to focus on the term vocational education because of its longe vity and historical significan ce (p. 7). Both Bragg and Cohen and Brawer (2003) agree that many terms have been us ed to describe work-related education and that the terminology has ne ver been exact (Cohen & Brawer, p. 222). The overall interest and focus of the published works was not necessarily on the particular term to be used. Instead, a term was used consistently to establish a logical benchmark for assessing change from the past to the present and to the future (Bragg, p. 7). This is essentially why the researcher chos e to refer to an original term work-related education within the boundaries of this study to be inclusive, not limiting, and without any bias of past terminology. Learning College Theory and Work-related Education Cohen and Brawer (2003) indicated that community colleges today and in the near future will continue to realiz e sufficient, if not increased, enrollments. They noted that the absolute number of 18 year olds in the United States peaked at 4.3 million in 1979, bottomed at 3.3 million in 1992, and is project ed to regain the 1979 level by 2009 (p. 405). However, two major reform reports in 1983 and 1993 gave reasonable concern as to whether or not community colleges, as pa rt of the national higher education system, were up to the task to serve the students of those decades and in the future. Recent Attempts at Educational Reform The National Commission on Excellence in Education (1983) presented its viewpoint of American education in A Nation at Risk: The Imperative for Educational Reform as:

PAGE 42

32 We report to the American people that while we can take justifiable pride in what our schools and colleges have historica lly accomplished and contributed to the United States and the well-being of its pe ople, the educational foundations of our society are presently being eroded by a risi ng tide of mediocrity that threatens our very future as a nation and as a people. (para. 2) In the next decade, the Wingspread Gr oup on Higher Education (1993) reiterated the general alarm of A Nation at Risk in its report, An American Imperative, which noted: A disturbing and dangerous mismatch exis ts between what American society needs of higher education and wh at it is receiving. Nowher e is the mismatch more dangerous than in the quality of unde rgraduate preparation provided on many campuses. The American imperative for the twenty-first century is that society must hold higher education to much higher expectations or risk national decline. (Higher Expectations for Higher Education, para. 1) As a precept to his concept of the lear ning college, OBanion (1997) addressed the reform efforts of the past noting that a great deal of reform effort . focused on the traditional architecture of education. . . The traditional education system is based, as has been noted, on an architecture that is time-bound, place-bound, efficiency-bound, and role-bound, undergirded by a grading system that assigns only 5 of 26 possible letters in the alph abet to designate amount and kind of learning achieved. (p. 63) Learning College Theory In response to an education system b ound and restricted by traditions, OBanion, a self-described zealot for Humanistic Educa tion, proposed a new way of thinking. His view would place the learner first and thus upset the established system and replace it with a new model. He based this new mode l on the writings of Dewey, Rodgers, Combs, and others pertaining to the natural educativ e process, client-cen tered therapy, and the humanistic education movement, respectively (pp. 42-44). Other au thors and researchers also recognized that a new era for community colleges was necessary to unbind the ropes

PAGE 43

33 of failed reform and forces of resistance. How community colleges would have to respond to increasing enrollment and satisfy student needs in a new era of community colleges was noted in the late 1990 s by Alfred and Carter (2000): By todays standards, however, first generation institutions could not survive. Their strategy was to "develop and deliver" and factors of demand, competition and quality were relatively insigni ficant as part of this stra tegy. . Growth was their focus and a comprehensive institution with many offerings was a natural, though inefficient, organizational form. To be fa ir, they did plan and were sensitive to market dynamics and student needs, but now here near the extent to which todays high performing organizations do these things. (para. 1) How community colleges respond and ensure they are indeed high performing required concentrated focus on what Alfred and Carter (2000) referred to as market dynamics and student needs (para. 1). As one scholar rhetorically asked, Is education to be organized around institutions, credit, and credentials . or is education to be organized around learners as an optimal system for distributing knowledge and encouraging its utilization? (p. 11). One widely recognized theoretical response was OBanions (1997) model for community colleges of the future titled A Learning College for the 21st Century, a new concept in the late 1990s. He described this concept as being built on the long-established values in th e community college, values that place a premium on quality teaching. . . (p. xvi). In the foreword to OBanions modern classic, Patricia Cross de scribed how OBanion offered a compelling rationale for focusing the attention of higher education on student learning, i.e., on creating the learning college. . Community colleges w ill be bellwether institutions if they adopt OBanions vision for the learning colle ge (OBanion, 1997, p. x). In his book, OBanion provided a framework of the reform movements of the past decade and the emerging focus on learning (p. xiv). He pr esented a new model for education designed to help students make passionate connections to learning, and he identified six key

PAGE 44

34 principles that formed the emerging definition and character of the l earning college . . (p. xiv). Again, OBanion (1997) refe rred to many other critics besides himself (for example, Cross, Marchese, Daggett, Leonard, Ge rstner) who criticized the reform efforts of the past two decades as mismatched, f alling short, and even detrimental. OBanion summed up the reform effort prompted by A Nation at Risk in 1983 as the spectacular failure (p. 6). OBanion conc luded that reforms were too focused on addons or modifications to the current system, and did not address core issues that alluded to the root cause that the institution itsel f was the problem. He further described the reform efforts in the 1980s as trimming the branches of a dying tree (OBanion, p. 7). In direct response, OBanion proposed a fo cused theory for the community college based on the assumption that . the learning college places learning first and provides educational experiences for lear ners anyway, anyplace, anytime (p. 47). OBanions (1997) theory was used as the theoretical framework for this study and was based on the assumption that educat ional experiences are designed for the convenience of the learners rather than for the convenience of institutions and their staffs (p. 47). The focus of this study, th at is, the establishment of a consolidated position on and a holistic approach to what constitutes work-related education, was anticipated to reveal how a common definition or lack thereof was integrally related to policy development and funding decisions. The th eory of the learning college and its six principles served as a foundation to understa nd the process. The principles provided a framework for developing, evaluating, and synthesizing the complex dynamics that

PAGE 45

35 influence how work-related education could be viewed in higher education and at the federal, state, and local levels of government. Six Principles of the Learning College Again, OBanion proposed a focused theory for the community college based on the assumption that . the learning college places learning first and provides educational experiences for lear ners anyway, anyplace, anytime The framework of the learning college is based on six key principles (OBanion, 1997, p. 47): Principle I: The learning college creates substantive change in individual learners. This principle was described by OBani on as self-evident and an embedded value undergirding all other principles (p. 48). This principle was integrated into the research study to determine if work-related education should create substantive change in its learners. This principle was used to "k indle" (stimulate) new ways of seeing, thinking, and doing--in dramatic first events and new discoveries, and also to "kindle" (stimulate) new ways of seeing, thinking, and doing--incrementally in day-to-day experiences. Principle II: The learning college engages lear ners as full partners in the learning process, with learners assuming pr imary responsibility for their own choices. This principle was described by OBanion: A series of services will be initiated to prepare the learner for the experiences and opportunities to come (p. 49). This principle was integrated into the res earch study to determine if work-related education should communicate that students are full partners in the creation and implementation of their learning experiences. This principle was used to determine: if students will assume primary responsibility for making their own c hoices about goals and options; if students

PAGE 46

36 should be required to participat e in a structured induction/or ientation process; and if a personal learning plan or negotia ted contract should be requ ired for students (pp. 49-51). Principle III: The learning college creates an d offers as many options for learning as possible This principle was described by OBanion as options regarding time, place, structure, and methods of delivery (p. 52). This principle was integrated into the research study to determin e if work-related education s hould offer a full array of options to accommodate individu al differences in learning st yles, rates, aptitudes, and prior knowledge and what options should be o ffered (p. 52). In addition, the research was augmented by other statements which comp lemented OBanions descriptors. This research was to determine if work-related educational options should be: seamless, that is, not operated in isolation, so students can ma ke reasonable changes in their programs; trackless, that is, same beginning courses for several programs so students can explore before committing to a single track; and/or cla ssless, that is, similar skills within same programs at other institutions, thereby provi ding mobility for students who may change institutions (Hamm & Mundhenk, 1995, p. 13). Principle IV: The learning college assists lear ners to form and participate in collaborative learning activities. This principle was described by OBanion as transforming the traditional institution ideal of a community of scholars into a new ideal of community of learners. This principle was integrated into the research study to determine if work-related education s hould focus on creating communities among all participants (students, faculty, and other learning specialists) to : support individual learning; form and support learning commun ities in the workplace; establish learning communities and provide assessment services in the workplace.

PAGE 47

37 Principle V: The learning college defines the role s of learning facilitators by the needs of the learners This principle was framed by OBa nion to submit that if learners have varied and individual need s that require special attention, then it follows that the personnel employed in this enterprise must be selected on the basis of what learners need (p. 57). This principle was integrated into the resear ch study to determine if workrelated education personnel shoul d be hired on the basis of depa rtment or course needs or hired based on what learners need. This prin ciple was used to determine if work-related education students should also participate as le arning facilitators, that is, to capitalize on the resources students bring, to free professi onal staff for other roles, and to reduce personnel costs (O Banion, p. 60). Principle VI: The learning college and its lear ning facilitators succeed only when improved and expanded learning can be documented for its learners This principle was described by OBanion as the framework for documenting outcomes, both for the learner and for the learning facilitators (p. 60). This principle was integrated into the research study to determin e: if work-related educati on should require work-related educational competencies for entrance or exit; if portfolio assessment should be the primary means by which work-related learni ng is documented--should national or state standards not be available; and if community colleges s hould employ specialists or contract to develop "industry-based" sta ndards (similar to heal th care occupational programs). While OBanions (1997) six principles ref er primarily to process and structure, they were developed with the basic philosophy that the student is central in all activities within the scope of the edu cational enterprise (p. 61). Nora (2000) referred to

PAGE 48

38 OBanions notes about community colleges providing the ideal forum for the learning college with a caveat that different pract ices work differently on different student populations at different two-year colleges (A Bluepr int of Priorities for Action for Community Colleges, para. 3). Nora thus acknowledged the potentia l of this research direction of whether or not th e theoretical framework of th e learning college theory and its six principles could be identified a nd associated with work-related education. OBanion also recognized that other principles . must be considered. . Content, funding, and governance are examples of key issu es that must be addressed and for which principles must be designed (p. 61). This study sought to supplement OBa nions (1997) principles by examining additional complementary components of wo rk-related education. Accordingly, it was anticipated that such a comb ination of specific, theoreti cal principles and relevant, practical components could be identified, categorized, and ranked to derive a common terminology and definition for work-related education at community colleges. In addition to OBanions si x stated principles, components we re identified as: mission and organization; funding; needs assessment a nd documenting college success; instruction, programs, and delivery systems; coordinati on and planning; and national proclamation and national database for work-related education. Components of Work-related Education This section consisted of a review of the relevant lite rature to identify components of work-related education which could be cons idered complementary to the principles of the learning college (OBanion, 1997, p. 61). Th ese components were included as touch points for an expanded agenda to survey partic ipants on the research topic. They were also to seek convergence of agreement on what principles and component s, in total, could

PAGE 49

39 be identified, categorized, and ranked to derive a common definition for work-related education. Mission and Organization Mission and organization were descri bed by Hamm and Mundhenk (1995) in the form of three questions (p. 4): 1. Does the colleges mission statement focus in a significant way on workforce development? 2. Are those parts of the coll ege that deliver in-service upgrade workforce training and retraining, as well as noncredit pre-traini ng, explicitly a significant part of the mission? 3. Are they politically an important part of the organization? This component was integrated into the re search study to determ ine: if the mission statement should clearly claim the role of wo rk-related education equal to other mission tenants; and if work-related education should be politically centrally planned and funded as an important part of the organization. Gleazer (1968) advised that when a community college commits itself to occupational education . it is affirming an institutional viewpoint which affects every as pect of its operations (p. 79) Cohen and Brawer (2003) noted the following issue of merging work-rela ted education on an equal basis with the collegiate function: The full effects of vocational education as a primary function have yet to be discerned. The publics view of community colleges as agents of upward mobility for individuals seems to be shifting to ward a view of the institutions as occupational training centers. This narro wing of the colleges comprehensiveness could lead to a shift in th e pattern of support. (p. 251) Funding Funding was described by Hamm and M undhenk (1995) in terms of community colleges assessing the viability of work force development funding by developing

PAGE 50

40 strategies to modify the priorities of t hose who do control them (p. 5), such as the criteria for funding. They posed two questions (pp. 5-6): 1. Do funding mechanisms acknowledge the cen trality of workforce development? 2. Does the college make any efforts to infl uence funding formulas in order to include the needs of the emerging workforce as well as instructional innovation? This component was integrated into the res earch study to determine if funding formulas should be influenced to include the needs of the emerging workforce on state, regional, national, and global basis and/or to include the needs of inst ructional innovation in workrelated education. Cohen and Brawer (2003) found that f unds are often secure d through priorities established by state and federal agencies (p. 233). However, according to Merisotis and Wolanin (2000), trends identified in 1995-1996 for total institutional revenues for community colleges indicated significant sh ifts since 1980 toward external revenue sources and away from core state and local funding for basic operations. Merisotis and Wola nin (2000) found: Since 1980 the fastest growing revenue cat egories for community colleges have been government grants and contracts--f ederal, state, and local programs for training and research--and priv ate gifts from corporations and individuals. In fact, as a share of total revenues, these four cat egories grew from 2 percent in 1980 to 20 percent in 1996, a tenfold increase in less than two decades. At the same time, state and local appropriations for basic operations fell from 70 percent of total revenues in 1980 to 50 percent in 1996 (U.S. De partment of Education 1980 and 1996). These revenue trends suggested that the process of financing community colleges has migrated toward a more private, work force-oriented education model. As the focus of community colleges has broade ned to include more focused worker training, resources to pay for this traini ng have increased substantially. (Revenue Trends, para.1) Needs Assessment and Documenting College Success Needs assessment and documenting college success were described by Hamm and Mundhenk (1995) in the form of two questions (p. 5):

PAGE 51

41 1. How effectively does the college assess labo r market needs or use available local labor market data? 2. Does the college explicitly measure its success in terms of its contribution to the development of local and regional economy? This component was initially integrated into the research study to determine if community colleges should be the expert s at monitoring local labor markets and collecting data for program planning and need s assessment. The integration of this component was also to see if success shoul d be measured by work-related education program, certificate, or degree completion rate s, or if success should be measured in terms of job generation, upgrades, retraining, an d economic development. In addition to completion rates, student enrollments could be interpreted as an element of work-related education success (Cohen & Brawer, 2003): The number of students who are already employed and enter vocational programs only to get additional skills must be factored in, just as the students who obtain job certifications but find no jobs available to them should be tallied. Students who leave before completing the programs and en ter employment in the field for which they are prepared must be cons idered program successes. (p. 235) Cohen and Brawer (2003) found that t he college staff presumably initiates programs by perusing employment trends in th e local area and surveying employers (p. 233). In addition, they found that career pr ogram success can also be measured by the number of students who obtain employment in the field for which they were prepared (p. 234). With obtaining employment as the ke y endpoint of work-rela ted education, it is a higher risk pathway compared to a liberal arts education. The costs in tuition and foregone earnings may be the same for both, but occupational training is almost entirely wasted if there is no job at the end (Cohen & Brawer, p. 248). Overall, when

PAGE 52

42 considering needs assessment and docume nting college success, Cohen and Brawer (2003) found: Because vocational education has several purposes, the measures of success that can be applied to it vary. It prepar es people for specific jobs. How much do business and industry gain when their worker s are trained at public expense? It assists the disadvantaged and people with disabilities to become self-sufficient. How much is that worth to society? It aids economic development. How much does a locality or region gain thereby? It enhances individu al income generation and career mobility. What value has been added, person by person? Indicators of success and, indirectly, legislation an d funding depend on which purpose is being reviewed. (p. 239) Instruction, Programs, and Delivery Systems Instruction, programs, and delivery sy stems were described by Hamm and Mundhenk (1995) in terms of whether community colleges were providing only entrylevel job skills or rather learn ing to learn skills and an or ientation to lifelong learning. They posed two questions (p. 7): 1. Do programs train for discrete jobs or for job clusters? 2. Are the business development and workfor ce training centers part of the regular college programming? This component was initially integrated into the research study to determine if workrelated education should be on entry-level skills or on learning-to-learn skills. Orientation to lifelong learning is an example: if work-related education should focus on preparation for a single job requi ring a focused set of skills or on learning skills with broad application to several similar occupations; or if wo rk-related education should be on an equal footing with regular college pr ogramming or considered as add-on revenue centers (separate from tradit ional credit programs). Cohen and Brawer (2003) viewed the futu re for work-related education as positive in that vocational education will remain prominent. There can be no reversing the

PAGE 53

43 perception that one of the colleges prime functions is to train workers (p. 420). Merisotis and Wolani n (2000) found that: The rapid evolution of the workforce mean s that employers are increasingly turning to community colleges as essential centers of worker training. . The key question for community colleges is how to strike a balance between these direct worker training efforts and general education progr ams that provide students with broader skills, such as critical thinking. (E volving Workforce Needs and Employer Relationships, para. 1-2) Staffing Staffing was described by Hamm and Mundhenk (1995) in terms of whether community colleges were providing only entrylevel job skills or rather learning to learn skills and an orientati on to lifelong learning. They pos ed three questions (p. 8): 1. How do staffing patterns within workforce development areas match patterns in traditional credit programs? 2. What workplace experience do current instructors, counselors, and administrators have? 3. What percentage of the college staff serves both traditi onal and nontraditional students? This component was initially integrated into the research study to determine: if community colleges should establish a staffing pattern of both hard-money and full-time positions to ensure that work-related education programs receive their fair share (compared to credit programs); if work-rela ted education faculty and staff need real workplace experience to communicate effectiv ely with students; if work-related education advisory committees provide suffici ent "real world of work" input to faculty and staff; if student services and advising should be the same for work-related education students as it is for traditi onal credit students; if the student placement office should primarily focus on identifying career openings and pathways; or if the student placement office should primarily focus on short-term training for immediate employment.

PAGE 54

44 Cohen and Brawer (2003) found that co llege staffs established work-related education programs by perusing employment tr ends in the local area and surveying employers. They also noted that program coordinators are appointed and advisory committees composed of trade and employer representatives established (p. 233). Coordination and Planning Coordination and planning were intr oduced by Hamm and Mundhenk (1995) with two questions (p. 9): 1. Is the thinking and planning of the co llege primarily local or regional? 2. Can the college provide data that sup port regional or national claims about workforce development? This component was initially integrated into the research study to determine: if workrelated education programming and planning sh ould take into account regional, national, and global trends; if community colleges should cooperate with one another to accomplish regional and national work-related education planning collectively; and if community colleges should seek to build coali tions and partnerships with other colleges, organizations, and business to define roles and a vision for work-re lated education. In addition, this component engendered the politi cal overtones as sugge sted by Hamm and Mundhenks statement that colle ges are expected to be se nsitive to satisfying the demands of the district and their governing boards (p. 9). Vaughn (1994) stated that, to be effectiv e, community college leaders must focus on establishing political leadership and that the president ensures that the colleges mission moves in concert with the goals of the community, the state, and when appropriate, the nation (p. 73). Eaton ( 1994) made four suggestions about the presidential role in public policy:

PAGE 55

45 First, presidents must be pivotal in th e definition of the issue at hand. Second, presidents must be strategically positi oned to influence both locally based and state-based constituencies: they have a responsibility to the local community, local legislators, and the local press, as well as to their colleagues on the same level. Third, presidents must strengthen their st ate-level organizations to augment their individual efforts and build a sense of equ ity among institutions. Fourth, presidents need to be willing to take some risks to redesign and restructure the community college as part of the states higher education enterprise--when and where appropriate. (p.124) Furthermore, Eaton noted that community college presidents have a great responsibility in influencing public policy . . presidents need to take risk s in planning for the future in the areas of public policy and college governance (p. 136). Finlay, Niven, and Young (1998) found that vocational education and training (VET) systems were in fact prominent in viewing work-related education on a global basis: Many developed and developing nations ar e looking to their VET systems to provide a response to changes in the globa l economy. Our earlier research (Finlay and Niven 1996) indicated that some countri es are proactive with respect to these changes, adopting long-term strategies th at should benefit their economies. (p. 3) However, Cohen and Brawer (2003) indica ted that other indu strialized nations offered few insights of how to reform or im prove work-related education in the United States: Some countries depend on postsecondary in stitutions to carry the main burden, some on schools in the compulsory sector and others on adult education that is provided by other than formal educational in stitutions. . The greatest proportions of students in vocational programs in form al postsecondary structures are in Japan, Germany, France, and Italy (National Cent er for Education Statistics, 1994f). (p. 241) National Proclamation and National Database A national proclamation and a national data base for work-related education were described by Hamm and Mundhenk (1995) in te rms of preparedness at the local level and its relationship and visibili ty at the national level so that community colleges could

PAGE 56

46 have appropriate influence on policy, funding, le gislation, and rule-making (pp. 15-16) This component was initially integrated into the research study as statements seeking to determine: if a national proclamation should be created and promoted which defines the role of the community college in work-related education; and if a national database reflecting community college potential and ach ievement should be created to assist in identifying limitations and areas of growth and improvement. Brand (Finley et al., 1998) described the process of change for work-re lated education in the United States as complex, difficult, slow . possible, which was impacted on a national basis due to the decentralized nature of work-related educatio n in the 50 states. Brand also noted that despite federal legislation impasses, governor s and state and local o fficials are taking the lead in changing their programmes [ sic ] and simplifying, consolidating and improving them (p. 153). Summary Work-related education continue s to be a topic of intere st, as well as a topic of complexity and/or perplexity. Research about community colleges has produced a myriad of works on the broad topic of work -related education thr oughout the history of higher education in the United States. This to pic continues to be viable research today. The lack of a pointed study and demonstrated efforts to achieve a common definition for work-related education reflects a gap in curren t research and practice. This is evident by the varying terminology and individuality of work-related education programs in the United States. Specifically, little evidence was found in the litera ture that directly addressed the need for or value of a comm on definition for work-related education. Yet the numerous and repeated deficiencies have su rfaced that appeal to the value of pursuing a common definition. The literature review offered a basis for this study and future

PAGE 57

47 research, which could provide meaningful, repl icable information to shape and promote a common definition for work-relate d education in community colleges. More specifically, the literature review served as the fo undation upon which the six learning college principles and other components id entified in the litera ture could be used as a catalyst for this research study to take place. This resear ch could ultimately contribute to efforts to derive a common definition fo r work-related education.

PAGE 58

48 CHAPTER 3 METHODS The Setting This study sought to determine if the De lphi technique could be effectively employed in an educational forum for lead ers of community colleges. The purpose would be to reach levels of agreement and consensus on what principles and components could be identified to derive a common definition for work-related education. Epistemologically, the Delphi technique coul d be derived from constructionism where the research results and conc lusions essentially represent a shared meaning based on the interactive process of the Delphi tec hnique (Stewart, 2001, p. 923). The research included the use of the Delphi technique to collect data from a panel of experts to determine if work-related education conforme d to the six principles of the learning college (O'Banion, 1997). Nora (2000) refe rred to OBanions notes about community colleges providing the ideal forum for the learning college with a caveat: Different practices work differently on di fferent student populations at different two-year colleges (A Blueprint of Priorities for Action fo r Community Colleges, para. 3). This acknowledges the potential of th is research direction of wh ether or not the theoretical framework of the learning college theory and its six principles coul d be identified and associated with work-related education. Furthermore, this study sought to supp lement OBanions (1997) principles by examining additional, complementary component s of work-related e ducation. This study would determine if, in total, qualitative prin ciples and components could be identified,

PAGE 59

49 categorized, and ranked to de rive a common terminology and definition for work-related education at community colleges. The use of qualitative data in e ducational research was recognized as important to the study for an understanding of educational phenomena and testing hypotheses. Qua litative data also provided a natu ral basis for interpretation with explanations emerging from intensive exam ination of the data (Tuckman, 1999). Linstone and Turoff (1975) summarized that the Delphi may be characterized as a method for structuring a group communication proce ss, so that the process is effective in allowing a group of individuals, as a whole, to deal with complex problems (p. 3). By participating in this study, the community co llege leaders acted as a panel of experts assisting in the research to derive a common definition fo r work-related education at community colleges. Primary and secondary re search questions were developed to guide this study and execute the methods. Primary Questions 1. Which, if not all, of OBanions six princi ples of the learni ng college could be associated with work-related education? 2. What other components could be identifie d for the work-related education function at community colleges? 3. What were the most strongly advocated principles and components supporting work-related education? Secondary Questions Additionally, secondary research questions were identified that could be answered as a result of this study. These questions were addressed based on the compilation of answers to the primary research questions. 1. Could a selected group of community colle ge leaders reach consensus, using a Delphi technique, on what principles and co mponents could be identified to derive a common definition for work-related education?

PAGE 60

50 2. Could meaningful relationships be confir med between the six principles and the identified components to derive a comm on definition of work-related education? This research study evolved into four stages: First Stage The first stage consisted of a review of the literature pertaining to community colleges and the role they fulfill in prepari ng the workforce. In conjunction with the literature review, a theoretical framework was identified in past literature which focused on the six principles outlined in OBanion s model of the learning college (OBanion, 1997). Additionally, specific work-related e ducation components were identified based on work contained in a National Council fo r Occupational Education monograph (Hamm & Mundhenk, 1995, pp. 4-9). These components were included in the survey to complement OBanions theory for those area s not addressed: O Banion advised that these components were key issues that must be addressed and for which principles must be designed (OBanion, p. 61). Second Stage This stage consisted of developing an initial mixed methods Delphi technique survey of Likert scale items, open-ended items, and open-ended comment blocks to assimilate data from the panel of experts. These data were base d on the principles and components initially identified. This initial identificati on process by the researcher ensured that the th ree-round limit of the Delphi technique was maintained by complying with the following three pro cedures (Linstone & Turoff, 1975, p. 88): 1. The monitor team [researcher] devoting a considerable amount of time to carefully pre-formulating the obvious issues; 2. Seeding the list with an initial range of options but allowing the respondents to add to the lists;

PAGE 61

51 3. Asking for positions on an item and underlying assumptions in the first round. This initial survey design ensured that all the obvious statements and issues had been included to the extent possible and that participants were being asked to supply the more subtle aspects pertaining to each topic (p. 88). The initial survey was juried by a community college vice president with re sponsibilities for work -related education, a psychometric analyst, and tw o institutional researchers. The open-ended comment blocks allowed participants to suggest additions, deletions, or changes in the wording of stat ements, which were then introduced as new items. These new items were developed by the re searcher after a careful distillation of all the qualitative responses, as presented in the appendix. Linstone and Turoff (1975) also found that the ratings on items were sensitive to wording, and because of this property, the material can mushroom in size after the first round: If the respondents feel str ongly about the issues, and this should be the case, they will generate a large amount of written material. If th ey are provided a certain number of items to deal with on the first round then each of them will make approximately the same number of writte n comments or additions in response. These must be abstracted carefully and duplications among the respondents eliminated. (pp. 92-93) Third Stage The Delphi technique was conducted in a se ries of three rounds to facilitate a detailed critical examination and structured communication process to focus attention on the problem. Both frequencies and the groups optional qualitative responses were shared with the participants in Rounds Two and Three. They supported the convergence of agreement on which principles and com ponents could be identified to derive a common definition for work-related education. Su rveys were offered in either an online electronic format or hardcopy format. The on line web format was the first choice by 100

PAGE 62

52 percent or 20 out of 20 total participants. Participants self-identified so the researcher could verify receipt of each participants survey and facili tate any follow-up as required. Fourth Stage The fourth stage consisted of the fina l compilation and reporting of the data, the analysis of the data, and the presentation of the results. Bo th quantitative and qualitative analyses were provided as assimilated from the panel of experts. The researcher used the results and analyses to draw conclusions and bring forward recommendations for future research. The Participants A specific group of community college lead ers were identified to comprise the panel of experts. These leaders were identifi ed as the colleges chief executive officers (CEOs), academic affairs officers, busines s/industry liaison officers, continuing education officers, or occupational educa tion officers (American Association of Community Colleges [AACC], 2004, p. 3) at th e 20 colleges listed in Table 1-2 whose CEOS comprised the League for Innovati on in the Community College Board of Directors. The League for Innovation is an international orga nization dedicated to catalyzing the community college movement (League for Innovation in the Community College [League], 2004, About the League, preface). Twenty (19 in the United States and 1 in Canada) CEOs from some of the most influential, resourceful, and dynamic commun ity colleges and districts in the world comprise the Leagues board of directors (L eague, para. 3) The League has more than 700 member institutions from 10 different countries and has pa rtnerships with more than 100 corporations. The list of the League s Board of Directors was posted at: http://www.league.org/league /about/board_of_directors.htm Some institutions were

PAGE 63

53 organized with a chancellor as CEO with mu ltiple presidents; other institutions had a president as the CEO. By requesting the pe rsonal participation from each CEO plus his primary administrator responsible for work-re lated education, the to tal possible sample size consisted of 40 possible pa rticipants of which 20 community college leaders agreed to participate in the study. The participants were recruited by a personal communica tion signed by the president and a president emeritus, respectively, of one of the colleges which was represented as one of the members of the Leagues Board of Directors. The personal communication explained the Delphi techniqu e and specifically requested a commitment from all the CEOs to personally particip ate in the study along with the primary administrators who were res ponsible for work-related educat ion at their colleges. The topic of study and the time involved to partic ipate were communicated to the invitees. All CEOs received a postcard to return confir ming their interest to participate in this research study plus requesti ng their electronic mail addres ses for communication. The postcard also provided: space to confirm cont act information (name, position, electronic mail address) for all the administrators who would participate in th e research study; the choice of completing the surveys using th e online web format; and the choice of receiving an electroni c copy of the finished research study. The researcher obtained informed consent by way of a follow-up letter once interest was confirmed by the return postcard. The informed consent document was mailed to all voluntary participants. All pa rticipants were informed that they would not have to answer any question they did not wish to answ er. After the primary investigator received a signed copy of the informed consent by ma il, the first-round survey was announced via

PAGE 64

54 electronic mail with the surveys corres ponding web link. The participants were informed that only the researcher would have access to the data and the self-identifiers were to be removed during final analysis. The participants were also advised that their identities would be kept confidential to th e extent provided by law and their identity would not be revealed in the final dissertati on. All participants were advised that there were no anticipated risks, compensation, or othe r direct benefits as a participant in this interview. The participants were advised that they were free to withdraw their consent to participate and could disconti nue their participation at a ny time without consequence. The gender proportion was 65 percent male and 35 percent female. Nine or 45 percent were CEOs, and the other 11 or 55 percent we re primary administrators responsible for work-related education at their colleges. Th ree of the participants were between 46 and 50 years old. Nine were between 51 and 55 years old. Five were between 56 and 60 years old. Two were between 61 and 65 year s old, and one was 66 years old or older. Tasks and Materials The Delphi technique was the method c hosen by which the panel of experts provided levels of agreement, opinions, and be liefs regarding a specific set of statements in three successive rounds of surveys. The mo st successful studies are the result of three rounds of data collection as three rounds pr oved sufficient to attain stability in the responses; further rounds tended to show very little change and excessive repetition was unacceptable to participants (Linstone & Turoff, 1975, p. 229). The researcher compiled the data, tested for validity, and reported back to the panel between each round. The panel was asked to respond to the statements in each round, as well as provide optional, qualitative comments. The data were then collected and analyzed with additions, deletions, and refinements, as deemed appr opriate from the responses and quantitative

PAGE 65

55 analyses. The survey content was the task because all participan ts received the same instructions and experienced the same activiti es. The content and task were therefore constant for all participants (Tuckman, 1999). The Delphi technique followed the proce dures of using self-r eported data. The procedure used for establishing the Delphi technique was to ju ry the proposed instrument. The instrument was juried by a community coll ege vice president with responsibilities for work-related education, a psychometric analyst, and two institutiona l researchers. The instrument was juried on the basis of their reading comprehension of the statements, ease of completion, length of time required to complete the survey, and the manageability of the web-based technology. The sample group was offered a choice of a mailed hardcopy or an electronic copy of the surveys. All the part icipants chose to respond by acce ssing an electronic copy of the survey, and they were provided links via electronic mail to the web-based electronic surveys. No one chose to participate using a hardcopy survey. The sample group responded to the initial survey, Round One, and the survey data were compiled with the item frequencies/perc entages redistributed to the sample group. All responses to the open-ended questions in Round One were compiled and provided anonymously in an attached file. New and re vised statements were developed from the responses and added to Round Two. In the second survey, Round Two, with frequencies posted for the items, participants were asked to consider the fre quencies and to rate each of the Likert scale survey items in an effort to seek convergence of agreement on the items. In addition, participants were advised to provide their underlying reason s for any statement(s) with

PAGE 66

56 which they may have taken exception w ith the converging group view. Open-ended blocks were provided after each set of stat ements, which pertained to each specific principle and component for participants, to give their reasons for any exceptions. The data were assimilated a second time, making note of changes that occurred since Round One. The item data were co mpiled with the item frequencies/percentages redistributed to the sample group. As in Round One, the responses to the open-ended questions in Round Two were compiled and pr ovided anonymously in an attached file and once again distributed to the particip ants as a final surv ey, Round Three, via electronic means. Participants were given a final opportunity to validate or revise the data previously submitted. Participants were asked to justify their responses, particularly in cases where their responses differed from the majority of the participants. At this point, the data were then orga nized according to the statistic al assumptions. This last group of data were compiled and analyze d. The data between Round One and Round Two were analyzed, as were the data between Round Two and Round Three, to determine the effects of the responses on the statements in the subsequent surveys. The iterative process of the Delphi tec hnique allowed for data collection to be implemented in a non-threatening manner. The group was afforded opportunities to change its ratings in subsequent rounds. The group could reach further levels of agreement on specific principles and component s, which could be modeled to derive a common definition for work-related education. General Operational Design The study employed Internet-based survey research to examine if and how community college leaders could reach agreemen t using a series of surveys to aggregate their knowledge, judgments, and opinions as a pane l of experts. The panel of experts was

PAGE 67

57 comprised of community college CEOs and th eir respective administrators responsible for work-related education in order to addr ess the complex question of how to define work-related education. The panel of experts communicated their knowledge and experience through a three-round iterative proce ss. They used the Delphi technique to reach agreement on which principles, component s, and other aspects could be identified, prioritized, and applied to a common te rminology and definition for work-related education. Differential outcomes were considered based on the participants position/capacity at their colleges and characteristics of their colleges. The operational design represented multi-fact or repeated measures and longitudinal research with three levels labeled as Round One, Round Two, and Round Three (e.g., repeated measures: time-1, time-2, time-3). The design included the total score, the principles score, and the components score factors whereby the pa rticipants (CEOs and administrators) responded to the research st atements. They then reached levels of agreement through the iterative proc ess of the Delphi technique. Data Collection Round One Once interest was confirmed by the voluntar y participants, the informed consent document was mailed to all participants a dvising them that the web link to Round One would be electronically mailed upon receipt of the signed informed consent letter. The web link to Round One was electronically ma iled to all participants. The introductory electronic mail to the first round survey welcomed the pa rticipants to the study and explained the general procedures to follow. All participants were asked to read the rationale, directions, and instru ctions before attempting to complete their surveys. Upon

PAGE 68

58 opening the web link, each participant was asked to self-identify with the first five letters of the last name for the researcher to verify receipt of survey results. The Round One survey consisted of thr ee sections. The first two sections contained 46 statements, which served as a ta sk to be completed by each participant. The third section contained demographic survey it ems. The first section of 21 statements pertained to work-related edu cation and OBanions (1997) six principles of the learning college. The second section of 25 statements contained other key issues, areas of focus, and components of work-related education, as were initially identifie d in the literature review. A Likert scale was placed immediat ely below each statement. The Likert scale was self-explanatory. Participants were asked to rate each statement by checking the perceived agreement with each statement. The rating scale had as the lower anchor, Strongly Disagree, and was assigned a negati ve three (-3) value, and the higher anchor, Strongly Agree, was assigned a positive three (+ 3). The options were coded as: Strongly Disagree = -3; Disagree = -2; S lightly Disagree = -1 ; Slightly Agree = +1; Agree = +2; and Strongly Agree = +3. A neutral option or response choice of "undecided," "no opinion," "uncer tain," or "don't know" was inte ntionally left out. It was feasible to use a response s cale with an even number of re sponses and no middle, neutral, or undecided choice because most particip ants, as experts, had an opinion and corresponding level of agreement for the items. Otherwise, as per the informed consent, it was clear to the participants that they did not have to respond to every statement. But they could take a no judgment view, whic h is a practice commonl y applied to Delphi studies (Linstone & Turoff, 1975, p. 90; Adler & Sainsbury, 1996, p. 188).

PAGE 69

59 After each set of statements pertaining to a specific principle or component subsection, participants had an opportunity to write optiona l statements of opinion or comment, if they so desired. The second section was followed by a final, open-ended question as an opportunity to li st any other items that the participants believed would contribute to a common definiti on for work-related education. The intent of this final question was to encourage reflection beyond the initial survey with an additional purpose to provide points of reference. These point s of reference would assist with the data analysis where lack of agreement or consen sus existed. The third section of Round One consisted of three demographic questions pe rtaining to current work capacity, gender, and age. Six days after the initial notification was sent via electronic mail, self-identified survey results were reconciled to the list of confirmed participants. A generic reminder request was sent via blind copy electronic mail to those who had not yet completed Round One. Verification was made on an ev ery-other-day basis until all expected responses had been received up to nine days from the original notif ication. On the ninth day, individual personalized electronic mails we re sent to the remaining six participants who had yet to complete Round One. Six of the seven participants responded by the 13th day from initial notification. The 20th participant made contact and advised technical difficulty in accessing the web survey. That participant was advised and subsequently responded to Round One. Survey results for the 20 participants we re downloaded into spreadsheet and text files. Likert scale items and demographic in formation were analyzed statistically using Excel, SPSS, and SAS programs. Open-ended comments and opinions were interpreted

PAGE 70

60 as subjective information, which had characteri stics relevant to th e research questions. This subjective information was developed and aggregated as revised with additional survey items for new statements in Round Two. Six statements were revised and replaced, 16 new statements added, and in to tal the number of statements increased from 46 to 62. Round Two The Round Two survey was electronically mailed following the analysis of Round One, posting of frequencies, and aggrega tion of revised and new statements. The electronic mail for the second r ound survey thanked the particip ants for their support of the study and explained the general pro cedures to follow for Round Two. All participants were asked to read the rati onale, directions, and instructions before attempting to complete their surveys. Upon opening the web link, all participants were asked to self-identify with the fi rst five letters of the last name for the researcher to verify receipt of survey results. Round Two consis ted primarily of 40 out of 46 of the same items in Round One plus the open-ended comm ents and opinions, which were developed and aggregated as 22 revised and additional survey items. A Likert scale was placed immediately below each statement. All the anonymous open-ended responses and comments were contained in a Microsoft Word document file, which was also provided via an attachment to the elec tronic mailing. For those statements which did not change, the frequencies/percentages were placed in fr ont of the respective Li kert scale options. The frequencies identified the relative posit ion of the group consensus in relation to the Likert scale options for ease of understa nding and reevaluation for convergence of agreement.

PAGE 71

61 During Round Two, each participant was given an opportunity to re-rate each of the 40 original statements with knowledge of the group's convergence of agreement. It was explained that although consensus was desi rable, they should not have felt compelled to re-rate according to the gr oups ratings. However, particip ants were advised that if they differed markedly to the groups ratings, they should have given careful reappraisal to those particular statements. As in Round One, the Likert scale was self-explanatory. Participants were asked to rate each stat ement by annotating their perceived agreement with each statement. The rating scale had as the lower anchor, Strongly Disagree, and was assigned a negative three (-3) value, and the higher anchor, Strongly Agree, was assigned a positive three (+ 3). The options were coded as: Strongly Disagree = -3; Disagree = -2; Slightly Disagree = -1 ; Slightly Agree = +1; Agree = +2; and Strongly Agree = +3. A neutral opti on or response choice of "undecided," "no opinion," "uncertain," or "don't know" was inte ntionally left out and deemed reasonable as virtually all participants had an opini on and corresponding level of agreement for the items. Otherwise, as per the informed consent, it was clear to the participants that they did not have to respond to every question. But the participants could take a no judgment view, which is a practice commonl y applied to Delphi studies (Linstone & Turoff, 1975, p. 90; Adler & Sainsbury, 1996, p. 188). Again, participants were advised to provide their underlying reasons for any stat ement(s) with which they may have taken exception with the converging group view. An open-ended block was provided after the two individual principle and component subs ections for participants to provide their reasons for any exceptions. The third s ection of Round Two consisted of: three

PAGE 72

62 classification questions pertaini ng to the organization; size; location of the participants college or district; and one demographic ques tion pertaining to curr ent work capacity. One week after the initial no tification was sent via electronic mail, self-identified survey results were reconciled to the list of confirmed participants and a generic reminder request was sent via blind copy electronic mail to those who had not yet completed Round Two. Verification was made on an every-other-day basi s until all expected responses had been received up to 11 days from the original notification. On the 11th day, individual personalized electronic mails we re sent to the remaining six participants who had yet to complete Round Two. Four of the six participants responded by the 22nd day, and one participant responded by the 29th day from the initial notification of Round Two. Survey results for the 19 participants we re downloaded into spreadsheet and text files. Likert scale items and demographic in formation were analyzed statistically using Excel, SPSS, and SAS programs. Open-ended comments and opinions were interpreted as subjective information, which had characteri stics relevant to th e research questions. This subjective information was developed and aggregated as revised and additional survey items for new statements in Round Th ree. The data obtained from Round Two were analyzed using descriptive statistics. A criterion was set so th at any statement not scoring an overall positive mean was exclude d from Round Three. Six statements not achieving the criterion were eliminated. In addition, one statement was revised into two statements, and these two new statements were added. In total, the number of statements decreased from 62 to 59.

PAGE 73

63 Round Three The Round Three survey was electronically mailed following the analysis of Round Two. The electronic mail for the Round Three survey thanked the participants for their support of the study and explained the genera l procedures to follow for Round Three. Each participant was asked to read the rati onale, directions, and instructions before attempting to complete their survey. Upon opening the web link, each participant was asked to self-identify with the fi rst five letters of the last name for the researcher to verify receipt of survey results. Round Three cont ained the statements from Round Two--less the six statements not scoring an overall positive mean--plus the new and revised statements. A Likert scale was placed imme diately below each statement. Frequencies for all statements in the form of percentages or levels of agreement were placed in front of the respective Likert scale options. The fr equencies highlighted the relative position of the group consensus in relation to the Likert scale options for ease of understanding and reevaluation for convergence of agreement. A Microsoft Word document file containing all the anonymous open-ended responses a nd comments was also provided via an attachment to the electronic mailing. With Round Three, each participant was gi ven a final opportunity to re-rate each statement with knowledge of the groups decision. It was explained that although consensus was desirable, the pa rticipants should not have felt compelled to rate according to the groups rating. However, participants we re advised that if they differed markedly to the mean rating, they should have given ca reful reappraisal to that statement. In addition, participants were give n the opportunity to explain th eir reasons for their ratings on Round Three, but were not compelled to do this.

PAGE 74

64 As in Round One and Round Two, the Likert scale was self-explanatory. Participants were asked to rate each stat ement by annotating their perceived agreement with each statement. The rating scale had as the lower anchor, Strongly Disagree, and was assigned a negative three (-3) value, and the higher anchor, Strongly Agree, was assigned a positive three (+ 3). The options were coded as: Strongly Disagree = -3; Disagree = -2; Slightly Disagree = -1 ; Slightly Agree = +1; Agree = +2; and Strongly Agree = +3. A neutral option or response choice of "undecided," "no opinion," "uncertain," or "don't know" was inte ntionally left out and deemed reasonable because virtually all particip ants had an opinion and corresp onding level of agreement for the items. Otherwise, as per the informed c onsent, it was clear to the participants that they did not have to respond to every questi on, but they could take a no judgment view, which is a practice commonly applied to Del phi studies (Linstone & Turoff, 1975, p. 90; Adler & Sainsbury, 1996, p. 188). Again, partic ipants were advised to provide their underlying reasons for any statement(s) with which they may have taken exception with the converging group view. An open-ended bl ock was provided after each of the two sections for participants to provide their reasons for any ex ceptions. The third section of Round Three consisted of one demographic question pertaining to the current work capacity of the participant. One week after the initial notification was sent via electronic mail, the selfidentified survey results were reconciled to the list of confirme d participants, and a generic reminder request was sent via blind c opy electronic mail to those who had not yet completed Round Three. Verification was ma de on a daily basis until all expected responses had been received up to two weeks from the original notif ication. Individual,

PAGE 75

65 personalized electronic mails were sent to the remaining five participants who had yet to complete Round Two. Verification was ma de on a daily basis until all expected responses had been received. Two of the remaining five participants responded by the 17th day from initial notifi cation of Round Three. Communication Process During the course of the study, telephone contacts we re initiated, and numerous mailings and electronic mail messages were received and sent. Only work phone numbers, work addresses, and electronic mail accounts were accessed. The researcher offered both work and mobile phone numbers along with an electronic mail address to facilitate frequent and cord ial contact with the participants to achieve the maximum response rates for the three De lphi rounds. Personal commun ications in the form of thank-you cards were sent to every participan t after the closure of Round Three. Those participants who requested an electronic copy of the final study were advised that copies would be forthcoming after the successful defe nse of the dissertation and its subsequent publication. Data Management and St atistical Procedures The data from the surveys were analyzed after each round was downloaded. Data were managed and entered into a spreadsh eet denoting individual responses to each statement and open-ended questions. Speci fic comments and responses to the openended questions, which were downloaded in to a spreadsheet, were subsequently transferred to a Word document for qualitativ e analysis. The use of qualitative data in educational research is recognized as important to the study and understanding of educational phenomena, testing hypothes es, and providing a natural basis for interpretation with explanations emerging from intensive examination of the data

PAGE 76

66 (Tuckman, 1999). Sample characteristics incl uded: the number of participants (n) and demographics, which included each participan ts position at the coll ege; his gender; his age; and classification data on his respect ive college, which included the organization (number of campuses), size (number of student s), and location of the college/district (urban, suburban, or rural). The data analysis and statistical procedur es that were employed by the researcher focused on the primary purpose of this study: to test if work-relate d education conformed to the six principles of the learning college (OBanion, 1997). The other purposes of this were to see if other complementary compone nts of work-related e ducation--in total, principles and components--could be identi fied, categorized, and ranked to derive a common terminology and definition for work-rela ted education at community colleges. This study tested the applicati on of the Delphi technique survey method to determine if the Delphi technique could be effectively applied to an educ ational forum for leaders of community colleges. The purpose would be to achieve consensus and support rationale for establishing a consolidated position on and a holistic approach to what constitutes work-related education. As such, by focu sing on the purpose of reaching consensus, levels of agreement, and further identifying issues pertaining to work-related education, this study did not address whether or not the extreme answer was th e correct answer. Based on the research questions guiding the study, the right answer would instead be determined by whether or not the experts r eached consensus and levels of agreement on the various constructs. Reliability of the Instrument Reliability refers to the consistency of such measurements when the testing procedure is repeated on a population of individuals or gr oups (American Psychological

PAGE 77

67 Association, American Educational Re search Association, & National Council on Measurement in Education [APA, AERA, NCME ], 1999). Reliability also refers to the extent to which the responses are free of measurement error. As such, the responses should be the same every time the measurement is repeated on the same group, sample, or population. To achieve reliable results, th e scale and instrument were constructed so as to minimize random error in responses. The study focused on the proportion of the experts who responded to item stems (stateme nts) according to the scale scores. That Rounds Two and Three of the Delphi afforded the experts an opportuni ty to change their initial ratings in light of the new informati on further ensured that the results could be used for well-founded conclusions. Validity of the Instrument Validity refers to the appropriateness of use and the proposed interpretation of the scores for a given purpose under a prescrib ed set of conditions. Validity is the most fundamental consideration in developing and evaluating the extent to which an instrument is doing what it is supposed to do. Crocker and Algina (1986) refer to Cronbachs description of valid ation as the process by which a test developer or test user collects evidence to support the types of inferenc es that are to be dr awn from test scores (p. 217). Validation begins w ith an explicit statement about the proposed interpretation of the scores. There is no single all-inclus ive form of validity. Validity is instead a matter of degree with types of evidence adding weight to validity, described as content, criterion-related, or constr uct validity. These three types of evidence are only conceptually independent, and rarely is just one of them important in a particular situation.

PAGE 78

68 The types of evidence describe the exte nt to which the data obtained are systematically representative of the true st ate of affairs, and they describe if the assessment items give information about wh at the items were intended to provide (Penfield, 2003). Content validity describe s how well the content of the scale matches the content domain intended to be measured by the scale. In other words, it makes human judgments about whether or not the content of the items covers the major facets related to the knowledge areas. Content validity addresses f eatures of the test, not the scores. In fact, content validation often occurs before scores are even obtained. Crocker and Algina (1986, p. 218) outlined the fo llowing steps for content validation: 1. Defining the performance domain of interest; 2. Selecting a panel of qualified e xperts in the content domain; 3. Providing a structured framework for th e process of matching items to the performance domain; and 4. Collecting and summarizing the data from the matching process. Content validity was essentially built-in with the juried expert review survey and also with each round of the Delphi techni que. This was by virtue of the development of the content of the scale matching the c ontent domain, as conveyed by the experts responses and what they considered to be th e constructs of interest. Other types of evidence related to content validity existed, su ch as face validity, which is really not as antidotal as perceived by some (that is, doe s it look professional, serious, worth taking, and so forth). Face validity was appropriate in this case as a type of evidence in which items appeared to measure a construct that was meaningful to laypersons and may have served to motivate participants to perform their best since the instrument appears to measure a meaningful constr uct (Crocker & Algina, 1986, p. 223). Criterion-related

PAGE 79

69 validity pertains to the accuracy of decisions linked to the va lidity of the scores. For the purposes of this study, these two secondary research questions were asked: 1. Could a selected group of community colle ge leaders reach consensus, using a Delphi technique, on what principles and co mponents could be identified to derive a common definition for work-related education? 2. Could meaningful relationships be confir med between the six principles and the identified components to derive a comm on definition of work-related education? Construct validity was used to determine whether or not the items of the scale measure the constructs they ar e supposed to measure. Cons truct validity addresses the degree to which scores represent the unobserva ble trait operationalized through the items. Internal validity claims were met by follo wing established procedure for the Delphi technique to answer in ferential questions about the scores and further define and distill the data to well-founded conclusions. It would have been most difficult, if not impossible, to incorporate a comparison group into the Delphi research design to establish certainty of the instrument. Extern al validity was depende nt on the selection of the experts as a representativ e body, whose scores may or may not be generalized to all community colleges in a particular sample, group, or the population. Analytical Procedures Descriptive statistics, t-tests, the Duncan s multiple-range test, and the parametric correlation were all statistical pr ocedures and tests used to exam ine the data in this study. The data in each phase were analyzed in terms of the statemen ts means, standard deviations, and confidence intervals, in addi tion to the participants responses to openended questions for each item. The iterative st ructure of the Delphi technique developed what the panel of experts identified as the co ntent domain and what they considered to be the constructs of interest. Again, the part icipants responses to open-ended questions

PAGE 80

70 were given considerable weight for revi sing, deleting, and adding statements in the subsequent rounds. Negative means gave an in dication of how individual statements did not support agreement of specific principles or components and their lack of association with work-related education. After Round Two, those statem ents, which did not meet the criterion of an overall positive mean, were ex cluded from Round Three. This criterion acknowledged that participants did not endorse specific statements as descriptive of the research questions and c onstructs of interest. Descriptive Statistics Descriptive statistics, which are those statistics used to best answer questions about or describe the parameter of interest, were analyzed for every item over the participants, which included the measures of central te ndency-mean, median, mode--plus confidence intervals, standard deviation, frequencie s, including cumulative frequencies. Penfield (2003) defined the mean as a common method of obtaining a representative value for th e average of a group of scor es (p. 72). Tuckman (1999) defined the median as the score in the middl e of a distribution: 50 percent of the scores fall above it, and 50 percent fall below it (p. 288). Furthermore, The median defines the middle of the distribution and is not as se nsitive to extreme scores as is the mean (Tuckman, 1999, p. 289). The mode is a meas ure of central tendency, which describes the most frequently occurring observation. No calculation is involved to identify the mode. It may not exist or there could be more than one modal value, which suggests a bi-modal observation, that is, the indicati on of two separate distributions. Penfield (2003) described a confidence inte rval as an interval that has a certain level of confidence of contai ning the value of interest (p. 110). The lower and upper bounds of the confidence interv al are useful in specifyi ng with a certain degree of

PAGE 81

71 confidence, such as 95 percent confident, th at the interval around the sample mean is expected to contain the population mean (C rocker & Algina, 1986, p. 433). The measure of dispersion of the participan ts responses used in this study was the standard deviation. Penfield (2003) defined the sta ndard deviation as a measure of the amount of spread in a group of scores, which equals the typical dist ance that each score in the group lies from the group mean (p. 72). Frequencies and cu mulative frequencies were used to verify and analyze the Likert scale da ta. The frequencies for the statements in the form of percentages were provided to the particip ants for comparability and convergence of agreement between Round One and Round Two and between Round Two and Round Three. Parametric Statistical Tests A large amount of the quantitative portion was descriptive due to the nature of the Delphi technique and driven by the small number of participants (n). An independent ttest was used as an investigation of any di fferences between the two means for CEOs and other administrators. This was to ascertai n the probability that any difference between them reflected a real difference between the gr oups of participants rather than a chance variation in the data (Tuckman, 1999, p. 300). The t-test was an effective tool for predicting any statistical differe nces between the means for the scores (total score and the individual scores for the prin ciples and the components) as the dependent variables from the independent variables (CEOs and administra tors). An analysis of variance could not be used for factor analysis since the particip ant sample size (n) in the study was too small to permit a meaningful factor analysis. Three primary questions were posed: 1. Did the average total score depend upon the job capacity of participant (CEOs versus other administrators)?

PAGE 82

72 2. Did the average score for the principles of the learning college subsection depend upon the job capacity of participant (C EOs versus other administrators)? 3. Did the average score for the other id entified components subsection depend upon the job capacity of participant (C EOs versus other administrators)? As a follow-up to the independent t-tests, a Duncan's test (Duncan's multiple range test) was employed as a type of multiple comparison test used to make pair-wise comparisons of means that are not signifi cantly different among themselves. The Duncan's test provided output that was esse ntially a "picture" of which pairs of means were significantly different as a post hoc test. The Duncans test was employed to determine if statistical diffe rences existed among the average total scores with any statistical significance between Round On e and Round Two, between Round Two and Round Three, and between Round One and Round Three. Correlation coefficients were used to confirm statistical significance for th e principles of the learning college and the components identified, as pertaining to work -related education. A correlation matrix was constructed in which the correlation between ev ery pair of variables was computed. Then the variables were organized into a matrix to facilitate inspection and comparison of each for significance. Summary The aggregate data from all three ro unds were collected, compiled, and the participants comments were noted after each round. The aggregate data and comments were shared with the participants for re view and reflection when they received each subsequent round. Participants had the opportu nity to change their minds at any time during Round Two or Round Three to allow time to reflect on an issue. After the final data were compiled, the quantit ative analysis of data was completed using a statistical program (SAS). The qualitative analysis took into consideration the compilation of

PAGE 83

73 comments made throughout the collection pr ocess. The Delphi technique was recognized as an appropriate study design a nd assessment to make important decisions about educational policy (Clayton, 1997, p. 373). Internal validity was a function of developing an appropriate survey instrument and administering th e surveys over a threeround iterative process, while compiling and redistributing the aggregate data to the participants after each round. External vali dity was a function of determining if the results obtained answered the research ques tions and further the process of deriving a common definition for work-related education. External validity was also viewed in terms of whether or not a common definiti on would extend beyond the sample and apply to a larger sample of th e population. When designing a research project, the two principle types of validity, inte rnal and external, must be ba lanced to obtain conclusive results (internally), but they must still repres ent a more global reality (externally) for the results to be generalized to other groups, a larger samp le, or the population (Tuckman, 1999, p.10). Detailed reporting of the data analysis is contained in Chapter 4.

PAGE 84

74 CHAPTER 4 PRESENTATION AND ANALYSIS OF DATA Introduction The primary purpose of this study was to te st if work-related education conformed to the six principles of the learni ng college theory (OBanion, 1997, pp. 47-61). Furthermore, this study would supplement OBanions learning college principles by examining additional complementary compone nts of work-related education. The purpose would be to determine if in tota l the principles and components could be identified, categorized, and ranked to de rive a common definition for work-related education at community colleges. OBanion recognized that there are certainly other principles that must be cons idered. . Content, funding, and governance are examples of key issues that must be addressed and for wh ich principles must be designed (p. 61). Included in the study were the learning colle ge principles, components identified in the existing literature, and other related component s, which were developed by participants through the iterative process of the Delphi technique. This st udy tested the a pplication of the Delphi technique to determine if it coul d be effectively applie d to an educational forum for leaders of community colleges to ach ieve consensus and levels of agreement. This in turn could support a rationale fo r establishing a consolidated position on and a holistic approach to what constitutes work -related education. Such a consolidated position could facilitate clarity and consistency in policymaking at the federal, state, and local levels. By participati ng in this study, the community college leaders served as a

PAGE 85

75 panel of experts assisting in the research to derive a common definition for work-related education at community colleges. Specifically, this study sought answers to primary and secondary research questions, which were developed to guid e this study and execute the methods: Primary Questions 1. Which, if not all, of OBanions six princi ples of the learni ng college could be associated with work-related education? 2. What other components could be identifie d for the work-related education function at community colleges? 3. What were the most strongly advocated principles and components supporting work-related education? Secondary Questions Additionally, secondary research questions were identified that could be answered as a result of this study. These questions were answered through a compilation of the answers to the primary research questions. 1. Could a selected group of community colle ge leaders reach consensus, using a Delphi technique, on what principles and co mponents could be identified to derive a common definition for work-related education? 2. Could meaningful relationships be confir med between the six principles and the identified components to derive a comm on definition of work-related education? Results of the Delphi Technique This primarily qualitative, mixed-methods study was designed to be experimental. The Delphi technique was used to survey the perceptions of community college CEOs and administrators responsible for work-relate d education. They served as a panel of experts to confirm levels of agreement on t hose principles and components from which a common definition for work-related education co uld be derived. The initial survey was juried by a community college vice presiden t with responsibilit ies for work-related

PAGE 86

76 education, a psychometric analys t, and two instituti onal researchers. Revisions to the format and content were made based on th e recommendations of the jury. One-hundred percent of the participants chose to do the survey electronically, and they were provided the web-based electronic surveys. No one c hose to participate using hardcopy surveys. The participants results from the three-round Delphi techniqu e were collected through an online service based on an open-source projec t, which originated at Virginia Tech (http://www.opensource.isc.vt.edu/products/surve y/). The online service was titled: Survey A web-based survey tool, and was available via the Co llege of Education website at the University of Florida. When each round was completed, the researcher downloaded the survey responses data into a pop-up window and saved the raw data as a text file. The survey data in the text f ile were exported to Mi crosoft Excel via the import external data fu nction for the initial analysis and data coding. Quantitative statistical analysis tools were applied to each statement in each round, including the measures of central tende ncy--mean, median, mode--plus confidence intervals, standard deviation, frequencies, including cumulative frequencies. Goldstein (1975) identified the means, standard deviations, percentage distributions as appropriate descriptive statistics used in Delphi research st udies and these statistics for each statement in each round were included in this report (Linstone & Turoff, 1975, p. 222). Each statement was analyzed using Ex cel and SAS programs to provide empirical data to alleviate concerns rega rding the use of the Delphi te chnique. Each statement also addressed concerns about inconclusive ev idence, which was subsequently supported by statistical data analysis.

PAGE 87

77 The qualitative responses were collected through the use of open-ended comment boxes on the web-based survey tool. The quali tative responses data were simultaneously downloaded into a pop-up window along with th e survey raw data and saved as a text file. The qualitative data in the text file were exported from Micros oft Excel to Microsoft Word for aggregation, analysis, and presenta tion in the Appendix. Additionally in Round One, one participant electronically mailed e xpanded reflections rega rding the principles, components, and perceptions. These comment s were included in the qualitative data response results in the Appendix. Selection and Confirmation of Participants The participants in this study were al l current employees of public community colleges. Twenty community college CEOs and 20 of their administrators, respectively, for a total of 40 community co llege leaders, were invited to participate in the study. These community college leaders were define d as the experts on work-related education within a population of public community colleges. The sample included the colleges chief executive officers (CEOs), academic affa irs officers, business/industry liaison officers, continuing education officers, or occupational ed ucation officers (American Association of Community Co lleges [AACC], 2004, p. 3) at the 20 colleges listed in Table 1-2 whose CEOS comprised the League for Innovation in the Community College Board of Directors. Out of the 40 potential participants, 20 agreed to participate. According to Clayton (1997), the group si ze may vary with -30 people for a homogeneous . and 5-10 people for a heterogeneous population, which is an acceptable number of participants for using a Delphi technique in educational studies (p. 8). Ziglio (1996) noted that good results ca n be obtained even with small panels of 1015 individuals (p. 14). The final group of 20 participants pr ovided a stratified sample of

PAGE 88

78 CEOs and administrators from varied commun ity college classifica tions of organization, size, and location (Community College Survey of Student Engageme nt [CCSSE], 2005). Eighteen participants were from the United States and two were from Canada. The American participants colleges represen ted a geographically diverse sample. The geographic diversity was demonstrated by memb ership in five out of the six regional accrediting organizations, as recognized by the Council for Higher Education Accreditation (CHEA) and the U.S. Depa rtment of Education (USDE). The 18 participants at American colleges included: Seven from the North Central A ssociation of Colleges and Schools Five from the Southern Asso ciation of Colleges and Schools Three from the Middle States Commission on Higher Education Two from the Western Associ ation of Schools and Colleges One from the Northwest Commissi on on Colleges and Universities The participants represented colleges wh ich were 42 percent urban and 58 percent suburban; no colleges were considered rural. Eighty-nine percent of the participants were employed at colleges with extra large enrollments (15,000 or more students); only 11 percent of the participants were employed at colleges wi th a large enrollment (8,000 to 14,999 students); no colleges were considered medium or small, e.g. 4,500 to 7,999 or fewer than 4,999 students). In terms of organi zational size, 58 percen t were from a multicampus organization, 37 percent were from a single campus, and the remaining 5 percent portrayed a single member college in a mu lti-college system. The gender proportion was 65 percent male and 35 percent female. Nine or 45 percent were CEOs, and the other 11 or 55 percent were primary administrators re sponsible for work-relate d education at their colleges. Three of the participants were between 46 and 50 years old. Nine were

PAGE 89

79 between 51 and 55 years old. Five were between 56 and 60 years old. Two were between 61 and 65 years old, and one was 66 years old or older. Response Rates to Delphi Surveys The response rate for Round One was 100 pe rcent. All 20 of the community college leaders (nine CEOs and 11 administrato rs) who agreed to participate, actually completed Round One. An initial response rate of more than two-thirds was considered high for a Delphi study and showed significant in terest on the part of the panel of experts (Jillson, 1975, p. 132). Considering the intensity of the schedules of these top community college leaders, this researcher anticipated that not all participants would complete all rounds, and this was the case in this st udy. The response rate for Round Two was 85 percent. Seventeen out of the 20 who agreed to participate, (7 CEOs and 10 administrators) actually co mpleted Round Two. The response rate for Round Three was 75 percent. Fifteen out of the 20 who agr eed to participate (six CEOs and nine administrators) actually comp leted Round Three. In all th ree rounds, all or 100 percent were able to respond through th e web-based survey format. The aggregate response rate for the three rounds was acceptable consideri ng that Round Threes sample size was 15 with -30 people for a homogeneous populatio n as an acceptable sample size for using a Delphi technique in educational studies (Clayton, 1997, p. 8). Ziglio (1996) noted that good results can be obtained even with small panels of 10-15 individuals (p. 14). A review of other Del phi research (Linstone & Turo ff, 1975; Jillson, 1975) and Delphi-based dissertations (Smith, 1975; Ne mr,1977; Lewis, 1984) revealed a reduction in response rates from the first to the fina l round, particularly t hose involving voluntary participation (Jillson, p. 132).

PAGE 90

80 Verification of the Accuracy of the Delphi Technique The Delphi technique was used as a co mmunications structure aimed at producing a detailed critical examination and discussion with certain quantification of the participants viewpoints (Turoff & Hiltz, 1996, p. 56-57). This communication process required the participants, serving as a panel of experts, to rate statem ents in a series of iterative surveys to quickly identify levels of agreement and disagreement. The groups levels of agreement for each round were shar ed in subsequent surveys (Round Two and Round Three). The response rating frequencies or percentages for each statement, which were integrated in the subsequent surveys, focused on which principles and components could be identified, categorized, and ranked to derive a common definition for workrelated education. To verify th e accuracy in using the Delphi technique for this research study and to present the result s of the three-round process in a comparative depiction, response ratings from all three iterative rounds were presented in a round-by-round evaluation of the data with the appropriate de scriptive statistics. These response ratings frequencies, means, and standa rd deviations by survey statem ent were reported for each of the six principles of the learning co llege and for each of the seven identified components of work-related education, as su mmarized and presented in Table 4-5. The researcher shared the groups viewpoints after each round with the participants for self-comparison and as a point of refere nce for considering the other participants views. The frequencies reported were base d on the aggregate of the total number of responses and their corresponding ratings in each particular round. In addition to providing the frequencies, the qualitative responses were co llected through use of openended comment boxes on the web-based survey tool. These qualitative responses were also shared in the subsequent rounds (R ound Two and Round Three), as summarized and

PAGE 91

81 presented in the Appendix. The Delphi t echnique and these procedures offered each participant an opportunity to reconsider hi s position in light of the groups views in addition to considering new items that were in troduced. That the second and third rounds of the Delphi survey technique afforded the panel of experts these opportunities to change their ratings in light of new informa tion further ensured that the results could be used for well-founded conclusions. Content va lidity was essentially built-in with each round of the Delphi technique. Content va lidity was verified by virtue of the development of the content of the scale matc hing the content domain, as conveyed by the panels expert responses and what the particip ants considered to be the constructs of interest. The merger of a De lphi technique, the research procedures, and the web-based process verified and promoted reaching a supe rior group view of the task at hand through the phenomenon collective intelligence (Turoff & Hiltz, 1996, p. 80). Round One Results Twenty participants rated each of 46 statem ents according to the following scale: Strongly Disagree, Disagree, Slightly Disagree, Slightly Agree, Agree, or Strongly Agree. Any absence of a rating for a statement was classified as no judgment. As in each round of the Delphi, participants were asked to complete three sections. Instructions and statements were provided for each section as a task to be completed. After each subsection for a princi ple or component, the participants were asked to list their reactions, initial thoughts, comments, and/or recommended changes to any statements pertaining to the specific pr inciple or component. The participants who rated the eighth statement either Agree or Strongly Agree for work-related education should offer a full array of options to accommodate individual differences in learning styles, rates, aptit udes, and prior knowledge were asked to respond to an open-

PAGE 92

82 ended question, what options should be offe red. At the end of surveys first and second sections (before demographics data sec tion), the participants were asked to list any other principles, components, changes in service delivery, and innovative ways of thinking which they believed would contribu te to a common definition for work-related education. Finally, the participants were asked to respond to a third section which requested demographic data, as detailed earlier in this chapter. The percentages, means, and standard deviations reported in Table 45 were based on an aggregate of the total number of participants who responded by r ound and the correspond ing group ratings. Not all percentages equated to 100 percent due to fractiona l rounding or in those cases where participants chose not to respond (c lassified as no judgment) to specific statements. Open-ended comments and opi nions were interpreted as subjective information, which had characteristics relevant to the research questions. This subjective information was developed and aggregated as revised with additional survey items for new statements in Round Two. Six statem ents were revised and replaced, 16 new statements added, and in total the number of statements increased from 46 to 62. At the conclusion of Round One, the participants we re electronically maile d the web link to the follow-up Round Two survey. The follow-up su rvey contained revisions based on the panel of experts recommendations along w ith the qualitative data and comments compilation from Round One. The informa tion in the Appendix reflects the raw qualitative responses and comments from Round One which were analyzed to refine the survey instrument for Round Three. The inform ation in Table 4-5 reflects the levels of agreement for each statement in each round, as rated by the panel of experts.

PAGE 93

83 Round Two Results Seventeen participants rated each of th e 62 statements in Round Two according to the following scale: Strongly Disagree, D isagree, Slightly Disagree, Slightly Agree, Agree, or Strongly Agree. Round Two consisted primarily of 40 out of 46 of the same items in Round One, which were presented with the participants aggregate frequencies or percentages fr om Round One for the Likert scale responses. In addition, the open-ended comments and opinions were analyzed from which 22 revised and additional survey items were developed and aggregated into Round Two. Additionally, an absence of any rating for a statement was classified as no judgment. As in each round of the Delphi, participants were asked to complete three secti ons. Instructions and statements were provided for each section as a task to be completed. After each section for the principles and the components, the participants were as ked to provide their underlying reasons for any statements with wh ich they may have taken exception with the converging group view. These statements ma y have pertained to a specific principle or component, as reported in the Appendix. Finally, the participants responded to a third section, which requested demographic and colleg e classification data, as detailed earlier in this chapter. The percentages reported in Table 4-5 were based on an aggregate of the total number of participants who re sponded by round and th e corresponding group ratings. Not all percentages equated to 100 pe rcent due to fractional rounding or in those cases where participants chose not to respond (classified as no j udgment) to specific statements. Open-ended comments and opi nions were interpreted as subjective information, which had characteristics relevant to the research questions. This subjective information formed the basis for revised and additional survey items which were aggregated as new statements in Round Thr ee. The data obtained from Round Two were

PAGE 94

84 analyzed using descriptive statistics. A criteri on was set so that any statement not scoring an overall positive mean was excluded from Round Three. Six statements not achieving the criterion were eliminated. In addition, one statement was revised into two statements, and these two new statements were added. In total, the number of statements decreased from 62 to 59. At the conclusion of Round Two, the participants were electronically mailed the web link to the follow-up Round Three survey. The follow-up survey contained revisions based on the panel of experts recommendations along with the qualitative data and comments compilation fr om Round Two. The information in the Appendix reflects the raw qualitative respons es and comments from Round Two, which were analyzed to refine the survey instru ment for Round Three. The information in Table 4-5 reflects the levels of agreement fo r each statement in each round, as rated by the panel of experts. Round Three Results Fifteen participants rated a possible 59 statements according to the following scale: Strongly Disagree, Disagree, Slightly Disagree, Slightly Agree, Agree, or Strongly Agree. Each round of the Delphi technique requested th e participants to complete three sections. Instru ctions and statements were provided for each section as a task to be completed. Round Three containe d the statements from Round Two less the six statements not scoring an overall positive mean. Round Three also contained revised and new statements based on qualitative inputs Thirty-five statements carried over from Rounds One and Two. As in the two prev ious rounds, open-ended comments and opinions were interpreted as s ubjective information, which had characteristics relevant to the research questions. For Round Three, a cr iterion was set so that any statement not scoring an overall positive mean in Round Two was excluded from Round Three. Six

PAGE 95

85 statements not achieving the criterion were eliminated. Again, this subjective information was developed and aggregated as revised and new statements in Round Three. From the subjective information, one statement was revised into two, and two new statements were added. In total, the num ber of statements decreased from 62 to 59. The data obtained from Round Three were an alyzed using descri ptive statistics. Additionally, an absence of a ny rating for a statement was classified as no judgment. An open-ended comment block was provided after each of the two major sections for the principles and the components. Participants were given an opportuni ty to provide their underlying reasons for any statements with which they may have taken exception with the converging group view. These statements pertained to any speci fic principles or components. The percentages reported were ba sed on an aggregate of the total number of participants who responded by round and th e corresponding group ra tings. Not all percentages equated to 100 pe rcent due to fractional roundin g or in those cases where participants chose not to res pond (classified as no judgment ) to specific statements. The information in the Appendix from Round Th ree reflects the participants reasons for taking exception with the groups views and further general comment about the workrelated education topic. The information in Ta ble 4-5 reflects the levels of agreement for each statement in each round as rated by the panel of experts. Differences in Responses by Subgroups and Rounds An independent t-test was used as an i nvestigation of any differences between the two means for CEOs and other administrators. The t-test was to as certain the probability that any difference between them reflected a real difference between the groups of participants rather than a ch ance variation in the data (T uckman, 1999, p. 300). The t-test was an effective tool for pred icting any statistical differen ces between the means for the

PAGE 96

86 scores (total score and the i ndividual scores for the principl es and the components) as the dependent variables from the independent va riables (CEOs and administrators). An analysis of variance could not be used for fact or analysis since the participant sample size (n) in the study was too small to permit a m eaningful factor anal ysis. Three primary questions were posed: 1. Did the average total score depend upon the job capacity of participant (CEOs versus other administrators)? 2. Did the average score for the principles of the learning college subsection depend upon the job capacity of participant (C EOs versus other administrators)? 3. Did the average score for the other id entified components subsection depend upon the job capacity of participant (C EOs versus other administrators)? As noted in the Tables 4-1, 4-2, 4-3, statistical differences between the CEO scores and other administrator scores were not present. Table 4-1 T-test of Average Total, Princi ples, and Components Scores for CEOs and Administrators during Round One Average Scores CEO Administrators Total Score 71.3 64.9 Principles Score 37.6 31.2 Components Score 33.8 33.7 Statistical differences between CEO scores and other admini strator scores were not present at p< 0.05. Table 4-2 T-test of Average Total, Princi ples, and Components Scores for CEOs and Administrators during Round Two Average Scores CEO Administrators Total Score 103.4 100.9 Principles Score 55.4 52.1 Components Score 48.0 48.8 Statistical differences between CEO scores and other admini strator scores were not present at p< 0.05. Table 4-3 T-test of Average Total, Princi ples, and Components Scores for CEOs and Administrators during Round Three Average Scores CEO Administrators Total Score 131.0 120.6 Principles Score 69.8 63.3 Components Score 61.2 57.4 Statistical differences between CEO scores and other admini strator scores were not present at p< 0.05.

PAGE 97

87 Given the absence of statistical signifi cant differences between CEO scores and other administrator scores, as presented in Tables 4-1, 4-2, 4-3, another means test was run as a follow-up on the entire group betw een Rounds One, Two, and Three. The means test conducted on the entire group was the Duncan 's test (Duncan's multiple range test), which, as a type of multiple comparison tests, was used to make pair-wise comparisons of means that were not significantly different between each other. The Duncan's test, as presented in Table 4-4, provided output that was essentially a "picture" of which pairs of means were significantly diffe rent as a post hoc test. Table 4-4 Duncans Multiple Range Test of Scores between All Three Rounds Round One Round Two Round Three Total Score 60.7* 66.6 72.7* Principles Score 26.4* 29.6 32.9* Components Score 34.3 36.9 39.8 Denotes statistical significance at p< 0.05. The Duncans test was employed to determine if statistical differe nces existed among the average total scores with any statistical si gnificance between Round One and Round Two, Round Two and Round Three, and R ound One and Round Three. Statistical differences were not evident between Round One and Round Two or between Round Two and Round Three. However, statistical significance was evident in average total scores between the first and third rounds. Th is significance indicated that measurable progress was made in reaching consensus from the first round to the final results. Table 4-5 Descriptive Statis tics of Statements Compared Across All Three Rounds Section A: Work-related Edu cation and the Learning College According to O'Banion (1997), "The learning college places learning first and provides educational experiences for lear ners anyway, anyplace, anytime." Please consider if work-related education has a place in the lear ning college and how each of the following six principles may or may not apply to work-related educational experiences.

PAGE 98

88 Table 4-5 Continued 1. Work-related education should create substantive change in its learners. Round One Round Two Round Three Strongly Disagree 5% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 6% 0% Slightly Agree 0% 0% 13% Agree 50% 59% 47% Strongly Agree 45% 35% 40% Mean 2.200 2.176 2.267 Standard Deviation 1.323 0.951 0.704 2. Work-related education should "kindle" (stimulate) new wa ys of seeing, thinking, and doing--in dramatic first ev ents and new discoveries. Round One Round Two Round Three Strongly Disagree 5% 0% 0% Disagree 0% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 5% 6% 7% Agree 40% 41% 20% Strongly Agree 45% 53% 73% Mean 1.522 2.471 2.667 Standard Deviation 0.667 0.624 0.617 3. Work-related education should "kindle" (stimulate) new wa ys of seeing, thinking, and doing--incrementally in day-to-day experiences. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 5% 6% 7% Agree 60% 71% 60% Strongly Agree 35% 24% 33% Mean 2.300 2.176 2.267 Standard Deviation 0.571 0.529 0.594 Principle II: According to O'Banion, "The le arning college engages learners as full partners in the learning proce ss, with learners assuming primary responsibility for their own choices."

PAGE 99

89 Table 4-5 Continued 4. Work-related education should communicate that students are full (and activ e) partners in the creation and implementation of their learning experiences. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 5% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 0% 6% 0% Agree 30% 18% 20% Strongly Agree 60% 76% 80% Mean 2.250 2.706 2.800 Standard Deviation 1.372 0.588 0.414 5. Work-related education should communicat e that students will assume primary responsibility for maki ng their own choices about goals and options. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 5% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 5% 6% 6% Agree 40% 35% 33% Strongly Agree 45% 59% 67% Mean 2.050 2.529 2.667 Standard Deviation 1.356 0.624 0.448 6. Work-related education should require st udents to participate in a structured induction/orientation process. Round One Round Two Round Three Strongly Disagree 10% n/a n/a Disagree 5% n/a n/a Slightly Disagree 15% n/a n/a Slightly Agree 15% n/a n/a Agree 30% n/a n/a Strongly Agree 25% n/a n/a Mean 0.950 n/a n/a Standard Deviation 2.038 n/a n/a

PAGE 100

90 Table 4-5 Continued 6.1. Work-related education orientation should be tailored to the indi vidual learner--some begin after a single point of engagement, while others may continue orientation for a few days or a few weeks. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 6% 0% Slightly Disagree n/a 6% 13% Slightly Agree n/a 18% 7% Agree n/a 47% 47% Strongly Agree n/a 24% 20% Mean n/a 1.647 1.600 Standard Deviation n/a 1.367 1.298 6.2. Work-related education orientation should offer many formats (flexible times, onsite/workplace, group, one-on-one, self -guided, mentoring, on-line, etc.). Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 0% Slightly Agree n/a 18% 7% Agree n/a 29% 13% Strongly Agree n/a 53% 73% Mean n/a 2.353 2.533 Standard Deviation n/a 0.786 0.915 7. Work-related education should require a pers onal learning plan or negotiated contract for students. Round One Round Two Round Three Strongly Disagree 5% 0% 0% Disagree 5% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 10% 12% 0% Agree 60% 82% 100% Strongly Agree 15% 6% 0% Mean -0.950 1.941 2.000 Standard Deviation 0.999 0.429 0.000

PAGE 101

91 Table 4-5 Continued 7.1. Work-related education should assess prio r learning to ensure students' learning experiences are not duplicative. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 0% Slightly Agree n/a 12% 13% Agree n/a 47% 67% Strongly Agree n/a 41% 20% Mean n/a 2.294 2.067 Standard Deviation n/a 0.686 0.594 Principle III: According to O' Banion, "The learning college creates and offers as many options for learning as possible." 8. Work-related education should offer a full a rray of options to accommodate individual differences in learning styles, rates, aptitudes, and prior knowledge. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 25% 12% 7% Agree 35% 24% 7% Strongly Agree 40% 65% 87% Mean 2.150 2.529 2.800 Standard Deviation 0.813 0.717 0.561 8.1. Standards and institutional reputation s hould be evident w ithin individualized learning options. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 0% Slightly Agree n/a 12% 0% Agree n/a 53% 73% Strongly Agree n/a 35% 27% Mean n/a 2.235 2.267 Standard Deviation n/a 0.664 0.458

PAGE 102

92 Table 4-5 Continued 8.2. Work-related education should follow an andragogical model of learning. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 6% 7% Slightly Disagree n/a 6% 0% Slightly Agree n/a 6% 0% Agree n/a 53% 67% Strongly Agree n/a 18% 20% Mean n/a 1.471 1.800 Standard Deviation n/a 1.419 1.265 9. Work-related educational opt ions should be seamless (not operated in isolation, so students can make reasonable changes in their programs). Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 25% 12% 7% Agree 40% 59% 53% Strongly Agree 35% 29% 40% Mean 2.100 2.176 2.333 Standard Deviation 0.788 0.636 0.617 10. Work-related education options should be trackless (same beginning courses for several programs, so students can explor e before committing to a single track). Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 15% 6% 0% Slightly Disagree 0% 0% 0% Slightly Agree 25% 24% 27% Agree 45% 65% 67% Strongly Agree 15% 6% 7% Mean 1.300 1.588 1.800 Standard Deviation 1.559 1.064 0.561 11. Work-related educational options should be classless (competency based), thereby providing mobility for students who may change institutions. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 10% 18% 13%

PAGE 103

93 Table 4-5 Continued Round One Round Two Round Three Slightly Disagree 5% 0% 0% Slightly Agree 20% 0% 0% Agree 50% 71% 67% Strongly Agree 15% 12% 20% Mean 1.400 2.176 1.667 Standard Deviation 1.465 0.529 1.543 Principle IV: According to O'Banion, "The le arning college assists learners to form and participate in collaborative learning activities." 12. Work-related education should focus on cr eating communities among all participants (students, faculty, and other learning sp ecialists) to support individual learning. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 5% 0% 7% Agree 40% 29% 20% Strongly Agree 55% 71% 73% Mean 2.500 2.706 2.667 Standard Deviation 0.607 0.470 0.617 13. Community colleges should form and support learning communities in the workplace via electronic forum (distance learning), vi deo-on-demand, interactive training modules, [and (added) hybrid credit/customized programs]. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 20% 6% 7% Agree 55% 76% 93% Strongly Agree 25% 18% 0% Mean 2.050 2.118 1.933 Standard Deviation 0.686 0.485 0.258 14. Community colleges should establish lear ning communities and provide assessment services in the workplace. Round One Round Two Round Three Strongly Disagree 0% n/a n/a Disagree 0% n/a n/a Slightly Disagree 5% n/a n/a

PAGE 104

94 Table 4-5 Continued Round One Round Two Round Three Slightly Agree 15% n/a n/a Agree 50% n/a n/a Strongly Agree 30% n/a n/a Mean 2.000 n/a n/a Standard Deviation 0.427 n/a n/a 14.1. Community colleges should establish le arning communities in the workplace. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0 7% Slightly Agree n/a 29 13% Agree n/a 71 80% Strongly Agree n/a 0% 0% Mean n/a 1.706 1.667 Standard Deviation n/a 0.407 0.816 14.2. Community colleges should assess the rele vance of course instruction in the workplace. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 6% 0% Slightly Agree n/a 0% 0% Agree n/a 35% 33% Strongly Agree n/a 59% 67% Mean n/a 2.412 2.667 Standard Deviation n/a 1.004 0.488 Principle V: According to O'Ba nion, "The learning college de fines the roles of learning facilitators by the needs of the learners." 15. Work-related education personnel should be hired on the basis of department or course needs. Round One Round Two Round Three Strongly Disagree 5% 0% 0% Disagree 25% 24% 13% Slightly Disagree 0% 0% 0% Slightly Agree 15% 0% 0% Agree 35% 53% 67% Strongly Agree 20% 24% 20%

PAGE 105

95 Table 4-5 Continued Round One Round Two Round Three Mean 0.800 1.294 1.667 Standard Deviation 2.093 1.929 1.563 15.1. Work-related education pers onnel should be hired as co urse content experts who adjust to learner and industry needs which change over time. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 0% Slightly Agree n/a 6% 0% Agree n/a 53% 53% Strongly Agree n/a 35% 47% Mean n/a 2.176 2.467 Standard Deviation n/a 0.809 0.516 15.2. Work-related education pe rsonnel should be hired based on their pedagogical content knowledge and who adjust to learner needs which change over time. Round One Round Two Round Three Strongly Disagree n/a n/a 0% Disagree n/a n/a 0% Slightly Disagree n/a n/a 7% Slightly Agree n/a n/a 7% Agree n/a n/a 73% Strongly Agree n/a n/a 13% Mean n/a n/a 1.867 Standard Deviation n/a n/a 0.915 16. Work-related education personnel should be hired based on what learners need. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 5% 6% 0% Slightly Agree 10% 12% 13% Agree 40% 24% 7% Strongly Agree 45% 53% 80% Mean 2.200 2.118 2.667 Standard Deviation 1.005 1.219 0.724

PAGE 106

96 Table 4-5 Continued 17. Work-related education students should also participate as lear ning facilitators--to assist other learners, to free professional staff for other roles and to reduce personnel costs. Round One Round Two Round Three Strongly Disagree 0% n/a n/a Disagree 10% n/a n/a Slightly Disagree 0% n/a n/a Slightly Agree 20% n/a n/a Agree 45% n/a n/a Strongly Agree 25% n/a n/a Mean 1.650 n/a n/a Standard Deviation 1.424 n/a n/a 17.1. Work-related education students should partic ipate as learning faci litators--to assist other learners. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 0% Slightly Agree n/a 18% 0% Agree n/a 59% 87% Strongly Agree n/a 12% 13% Mean n/a 1.706 2.133 Standard Deviation n/a 0.849 0.352 17.2. Work-related education students should partic ipate as learning faci litators--to free professional staff for other role s and to reduce personnel costs. Round One Round Two Round Three Strongly Disagree n/a 18% n/a Disagree n/a 18% n/a Slightly Disagree n/a 24% n/a Slightly Agree n/a 24% n/a Agree n/a 18% n/a Strongly Agree n/a 0% n/a Mean n/a -0.529 n/a Standard Deviation n/a 1.841 n/a Principle VI: According to O'Banion, "The le arning college and its learning facilitators succeed only when improved and expanded learning can be documented for its learners."

PAGE 107

97 Table 4-5 Continued 18. Community colleges should require work-r elated educational competencies for entrance. Round One Round Two Round Three Strongly Disagree 10% 12% n/a Disagree 50% 59% n/a Slightly Disagree 0% 6% n/a Slightly Agree 10% 0% n/a Agree 25% 18% n/a Strongly Agree 5% 6% n/a Mean -0.550 -1.000 n/a Standard Deviation 2.089 1.936 n/a 19. Community colleges should require work-rel ated educational competencies for exit (at multiple points [Round Two added]). Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 15% 0% 7% Agree 60% 71% 53% Strongly Agree 25% 29% 40% Mean 2.050 2.294 2.333 Standard Deviation 0.686 0.470 0.617 20. Portfolio assessment should be the primar y means by which work-related learning is documented. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 15% 12% 7% Slightly Disagree 15% 0% 0% Slightly Agree 35% 53% 0% Agree 30% 35% 87% Strongly Agree 5% 0% 7% Mean 0.650 1.000 0.867 Standard Deviation 1.565 1.225 0.834

PAGE 108

98 Table 4-5 Continued 20.1. Certifications and/or licenses should be the primary means by which work-related learning is documented. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 6% 7% Slightly Disagree n/a 12% 13% Slightly Agree n/a 12% 7% Agree n/a 65% 73% Strongly Agree n/a 6% 0% Mean n/a 1.353 1.267 Standard Deviation n/a 1.387 1.387 21.1. If national or state standards are not av ailable, community colleges should employ specialists or contract to develop standards. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 6% 0% Slightly Disagree n/a 24% 27% Slightly Agree n/a 24% 20% Agree n/a 35% 53% Strongly Agree n/a 12% 0% Mean n/a 0.941 1.000 Standard Deviation n/a 1.560 1.309 21.2. If national or state standards are not av ailable, community colleges should partner with industry to share cost s to develop standards. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 18% 7% Slightly Agree n/a 18% 20% Agree n/a 47% 60% Strongly Agree n/a 18% 7% Mean n/a 1.471 1.533 Standard Deviation n/a 1.328 0.990 Section B: Other Key Issues, Areas of Fo cus, Components of Work-related Education Please consider how community colleges shou ld define work-related education in the context of the institution, internal and ex ternal needs, and pol itical considerations.

PAGE 109

99 Table 4-5 Continued B-1. Mission and organization. Assess mission and organization in terms of the purpose and products of work-related e ducation (ends, not the means). 22. The mission statement should clearly claim the role of work-rela ted education equal to other mission tenants. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 5% 6% 0% Slightly Disagree 5% 0% 0% Slightly Agree 10% 6% 13% Agree 35% 41% 20% Strongly Agree 45% 47% 67% Mean 2.000 2.176 2.533 Standard Deviation 1.376 1.237 0.743 22.1. Work-related education should be integrated with general and transfer education (create an integrated model). Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 0% Slightly Agree n/a 6% 0% Agree n/a 76% 87% Strongly Agree n/a 18% 13% Mean n/a 2.118 2.133 Standard Deviation n/a 0.485 0.352 23. Work-related education should be, politica lly (centrally planned and funded), an important part of the organization. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 15% 6% 13% Agree 35% 29% 0% Strongly Agree 45% 65% 87% Mean 2.150 2.588 2.733 Standard Deviation 1.040 0.618 0.704

PAGE 110

100 Table 4-5 Continued 23.1. Work-related education should be autonomous. Round One Round Two Round Three Strongly Disagree n/a 0% 13% Disagree n/a 18% 27% Slightly Disagree n/a 18% 27% Slightly Agree n/a 35% 27% Agree n/a 24% 7% Strongly Agree n/a 6% 0% Mean n/a 0.471 -0.800 Standard Deviation n/a 1.625 1.612 23.2. Work-related education s hould retain an entrep reneurial perspective. Round One Round Two Round Three Strongly Disagree n/a n/a 0% Disagree n/a n/a 7% Slightly Disagree n/a n/a 0% Slightly Agree n/a n/a 27% Agree n/a n/a 53% Strongly Agree n/a n/a 13% Mean n/a n/a 1.600 Standard Deviation n/a n/a 0.980 B-2. Funding. Assess the viability of work-rela ted education in the context of funding priorities and strategies to modify the priorities of those who control the funding. 24. Community college funding mechanisms should acknowledge the centrality (deal effectively and fairly with all as pects) of work-related education. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 10% 0% 7% Agree 45% 41% 20% Strongly Agree 40% 59% 73% Mean 2.150 2.588 2.667 Standard Deviation 0.988 0.507 0.617

PAGE 111

101 Table 4-5 Continued 25. Funding formulas should be influenced to include the needs of the emerging workforce on state, regional, national, and global basis. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 10% 0% 0% Agree 45% 47% 27% Strongly Agree 40% 53% 67% Mean 2.150 2.529 2.733 Standard Deviation 0.988 0.514 0.458 26. Funding formulas should be influenced to include the needs of [how] instructional innovation [improves] work-relate d education. (Round Two added) Round One Round Two Round Three Strongly Disagree 0% 6% 0% Disagree 0% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 5% 0% 0% Agree 60% 65% 100% Strongly Agree 30% 29% 0% Mean 2.100 2.000 2.000 Standard Deviation 0.912 1.369 0.000 26.2. Colleges should partner with the private s ector and non-profits to obtain financial support for work-related education students. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 0% Slightly Agree n/a 0% 0% Agree n/a 65% 93% Strongly Agree n/a 35% 7% Mean n/a 2.353 2.067 Standard Deviation n/a 0.493 0.258 B-3. Needs assessment and documenting college success. Assess work-related education in terms of labor market data collecti on, exit requirements co mpared to industry expectations, and economic de velopment of the community.

PAGE 112

102 Table 4-5 Continued 27. Colleges should be the experts at monitori ng local labor markets and collecting data for program planning and needs assessment. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 10% 0% 0% Slightly Disagree 5% 6% 13% Slightly Agree 0% 6% 0% Agree 55% 82% 87% Strongly Agree 30% 6% 0% Mean 1.750 1.824 1.600 Standard Deviation 1.552 0.809 1.056 27.1. Colleges should rely on economists or researchers in higher education to monitor local labor markets and collect data fo r program planning and needs assessment. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 12% 7% Slightly Disagree n/a 29% 7% Slightly Agree n/a 18% 40% Agree n/a 35% 47% Strongly Agree n/a 6% 0% Mean n/a 0.529 1.133 Standard Deviation n/a 1.663 1.187 27.2. Colleges should rely on state and/or nati onal collection systems for labor market data for program planning and needs assessment. Round One Round Two Round Three Strongly Disagree n/a n/a 0% Disagree n/a n/a 7% Slightly Disagree n/a n/a 0% Slightly Agree n/a n/a 27% Agree n/a n/a 67% Strongly Agree n/a n/a 0% Mean n/a n/a 1.467 Standard Deviation n/a n/a 1.060

PAGE 113

103 Table 4-5 Continued 28. Success should be measured by work-related education program, certificate, or degree completion rates. Round One Round Two Round Three Strongly Disagree 5% 0% 0% Disagree 20% 12% 7% Slightly Disagree 0% 6% 7% Slightly Agree 35% 41% 73% Agree 25% 35% 13% Strongly Agree 10% 0% 0% Mean 0.600 0.824 0.800 Standard Deviation 1.818 1.334 1.014 29. Success should be measured in terms of job generation, upgrades, retraining, and economic development. Round One Round Two Round Three Strongly Disagree 0% n/a n/a Disagree 10% n/a n/a Slightly Disagree 0% n/a n/a Slightly Agree 15% n/a n/a Agree 35% n/a n/a Strongly Agree 30% n/a n/a Mean 1.550 n/a n/a Standard Deviation 1.538 n/a n/a 29.1. Success should be measured by job retentio n rates, employment rates, and increased wages. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 7% Slightly Disagree n/a 12% 0% Slightly Agree n/a 29% 27% Agree n/a 47% 67% Strongly Agree n/a 6% 0% Mean n/a 1.294 1.467 Standard Deviation n/a 1.105 1.060

PAGE 114

104 Table 4-5 Continued 29.2. Success should be measured by each st udent's educational attainment/skill acquisition--including those who complete one class and those who do not complete a certificate or degree. Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 0% Slightly Agree n/a 12% 13% Agree n/a 53% 67% Strongly Agree n/a 29% 20% Mean n/a 2.059 2.067 Standard Deviation n/a 0.827 0.594 B-4. Instruction, programs, and delivery system s. Assess work-related education in terms of instructional approaches, college programming, improvement processes. 30. The focus of work-related educati on should be on entry-level skills. Round One Round Two Round Three Strongly Disagree 5% 6% n/a Disagree 20% 29% n/a Slightly Disagree 45% 53% n/a Slightly Agree 20% 12% n/a Agree 5% 0% n/a Strongly Agree 5% 0% n/a Mean -0.450 -1.176 n/a Standard Deviation 1.572 1.015 n/a 31. The focus of work-related education s hould be on learning-to-learn skills, e.g., orientation to lifelong learning. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 5% 0% 0% Slightly Disagree 15% 6% 7% Slightly Agree 40% 71% 67% Agree 35% 24% 13% Strongly Agree 5% 0% 7% Mean 1.00 1.118 1.067 Standard Deviation 1.298 0.697 0.884

PAGE 115

105 Table 4-5 Continued 32. Work-related education should focus on pr eparation for a single job requiring a focused set of skills. Round One Round Two Round Three Strongly Disagree 15% 18% n/a Disagree 30% 29% n/a Slightly Disagree 40% 47% n/a Slightly Agree 5% 6% n/a Agree 10% 0% n/a Strongly Agree 0% 0% n/a Mean -1.200 -1.529 n/a Standard Deviation 1.436 1.007 n/a 33. Work-related education should focus on le arning skills with broad application to several similar occupations. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 20% 6% 13% Agree 55% 82% 73% Strongly Agree 20% 6% 13% Mean 1.850 1.882 2.000 Standard Deviation 0.933 0.600 0.535 33.1. Work-related education should reflect the needs of local businesses (not be limited to any specific level of skill development). Round One Round Two Round Three Strongly Disagree n/a 0% 0% Disagree n/a 0% 0% Slightly Disagree n/a 6% 0% Slightly Agree n/a 24% 13% Agree n/a 47% 73% Strongly Agree n/a 18% 13% Mean n/a 1.647 2.000 Standard Deviation n/a 1.057 0.535

PAGE 116

106 Table 4-5 Continued 34. Work-related education should be on an equal footing with regular college programming. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 5% 0% 0% Agree 45% 47% 40% Strongly Agree 45% 47% 60% Mean 2.250 2.353 2.600 Standard Deviation 0.967 0.786 0.507 35. Work-related education should be considered as add-on revenue cen ters separate from traditional credit programs. Round One Round Two Round Three Strongly Disagree 20% 12% n/a Disagree 25% 35% n/a Slightly Disagree 25% 29% n/a Slightly Agree 15% 18% n/a Agree 10% 6% n/a Strongly Agree 5% 0% n/a Mean -0.850 -1.059 n/a Standard Deviation 1.872 1.478 n/a B-5. Staffing. Assess work-related education in terms of staffing decisions, workplace experience levels, and student placement services. 36. Colleges should establish a staffing pattern (hard-money and full-time positions) relationship to ensure work-related educa tion programs receive their fair share. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 25% 18% 0% Agree 55% 59% 73% Strongly Agree 20% 24% 27% Mean 1.950 2.059 2.267 Standard Deviation 0.686 0.659 0.458

PAGE 117

107 Table 4-5 Continued 37. Work-related education faculty and st aff need real workplace experience to communicate effectively with students. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 20% 12% 13% Agree 45% 41% 13% Strongly Agree 30% 41% 73% Mean 1.950 2.176 2.600 Standard Deviation 0.999 0.883 0.737 38. Work-related education advisory committees provide sufficient "real world of work" input to faculty and staff. Round One Round Two Round Three Strongly Disagree 5% 0% 0% Disagree 5% 0% 7% Slightly Disagree 20% 12% 7% Slightly Agree 25% 47% 33% Agree 30% 41% 47% Strongly Agree 15% 0% 7% Mean 0.850 1.176 1.267 Standard Deviation 1.755 0.951 1.280 39. Student services and advising should be th e same for work-related education students as it is for traditional credit students. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 15% 6% 7% Slightly Disagree 15% 0% 0% Slightly Agree 20% 0% 0% Agree 40% 82% 93% Strongly Agree 10% 12% 0% Mean 0.850 1.882 1.733 Standard Deviation 1.694 1.054 1.033

PAGE 118

108 Table 4-5 Continued 40. The student placement office should primarily focus on identifying career openings and pathways. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 15% 6% 13% Slightly Agree 30% 0% 7% Agree 50% 94% 80% Strongly Agree 5% 0% 0% Mean 1.300 1.824 1.533 Standard Deviation 1.129 0.728 1.060 41. The student placement office should primarily focus on short-term training for immediate employment. Round One Round Two Round Three Strongly Disagree 10% 12% n/a Disagree 25% 29% n/a Slightly Disagree 30% 41% n/a Slightly Agree 20% 12% n/a Agree 15% 6% n/a Strongly Agree 0% 0% n/a Mean -0.600 -1.118 n/a Standard Deviation 1.667 1.364 n/a B-6. Coordination and planning. Assess whethe r the thinking and planning for workrelated education is focused on where students will find work or by other criteria. 42. Programming and planning should take into account regional, national, and global trends. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 5% 0% 0% Agree 45% 12% 7% Strongly Agree 50% 82% 93% Mean 2.450 2.706 2.933 Standard Deviation 0.605 0.772 0.258

PAGE 119

109 Table 4-5 Continued 43. Community colleges should cooperate with one another to accomplish regional and national work-related educa tion planning collectively. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 5% 0% 0% Agree 50% 24% 13% Strongly Agree 45% 71% 87% Mean 2.400 2.588 2.867 Standard Deviation 0.598 0.795 0.352 44. Community colleges should seek to build coalitions and partne rships with other colleges, organizations, and business to de fine roles and a vision for work-related education. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 0% 0% 0% Slightly Disagree 0% 0% 0% Slightly Agree 0% 0% 0% Agree 25% 6% 13% Strongly Agree 75% 88% 87% Mean 2.750 2.765 2.867 Standard Deviation 0.444 0.752 0.352 44.1. Each community college should be the pr imary coordinator between high schools, their college, and universities (circular linka ges) to eliminate duplication and gaps in student learning. Round One Round Two Round Three Strongly Disagree n/a 5% 0% Disagree n/a 0% 0% Slightly Disagree n/a 0% 7% Slightly Agree n/a 0% 0% Agree n/a 12% 13% Strongly Agree n/a 82% 80% Mean n/a 2.706 2.600 Standard Deviation n/a 0.772 1.056 B-7. National proclamation and national data base for work-related education. Assess work-related education in terms of a wide-s pread vision and supporti ng data that reflect the strengths and successes of community colleges.

PAGE 120

110 Table 4-5 Continued 45. A national proclamation should be created an d promoted which defines the role of the community college in work-related education. Round One Round Two Round Three Strongly Disagree 5% 0% 0% Disagree 0% 0% 0% Slightly Disagree 10% 6% 0% Slightly Agree 40% 59% 67% Agree 15% 6% 7% Strongly Agree 30% 29% 27% Mean 1.350 1.529 1.600 Standard Deviation 1.599 1.125 0.910 46. A national database reflecting community college potential and achievement should be created to assist in identifying limita tions and areas of growth and improvement. Round One Round Two Round Three Strongly Disagree 0% 0% 0% Disagree 5% 0% 0% Slightly Disagree 5% 0% 0% Slightly Agree 45% 59% 60% Agree 20% 24% 27% Strongly Agree 25% 18% 13% Mean 1.450 1.588 1.533 Standard Deviation 1.317 0.795 0.743 Confidence in the Accuracy of the Delphi Technique The accuracy of the Delphi technique fo r this research study was confirmed by a thorough examination of the results during th e three-round process. Upon comparing data from all iterative surveys by the appropriate descriptive statistics round-by-round, it was evident that the Delphi technique produ ced a communication st ructure aimed at producing a detailed examination and discussi on with a certain qua ntification of the participants viewpoints (Turoff & Hiltz, 1996). The levels of agreement were concentrated on those principles and compone nts which could be identified, categorized, and ranked to derive a comm on definition for work-related education. By sharing the

PAGE 121

111 overall group frequencies for the statements after each round, as well as the panel of experts qualitative responses, the participants could eff ectively contribute to the development of the content of the scale. Th e content of the scale progressively matched the content domain as participants pr ovided qualitative responses on what they considered to be the constructs of interest, t hus ensuring content validity. In this process, the Delphi technique and the survey proce dures offered participants opportunity to reconsider viewpoints and cons ider other items that were introduced. That the second and third rounds of the Delphi afforded the panel of experts an opportunity to change their initial ratings in light of this new info rmation further ensured that the results could be used for well-founded conclusions. Stat istically significant differences found in the average total scores between Round One and Round Three indicated that the panel of experts had fully developed the Delphi technique for this study. The levels of agreement and consensus that were further achieved de monstrated a level of confidence. This confidence, combined with the research pr ocedures and the study attributes, promoted reaching a superior group view of the task at hand through the phenomenon collective intelligence (Turoff & Hiltz, 1996, p. 80). Data Relationships to the Research Questions The purpose of this research was to co mplement and add to the body of knowledge pertaining to work-related education at comm unity colleges. The conceptual framework that guided this study was twofold. Firs t, from both theoretical and practical perspectives, this study was conducted to test if work-related education conformed to the six principles of the learning college (OB anion, 1997) and to determine whether or not the learning college principles could be supplemented by examining additional complementary components of work-related education. Second, the Delphi technique

PAGE 122

112 was used as a research method to determ ine whether or not the community college leaders who participated in this study as th e panel of experts could reach consensus and levels of agreement. This agreement c ould further add to th e body the knowledge--a relationship of principles and components by identifying, categorizing, and ranking such principles and components. This knowledge could model a common definition for workrelated education at community colleges. Specifically, this study addressed the following primary and secondary research questions: Primary Questions 1. Which, if not all, of OBanions six princi ples of the learni ng college could be associated with work-related education? 2. What other components could be identifie d for the work-related education function at community colleges? 3. What were the most strongly advocated principles and components supporting work-related education? Secondary Questions Additionally, secondary questions were id entified that could be answered as a result of this study. These questions were answered through a compilation of answers to the primary research questions. 1. Could a selected group of community colle ge leaders reach consensus, using a Delphi technique, on what principles and co mponents could be identified to derive a common definition for work-related education? 2. Could meaningful relationships be confir med between the six principles and the identified components to derive a comm on definition for work-related education? By participating in this study, the community college leaders acted as a panel of experts assisting in the research. The panels pur pose was to derive a common definition for work-related education at community colleges. The research ques tions and subsequent responses demonstrate the relationships that were devel oped from the data.

PAGE 123

113 Research Question Pertaining to Pr inciples of the Learning College The first research question was based on i nvestigating whether or not work-related education could be defined in terms of OBa nions (1997) six principl es of the learning college. The research question was framed as: 1. Which, if not all, of OBanions six princi ples of the learni ng college could be associated with work-related education? The analyses which follow were based on the data from Round Three, the final Delphi survey. With the exception of low leve ls of agreement and lack of consensus on how learning is documented and how competency standards are developed (Principle VI), the study answered the first re search question in a supportiv e manner for the first five principles of the learning college. It was apparent that the panel of experts generally agreed that the first five of the six principles of the learni ng college should apply to workrelated education, as identified. By Round Th ree, 28 out of the 29 statements pertaining to the principles of the learning college we re rated at means between Slightly Agree and Strongly Agree. The means should be viewed according to the rating scale, which was coded as: Strongly Disagree = -3; D isagree = -2; Slightly Disagree = -1; Slightly Agree = +1; Agree = +2; and Str ongly Agree = +3. A neutral option or response choice of "undecided," "no opini on," "uncertain," or "don't know" was intentionally left out and deem ed reasonable. Virtually all participants had an opinion and corresponding level of agreement for the items. Otherwise, as per the informed consent, it was clear to the participants that they did not have to respond to every question. The participants could take a no judgment view, which is a practice commonly applied to Delphi studies (Adl er & Sainsbury, 1996, p. 188). Supporting the validity of the Round Three instrument, only fo ur out of the 29 statements with overall

PAGE 124

114 levels of positive agreement posted greater than one standard deviation from their means. Analysis of all five statements with sta ndard deviations higher than one standard deviation showed a lack of consensus by th e panel of experts, as identified in the following tables. Principle I The three statements pertaining to Principle I-the learning college creates substantive change in its learners-were rated by the panel of experts at means above Agree and approaching Strongly Agree. The specific statements pertaining to Principle I were reviewed and compared with the means, the standard deviations, and the confidence intervals, as presented in Table 4-6. Table 4-6 Principle I Statements in Round Three Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval 1. Work-related education should create substantive 2.267 0.704 1.898 2.635 change in its learners. 2. Work-related education should "kindle" (stimulate) new ways of seeing, thinking, and 2.667 0.617 2.343 2.990 doing--in dramatic first events and new discoveries. 3. Work-related education should "kindle" (stimulate) new ways of seeing, thinking, 2.267 0.594 1.956 2.578 and doing--incrementally in day-to-day experiences. In particular, the statements standard devi ations were at or u nder 0.7 which suggested that the panel of experts viewed these specific statements and Principle I as complementing work-related education.

PAGE 125

115 Principle II Principle II-the learning college engages lear ners as full partners in the learning process, with learners assuming pr imary responsibility for their own choices -referred to a statement about the specific orientation format for new work-related education learners. By exception, specific st atements pertaining to Principle II were reviewed and compared with the means, th e standard deviations, and the confidence intervals, as presented in Table 4-7. Table 4-7 Principle II Statements by Exception in Round Three 6.1 Work-related education orientation should be tailored to the individual learner--some begin 1.600 1.298 0.920 2.280 after a single point of engagement, while others may continue orientation for a few days or a few weeks. 6.2 Work-related education orientation should offer many formats (flexible times, 2.533 0.915 2.054 3.013 on-site/workplace, group, one-on-one, self-guided, mentoring, on-line, etc.). The statement presented an orie ntation format that should be tailored to the individual learner and offered the option of a variable length of orientation e ngagement for a few days or a few weeks. This statement ( 6.1) was rated with a mean of 1.600 and a standard deviation of 1.298. Detailed analysis of the data revealed a lack of consensus across the scale by one-hal f of the participants. Statemen t 6.2, pertaining to orientation should offer many formats, was rated with a mean of 2.533 and standard deviation of Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 126

116 0.915, which indicates the panel of experts view ed this statement as a more acceptable alternative to Statement 6.1. Principle III Principle III-the learning college creates and offers as many options for learning as possible-contained two statements of in terest. By exception, specific statements pertaining to Principle III were reviewed and compared with the means, the standard deviations, and the confidence in tervals, as presented in Table 4-8. Table 4-8 Principle III Statem ents by Exception in Round Three 8.2 Work-related education should follow an 1.800 1.265 1.137 2.463 andragogical model of learning. 11. Work-related educational options should be classless (competencybased), thereby 1.667 1.543 0.858 2.475 providing mobility for students who may change institutions. The first statement was added during Round Tw o, based on a participants comment that work-related education shoul d follow an andragogical model of learning. This statement (8.2) was rated upwards to Agr ee, with a mean of 1.800 and a standard deviation of 1.265. A detailed comparison between rounds revealed that lack of agreement across the range in Round Two and was replaced by two-thirds consensus in Round Three. This suggested th at the andragogical model of learning may not be the best fit for work-related education. The second stat ement (11) of interest was carried forward from the initial survey and pertained to wor k-related education options [that] should be Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 127

117 classless or competency-based for learners to be able to change institutions without starting over to any great extent. This st atement (11) was rated with a mean of 1.667, Slightly Agree to Agree, and a standard deviation of 1.543. Again, the analysis suggests this work-related education option wa s not universally supported by the panel. Principle IV Principle IV-the learning college assists lear ners to form an d participate in collaborative learning activities-supported the study with two of the highest levels of agreement (Statement 12, mean 2.667; Statement 14.2, mean 2.667) and two of the smaller standard deviations (Statement 13, standard deviation, 0.258; Statement 14.2, standard deviation, 0.488). The specific stat ements pertaining to Principle IV were reviewed and compared with the means, th e standard deviations, and the confidence intervals, as presented in Table 4-9. Table 4-9 Principle IV Statements in Round Three 12. Work-related education should focus on creating communities among all participants (students, 2.667 0.617 2.343 2.990 faculty, and other learning specialists) to support individual learning. 13. Community colleges should form and support learning communities in the workplace via electronic forum (distance 1.933 0.258 1.798 2.069 learning), video-on-demand, interactive training modules, [and (added) hybrid credit/ customized programs]. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 128

118 Table 4-9 Continued -3 to +3 Deviation Interval Interval 14.1 Community colleges should establish learning communities 1.667 0.816 1.239 2.094 in the workplace. 14.2 Community colleges should assess the relevance of course 2.667 0.488 2.411 2.922 instruction in the workplace. Overall, the panel of experts was in agreem ent about Principle IV complementing workrelated education by creating learning comm unities to support individual learning and that those learning communities should extend to the workplace. Principle V Principle V-the learning college defines the role s of learning facilitators by the needs of the learners -contained a statement from the in itial survey, which pertained to hiring work-related education personnel on the basis of department or course needs. This statement (15) was rated with a mean of 1.667, Slightly Agree to Agree, and a standard deviation of 1.543. By exception, this and other specific statements pertaining to Principle V were reviewed and compared w ith the means, the sta ndard deviations, and the confidence intervals, as presented in Table 4-10. Table 4-10 Principle V Statemen ts by Exception in Round Three 15. Work-related education personnel should be hired on the basis of department 1.667 1.543 0.858 2.475 or course needs. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 129

119 Table 4-10 Continued 15.1 Work-related education personnel should be hired as course content experts who 2.467 0.516 2.196 2.737 adjust to learner and industry needs which change over time. 15.2 Work-related education personnel should be hired based on their pedagogical 1.867 0.915 1.387 2.346 content knowledge and who adjust to learner needs which change over time. 16. Work-related education personnel should be hired 2.667 0.724 2.288 3.046 based on what learners need. A detailed analysis of the data revealed the ratings were impacted by a lack of consensus, as well as a lack of agreement with 13 percent of the participants taking issue and disagreeing with this statement. This suggests that hiring personnel on the basis of department or course needs wa s not universally supported by th e panel of experts. This lack of agreement suggests that the panel of experts embraced the specific principle through stronger consensus on the other stat ements (15.1, 15.2, and 16), which focused specifically on learner needs. The panel of experts endorsed a nother statement (17.1) with a level of agreement (mean 2.133, standa rd deviation 0.352), which stated that work-related education students should participate as learning facilitators--to assist other learners. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 130

120 Principle VI The two statements which were rated at averages below Slightly Agree, pertained to Principle VI-the learning college and its learning facilitators succeed only when improved and expanded learning ca n be documented for its learners By exception, specific statements pertaining to Principle VI were reviewed and compared with the means, the standard deviations, and the confid ence intervals, as presented in Table 4-11. Table 4-11 Principle VI Statements by Exception in Round Three 20. Portfolio assessment should be the primary means by which work0.867 0.834 0.430 1.303 related learning is documented. 20.1 Certifications and/or licenses should be the primary means by which 1.267 1.387 0.540 1.993 work-related learning is documented. 21.1 If national or state standards are not available, community colleges should 1.000 1.309 0.314 1.686 employ specialists or contract to develop standards. 21.2 If national or state standards are not available, community colleges should 1.533 0.990 1.015 2.052 partner with industry to share costs to develop standards. In particular, these statements were about portfolio assessment as a primary means of documenting learning (Statement 20, mean 0.867, standard deviation 0.834) and community colleges employing specialists to develop standards (Statement 21.1, mean Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 131

121 1.000, standard deviation 1.309). The low m ean for Statement 20, which nearly all participants ranked as Slightly Agree with one outlier at D isagree, suggested that the panel of experts did not endorse portfolio a ssessment as the primary means to document learning for work-related educat ion. Likewise, the experts views did not universally support any of the alternatives, for example, use of certificates and licenses to document learning had one-fifth of the participants rating Slightly Disagree to Disagree (Statement 20.1, mean 1.267, standard deviation 1.387). A detailed analysis of the data revealed that the ratings fo r community colleges, which em ployed specialists to develop standards (Statement 21.1, mean 1.000, standard deviation 1.309), were impacted by onehalf of the participants diverging ratings ac ross the Likert scale from Slightly Disagree to Slightly Agree and the other half at Agr ee. This lack of consensus suggested that the panel of experts did not endorse comm unity colleges employing specialists or contracting as the primary means to deve lop standards for work-related education. Rather, the experts views were further supported by the higher mean posted for an alternative, for example, partnering with i ndustry to share costs fo r developing standards (Statement 21.2, mean 1.533, standard deviation 0.990). Research Question Pertaining to Co mponents of Work-related Education The second research question was based on investigating whether or not workrelated education could be defined in terms of components, as identified in the literature, and considered complementary to the princi ples of the learning college. The second primary research question was framed as: 2. What other components could be identifie d for the work-related education function at community colleges?

PAGE 132

122 The analyses that follow were based on the data from Round Three, the final Delphi survey. The study answered the second research question in a supportive manner. With the exception of the low levels of agreement and lack of consensus on needs assessment and documenting college success (Component 3), the study answered the second research question in a supportive ma nner for the other six components identified with work-related education. It was apparent that the panel of experts generally agreed that six of the seven components should apply to work-related education, as identified in the study. By Round Three, 28 out of the 30 statements in the components section of Round Three were rated at means between Slig htly Agree and Strongly Agree. The means should be viewed according to the ra ting scale, which was coded as: Strongly Disagree = -3, Disagree = -2, Slightly Disagree = -1, Slightly Agree = +1, Agree = +2, and Strongly Agree = + 3. A neutral option or response choice of "undecided," "no opinion," "unc ertain," or "don't know" was intentionally left out and deemed reasonable as virtually all participan ts had an opinion and corresponding level of agreement for the items. Otherwise, as per the informed consent, it was clear to the participants that they did not have to respond to every question. The participants could take a no judgment view, which is a pr actice commonly applied to Delphi studies (Adler & Sainsbury, 1996, p. 188). Testing th e validity of the Round Three instrument, 9 out of the 30 statements, with overall levels of positive agreement, posted greater than one standard deviation from their means. Anal ysis of all nine statements with standard deviations higher than one sta ndard deviation showed a lack of consensus by the panel of experts, as identified in th e Tables 4-12 through 4-19.

PAGE 133

123 Mission and Organization Three out of the five statements pe rtaining to the missi on and organization component were rated at averages between Agree and Strongly Agree, with standard deviations ranging from 0.352 to 0.743. By excep tion, specific statements pertaining to the mission and organization component were reviewed and compared with the means, the standard deviations, and the confiden ce intervals, as presented in Table 4-12. Table 4-12 Mission and Orga nization Statements by Exception in Round Three 23.1. Work-related education should be autonomous. -0.800 1.612 -1.645 0.045 23.2. Work-related education should retain an entrepreneurial perspective. 1.600 1.183 0.980 2.220 The two statements, which were rated with the lowest means, also had standard deviations greater than one. Statement 23.1 pertained to whether or not work-related education should be autonomous (mean -0.800, standard deviation 1.612), and Statement, 23.2 pertained to whether or not work-related ed ucation should retain an entrepreneurial perspective (mean 1.600, standard deviation 1.183). Both of these statements were additions to the initial survey based on part icipant qualitative input. A detailed analysis of the data and the consensus, which form ed on low levels of agreement with the statement, revealed that the panel of expert s did not endorse that work-related education should be autonomous. This was put in th e context of the institution, internal and external needs, or political considerations as framed by mission and organization. In addition, the entrepreneurial perspective statement was impacted by half of the Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 134

124 participants diverging ratings across the Likert scale from Disagree to Slightly Agree and Strongly Agree with the other half at Agree. This lack of consensus suggested that the panel of e xperts could not agree on an en trepreneurial perspective for work-related education. Funding It was apparent, based on the high levels of agreement and consensus reached by the panel of experts, that funding was a highly endorsed compone nt of work-related education. All four statements pertaining to the funding component had means at or above Agree to Strongly Agree with stan dard deviations ranging from zero standard deviation (Statement 26.1) to 0.617 (Statement 24) The specific statements pertaining to the funding component were reviewed and co mpared with the means, the standard deviations, and the confid ence intervals, as presented in Table 4-13. Table 4-13 Funding Statements in Round Three 24. Community college funding mechanisms should acknowledge the centrality (deal 2.667 0.617 2.343 2.990 effectively and fairly with all aspects) of workrelated education. 25. Funding formulas should be influenced to include the needs of the emerging workforce on 2.733 0.458 2.494 2.973 state, regional, national, and global basis. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 135

125 Table 4-13 Continued 26. Funding formulas should be influenced to include [how] 2.000 0.000 n/a n/a instructional innovation [improves] work-related education. 26.2 Colleges should partner with the private sector and non-profits to obtain 2.067 0.258 1.931 2.202 financial support for workrelated education students. Needs Assessment and Documenting Success Needs assessment. All three statements pertaining to the needs assessment subcomponent were rated at averages betw een Slightly Agree and Agree, with all posting standard deviations greater than one. The specific statements pertaining to the needs assessment subcomponent were review ed and compared with the means, the standard deviations, and the confidence inte rvals, as presented in Table 4-14. Table 4-14 Needs Assessment Subcom ponent Statements in Round Three 27. Colleges should be the experts at monitoring local labor markets and 1.600 1.056 1.047 2.153 collecting data for program planning and needs assessment. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 136

126 Table 4-14 Continued 27.1 Colleges should rely on economists or researchers in higher education to monitor local labor markets and collect data for 1.133 1.187 0.511 1.755 program planning and needs assessment. 27.2 Colleges should rely on state and/or national collection systems for labor 1.427 1.060 0.911 2.022 market data for program planning and needs assessment. Statement 27 (mean 1.600, standard deviati on 1.056) reached an 87 percent consensus rating of Agree with two ratings of disagr eement. This was the panel of experts leading subcomponent for community colleges taking ownership to monitor local labor markets and collect data for program pla nning and needs assessment. Statement 27.1 (mean 1.133, standard deviation 1.187) had a nearly 50/50 split between ratings of Slightly Agree to Agree, with two ratings of disagreement. Statement 27.2 (mean 1.467, standard deviation 1.060) reached 27 percent and 67 percent consensus, respectively, for ratings of Slightly Agree and Agree, with one rating of Disagree. Statements 27.1 and 27.2, which were revised stat ements from the init ial survey, lacked the high levels of agreement, compared to Statement 27. Documenting success. A detailed analysis of all three statements pertaining to the documenting success subcomponent revealed that Statement 28 (mean 0.800, standard deviation 1.014) reached 73 percent cons ensus, with a rating of Slightly Agree Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 137

127 and two ratings of disagreement. The speci fic statements pertaining to the documenting success subcomponent were reviewed and co mpared with the means, the standard deviations, and the confid ence intervals, as presented in Table 4-15. Table 4-15 Documenting Success Subcom ponent Statements in Round Three 28. Success should be measured by work-related education program, 0.800 1.014 0.269 1.331 certificate, or degree completion rates. 29.1. Success should be measured by job retention 1.467 1.060 0.911 2.022 rates, employment rates, and increased wages. 29.2. Success should be measured by each individual student's educational attainment/skill acquisition 2.067 0.594 1.756 2.378 including those who complete one class and those who do not complete a certificate or degree. It was apparent that the panel of expe rts did not wholehear tedly endorse success measured by completion rates in the context of exit requirements, as compared to industry expectations for documenting college success. However, the panel of experts moved toward higher levels of agreem ent on Statement 29.2 (mean 2.067, standard deviation 0.594), which stipulated: Success should be measured by each individual students educational attainment/skill ac quisition--including thos e who complete one class and those who do not comple te a certificate or degree. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 138

128 Instruction, programs, and delivery systems The instruction, programs, and delivery systems component consisted of four statements pertaining to the focus of wo rk-related education on learning skills and delivery of programs. The panel of e xperts generally endorsed this component subsection with ratings between Agree and Strongly Agree for the last three statements, as presented in Table 4-16. Table 4-16 Instruction, Programming, and De livery Systems Statements in Round Three 33. Work-related education should focus on learning skills with broad application to 2.000 0.535 1.720 2.280 several similar occupations. 33.1. Work-related education should reflect the needs of local businesses (not be 2.000 0.535 1.720 2.280 limited to any specific level of skill development). 34. Work-related education should be on an equal footing 2.600 0.507 2.334 2.866 with regular college programming. The exception was a lower level of consensu s at 67 percent and a rating of Slightly Agree for Statement 31. This statement pertai ned to an orientation to lifelong learning as a focus of work-related education (mean 1.067, standard deviation 0.884), as depicted in Table 4-16. Staffing The staffing component consisted of five st atements pertaining to assessing workrelated education in terms of staffing decisions, workplace experience levels, and student Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 139

129 placement services. The panel generally endorsed this component subsection with averages between Agree and Strongly Agree for the first two statements and between Slightly Agree and Agree for the remaining st atements, as presented in Table 4-17. Table 4-17 Staffing Statements in Round Three 36. Colleges should establish a staffing pattern (hard-money & full-time positions) 2.506 2.267 0.458 2.027 relationship to ensure workrelated education programs receive their fair share. 37. Work-related education faculty and staff need real workplace experience to 2.600 0.737 2.214 2.986 communicate effectively with students. 38. Work-related education advisory committees provide sufficient "real world of work" 1.267 1.280 0.596 1.937 input to faculty and staff. 39. Student services and advising should be the same for work-related education 1.733 1.033 1.192 2.274 students as it is for traditional credit students. 40. The student placement office should primarily focus 1.533 1.060 0.978 2.089 on identifying career openings and pathways. By exception, Statement 38 (mean 1.267, standa rd deviation 1.280), which pertained to the value of work-related education advisory committees, did not achieve high levels of agreement beyond the one-half of the particip ants who rated it at the Agree level. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 140

130 Coordination and Planning The coordination and planning component consisted of four statements which assessed whether or not the thinking and planning for wo rk-related education was focused on where students would find work or find work by other criteria. This was one of the strongest components in terms of high le vels of agreement. The panel of experts fully endorsed this component subsection wi th all averages upwards or nearly at Strongly Agree for all statements, as presented in Table 4-18. Table 4-18 Coordination and Pla nning Statements in Round Three 42. Programming and planning should take into account regional, national, 2.933 0.258 2.798 3.069 and global trends. 43. Community colleges should cooperate with one another to accomplish 2.867 0.352 2.682 3.051 regional and national work-related education planning collectively. 44. Community colleges should seek to build coalitions and partnerships with ot her colleges, 2.867 0.352 2.682 3.051 organizations, and business to define roles and a vision for work-related education. 44.1 Each community college should be the primary coordinator between high schools, their college, and 2.600 1.056 2.047 3.153 universities (circular linkages) to eliminate duplication and gaps in student learning. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 141

131 Eighty percent of the participants rated St atement 44.1 Strongly Agree to community colleges assuming the role of primary coordi nator for circular linkages. Only one participant rated the statement Slightly Disagree. National Proclamation and National Database for Work-related Education The final component consisted of two specific statements pertaining to the promotion of a wide-spread vision and suppor ting data to reflec t the strengths and successes of work-related education at comm unity colleges. The panel of experts responded to this component subsection with rating averages upwards of Slightly Agree to Agree, as presented in Table 4-19. Table 4-19 National Proclamation and Nationa l Database Statements in Round Three 45. A national proclamation should be created and promoted which defines the 1.600 0.910 1.123 2.077 role of the community college in work-related education. 46. A national database reflecting community college potential and achievement 1.533 0.743 1.144 1.923 should be created to assist in identifying limitations and areas of growth and improvement. Research Question Pertaining to Strong est Advocated Principles and Components The third research question was base d on confirming those principles and components pertaining to work-related educati on which received the panel of experts highest support. The research question was framed as: 3. What were the most strongly advocated principles and components supporting work-related education? Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 142

132 Strongest Advocated Principles With the sole exception of Principle VI, the ot her five principles of the learning college were supported by one or more statements rated upwards of Strongly Agree. A total of eight out of the 29 statements for the five principles garnered high levels of agreement, as presented in Table 4-20. Table 4-20 Strongest Advocated Principles in Round Three Principle I 2. Work-related education should "kindle" (stimulate) new ways of seeing, thinking, 2.667 0.617 2.343 2.990 and doing--in dramatic first events and new discoveries. Principle II 4. Work-related education should communicate that students are full (and active) 2.800 0.414 2.583 3.017 partners in the creation and implementation of their learning experiences. 5. Work-related education should communicate that students will assume primary 2.667 0.488 2.411 2.922 responsibility for making their own choices about goals and options. 6.2. Work-related education orientation should offer many formats (flexible times, 2.533 0.915 2.054 3.013 on-site/workplace, group, one-on-one, self-guided, mentoring, on-line, etc.). Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 143

133 Table 4-20 Continued Principle III 8. Work-related education should offer a full array of options to accommodate 2.800 0.561 2.506 3.094 individual differences in learning styles, rates, aptitudes, and prior knowledge. Principle IV 12. Work-related education should focus on creating communities among all 2.667 0.617 2.343 2.990 participants (students, faculty, and other learning specialists) to support individual learning. 14.2. Community colleges should assess the relevance 2.667 0.488 2.411 2.922 of course instruction in the workplace. Principle V 16. Work-related education personnel should be hired 2.667 0.724 2.288 3.046 based on what learners need. By the completion of Round Three, patterns had emerged with respect to the most strongly advocated principles and components. Eighteen out of the 59 statements in Round Three were rated at averages at or above 2.5, with upper 95 percent confidence intervals in the 2.9 to 3.0 ra nge of Strongly Agree. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 144

134 Strongest Advocated Components Ten out of the 30 statements pertaining to the components in Round Three were rated at averages at or above 2.5, with upper 95 percent confidence inte rvals in the 2.9 to 3.0 range of Strongly Agree. By exception, there were two out of seven component subsections which did not have any stat ements rated at or above 2.5. These two component subsections were Needs Assessm ent and Documenting College Success and National Proclamation and National Database for Work-related Education. The other five component subsections were supported by one or more statements rated upwards of Strongly Agree. A total of 10 out of th e 30 statements garnered high levels of agreement, as presented in Table 4-21. Table 4-21 Strongest Advocated Components in Round Three Mission and Organization 22. The mission statement should clearly claim the role 2.533 0.743 2.144 2.923 of work-related education equal to other mission tenants. 23. Work-related education should be, politically (centrally 2.733 0.704 2.365 3.102 planned and funded), an important part of the organization. Funding 24. Community college funding mechanisms should acknowledge 2.667 0.617 2.343 2.990 the centrality (deal effectively and fairly with all aspects) of work-related education. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 145

135 Table 4-21 Continued 25. Funding formulas should be influenced to include the 2.733 0.458 2.494 2.973 needs of the emerging workforce on state, regional, national, and global basis. Instruction, Programs, and Delivery Systems 34. Work-related education should be on an equal footing 2.600 0.507 2.334 2.866 with regular college programming. Staffing 37. Work-related education faculty and staff need real 2.600 0.737 2.214 2.986 workplace experience to communicate effectively with students. 43. Community colleges should cooperate with one 2.867 0.352 2.682 3.051 another to accomplish regional and national work-related education planning collectively. 44. Community colleges should seek to build coalitions 2.867 0.352 2.682 3.051 and partnerships with other colleges, organizations, and business to define roles and a vision for work-related education. 44.1 Each community college should be the primary 2.600 1.056 2.047 3.153 coordinator between high schools, their college, and universities (circular linkages) to eliminate duplication and gaps in student learning. Mean Lower Upper Scale 95% 95% Range Standard Confidence Confidence -3 to +3 Deviation Interval Interval

PAGE 146

136 Consensus Reached by the Panel of Experts The first of the secondary research ques tions was based on determining the impact of the Delphi technique in confirming wh ether or not the panel of experts reached consensus on six principles and seven components to derive a common definition for work-related education. The research question was framed as: 1. Could a selected group of community colle ge leaders reach consensus, using a Delphi technique, on what principles and co mponents could be identified to derive a common definition for work-related education? A review of the standard deviations in Table 4-5 for the 29 statements, which pertained to the principles of the learning co llege, revealed that the panel of experts reached consensus on 23 statements or 79 perc ent of these statements by posting less than one standard deviation. A re view of the standard deviat ions in Table 4-5 for the 30 statements, which pertained to the components, revealed that the panel of experts reached consensus on 20 statements or 67 percent of these statements by posting less than one standard deviation. Forty-three out of the 59 statements or 73 percent of these statements, which posted less than one standa rd deviation indicated that the Delphi technique was effective in generating consensus. Relationships between the Principles and Components The last of the secondary research questi ons was based on investigating whether or not relationships could be confirmed between OBanions (1997) six principles of the learning college and the seven components iden tified in the litera ture and through the Delphi technique. The research question was framed as: 2. Could meaningful relationships be confir med between the six principles and the identified components to derive a comm on definition of work-related education?

PAGE 147

137 Correlation coefficients were used to analy ze the relationships of the means between the principles of the learning college and the components identified as pertaining to workrelated education. The correlations were orga nized into a matrix showing 42 correlations to facilitate inspection and comparison of each variable as shown in Table 4-22. Table 4-22 Correlation Matrix of Princi ples and Components in Round Three C 1 C 2 C 3 C 4 C 5 C 6 C 7 P I 0.29 0.75* 0.72* 0.61* 0.21 0.77* -0.04 P II 0.34 0.50* 0.36 0.43 0.25 0.62* 0.28 P III 0.62* 0.69* 0.27 0.68* 0.12 0.68* 0.32 P IV 0.41 0.67* 0.24 0.43 0.38 0.65* -0.12 P V 0.27 0.69* 0.36 0.54* 0.52* 0.72* -0.20 P VI 0.45 0.50* 0.72* 0.09 0.17 0.47 -0.10 Correlation is significant at the 0.05 level, p<0.05 P I = Principle I C 1 = Mission and organization P II = Principle II C 2 = Funding P III = Principle III C 3 = Need s assessment and docume nting college success P IV = Principle IV C 4 = Instruction, programs, delivery systems P V = Principle V C 5 = Staffing P VI = Principle VI C 6 = Coordination and planning C 7 = National proclamation and national database Both positive and negative corr elations were found. Significa nt correlations were all positive. Significant correlations were annotat ed in Table 4-22 by a single asterisk where the p-value was less than 0.05. Relationship of Components to Principles Principle I -the learning college creates substantive change in individual learners (P I)--had positive significance between the components pertaining to funding (C 2), needs assessment and documenting colle ge success (C 3), instruction, programs, and delivery systems (C 4), and coordination an d planning (C 6), as shown in Table 4-23. Principle II -the learning college engages lear ners as full partners in the learning process with learners assuming primary responsibility for their own choices

PAGE 148

138 (P II)--had positive significance between the components pertaining to funding (C 2) and coordination and planning (C 6), as depicted in Table 4-23. Principle III -the learning college creates and offers as many options for learning as possible (P III)--had positive significance between the components pertaining to mission and organization (C 1), funding (C 2), instruction, programs, and delivery systems (C 4), and coordination and planni ng (C 6), as depicted in Table 4-23. Principle IV -the learning college assists lear ners to form an d participate in collaborative learning activities (P IV)--had positive significance between the components pertaining to funding (C 2) and c oordination and planning (C 6), as depicted in Table 4-23. Principle V -the learning college defines the role s of learning facilitators by the needs of the learners (P V)--had positive significance between the components pertaining to funding (C 2), in struction, programs, and deliver y systems (C 4), staffing (C 5), and coordination and planning (C 6), as depicted in Table 4-23. Table 4-23 Significance between Princi ples and Components in Round Three Components Principle I = C 2 C 3 C 4 C 6 Principle II = C 2 C 6 Principle III = C 1 C 2 C 4 C 6 Principle IV = C 2 C 6 Principle V = C 2 C 4 C 5 C 6 Principle VI = C 2 C 3 C 1 = Mission and organization C 2 = Funding C 3 = Needs assessment an d documenting college success C 4 = Instruction, programs, delivery systems C 5 = Staffing C 6 = Coordination and planning C 7 = National proclamation and national database

PAGE 149

139 Principle VI -the learning college and its learni ng facilitators succeed only when improved and expanded learning ca n be documented for its learners (P VI)--had positive significance between the components pertaining to funding (C 2) plus needs assessment and documenting college success (C 3), as depicted in Table 4-23. Summary The Delphi technique was selected as the methodology for this research study for its potential to reach levels of agreement a nd consensus among a pane l of experts within a field to derive a common definition for work -related education. This research study was built on the theoretical foundation of the pr inciples of the learning college along with a practitioners perspective of those compone nts of work-related e ducation as identified in the literature that pertaine d to community colleges. The research sample consisted of 20 comm unity college leaders who were currently the chief executive officers or other administra tors at the colleges whose CEOs comprise the League Board of Director s. The three rounds of the survey instrument were webbased and administered via the Internet. To verify the accuracy of using the Delphi technique for this research study, the results of the three-round pro cess were shown in a comparative depiction. The response ratings were presented in a round-by-round eval uation of the data w ith the appropriate descriptive statistics. T-tests were used to confirm the absence of statistical significant differences between CEO scores and other admi nistrator scores. The Duncan's test was used and it found that statistical differences were evident between average total scores in the first and third rounds, whic h indicated that the panel of experts had fully developed the Delphi process for this study. This Dun cans test supported a level of confidence, which, when combined with the research pr ocedures and the study attributes, promoted

PAGE 150

140 reaching a superior group view of the task at hand through the phenomenon collective intelligence (Turoff & Hiltz, 1996, p. 80). A statistical investigation of the relationshi ps between the principles of the learning college and the identified components of work-related education confirmed that significant relationships existed between the six principles and th e seven components. This examination found that: Principle I related significantly to mi ssion and organization, funding, instruction, programs, and delivery systems, and coordination and planning. Principle II related significantly to funding and coordina tion and planning. Principle III related significantly to mi ssion and organization, funding, instruction, programs, and delivery systems, and coordination and planning. Principle IV related significantly to funding and coordination and planning. Principle V related significantly to f unding, instruction, programs, and delivery systems, staffing, and coordination and planning. Principle VI related significantly to funding and needs assessment and documenting college success. Six of the seven components under study were found to relate significantly to one or more of the principles of the learning college. Chapter 5 presents conclusions based on the results from the data that were compiled and the relationships which were identified to expand the existing body of knowledge pertaining to work-related educat ion. A prototype model of a common definition for work-related education is outli ned, the commonality of certain components is identified, recommendations for further re search are offered, and implications for community college leaders and policymakers are presented.

PAGE 151

141 CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS Introduction Work-related education conti nues to evolve as a key aspect of the community college mission. Bragg (2001) found that pres ent-day community colleges have a major responsibility for preparing the nations current and future midskilled workforce comprising three-quarters of employees in the Un ited States (p. 5). A central part of this research was to complement and add to the body of knowledge pertai ning to work-related education at community colleges. The conceptual framework that guided th is study was twofold. First, from the theoretical and practical pers pectives, the primary purpose of this study was to test if work-related education conformed to OBani ons (1997) six principles of the learning college and, furthermore to determine wh ether or not the pr inciples could be supplemented by examining additional comp lementary components of work-related education. Second, the research was conducte d with a Delphi tech nique to determine whether or not the community college leaders who participated in this study as the panel of experts could reach agreement on the identif ied principles and components. The point was to further the knowledge of these relations hips, which in total could be modeled to derive a common definition for work-relate d education at comm unity colleges. Specifically, this study addressed the fo llowing primary and secondary research questions:

PAGE 152

142 Primary Questions 1. Which, if not all, of OBanions six princi ples of the learni ng college could be associated with work-related education? 2. What other components could be identifie d for the work-related education function at community colleges? 3. What were the most strongly advocated principles and components supporting work-related education? Secondary Questions Additional secondary questions were identified that could be answered as a result of this study. These questions were addr essed through a compilati on of the answers to the primary research questions. 1. 1. Could a selected group of community college leaders reach consensus, using a Delphi technique, on what principles and co mponents could be identified to derive a common definition for work-related education? 2. 2. Could meaningful relationships be conf irmed between the six principles and the identified components to derive a comm on definition of work-related education? The research questions were first addressed through an exte nsive search of relevant literature on work-related education. The lit erature review examined classical and current literature in the field of vocational and occupational educati on as it pertained to postsecondary education, specifi cally community colleges. Th e literature review brought forth the consistent message that community colleges demonstrated flexibility in adapting to social and economic challenges f acing communities, states, regions, and the nation (Campbell, Leverty, & Sayles, 1996, p. 172) However, the findings of this study indicated that the conceptual ambiguity, wh ich has historically surrounded work-related education, has endured. Community colleges ha ve experienced an evolutionary mix of educational innovation, which contributed to a lack of consistency in defining workrelated education at their inst itutions. Varying terminology and a lack of a wide-spread

PAGE 153

143 definition for work-related education have imp acted the consistency a nd centrality of the function. These variations on the work-re lated education theme for the nations community colleges have been peculiar to each state based on a states individual needs, including how policy was made and how funding was allocated. Campbell et al. (1996) found that state funding for higher education reflects each states preference for higher education among other services (p. 174). Bran d (1998) described the process of change for work-related education in the United Stat es as complex, difficult, slow, which, on a national basis, has been impeded by the decen tralized nature of work-related education in the 50 states (p. 153). Beyond the observa tions and deficiencies which surfaced from the review, the literature confirmed a ga p in the current knowledge. This did not specifically reveal pointed evidence or de monstrate successful efforts to pursue and achieve a common definition for work-related education. This study was initiated to add to the body of knowledge, which reflected a gap in current research and practice, and also to establish a foundation and catalyst to mode l principles and components to derive a common definition for work-related education. The Delphi technique provided the methodology by which the panel members could communicate their opinions, beliefs, and agreement about work-related education. The participants responses pe rtained to the principles of the learning college and the components identified with work-related educa tion. The responses we re used to support a detailed examination and discussion with certain quantification of the participants viewpoints. The survey method and methodology lent itself to esse ntially built-in content validity by virtue of the participan ts development of the content of the scale matching the content domain. The panel of experts responses and inputs over the

PAGE 154

144 iterative three rounds of the Delphi technique further ensured that the results could be used for well-founded conclusi ons and relationships. The following observations were made at the conclusion of this study: The aggregate response rates, based on the number of participants, met the general guidelines for using the Delphi tech nique in educational studies. The study concluded with answers to all rese arch questions in a supportive fashion. It was apparent that the panel of expert s generally agreed that five of the six principles of the learning college sh ould apply to work-related education. It was apparent that the panel of expert s generally agreed that six of the seven components identified should appl y to work-related education. A statistical investigation revealed significance between the principles and several of the components identified in the study. The professionalism and commitment by th e panel of community college CEOs and senior administra tors were noteworthy by their co ntributions to the success of this research study. Model of Work-related Education The review of both the quantitative and qualitative data, which were collected during the three rounds of the Delphi study, revealed that th e panel of experts generated levels of agreement and consensus on specific principles of the lear ning college and other components. These principles and component s could be assimilated into a model to derive a common definition for work-related education. The im plications of this study suggest that current-day community college l eaders could use a participatory model, such as presented in this study, to pursue an incr eased presence and improved levels of support for community colleges in the United States. This model addresses th e six principles of the learning college and attempts to captur e relationships between each principle and identified components of work-related e ducation. However, understanding all the elements and relationships between each othe r to derive a common definition for work-

PAGE 155

145 related education may be more than can be expe cted at this juncture. This model is an effort to establish a consolidated position on and a holistic approach to what constitutes work-related education. Obtaining feedback on the prototype model drawn from this research could provide a starting point for more questions, further research, and an appropriate dialogue about the endorsemen t of a common definition for work-related education nationwide. Principle I: The learning college creates substantive change in individual learners This principle was supported by signifi cance with four out of the seven components identified with work-related education: Funding Needs assessment and documenting college success Instruction, programs, and delivery systems Coordination and planning Such an inclusive relatio nship could be explained based on the comprehensive nature for this first princi ple of the learning college. According to OBanion (1997), Principle I of the learning college is the embedded value undergirding all other principles (p. 48). This prin ciple symbolizes formal schooli ng to learners, and as such may also focus on those indirect proce sses which support formal schooling. Principle I demonstrated significance with funding (C 2) which recognized how the viability of the learning college could be de pendent on the ability of community colleges to influence funding priorities based on ne gotiating with those who control funding. Modification of funding strate gies, in support of work-related education, was found to be ultimately more likely to be a state or national effort (Hamm & Mundhenk, 1995, p. 5). Community colleges, however, still need to assess their work-related education efforts in the context of funding prioriti es that support an environment of substantive change for

PAGE 156

146 work-related education learners. In addition, there was 100 percent agreement (Statement 26, mean 2.000, standard deviati on 0.000) that funding formulas should be influenced to support innovative, instructiona l improvements for work-related education, as illustrated in Table 4-13. Considering th at work-related educa tion does not always integrate well into credit-bas ed funding formulas, there was also general agreement (93 percent) that community colleges may need to seek different funding avenues. These sources could include partnering with the priv ate sector and non-pr ofits (Statement 26.2) to satisfy learning college funding needs fo r work-related education. However, the ability of community colleges to establish new services and innovative practices would require dedicated efforts to influence the pr iorities of those who control the funds and funding formulas. Cohen and Brawer (2003) noted that funds are often secured through priorities established by state and federal agencies (p. 233). Again, Campbell et al. (1996) found that state funding for higher educ ation reflects each states preference for higher education among other services (p. 174). Brand (1998) descri bed the process of change for work-related education in the Unite d States as complex, difficult, slow. Needs assessment and documenting college success (C 3) revealed significance as to what learners fundamentally (and work-re lated education stakeholders) need and how community colleges document learner succe ss including degrees and certificates. Documenting college success should also in clude other exit point achievements for learners --those who did not--and never inte nded to--graduate, yet still experience substantive change at community college s. Cohen and Brawer (2003) found that students who enter vocational prog rams only to get additional skills must be factored in, just as students who obtain j ob certifications but find no jobs available (p. 235).

PAGE 157

147 Additionally, they found that because work-related educati on has several purposes, the measures of success that can be applied to it vary, and that indi rectly, legislation and funding depend on which purpose is being reviewed in determining the value added (p. 239). Instruction, programs, and delivery systems (C 4) demonstrated significance, which recognizes that in order to create substantiv e change, the standard methods of delivering work-related education may not address the developing needs of learners. OBanion described these needs as developmental task s and how learning kindles new ways of seeing, thinking, and doing--in dramatic even ts and incrementally in day-to-day experiences (p. 48). In addition, this co mponent focused on participants responding with strong agreement that work-related e ducation should be on an equal footing with regular college programming. The participants also responded that the most effective instructional approaches are those which focu s on learning skills with broad application to several similar occupations while ensuring that the needs of business are satisfied, as presented in Table 4-16. Meri sotis and Wolanin (2000) noted that the key question for community colleges is how to strike a balanc e between direct worker training efforts and general education programs (para. 2). Cohe n and Brawer (2003) not ed that there can be no reversing the perception that one of the colleges pr imary functions is to train workers (p. 420). The significance on the coordination and pl anning component (C 6) assesses the community colleges sensitivity to satisfyi ng the needs of people, both learners and stakeholders. There was strong agreement acr oss all component aspects (Statements 42 44.1), as shown in Table 4-18. This suppor ted the thinking that community colleges

PAGE 158

148 should be the primary coordinators of work -related education between high schools, the college, and universities (circu lar linkages) to eliminate ga ps in student learning. In addition, there was strong agreement that community colleges should focus on where learners will find work. They agreed th at community colleges should seek to build coalitions and partners hip with other colleges, organizati ons, and business to define roles, establish a vision, and meet planning objectiv es for work-related education. Hamm and Mundhenk (1995) emphasized the concerns of whether or not the thinking and planning of the college was primarily local or regi onal, and they emphasized the potential for regional and national recognition of work-rela ted education through data support (p. 9). Finlay, Niven, and Young (1998) noted the inte rnational trend of other nations workrelated education systems in providing a response to changes in the global economy and the adoption of long-term strategies w ith respect to these changes (p. 3). Again, based on such a relationship with significance found in four out of seven components, it was appropriate how OBanion de scribed this Principl e I of the learning college: This first principle must form the framework for all other activities (p. 49). Principle II-the learning college engages lear ners as full partners in the learning process with learners assuming primar y responsibility for their own choices. This principle was supported by significan ce with two out of the seven components identified with work-related education: Funding Coordination and planning Principle II demonstrated significance between the co mponents pertaining to funding (C 2) and coordination and planning (C 6), as depict ed in Table 4-23. Such a relationship could be explaine d based on the concentration on services that must be

PAGE 159

149 initiated: at the point a learner chooses to engage the learning college (OBanion, 1997, p. 49). This principle addresses serv ices which OBanion described as full assessment of the learners abilities, achie vements, values, needs, goals, expectations, resources, and environmental/situational li mitations (pp. 49-50). In addition, this principle puts the responsibil ity on community colleges to pr ovide orientation or rather the process of engagement in the new l earning environment, based on a variety of formats in such a way that the process meet s the needs of each indi vidual learner (pp. 5051). Funding (C 2) also showed significance with Principle II. It recognized how the viability of the learning college could be de pendent on funding the services that should be taken into consideration and the requirement to fund specialists as an innovation and improvement for work-related education (S tatement 26, mean 2.000, standard deviation 0.000). These specialists were identified by O Banion to monitor the services, provide new technology training, develop learning colla borations, locate learning resources, and navigate the learning system and thus approve a learners readiness to fully engage the learning opportunities provided (p. 50). Th is would support the centrality of workrelated education (Stateme nt 24, mean 2.667, standard deviation 0.617), as displayed in Table 4-13. Again, as found with Principle I, ther e was significance on the coordination and planning component (C 6), which assesses comm unity colleges sensitivity to satisfying the needs of the learners. There was st rong agreement across all component aspects (Statements 42 44.1), as shown in Table 4-18. This supported the thinking that community colleges should be the primary coordinators of wo rk-related education

PAGE 160

150 between high schools, the colleg e, and universities (circular li nkages) to eliminate gaps in student learning. In addition, there was str ong agreement that community colleges should focus on where learners will find work. They agreed that community colleges should seek to build coalitions and partnership with other college s, organizations, and business to define roles, establis h a vision, and meet planning objectives for work-related education. Principle III-the learning college creates and offers as many options for learning as possible. This principle was supported by significance with four out of the seven components identified w ith work-related education: Mission and organization Funding Instruction, programs, and delivery systems Coordination and planning Principle III demonstrated significance w ith the components pertaining to mission and organization (C 1), funding (C 2), instruction, programs, and delivery systems (C 4), and coordination and planning (C 6), as depi cted in Table 4-23. Such a relationship could be explained based on community college programs which, according to OBanion (1997), offered many options for learning regar ding time, place, structure, and methods of delivery (p. 52). The college programs could incorporate the la test in technique, technology, and training materials. OBanion stated that t o manage the activities and progress of thousands of learners engage d in hundreds of learning options at many different times, at many different levels, expert systems were needed. An example based on such developments was the MiamiDade Colleges Synergy Integrator, which was implemented to manage the edu cational enterprise (p. 54).

PAGE 161

151 The significance on mission and organization (C 1) recognized the emphasis that participants placed on claiming the role of work-related education equal to other mission tenants (Statement 22, mean 2.533, standard deviation 0.743). The value of centrally planning and funding work-related education was also identified as an important part of the organization (Statement 22.1, mean 2.733, standard deviation 0.704). In addition, participants supported an integrated model for work-related educa tion with general and transfer education (Stateme nt 22.1, mean 2.133, standard de viation 0.352). Gleazer (1968) and Cohen and Brawer (2003) found that when work-related education was acknowledged as a primary f unction, it required a commitment by a community college that affected every aspect of its operations and could lead to shifts in the pattern of support. Funding (C 2) demonstrated significance, which recognized how the viability of the learning college could be dependent on fundi ng formulas which should be influenced to include the needs of how in structional innovation improve s work-related education (Statement 26, mean 2.000, standard deviation 0.000). Funding also demonstrated that community college funding mechanisms s hould acknowledge the centrality of workrelated education (Stateme nt 24, mean 2.667, standard deviation 0.617), as displayed in Table 4-13. Another component with si gnificance was instruction, programs, and delivery systems (C 4). This component re cognized that the standard methods of delivering work-related education may not include as many options as possible, and that community colleges would need to crea te and offer options that were seamless, trackless, and classless so work-related educat ion would not be operated in isolation

PAGE 162

152 and prevent learners from making reasona ble changes in their programs (Hamm & Mundhenk, 1995, p. 13). OBanion (1997) described programs which should accommodate differences in learning styles, learning rate s, aptitudes, and prior knowledge while maintaining educational quality (p. 52). In addition, this component focu sed on participants responding with strong agreement that work-rel ated education should be an equal footing with regular college programming. This com ponent also found that the most effective instructional approaches are those which focu s on learning skills with broad application to several similar occupations while ensuring that the needs of business are satisfied, as presented in Table 4-16. As found with Principles I and II, there was significance with Principle III to the coordination and pla nning component (C 6), which assesses the community colleges sensitivity to satisfying th e needs of the learners and others. There was strong agreement across all component aspects (Statements 42 to 44.1), as shown in Table 4-18. This supported the thinking that community colleges s hould seek to build coalitions and partnerships . to define roles and a vision for work-related education (Statement 44, mean 2.867, standard deviation 0.352), based on the e xpectations of the learning college which should create a nd offer as many options as possible. Principle IV-the learning college assists lear ners to form an d participate in collaborative learning activities. This principle was supporte d by significance with two out of the seven components identif ied with work-related education: Funding Coordination and planning Principle IV demonstrated significance w ith the components pertaining to funding (C 2) and coordination and pla nning (C 6), as depicted in Ta ble 4-23. Such a relationship

PAGE 163

153 could be explained based on community colle ge programs, which, according to OBanion (1997), required transformation of the traditional institution or community of scholars into a community of learners. The focus on creating learning communities would innovatively and purposefully restructure th e curriculum to link together courses. Learners would then find gr eater coherence in what they are learning as well as increased intellectual interac tion with faculty and fellow students (OBanion, p. 56). The significance on funding (C 2) recogni zed how the viability of the learning college could be dependent on funding formulas which should be influenced to include the needs of how instructi onal innovation improves work-rela ted education (Statement 26, mean 2.000, standard deviation 0.000). Th is would include transformation to learning communities and collaborative learning activities, and that community college funding mechanisms should acknowledge the centrality of work-related education (Statement 24, mean 2.667, standard deviati on 0.617), as displayed in Table 4-13. As found with all the other principles, ther e was significance with Principle IV to the coordination and planning component (C 6) which assesses the community colleges sensitivity to satisfying the needs of the lear ners and others. Ther e was strong agreement across all component as pects (Statements 42 to 44.1), as shown in Table 4-18. In particular, there was strong agreement that community colleges should seek to build coalitions and partnerships . to define roles and a vision for work-related education (Statement 44, mean 2.867, standard deviat ion 0.352). This would include the transformation to a learning college as a vision ary direction for work-related education.

PAGE 164

154 Principle V-the learning college defines the role s of learning facilitators by the needs of the learners. This principle was supported by significance with four out of the seven components identified w ith work-related education: Funding Coordination and planning Instruction, programs, and delivery systems Staffing Principle V also demonstrated signifi cance with the components pertaining to funding (C 2), instruction, programs, and de livery systems (C 4), staffing (C 5) and coordination and planning (C 6), as depicted in Table 4-23. Such a relationship could be explained based on OBanions (1997) descri ption that everyone employed in the learning college will be a learning facili tator. This description suggests the comprehensive nature of this principle. The significance of this principle with f unding (C 2) recognized how the viability of the learning college could be dependent on funding contracts with many learning specialists and educators of the future labeled learning consultants. This principle could be supported by funding formulas that could be influenced by such innovation and improvement for work-related educa tion (Statement 26, mean 2.000, standard deviation 0.000). These learni ng specialists were identified by OBanion (1997) to produce specific products or deliver specific se rvices. These services ranged from needs assessment to learning options creation, and fr om creating the colla borative networks to establishing learning in th e workplace (p. 59). The significance on instruction, programs, and delivery systems (C 4) recognized that work-related education should be asse ssed in terms of inst ructional approaches, college programming, and improvement proce sses. OBanion (1997) put the ownership

PAGE 165

155 on everyone employed in the learning college. He took a clearly innovative approach to new roles because learning faci litators would best describe th e educator of the future as mentors, facilitators of inquiry, archite cts of connection, man agers of collaboration and integration, as well as learners themselves participating as lab assi stants or tutors to assist other learners (pp. 59-60). Finally, the significance on the staffing com ponent with Principle V was a natural relationship, considering the explicit focus on staff and faculty roles in this principle. The panel of experts expressed general agreement and endorsed this component subsection with all averag es between Slightly Agr ee and Strongly Agree, as presented in Table 4-17. That work-related education faculty and staff need real workplace experience to communicate effectiv ely with students was the statement of strongest agreement (mean 2.600, standard deviation 0.737). The participants recognized the value and multi-role aspects of employing specialists and learning consultants, as designated in Principle V. Cohen and Brawer (2003) posed that program success could be subject to how community co lleges appoint the program coordinators, and how these colleges compose those adviso ry committees responsible for work-related education programs (p. 233). There was significance on the coordinati on and planning component (C 6), which assesses the community colleges sensitivity to satisfying the needs of people--both learners and stakeholders. There was st rong agreement across a ll component aspects (Statements 42 44.1), as shown in Table 4-18. The all-embracing goal of this principle could be supported by community colleges seek ing to build coaliti ons and partnerships with other colleges, organizat ions, and business to define roles and a vision for work-

PAGE 166

156 related education (Statement 44, mean 2.867, standard deviation 0.352). Again, based on such a relationship with high significance f ound in four out of the seven components, it was appropriate how O’Banion (1997) describe Principle V of the learning college: “The goal is to have every employed person thinking about how his or her work facilitates the learning process” (p. 58). Principle VI-the learning college and its learni ng facilitators succeed only when improved and expanded learning can be documented for its learners. This principle was supported by significant correlations with tw o out of the seven components identified with work-related education: Funding Needs assessment and documenting college success Principle VI demonstrated significance w ith the components pertaining to funding (C 2) and needs assessment and documenting co llege success (C 3), as depicted in Table 4-23. Such a relationship in support of work-related education could be based on O’Banion’s (1997) goal of documenting “what learners know and what they can do and to use this information as a primary measur e of success for the learning facilitators and the learning college” (p. 60). This principle could be s upported by funding formulas that could be influenced by such “innovation” and improvement for wo rk-related education (Statement 26, mean 2.000, standard deviation 0.000). The significanc e on funding (C 2) recognized how the viability of the learning college could be dependent on whether or not “funding formulas should be influenced to include the needs of the emerging workforce on state, regional, national, and global basis” (Statement 25, mean 2.733, standard deviation 0.458), as displayed in Table 4-13.

PAGE 167

157 The significance on the needs assessme nt and documenting college success component presented a natural relationship, if not explicit focus, on documenting success. Statement 29.2 (mean 2.071, standard deviat ion 0.616) said that “success should be measured by each individual student’s educa tional attainment/skill acquisition--including those who complete one class and those w ho do not complete a cer tificate or degree,” which integrates well with O’Banion’s (1997) viewpoint that “learners will be encouraged to add competencies and goals be yond those established in the standards” (p. 60). Again, based on the relationships with significance found with these two components, it was appropriate how O’Banion described Principle VI of the learning college as: “well-designed to support the goals and structures of the learning college” (p. 61). Commonality of Components across Principles The commonality of certain components was conveyed through the analysis of the data (Table 4-13 and Table 418) and then linked to the conclusions presented in the previous section, which detail ed the prototype m odel of work-related education drawn from the research. Specifically, the funding component and the coordination and planning component were found to have signif icance across the majo rity of principles (Table 4-23). Funding was shown to be statistically significant at the 0.05 level across all principles. The participants fully endorse d the funding component with means at or above “Agree” to “Strongly Agree” and with standard deviations ranging from zero standard deviation to 0.617. These high leve ls of agreement and consensus recognized the commonality of the funding component across all principles indicating that funding is a critical issue to deve loping a common definition fo r work-related education.

PAGE 168

158 Recognizing the impact of the funding issue at community colleges, Cohen and Brawer (2003) forecasted that the “form of the commun ity college will not change . all current services will continue to be provided, with growth or shifting emphases depending on funding and different population bases” (pp. 404-405). Hamm and Mundhenk (1995) stated that community colleges “need to assess the viability of their workforce development efforts in the context of fundi ng priorities . may want to consider proposing different approaches to funding, t hough they should do so in a coordinated, regional way” (p. 6). Honeyman and Bruhn (1996) noted that “outcome measurements assumed a new importance with the growi ng public demand for accountability” in the 1990s (p. 27). Cohen and Brawer (2003) fo recasted that work-related education will remain prominent while still recognizing that “more than in any othe r area, the specter of institutional accountability looms over the occupational programs” (p. 420). Proposing different approaches to funding indicat es that new, non-traditional outcome measurements on a regional and national basi s could carry forward a compelling case to policymakers and the general public for deve loping a common definition for work-related education. An issue with propos ing different approaches to funding rests with “challenge of defining priorities among a potentially infinite set of individual training and educational agendas (Palmer, 1996, p. 194). Such a challenge could be effectively answered by a model and common definition fo r work-related education that is widely understood and universally accepted as an instit utional purpose of co mmunity colleges. Coordination and planning was shown to be statistically signifi cant at the 0.05 level across five of six principles (Table 4-22) and was correlated to a value of 0.47 with Principle VI which was statisti cally significant with increased tolerance at the 0.10 level.

PAGE 169

159 The participants fully endor sed the coordination and pla nning component with means upwards or nearly at “Strongly Agree” and with standard deviations ranging from 0.258 1.056 standard deviation. These high levels of agreement and consensus recognized the commonality of the coordination and planning component across all pr inciples indicating that coordination and planning are also criti cal issues to devel oping a common definition for work-related education. Cohen and Brawer (2003) forecasted that “the trend toward greater state-level coordination will continue at a slow pace” (p. 413). Ashworth (1972) stated that “public appreciation for general as well as specialized education is also necessary” and that the general public can not be neglected nor ignored by institutions of higher education: Another area worthy of study is how th e higher education co mmunity can gain increasing support from society. If gove rnment ultimately is directed and controlled by the people, their understandi ng of the needs and prerequisites of higher education would be the best protecti on against government interference . . continued public support cons titutes a safeguard and a re source which our colleges and universities dare not permit to wane. (p. 137) Hamm and Mundhenk (1995) stated that in mo st cases, work-relate d education “issues and demands tend to stretch beyond sponsorship borders.” They advocated that workrelated education “must take into account regional and national trends” which acknowledged that learners may find work beyond their respective community college districts to other parts of the nation or even globally. They also advocated that community colleges must “be involved in ma king both regional and national cases for their role” to initiate and accomplish regi onal and national planning collectively for work-related education (p. 9). Hamm and Mundhenk (1995) also beget the political overtones of this component in that “colleges ar e expected to be sens itive to satisfying the demands of the district and their governing boards (p. 9). Vaughn (1994) stated that, to

PAGE 170

160 be effective, community college leaders must focus on establishing political leadership and that “the president ensures that the coll ege’s mission moves in concert with the goals of the community, the state, and when appropr iate, the nation (p. 73). Eaton (1994) noted that “community college presidents have a great responsibility in influencing public policy . . Presidents need to take risks in planning for the future in the areas of public policy and college governance” (p. 136). Community colleges cooperating with one another as well as building coalitions and partnerships with business/industry, policym akers, and the genera l public could set in motion a model and common definition for wo rk-related education that is widely understood and universa lly accepted. Suggestions for Further Research This study did not surface previous resear ch findings where the specific issue of pursuing a common definition for work-related education was addressed. The literature showed consistent evidence that the nation ha s in the past century turned to community colleges for solutions to its workforce pr oblems and shortfalls. Several community college leaders and higher education research ers have echoed the dilemma and pitfalls of not having a common definition for work-relate d education. They indicate a lack of clarity and consistency in policymaking, partic ularly funding decisions, at the federal, state, and local levels. This researcher did not find any dedi cated studies which addressed this specific issue, and also did not find the potential in pursuing a common definition model for work-related education at community colleges. The results of this study suggest several areas for fu rther research to facilitate a consolidated position on and a common definition for work-related educat ion at community colleges. These areas include:

PAGE 171

161 Carry forward the Round Three survey instrument with a larger sample by approximating five times the number of participants/respondents as there are statements, for example, 60 statements a nd 300 participants, for greater statistical power. Replicate this study with additional stakehol ders in areas that are both internal and external to the community college setting, including internal constituents, such as other administrative and managerial sta ff, that is, vice presidents, deans and directors. Also, use external stakeh olders, such as community members and businesses, university partners, and wo rk-related education student groups. Replicate this study with state and fede ral government agencies responsible for work-related education administ ration, policymaking, and funding. Replicate this study with other community college associations or organizations on a national level, including affiliate councils of the American Association of Community Colleges, such as the Nati onal Council for Workforce Education, to further the body of knowledge from a broader geographical perspective. Examine the principles and components with a future perspective to facilitate the priority, focus, and applicability of new work-related education programs in the next decade. Implications for Community College Leadership Community college leaders can better meet expectations and further the community college role as Hamm and Mundhenk (1995) de scribed the community college--“preparer of the nation’s workforce”--by investing in furt her research, such as is presented in this study and offered in the prototype model. The conclusions of this study may help to more effectively present what is salient to all or most of work-re lated education including principles of the learning college” an d components identifie d through the Delphi technique--in particular the commonality of components pertaining to funding and coordination and planning. While it is not asse rted in the model that every principle was equally critical to work-related educati on, community college leaders could derive-through a thoughtful self-analysis of and consol idated response to each principle and the related components--a common definition of work-related education within a national

PAGE 172

162 context. Community college leaders should gi ve due diligence to th e value in pursuing a common definition for work-related education and a consolidated front for community colleges to uphold their role as “pre parer of the nation’s workforce.” Implications for Policymakers The conclusions of this study affirm that a common definition for work-related education is feasible by identifying, ra nking, and modeling th e principles and components specific to work-related education. The implications for policymakers are to recognize the impact such a common definition would have if implemented nationwide. The history of work-related e ducation at community colleges in the United States has shown that discourse in how work-related e ducation policy is made and how funding is distributed has a fragmented impact on how the work-related educa tion is conducted. Policymakers can ultimately improve overall support of work-related education at community colleges through a favorable r eception of a common definition for workrelated education and suppor t of a model which addr esses the commonality of components as conveyed in this study. The im plications for policym akers are that such areas of commonality indicate trends for cha nge for how policy is developed pertaining to funding issues as well as overall coordination and pla nning. Policymakers should consider these two areas of commonality as touch points to justify funding to community colleges and for pursuing a common definition for work-related education which is based on coordination and planning at their specific legislative levels. In addition, “demonstrated flexibility in adapting to social and economic challenges” by community colleges was a consis tent message in the literature. For these most obvious reasons, this “demonstrated flex ibility” was construed as an asset. However, such “flexibility” can also be a de triment if it is used by policymakers as an

PAGE 173

163 agent for change based solely on the polit ics of the day or upon changes in party dominance. The ramifications of inconsiste nt policy pertaining to funding as well as coordination and planning woul d distract, degrade, and frag ment any direction towards a nationwide understanding and the value of a common definition for work-related education The overall implication for policymakers is that, as long as work-related education can be subject to changes in terminol ogy, definition, funding methodology, and planning and coordination efforts, community colleges will be restricted from fulfilling their potential as the “preparers of the na tion’s workforce.” By supporting a common definition for work-related education, policymak ers will have satisfied their obligation of service to students – learners, community co lleges, and taxpayers alike in a fair and consistent manner across the nation.

PAGE 174

164 APPENDIX QUALITATIVE RESPONSES FOR ALL THREE ROUNDS The qualitative responses were collected through use of open-ended comment boxes on the web-based survey tool. The quali tative responses data were simultaneously downloaded into a pop-up window along with th e survey statement data and saved as a text file. The qualitative data in the text file were exported from Microsoft Excel to Microsoft Word for aggregation and anal ysis. Additionally in Round One, one participant electronically mailed expanded reflections regardin g the principles, components, and perceptions. These comment s were included in the qualitative data results. Open-ended comments and opinions were in terpreted as subjective information, which had characteristics relevant to the res earch questions. This subjective information was developed and aggregated as revised and additional survey items for new statements in the subsequent rounds. Round One Qualitative Data Provided by Participants List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) pertaining to Principle I: I believe that since work-related education is so central to the immediate desire of students to achieve and grow that there are even more "aha" moments than in required general education. The fact that learning is in "context" makes it more meaningful for the student. Any kind of learning should accomplish new ways of seeing, thinking, and doing. Isn't that what le arning is all about?

PAGE 175

165 The wording in item 2 and 3 and the reversing of the scale may incorrectly send the message that they are mutually exclusive or antithetic when, in truth, they can both occur. Studies of how learning takes pl ace clearly document that learning can both be "all or none" and "s uccessive approximation." Most work-related education requires subs tantial "hands on" learning. Actual experience is an important part of a h ealthy learning environment. Students are also more likely to work in teams and in teract with mentors and each other in work related settings. At our institution--learning is learning is learning and we work hard to treat all learning and all learners as priority. Work-related or real world expe rience is vital to success. Significant positive outcomes should be the goal of work-related education. Substantive change may include increased creativity, improved critical thinking and enhanced academic and technical skills. One should be looking for long term growth potential not just quick hit skill sets or material learned by rote. Experience is always an asset and helps to link theory to practice. 1 & 2. “Substantive change” and “dramatic first events and new discoveries” are learner specific and should not be the same for each student. For some learners, [I] strongly agree that in a good course th is will happen and for others I would disagree that this should be an expect ed outcome depending on where they are in life and personal experiences. It is in work-related education that the hi ghest level of learning occurs. It is in "doing" that all the pieces of inform ation often come t ogether. Work-related education also provides a solid method of evaluation to ensure there is understanding of the content. Shouldn't all education accomplish these ac tivities too? Whether the content/course is “work-related” shouldn't change these core student growth goals. If Agree or Strongly Agree that students should part icipate in a structured induction/orientation process, what form at and time frame should be offered? Flexible time frames to meet the needs of working students. Format should be participative, using the life experiences of students. depends on purpose and content.

PAGE 176

166 Most work related programs require an ini tial orientation and safety training before a student can begin to work in a lab or clinical setting. Is there evidence that orientation makes a difference? In either of two forms, a pre-program orie ntation or a concurre nt orientation with courses. Mentors should be a part of the experience. Early in the semester or quarter with continued support available. I think it depends on the nature, format a nd extent of the work-related education. As soon and as possible and we have found on-line is good. This needs to be very flexible. On-line c ould work and it may require an instructor. Face-to-face or online as an introduction (immediately prior to the learning experience). Participatory format over a couple different time frames so student can digest info and then discuss later. Optional workshop after wo rk hours on work site. Three hours. “Require students” Strongly agree that so me need an induction or orientation while others do not--or if they do it would be of a different t ype therefore disagree. All dependent on assessment, prior knowledge /skills/ability, and educational goal. Paper orientation as well as an on-site orientation. Variable requirement based upon the depth/content/format of the program. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) pertaining to Principle II: The nature of the current job market requires that workers mange their career development. This process should start with and be embedded in the teaching and learning process. Proactive decision-m aking about careers should begin in the career preparation phase. I have questions about whether "full" part ner is the right choi ce of word. Perhaps "active" partner?

PAGE 177

167 Given that most community college admini strators, faculty and students are most familiar with the role relationships play ed by each under the "p edagogical model of learning," there must be a thoughtful and thorough set of guidelines promulgated as a "Learning Contract" when following an “andragogical model of learning." Work-related education programs typically engage students in work settings or laboratories that simulate work settings. This allows the student to experience firsthand the work environment and to ma ke good choices about their learning. Mandatory induction and negotiated contra cts may restrict student choice and interest. Students should have personal le arning plans, but educ ation is not a labor union. The answers to the questions above really depend on who the client/sponsor is for the work-related learning. If it is an individual learner, then certainly he/she should be an active partner th roughout the process. If, on the other hand, the client/sponsor is the employer, then there are two levels of "partner engagement" and they are quite different from the former. If the employer is sponsoring the acquisition of mission critical skills and competencies, industry certification, etc., then our first obligation as a learning solutions provider is to meet the employer's requirements, and then in that context, engage the individual employees/learners as partners in the learning process to m eet the client/sponsor 's objectives and requirements. Hopefully, the objectives are not mutually exclusive between the client/employer and the employ ees/learners so as to create a conflict and thus result in an unsatisfactory situ ation for all concerned. Students need to be engaged in the learning process. A provision that the contract may be modified so that students may make reasonable changes is helpful. Student must be a partner in the design a nd the activity of the learning experience, but [it] must be recognized that the pr ocess needs to provide for the faculty to impart their expertise in a positive manner. Knowledge base of each--student and faculty can't be compromised. Students need a connection to the work re lated education and a learning plan would assist in this effort. Sometimes in work-related education, the objectives are the same for all students. While interests and needs can be addre ssed, all students typically have similar course goals, objectives, and competencies. Similar to my prior statement, these recommendations should apply to almost all learning activities.

PAGE 178

168 If Agree or Strongly Agree, what options should be offered, e.g. portable modules, learning communities, stand-alone expert system s that respond to learner idiosyncrasies, others already established, or others yet to be designed? I believe that assessment of prio r learning, credit through competency demonstration and other methods of assessi ng skills should be an integral part of the learning and credentiali ng process. Distance learni ng, hybrid (classroom and on-line) courses and other instruction desi gned so that students can learn at their own pace is critical. All of the above. In addition to those cited above, there s hould also be an option where the faculty member provides the individual learner with recommendations for sources and resources based on the learner's desired goals. Open entry/open exit labs help to accomm odate individual differences in prior knowledge and rates of learning. Most co mmunity college faculty are aware of differences in learning styles and ca n accommodate individual student needs. Options that are best practices and have tested models and research behind them. All of the above plus clusters, accelerate d cohort modes, distance learning, etc. Every option should be utilized. Elearning, mentoring, interaction, etc. Prior knowledge may be assessed so th at the advanced student's learning experience will be enhanced, not duplicative. Learning communities benefit both the advanced and regular student. As many options as possible should be offe red, but it must be done in a manner that it doesn't drive upward the cost of education beyond cost goal. 8, 9, 10 & 11. Full array of options, seamless, trackless, classless--agree for some disciplines disagree for others. All de pends on nature of the program. Some disciplines are focused while others less so. There is, and needs to be, a big difference between allied health programs and others like agriculture or computer science. Self-paced options; chunking of the curriculum; small group interaction. Options could be as simple as the wo rk-related location and learning focus. Additional technology could also be used to accommodate individual differences.

PAGE 179

169 In addition to these noted potential deliv ery modalities, differential and flexible funding processes need to follow whatever fl exible educational delivery options are implemented to support the institution's initiatives. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) pertaining to Principle III: Related to item 11, if programs are truly co mpetency based then this can easily be accomplished if institutions are truly con cerned for students. By using a cluster concept item 10 could be accomplished. We can't allow the curriculum to be "owned" by a narrow program/department/occupation. Our systems need to change so that we are looking at it from the student's point of view not from our own sense of convenience. This principle will require greater flexibil ity than most public institutions will be allowed by state agency and/or legislature. Michigan community colleges have been good about coordinating various workrelated programs. The foundation courses for many work related programs are the same or similar. Only trackless options where it ma kes sense within certain fields. I've answered these questions from a cor porate client service point of view, not from the perspective of individual students pursuing career-entry re lated programs. Michael, a suggestion: it would have b een helpful to include your description of the context for this series of questions e.g. corporate/workpl ace based training or post-secondary programs or other. Having multiple options is missing in most programs. Seamless, trackless and classless options are essential to student matriculation in an ever changing environment. The matter of standards and institutional repu tation still are important, so that while a program can be built for each student's needs, they cannot set the standard which can reduce to the lowest denominator if there is too much flexibility. The merits of work-related options should be evident so discussion can occur and the linkage identified. Often work-related experiences built fr om simple to complex. Therefore, a trackless system would not work. There ar e often differences in programs and the classless concept is not in place. It is difficult for programs to all be similar. As above, if “work-related” is dele ted, the statement is still true.

PAGE 180

170 List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) pertaining to Principle IV: Linking the training to the workplace is critical, whether by electronic means or through hybrid credit/customized training programs. Using assessment in the workplace either as part of a course of instruction or as assessment of prior learning is an important piece of assuring the re levancy of the instructional program and creating a seamless environment between the "classroom" and the workplace. Research shows that collaborative learning results in increased learning Nothing to add. Electronic media has provided increa sed opportunities for many students. Learning communities are only one method. Connections need to be made between the training program and the workplace. Assessment is needed, we to be willing to take this step to evaluate the learning success. These are all great ideas, but can an ins titution afford the many options and is it feasible with faculty workload. There will have to be a balance. Community college[s] should take a lead ership role in th is experience and connecting to the community. “Should focus on creating communities” implie s the exclusion of individual work. Depending on the nature of the discipline, work-related curriculum should focus on successfully working in communities or individually depending on what the workrelated situation demands. Agree that communities should be formed but disagree that the technology solutions suggested in the question are the only, best, or needed way to do that for all disciplines. “Assessment services in the workplace”: some areas yes, others no--or only if requested. Sometimes in a work-related setting, the electronic format is not needed. The hands-on focus supercedes the technological format. Providing workplace assessment services requires support, operational and financial, from the external organi zation. Not routinely easy to obtain.

PAGE 181

171 List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) pertaining to Principle V: Department and course needs should align wi th what the learner needs and what the industry needs but often they do not. Wh at learners need to know and what industry defines as essential skills should drive where resources and personnel are applied. Work-related students can and should be used to assist others learning as well as to enhance their own skills not to reduce personnel costs. I dislike the word “facilitator” in this cont ext but have not been able to figure out a better term! Nothing to add. Students should participate as facilitators who can be of assistance. Otherwise the skills gap could worsen if untrained st udents are put into a facilitator role. Principle V should be reconsidered in li ght of research s howing distinctions between experts and novices, pa rticularly in the workplace. The word "facilitators" fails to capture the role of the expert with regard to a novice. This relationship is more than facilitation. Facilitated learning is the best way to engage students. The student's role as learning facilitator will be one of their most valuable learning experiences. Personnel must have expertise, it's not the student revolution of the 1970s where learners dictate what th ey should know and learn. 15 & 16. Implies one or the other. S uggest faculty should be course content experts based on department needs that can adjust to individual learner needs that change over time. Work-related education student s, if in a learning role should have the needed mentorship and not be expected to replace other personnel. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) pertaining to Principle VI: Competency attainment, as defined by industry skill standards, should be the criteria for exit, although there must be multiple exit points based on sequences or clusters of competencies. Certifications where available should be the method of demonstrating competencies. Certificati ons and licenses, wh ere available should

PAGE 182

172 be the primary documentation of lear ning but portfolios are the preferred mechanism for presenting documented learning. I don't think we should land on a specific assessment method. The assessment method should be tied to what is ap propriate for the content and the pedagogy. Nothing to add. Paying to establish standards should not burden the community colleges. Industry should partner with community colleges to establish standards. Re #21 industry certification systems or standards are also used as benchmarks. Again there are so many types of work-rela ted learning it is hard to answer these questions categorically. Having work-related educational competen cies for completing the program should be a requirement. Assessment is very complicated--it is needed, but one “glove" will not fit all. Standards need to be determined but instit utions may be able to do so on their own. Multiple measures should be the primary means of documenting learning and portfolios can be an im portant part of those measures in most disciplines but not all. While portfolio assessment is excellent, there are other means to assess workrelated education. Standards can be set by faculty as opposed to employing specialists. While 'portfolio' is a potential wide-rangi ng concept, defining it relative to each program/institution and career field will be very difficult for diverse work environments. Community colleges recei ve minimum resources to accomplish incoming assessment of student readiness to learn, etc. Additional assessments would be valuable but time-consuming for the student and expensive for the college. Many adult learners would resist the additional assessment components inasmuch as they are not seeking degrees, just focused coursework. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) concerning mission and organization: General and transfer education and wo rk-related education should support each other rather than compete for institutional resources. All education is, ultimately, work-related education. Nothing to add.

PAGE 183

173 Work-related education has always been a hallmark of the comprehensive community college. A mission should remain broad. Vision a nd values can capture work-related education. Also work-related educa tion should include an entrepreneurial orientation which can be diminished if "centrally planned and funded." If work-related education is not included in the mission it will never be integrated in the organization. There is a place for work-related educati on in the CC mission, but it should not be the driver of the mission--it is one element. Work-related education is one means of e ducation, but other mean s may be just as valid. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) concerning funding: I'm not sure how you measure or admini ster funding based on instructional innovation, at least through a formula. F unding formulas should take into account the critical mission that community colleges play in our economy. Nothing to add. Funding needs to be linked to work-related education. Again, CCs are not all alike and they serve different needs and populations. Many are primarily transfer institutions. Ther e goals are different. Funding should be driven by college mission. It's about learning which may or not incl ude innovation--certain ly not for its own sake just as it improves lear ning in necessary areas of study. Need change very rapidly; a single focus formula would not be flexible. "Emerging workforce" is only one compone nt of a community college audience. "Transitioning" and "incumbent" workers ar e also vital and an immediate priority. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) concerning needs asse ssment and documenting success: Certificates and degrees are only one indicator of the e ffectiveness of work-related education in an institution. Customer satisfaction, empl oyment rates, retention

PAGE 184

174 rates and increased wages are also keys. Measuring accountability for work-related education is a very complex process that we have not spent enough time on. 28 Depends on the intent of the student. Especially in work-related education, certificates and degree completion are not necessarily the best measures. 29I don't know that we generate jobs. We help attract jobs because we assist in the development of a trained workforce (the #1 reason why businesses locate in a given area) but it's the businesses that generate the jobs. There is also a "value-added" success m easure when the learner acquires his/her desired learning even when that may be only one course. Industry is responsible for generating jobs, upgrading employees and communities with business are responsible for economic development as is the state. Question 29 is a very bad item. For question 27, economists who are prominent in monitoring local labor markets and researchers in higher education, including community colleges, should be the experts. Commun ity colleges cannot do this by themselves. #28 relevance and responsiveness to indus try needs would be a more relevant measure. Colleges can rely upon experts from othe r organizations; they don't need to replicate expertise available in other spheres. Our success is when our students are successful. Measures of success should consider a student's educational attainment, skill acquisition and employment even if cer tificate or degree is not completed. #28 & 29 are not clear. Success of what--t he college, the work-related program. The work related activity/program will be just a part of the college. Success for it should be measurement of predetermined goals. All of the above me rit consideration. #27. Colleges need to be active partners with others in the local community in collecting data and looking at trends and experts in translating that information into effective educational plans. #28. Success should be measured by multiple means, including those listed but those exclusively. Some students in programs are recruited prior to graduation because of the labor needs. They may not complete the progr am, but they have a good job (which met their goal).

PAGE 185

175 Success is a multi-layered concept. MANY/mo st adult participan ts are not enrolled in degree or formal lengthy certificate programs, but need/want short, very short, assistance. Success needs to be measured at the user/student level, not at the program completion level. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) concerning instruction, programs, and delivery systems: There can be no either/or choices relative to entry level vs. advanced skills or single occupation vs. cluster. So me populations need to become quickly engaged in the workplace through basic entry-level skills, but we then need to have a pathway for them to advance and grow. Treating wo rk-related education as the add-on and traditional credit as the funded core will prob ably lead to weakness in both. Credit is one form of credentialing. There are ot hers. It is counterpr oductive to view this as a dichotomy. Work-related education programs should re flect the needs of businesses in the community and should not be limited to any specific level of skill development Nothing to add. The relationship between work-related edu cation and regular college programming often depends on the community the college serves. Regarding item 35; If business and industr y needs learning customized to their company it can be part of a revenue genera tion center. This t ype of learning is aligned with the business goals a nd objective for their organization. Work-related brings into consideration the real world. Balance should be considered. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) concerning staffing: Staffing patterns and budgeted positions s hould be based on the needs of students, not the traditional political pecking or der within the institution. Advisory committees should provide real input to faculty but they often don't. Faculty can influence the selection and i nput of advisory council me mbers to support their point of view. There are multiple sources of real world input. Student services should be equivalent but not necessarily delivered in the same way to work-related students. Nothing to add. #38 also need market scanning on a regular basis to ensure curriculum relevance and to understand how the workplace is ch anging in terms of processes, etc.

PAGE 186

176 Work-related education faculty and sta ff need real workplace experience to communicate effectively with students. This statement is one of the most important issues for work-related education. Work-related [education] should be cons idered permanent and support and funded as same. The questions indicate that work-related is frequently not 'credit' programming. Untrue. Work-related does not preclude tr aditional cred it recognition and valuation. List your reactions, initial t houghts, comments, and/or any recommended changes to any statement(s) concerning c oordination and planning: Building partnerships with business, ec onomic development and other educational institutions are the key to effectiv ely addressing workforce needs locally, regionally, and globally. Collaboration and partnership are essential in these times of constrained resources. Nothing to add. Community colleges should be externally focused. Partnerships with business and industry that help with providing real world education are the most critical issue for success. These are all important to quality work-related educa tion program[s]. Coalitions help augment funding and provide a different perspective. In these times of decreased funding, it is very important for community colleges to work together to meet the needs of the state. Please list any other principles, components, changes in service delivery, and innovative "ways of thinking" which you believe would c ontribute to a common definition for workrelated education. We need to focus on integrating general a nd transfer education with work related education and integrating credit and noncredit work-related ed ucation. Our focus should be more on creating an integrated m odel than defining what is and what is not workforce education. If we can accomp lish this then there will be no need for there to be winners and losers within th e institutions based on academic vs. career or credit vs. non-credit.

PAGE 187

177 Given the demographic and socioecono mic changes taking place in the United States, community colleges must develop e qual partnerships with the private sector and non-profit organizations to provide f unding resources that permit work-related education students to pursue their le arning without financial restraints on tuition/books. See attached. As you can see from my att ached responses I am not one for lumping all of work-related academic programs into single or simple uniform definitions. I believe the greatest need is to have a common set of definitions for each of the multiple roles that community colleges provide in work-related programs. National proclamations don't do much for us. #45 the results of college work in this aspect should demonstrate value-add to industry. We just need to get people to recognize th at in this age of th e Knowledge Worker learning is learning is learning. Community colleges will vary across the count ry. I believe it will be difficult to have a national database that adds value. The community college should assume the primary role in c oordinating curricular linkages (in work-related education) be tween high schools, community colleges and universities to eliminat e duplication and gaps in learning experiences for students. It is important to rec ognize work-related education programs will compete with other educational programs within the inst itution for resources. Any attempt to promote the learning college as an end-al l or work-related education as primary will be a disservice to the overall mission of the CC. These programs must be part of the overall programming in the CC. Work related experiences position student s for the real world. Theories and experience help them to succeed. Many of the questions are worded in a wa y that is reductionism in nature (the course should XY, or the role of is AB) as opposed to a more inclusive ”one of the essential roles of the course should be to XY something” compound answer. Work-related education at a community college is so diverse that it [is] difficult to answer many of your questions without qualifying each answer. For example, I may strongly disagree with something if I were answering for mature degree holding students enrolled in a course fo r skill enhancement while strongly agreeing with the same question if it were to appl y to a young first time college student.

PAGE 188

178 Similar, some programs need to be very skill based and narrowly focused while in other programs are a balance between focused skill development and broader based (almost GE like) in focus. #42. This is the historic debate on the definition of “community” in “community” college. Should a community college ex ist to meet the needs of the local community within the local community or prepare learners within the local community to go out into the bigger worl d? The answer depends on the local community and the nature of the college mission and funding source. As a result the answer is very campus centered and indi vidual program related. If one college has the only program in the state teaching X then the answer is different for that program than for one where multiple options exist to get the same program. Perhaps the majority of community colle ge programming is “work-related,” and much of the liberal arts enrollment is driven by work-related student enrollment to complete degrees and certificates. Comm unity colleges are broad in focus and format, diverse in size and location. Attempting to make all into one will be counterproductive due to differential funding and operational realities. Perhaps the dialogue should be centered on not “work -relatedness” but “meeting the needs of our student/community/region” to enhance our economic competitiveness and individual success. Round Two Qualitative Data Provided by Participants Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging group view pertaining to Pr inciple I (please identify by statement #): Although 2 and 3 are not mutually exclusive, I believe that workforce education is somewhat more incremental rather than sudden and dramatic. No change from my last response. I strongly believe that experiential learni ng is essential and th at's why I answered “strongly agree” for item #3. Please provide your underlying reasons fo r any statement(s) with which you may take exception with the converging group vi ew pertaining to Pr inciple II (please identify by statement #): Work related education should be tailor ed, based on the needs and prior learning experiences. I'm not sure what was meant by "negotiated contract" in 7.

PAGE 189

179 Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging gr oup view pertaining to Prin ciple III (please identify by statement #): #11. This is not an either/or proposition. Transferability will depend on articulated competencies and associated courses or internships. #11. Some work-related programs need to have the structure of coursework and its passage for accreditation. #10. While there may be some common c ourses between programs, many workrelated programs have specific courses and there would be limits in the number of common beginning courses. #9. While seamless programs would be a wonderful goal, it is probably not reasonable for many programs to be highly seamless. If by “classless” you mean less or no relian ce on seat time as a measure in favor of competencies, then I strongly agree. Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging gr oup view pertaining to Prin ciple IV (please identify by statement #): If coursework in the workplace is to be part of the students’ performance either through prior learning assessment or part of a formal educational program, then we must assess its relevance. #14.1. Establishing learning communities in the workplace is not easy or inexpensive. Nice concept but pr obability of success is low. Please provide your underlying reasons fo r any statement(s) with which you may take exception with the converging group view pertaining to Principle V (please identify by statement #): #16. Work-related education personnel should be hired based on their pedagogical content knowledge as well as the needs of learners. WR students usually end up as informal learning facilitators for their fellow students but individual schedules may limit how formal this arrangement could be.

PAGE 190

180 Student participation as facilitators is a valuable learni ng technique, but is questionable that they can be used to the extent of reducing personnel costs or responsibilities due to professional responsibilities and requirements. #17.2. Students should help facilitate learning but for other reasons #17.2. The reason for having students involved as learning facilitators should not be based on reduction of cost and/or free faculty time, rather it should be based on the concept of team-building skills. Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging gr oup view pertaining to Prin ciple VI (please identify by statement #): The assessment method used should depend on its validity and reliability. Portfolio may be one of many assessment methods to be considered. Portfolios are great and help document le arning but credentials, particularly competency based certifications and licenses should be primary. #20 and 21. I don't think there is one right way. Multiple methods to meet multiple needs. Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging group view pertaining to concerning mission and organization (please identify by statement #): #22. A mission statement should be broad in scope that encompasses various means of education delivery. Specific mention work-related education should be addressed in the institution's values or goals. #23.1 is two opposing questions in one quest ion. I agree with one and disagree with the other. Please separate so we can render our opinions. WR education should have an entrepreneur ial aspect but should not be totally autonomous from the rest of the institu tion. I consider WR education to be everything from regularly sc heduled credit programs to cutting edge consulting and training. #23.1. compound question. #23 is a bad question, merging autonomy and en trepreneurism [sic]. It should have the entrepreneurial perspective, but it should not be autonomous.

PAGE 191

181 Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging group view pertaining to fu nding (please identify by statement #): None Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging group view pertaining to needs assessment and documenting success (please identify by statement #): #27. It is doubtful that community college s have the faculty or staff with the intellectual expertise to be experts with m onitoring labor markets. There are others who do this as their sole res ponsibility such as the state or federal Bureaus of Labor Statistics. Success should be measured in multiple ways. Program completion should not be the only or even primary means. #27.1. The need to rely on higher ed re search/economists instead of other research/economists at the state or federal level is unclear. Good data [are] good data where the source and external data can present solid indications of economic and workforce requirements. Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging group view pertaining to instruction, programs, and delivery systems (please identify by statement #): 35. Funding for this type of education needs to be flexible and treating work-related education as a revenue center allows for such flexibility. #35. Some programs will not generate additional revenue, but serve a strong community need. #31 The workforce is changing so rapidly th at we need to educate problem solvers and thinkers to be prepared for the jobs of the future. #32. Many programs do focus on a specific job, but the career will evolve over time and change. The graduates need to be able to keep changing according to the labor needs.

PAGE 192

182 RE 34. We shouldn't look at WR educati on as just another stovepipe in the institution. It should be integrated with other college programming. #35. Primary programs related to work-relate d education should be part of the traditional credit programs. There may be opportunity to add revenue separate from the traditional credit programs, and some colleges may want to take advantage of this additional revenue. Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging group view pertaining to staffing (please identify by statement #): #41 The student placement office should have a broader function to also assist graduates in finding jobs. Please provide your underlying reasons for a ny statement(s) with which you may take exception with the convergi ng group view pertaining to coordination and planning (please identify by statement #): None. Please provide your underlying reasons for a ny statement(s) with which you may take exception with the converging gr oup view pertaining to a wi de-spread vision or national proclamation for work-related educa tion(please identify by statement #): Re. 45. Defining roles also limits roles whic h could be a detrimen t as the needs of the workforce change. Round Three Qualitative Data Provided by Participants #14.1. Community colleges should work with those in the workpl ace to determine if learning communities are appropriate. #6.1. Orientations should be standard conten t to ensure information is received and all students should be able to complete a re quired orientation within a fairly similar amount of time. #20.1. Work-related learning should be docum ented via program competencies, not an outside certification or license.

PAGE 193

183 #21.1. Community colleges can determine st andards via surveys, etc, outside specialists are not always needed. These qualitative comments were helpful in providing direct ion during the three rounds. Based on the quantity and quality of the responses, it was evident that the participants took the research process seriously and regard ed their participation as a commitment toward furthering the body of knowledge in the area of work-related education. They made specific comments, usef ul insights, and a collective focus, which was used to refine the Delphi technique survey instrument over the three rounds.

PAGE 194

184 LIST OF REFERENCES Adler, M., & Sainsbury, R. (1996). Alterna tive approaches to the computerization of social security: Reflections on a Delphi exer cise. In M. Adler, & Ziglio, E. (Eds.), Gazing into the oracle: The Delphi met hod and its application to social policy and public health (pp. 176 -192). London: Kinsley Publishers. Adler, M. & Ziglio, E. (Eds.). (1996). Gazing into the oracle: The Delphi method and its application to social policy and public health. London: Kingsley Publishers. Alfred, R., & Carter, P. (2000). Contradictory colleges: Thriving in an era of continuous change. [Electronic version]. Wash ington, DC: Community College Press. American Association of Comm unity Colleges (AACC). (2004). AACC membership directory 2004 Washington DC: Community College Press. Ashworth, K. H. (1972). Scholars and statesmen: Hi gher education and government policy. San Francisco: Jossey-Bass, Inc. American Psychological Association, Ameri can Educational Research Association, & National Council on Measurement in Edu cation (APA, AERA, & NCME). (1999). Standards for educational and psychological tests Washington, DC: American Educational Research Association. Baker, G. A., III, Dudziak, J., & Tyler, P. (Eds.). (1994). A handbook on the community college in America: Its history, mission, and management. Westport, CT: Greenwood Press. Bohlen, C. H. (2004, March 4). Statement of the American association of community colleges to the senate health, education, labor, & pensions committee on the reauthorization of the higher education act and the nation’s workforce. [Electronic version]. Washington, DC: Community College Press. Bragg, D. D. (Ed.). (2001). The new vocationalism in community colleges. San Francisco: Jossey-Bass. Brand, B. (1998). The process of change in vocational education and training in the United States. In I. Finlay, S. Niven, & S. Young (Eds.). Changing vocational education and training: An inte rnational comparative perspective (pp. 137-155). London: Routledge.

PAGE 195

185 Bryant, D. W. (1996). Tech prep at the crossroads. Community College Journal of Research and Practice, 20, 413-425. Campbell, D. F., Leverty, L. H., & Sayles, K. (1996). Funding for community colleges. In D. S. Honeyman, J. L. Watte nbarger, & K. C. Westbrook (Eds.). A struggle to survive: Funding higher educ ation in the next century (pp. 172-186). Thousand Oaks, CA: Sage Publications, Inc. Center for Community College Policy (CCCP). (2002, September). State policies on community college workforce developmen t: Findings from a national survey. Denver, CO: Education Co mmission of the States. Clayton, M. J. (1997). Delphi: A technique to harness expert opinion for critical decision-making tasks in educat ion. [Electronic version]. Educational Psychology, 17(4), 373-387. Cohen, A. M., & Brawer, F. B. (1996). The American community college. (3rd ed.). San Francisco: Jossey-Bass. Cohen, A. M., & Brawer, F. B. (2003). The American community college. (4th ed.). San Francisco: Jossey-Bass. Community College Survey of Student Engagement (CCSSE). CCSSE national college characteristics. Retrieved January 13, 2005, from http://www.ccsse.org/survey/national.html Council for Higher Education Accreditation (CHEA). Regional accrediting organizations 2004-2005. Retrieved March 6, 2005, from http://www.chea.org/Directories/regional.asp Crocker, L. M., & Algina, J. (1986). Introduction to classical and modern test theory. Belmont, CA: Wadsworth Group. Crotty, M. (1998). The foundations of social research : Meaning and perspective in the research process. Thousand Oaks, CA: Sage Publications, Inc. Donohue, P. C. (2004, April 25). Economic forum: 2004 agenda for a regional economy. The Sunday Times pp. 6-12. Eaton, J. S. (1994). How presidents infl uence public policy. In Cohen, Brawer, & Associates (Eds.). (1994), Managing community colleges: A handbook for effective practice (pp. 123-140). San Francisc o: Jossey-Bass, Inc. Education Commission of the States (ECS). (2000, November). State funding for community colleges: A 50-state survey. Denver, CO: Education Commission of the States.

PAGE 196

186 Finlay, I., Niven, S., & Young, S. (Eds.). (1998). Changing vocational education and training: An internationa l comparative perspective London: Routledge. Gleazer, E. J., Jr. (1968). This is the community college. Boston: Houghton Mifflin Co. Goldstein, N. H. (1975). A Delphi on the future of the steel and ferro alloy industries. In H. Linstone & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 210-226) Reading, MA: Addison-Wesley. Hamm, R. E., & Mundhenk, R. (Eds.). (1995). American workforce development: A position paper: Community and technical co lleges prepare to meet the challenge. Columbus, OH: National Council for Occupational Education. Honeyman, D. S., & Bruhn, M. (1996). The fi nancing of higher education. In D. S. Honeyman, J. L. Wattenbarger, & K. C. Westbrook (Eds.). A struggle to survive: Funding higher education in the next century (pp. 1-28). Thousand Oaks, CA: Sage Publications, Inc. Honeyman, D. S., Wattenbarger, J. L., & Westbrook, K. C. (Eds.). 1996. A struggle to survive: Funding higher educ ation in the next century. Thousand Oaks, CA: Sage Publications, Inc. Jillson, I. A. (1975). The national drugabuse policy Delphi: Progress report and findings to date. In H. Linstone & M. Turoff (Eds.). The Delphi method: Techniques and applications (pp. 124-159) Reading, MA: Addison-Wesley. Koos, L. V. (1924). The junior college Vol. 1. MN: University of Minnesota. Koos, L. V. (1925). The junior-college movement. Boston: Ginn and Company. League for Innovation in the Community College (2004). About the league: A thumbnail sketch of the league. Retrieved August 31, 2004, from http://www.league.org/le ague/about/about_main.htm Lewis, D. E. (1984). Characteristics of selected De lphi studies and their perceived impact in higher education (Doctoral dissertation, Universi ty of Florida, 1984). Linstone, H., & Turoff, M. (Eds.). (1975). The Delphi method: Techniques and applications. Reading, MA: Addison-Wesley. Lombardi, J. (1992). A new look at vocational education. In A. M. Cohen (Ed.), Perspectives on the community college: Essays by John Lombardi. (pp. 79-87). Washington, DC: American Association of Community and Junior Colleges and the American Council on Education. Lucas, C. J. (1994). American higher education: A history. New York: St. Martin’s Press.

PAGE 197

187 Merisotis, J. P., & Wolanin, T. R. (2000). Community college financi ng: Strategies and challenges. [Electronic version]. Washington, DC: Community College Press. Moore, C. M. (1987). Group techniques for idea building. Vol. 9. Newbury Park, CA: Sage Publications, Inc. National Center for Educati on Statistics (NCES). (2001). Digest of Education Statistics, 2001, NCES 2002–130 Washington, DC: U.S. Department of Education. National Commission on Excellence in Education. (1983 ). A nation at risk: The imperative for educational reform [Electronic version]. Washington, DC: U.S. Department of Education. National Institute of Standard s and Technology/SEMATECH. (n.d.). e-Handbook of Statistical Methods. Retrieved January 30, 2005, from http://www.itl.nist.gov/div898/handbook/ Nemr, E. G. (1977). A modified Delphi study of envir onmental factors affecting decision making in Florida's public community colleges (Doctoral dissertation, University of Florida, 1977). Nora, A. (2000). Reexamining the community college mission. [Electronic version]. Washington, DC: Community College Press. O'Banion, T. (1997). A learning college for the 21st century Phoenix, AZ: The Oryx Press. Palmer, J. C. (1996). Funding the multipurpose community college. In D. S. Honeyman, J. L. Wattenbarger, & K. C. Westbrook (Eds.). A struggle to survive: Funding higher education in the next century (pp. 187-208). Thousand Oaks, CA: Sage Publications, Inc. Parnell, D. (1985). The neglected majority. Washington, DC: Community College Press. Penfield, R. D. (2003). The fundamentals of survey-based research. Unpublished manuscript, Department of Educationa l Psychology, University of Florida. Gainesville. Ratcliff, J. L. (1994). Seven Streams in the Historical Development of the Modern American Community College. In G. A. Bake r, III, J. Dudziak, & P. Tyler (Eds.). A handbook on the community college in Am erica: Its history, mission, and management (pp. 3-10) Westport, CT: Greenwood Press. Smith, A. J. (1975). A modified Delphi study of ob jectives for general education programs in Florida's p ublic community colleges (Doctoral dissertation, University of Florida, 1975).

PAGE 198

188 Stewart, J. (2001). Is the Delphi technique a qualitative method? Medical Education, 35(10), 922-923. Tuckman, B. W. (1999). Conducting education research. Belmont, CA: Wadsworth Group. Turoff, M., & Hiltz, S. R. (1996). Computer ba sed Delphi process. In M. Adler & Ziglio, E. (Eds.), Gazing into the oracle: The Delphi method and its application to social policy and public health (pp. 56 -85). London: Kinsley Publishers. Vaughn, G. B. (1994). Effective presidential leadersh ip: Twelve areas of focus. In Cohen, Brawer, & Associates (Eds.). (1994), Managing community colleges: A handbook for effective practice (pp. 123-140). San Francisco: Jossey-Bass, Inc. Wingspread Group on Higher Education. (1993). An American imperative: Higher expectations for higher education. [Electronic version]. Racine, WI: The Johnson Foundation. Wattenbarger, J. L. (1950). The organization, administ ration, and financi ng of public junior colleges in th e state of Florida (Doctoral dissertation, Un iversity of Florida, 1950). Witt, A. A., Wattenbarger, J. L., Gollattscheck, J. F., & Suppiger, J. E. (1994). America's community colleges: The first century. Washington, DC: Community College Press. Ziglio, E. (1996). Theoretical, methodological and practical issues arising out of the Delphi method. In M. Adler, & Ziglio, E. (Eds.), Gazing into the oracle: The Delphi method and its application to social policy and public health (pp. 3 -33). London: Kinsley Publishers.

PAGE 199

189 BIOGRAPHICAL SKETCH Mike Droll has served as managing direct or of the National Alliance of Community and Technical Colleges since 2002. He is pres ident-elect to the Fl orida Association for Institutional Research and an ad hoc member to the Florida Community College System strategic planning task force. In addition, Mike works as a consultant, concentrating on institutional effectiveness, strategic planni ng, and student learning outcomes assessment. He was the inaugural recipient of the Dr James L. Wattenbarger fellowship in 2004. Mike earned his Associate in Arts degree with distinction, Phi Theta Kappa, from Mesa Community College and then transferre d to Arizona State University where he earned a Bachelor of Science degree in Computer Information Systems and his commission in the U.S. Air Force. During his 20-year Air Force career, he earned a Master of Science degree in Operations Mana gement from the University of Arkansas. He also completed the Air Force Institute of Technology--Education with Industry and the Air University Academic Instructor Sc hool programs. Academic Instructor School preceded his appointment as an assistant profe ssor at the University of Louisville where he taught the leadership and management curriculum to Air Force ROTC cadets. Prior to returning to Florida in 2000, Mi ke taught high school accounting, computer programming, and software applications unde r the Troops to Teachers program. Mike transitioned to the University of Florida fr om a grant position at Santa Fe Community College where he served as a member of the successful reaffirmation leadership team in the Office of Institutional Research and Planning.


xml version 1.0 encoding UTF-8
REPORT xmlns http:www.fcla.edudlsmddaitss xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.fcla.edudlsmddaitssdaitssReport.xsd
INGEST IEID E20101204_AAAAEL INGEST_TIME 2010-12-05T01:00:16Z PACKAGE UFE0010111_00001
AGREEMENT_INFO ACCOUNT UF PROJECT UFDC
FILES
FILE SIZE 5175 DFID F20101204_AACSIH ORIGIN DEPOSITOR PATH droll_m_Page_151thm.jpg GLOBAL false PRESERVATION BIT MESSAGE_DIGEST ALGORITHM MD5
8d5abf0783ecc2caead3a9591c9131ac
SHA-1
f4498dd996375d16abdc66625f9bf89156a17e84
3823 F20101204_AACSHT droll_m_Page_173thm.jpg
f1525e74f82c49d9f353e01a44d7b3f1
85a2b0883054259780867eb12af53f0c6dfb5711
82882 F20101204_AACRFF droll_m_Page_129.jp2
639e6eda20bea07ded605272305b6341
4e0695a6d3e260dd8c5b08c5f9190b5df106c39f
72179 F20101204_AACRER droll_m_Page_113.jp2
85eae8ff869f8ab53ee4da739619b097
871030e4adcc2bcce99cbe370d8796a5cf3b69d7
56788 F20101204_AACQZL droll_m_Page_140.jpg
a6658800b9fc7a568ae3a840fa4cbc04
c71b02cf86abdb60babe2ffad1fb3508c556fd8d
74321 F20101204_AACQYW droll_m_Page_121.jpg
ce8d17441c124f803297622999f9eec7
7fbc7988cad095bb2381a3286d192cf4bbe17dc5
6388 F20101204_AACSII droll_m_Page_178thm.jpg
3ab6240eff90eebcd853deee9752df95
2695b60c486f546bd4119e06e78fc36e7f3be2f1
18688 F20101204_AACSHU droll_m_Page_174.QC.jpg
0c57e3e989b819d36289a1aef076c374
4872ce17a92e9916f50fce26035aabc180531fee
88619 F20101204_AACRFG droll_m_Page_130.jp2
755b6d40e29ef3eb761838c2d604d0dc
a8261c01922feead83683ae91b133534c8a5c5e8
84349 F20101204_AACRES droll_m_Page_114.jp2
ec3b544e0b5000095e3d3eec5b991820
811775c75c4a6c4cdac918e4430a27f0b9771d25
64110 F20101204_AACQZM droll_m_Page_141.jpg
e70d270c0572cfac0ab332732e1bf976
09d68c6bf6f200a61d1ac0b58dafc82f8523d9fb
68680 F20101204_AACQYX droll_m_Page_122.jpg
fbc61582ded9bd06eb57a947738aa4d7
d2e8ce7d338ef20d8549d3e29a87862ef14e9fcf
5404 F20101204_AACSIJ droll_m_Page_103thm.jpg
bb3724f8123469e2609da3ef4d17876c
f43d08436ed3bc00063240f4f4ee3e08a407d270
5867 F20101204_AACSHV droll_m_Page_025thm.jpg
ff7a62d3ad34923c53c92a411b1d90bf
511e6c4bd33eae72b3d33dfd9f6950641bc6b5d8
106696 F20101204_AACRFH droll_m_Page_131.jp2
ff993b27af19b2e9f34619a964de834a
5c3bc39ed32e193b1945abb63c1ef2618e9f27ee
74804 F20101204_AACRET droll_m_Page_115.jp2
eaa6520f8661387b3be6505465c3f70d
815c227dbf9e59141f4240e8b42c6aa7c5a4daed
50811 F20101204_AACQZN droll_m_Page_142.jpg
50e9ccef468fba987046ba6372cee131
33d66712632944d8e28616e01daf108c621d02c0
73772 F20101204_AACQYY droll_m_Page_123.jpg
bada3f815edbf8bccc00fad6fd165170
23e3888183f48ebb362677bb7bc8f8804b703936
6012 F20101204_AACSIK droll_m_Page_176thm.jpg
ec1cfb38f2eef60bb453ab8de06ebd1b
47bef76f2bbbade97bee81b2aa64500381b854e3
6365 F20101204_AACSHW droll_m_Page_184thm.jpg
b95f3ad7a2b4ced3e8bc2045ba804484
abc603333202f7a4e2364b555879af0801b40919
106150 F20101204_AACRFI droll_m_Page_132.jp2
bef0aebc60013b70b86d7473a3fef9f7
3183c4ea3b2d22fdf35195037572a2a7d9f1eb54
84965 F20101204_AACREU droll_m_Page_116.jp2
52a936654778e5f5fb7f281681da668e
a8d907143e394c93d4074b1d77d0a48cc167a5e6
45592 F20101204_AACQZO droll_m_Page_143.jpg
6cc2ee3b79cd61fe00e3a85b4150387c
308e33bf75c6f06835d4c7ea408bb0bce3d9a57b
57313 F20101204_AACQYZ droll_m_Page_124.jpg
602cd95acf53af049130155953480257
479b2154692b00207d2c084ab1f2684bb26d495e
23587 F20101204_AACSJA droll_m_Page_024.QC.jpg
ee649abb25ac899e287b861b2d649b1a
ebb25912e9ce7b8a82d651071b84f19f5b1dabe0
23361 F20101204_AACSIL droll_m_Page_088.QC.jpg
a53371ed86270f53fb1c685b7cad97a5
4bc8ba48940cc1b1acddf64df2e7c0f26ce36ae0
6046 F20101204_AACSHX droll_m_Page_126thm.jpg
91c99819c82f19b63299efadfe2685ec
08b88ecef6fce8fff43c3eddc762f009d22a419b
102615 F20101204_AACRFJ droll_m_Page_133.jp2
278dc36863d658e6c4ee41cb31013556
4ad522ea9461d5016be6cd9f7d3f7a8522a43f59
76859 F20101204_AACREV droll_m_Page_117.jp2
3f0a51950dbb2be069180859c4a8cbaf
a59bd338574c3e4e12c674cfce036bdb4fa0cfcd
69460 F20101204_AACQZP droll_m_Page_146.jpg
03b15f2b12c9df413aa4245eda96d305
46526708b7473eb7d053ddf83cfee3d00ff0b5ac
21328 F20101204_AACSJB droll_m_Page_027.QC.jpg
c82bfb8f5312b6216aee02599520d37e
38f9cd2241f624e70a976e3d7cca9d73aab26454
4801 F20101204_AACSIM droll_m_Page_198thm.jpg
a375d1a5e2d20f89f38c0e85483d0ec6
97c9ef5c1bbda72ee5a706824463978240e2958f
24532 F20101204_AACSHY droll_m_Page_081.QC.jpg
7df3f9314849df4615d0710f1d9d5e93
76af482a229b9699e2d50aadede5eccbeff4c90c
83491 F20101204_AACRFK droll_m_Page_134.jp2
2b584230adb89469f7b73cb465db7015
bf85f77993f6a8b74bdb1e18016f938153f837e2
81403 F20101204_AACREW droll_m_Page_118.jp2
d85742c06a10625f0f1ef92adbefbc18
04cd6aee38ac4e243e7d23ac4b8d25fc1dfc3ac4
69222 F20101204_AACQZQ droll_m_Page_147.jpg
967afa8fdc207c28722380d68b2a7c1e
d2967de9c5583d5673428b9c8387c9e229a2a7d7
294786 F20101204_AACSIN UFE0010111_00001.xml FULL
a3ba2a3e6d9670b3d6fcf436b1838500
bfdf2b7712560350e68048b31a0153d3e0126be6
6801 F20101204_AACSHZ droll_m_Page_154thm.jpg
d8ffc466fd61f5a820fe9c3a00103c55
fd2e972aff9fa16e6473244e9f92f31a7af589e3
75058 F20101204_AACRFL droll_m_Page_135.jp2
1ac711a4cd85b82252fe10570a0096b4
aa30a06c1945f313ce14a216683ececceabb03b9
95549 F20101204_AACREX droll_m_Page_119.jp2
5c5c71529b5d350777da35e59772f4f1
0469aec2a05297ff0913ef29e32ef9853eb31383
61164 F20101204_AACQZR droll_m_Page_148.jpg
2d88a83967cb722f7e7a329e90d4d955
ac20a60af23b6a76b493b7c5af158e6715123fea
5855 F20101204_AACSJC droll_m_Page_027thm.jpg
a7e196db393cccddb2cbd88e5765860c
bbb490bff3142bc6b8bef1a59404a0e4ca887576
1392 F20101204_AACSIO droll_m_Page_002thm.jpg
94f3d80ef9e850981a5f7235e6de17e3
9cadaa257e84ef49bdcfafba20001ea90c623542
93337 F20101204_AACRFM droll_m_Page_136.jp2
6a5d3d6a413d4defed6802b64ff849f8
b55a87ab171a25319d9a041eb098dd50843554ac
113088 F20101204_AACREY droll_m_Page_121.jp2
dd8d35ecdeb5141f82df17e30ae35a14
ef521705bc2ec83ada158d5ea66909e3edd8246f
72794 F20101204_AACQZS droll_m_Page_149.jpg
d32c2fadcd6f774891bd420f16f0431a
91b55f94e016f829ce2d686be5b28ffcc6a9a54a
110002 F20101204_AACRGA droll_m_Page_152.jp2
38800d38f092b3b5594ba93f24255fb5
149f3528783a75231963adf0cd3961a203b3195c
5278 F20101204_AACSJD droll_m_Page_028thm.jpg
8ed9acf57e501ff3b61c4185e751bee3
0002b88b9fd725c3ca4659e38d28f51177632805
17804 F20101204_AACSIP droll_m_Page_004.QC.jpg
0a19b483fa3c9d6f64b25d65ace5171c
604184c628ba2721975e819eecc9daa19311751f
88780 F20101204_AACRFN droll_m_Page_137.jp2
1e23fac3f75b4fe313a3282137154599
ee087c45eb827cd09c8fb88364175534e3a20f00
107947 F20101204_AACREZ droll_m_Page_122.jp2
fe870df9a4de8b590775efddc5cfb82b
dff62f41c22880d64e93b6ff137b7058794f66d2
64975 F20101204_AACQZT droll_m_Page_150.jpg
b07284164e8fff38c41be70c6d47800f
d397f3e68577e3a2c3c62fc8b2340b87574fc829
111836 F20101204_AACRGB droll_m_Page_153.jp2
17fd3bd6dec76ebb70e596cef540ddd7
87db4135446e64aea803c8949ef01135854e67ad
24267 F20101204_AACSJE droll_m_Page_029.QC.jpg
d2f065686828cd27fc51457a8626d7c7
e8d70ce6644aa419812fe8253a57c114e723c661
4631 F20101204_AACSIQ droll_m_Page_004thm.jpg
af5a299829061efa55d506f355aad8af
8a9e1ae1ce365322f35fe07073c5a6aadf1943b9
89493 F20101204_AACRFO droll_m_Page_138.jp2
bca962f923b1838110930002102b6e2c
081136c7a504618e429063d520e78b931314e92d
115554 F20101204_AACRGC droll_m_Page_154.jp2
a0c47479ff543cf0ee78245529dbf373
c93213200cd740bdd3f9b50d1c46171bc21191d1
6481 F20101204_AACSJF droll_m_Page_030thm.jpg
70382dc557cb693a5ad1205c52d484e4
c0cde8f966d7eb0f24f57e4502d46e8d2a7a9220
5628 F20101204_AACSIR droll_m_Page_005thm.jpg
f9b38d5c34c1b4ab19bf515aad444173
b46a6612d9cc11d12015d263a2d9e70b717b887c
86771 F20101204_AACRFP droll_m_Page_139.jp2
100b3576b22253c4ab6bc80916bb399d
5acbd538387616538aae0a259b3c9ab6a241c542
58952 F20101204_AACQZU droll_m_Page_151.jpg
31fee8715b0d38dbce23ccc393da8140
d458a605922203771a713c76f28169af0fd8c730
106830 F20101204_AACRGD droll_m_Page_155.jp2
e878f1c6440a26d99fcad5099eb74b9f
645bd9b87dbd0a0c4f7d011f391b7847cafa66e5
6476 F20101204_AACSJG droll_m_Page_031thm.jpg
78f5a703bf67319742982bf3e6defe6a
501681b322e16d87e48cd129f56817ca7b5922f9
5809 F20101204_AACSIS droll_m_Page_007thm.jpg
cc6cf961079b12bbc5a929e408e4beaf
6b88dadf931b705d108015b9b2e7d03adcf47a0d
83907 F20101204_AACRFQ droll_m_Page_140.jp2
aead0577238da5ca758aa640af78f485
b0ab903ed7859d64b1be02545b444b74c7fd17ff
71494 F20101204_AACQZV droll_m_Page_152.jpg
b573cf4fa6263ae6b5093cb0ec29dd2b
63c63675fccd1e0c20a6e011a061d9656b41e7f3
114406 F20101204_AACRGE droll_m_Page_156.jp2
c77e15075be46788632348355769572e
ee59bb4394f16dc983b62ac27cacfca60da94ce8
6778 F20101204_AACSJH droll_m_Page_033thm.jpg
e17731e1cb5a488d8b518b3dcadac539
84ee8118522213f7ef1ac6431539d11d7f696679
5245 F20101204_AACSIT droll_m_Page_010thm.jpg
664fc00448411872b088f5621c65ab3c
e72c911d0a7c6ff569a8f318a7031f5ca8dc4838
73531 F20101204_AACQZW droll_m_Page_153.jpg
fba05525b086fdad5ab47114550c7f3d
22f46a9f09bfb47f6b571d92fec91271fa7d2f65
107206 F20101204_AACRGF droll_m_Page_157.jp2
1c6a60f629138b93acdcc7f46c7b3552
0e39e650231b4328f5d4af2a822eff5514c9be24
96279 F20101204_AACRFR droll_m_Page_141.jp2
78279acb0cac33bd51521037c968fc9e
a928c9dcdb6f54893eb88eb5af8f4131d698587f
24247 F20101204_AACSJI droll_m_Page_035.QC.jpg
1fdfec580d6eefcfc62bd43341c5ca5b
7911d4193db219cf395af2208f44e05944b70930
25489 F20101204_AACSIU droll_m_Page_012.QC.jpg
6ad6acf414d7de217975b0ff08add416
fd79af47ef1ef206e1cc0ed876b61b3a6d4254e6
77885 F20101204_AACQZX droll_m_Page_154.jpg
2f1fd029bf432f6a7729a47e4b170698
7c91d4fc4d87231d724e6bebfc794c5cd5b80569
108598 F20101204_AACRGG droll_m_Page_158.jp2
17bf9390d81658c0dff0a76a2b0613f6
65dbb060eda99e11561c74af315e0826e841c805
67372 F20101204_AACRFS droll_m_Page_143.jp2
74d8c0734bf4be65b61e733665900482
a5fbc4961b8d68f4170085eac27b4e3a67b6b5cc
6416 F20101204_AACSJJ droll_m_Page_035thm.jpg
884646eae55e6da4805943b115475b00
3b2fbd86e59f4ef565f4c6fa990486885ab00f38
23850 F20101204_AACSIV droll_m_Page_014.QC.jpg
135eedd9712439a527eb78e15e955674
1f7f4ade7ae4f10cbd05463669e481b4fb821fbe
69306 F20101204_AACQZY droll_m_Page_155.jpg
0ea663ddfc0973562b2f87b9c59704f8
8d51b5ff0e7e47bb0020693496b6466f0f4a6af6
100879 F20101204_AACRGH droll_m_Page_160.jp2
2458d8ba9d2f8a3b5370e596cad5287c
dac38644f00658eb2dbf67143a16ae042e403bd2
85900 F20101204_AACRFT droll_m_Page_144.jp2
386d143f89462d6b03727cf959aed185
8e25bcf67a6a81222b6b95040c7e96805f04f0e9
22422 F20101204_AACSJK droll_m_Page_037.QC.jpg
857f4b28d5ed09841e72316be71d1874
24a14d3dab5966bbc163e9423b73f582322e5366
6550 F20101204_AACSIW droll_m_Page_014thm.jpg
4727faad61c1dac0aaa82c1b78f638b6
35515b32acdbb90fd8d411fb838267b89f31fcd4
74852 F20101204_AACQZZ droll_m_Page_156.jpg
da1391699c49ba12c5742bbb458543eb
624b9c572698628acbb8561d1ccc2f51778e18c7
107586 F20101204_AACRGI droll_m_Page_162.jp2
762a5250744d6ef3424fc5bf2caa750c
c97613ac356ac144118feaa6d3be4454bbca4af8
73830 F20101204_AACRFU droll_m_Page_145.jp2
44de0d856232599051e733bfa4dbc8bd
f3db1e4ac777358bca17df6d489286c579680f7a
22682 F20101204_AACSKA droll_m_Page_069.QC.jpg
399bae57f7ca368e8c3277bac09aaa2b
27f8ae6d08624d295ce6b47e130a31d915cbdcb6
23671 F20101204_AACSJL droll_m_Page_038.QC.jpg
1a5c639caa552ca539a6c5263b55b899
02ac5076a9025fc7b673202f890d4cee6c950e78
6586 F20101204_AACSIX droll_m_Page_016thm.jpg
8855d21f2baddb52c252c7bd48ac762f
7f40c72e1a34bdb750629db023110f530a7f334b
102530 F20101204_AACRGJ droll_m_Page_163.jp2
1f81c603f4b6935fbedbbc467d0a2164
b9e4d3893b8980f58b28534dd296ba800cac4843
105103 F20101204_AACRFV droll_m_Page_146.jp2
74cfb5740ab1dba3b88068f4b907dbfc
bf030353074b3de54bdf1cfb161ec93947d1c745
6402 F20101204_AACSKB droll_m_Page_069thm.jpg
6c545406ab6725cc6548bd2473ab44a0
88f28c809e8d6caf1397f21097f72e9132f7c1d2
6526 F20101204_AACSJM droll_m_Page_038thm.jpg
36577a442b12c00c40ec7f85eff02715
ec45702ef7af500a53e5248ab6018d128ef9f118
6537 F20101204_AACSIY droll_m_Page_017thm.jpg
56c4992aee168590354420bf6715332b
fba3e556b6c7624e45420c41216bf2f5d29da524
104418 F20101204_AACRGK droll_m_Page_164.jp2
7c91c0e0dbe5a010515bf2aa1e90b4d5
a08cf913d65cc9e4e206428448b221871a6486cb
104007 F20101204_AACRFW droll_m_Page_147.jp2
47665e58dc902d9dc99602dab39c1e6c
5116435e4d6a7ea3d6d108c4eaa706c13fa37be2
6263 F20101204_AACSKC droll_m_Page_070thm.jpg
421d8a978c308fdf20e3f88912fc9f69
2d7b262ee595ee1c37d6ea17835006251e304807
25464 F20101204_AACSJN droll_m_Page_043.QC.jpg
52a55d39bb7c27aae331528e152913f0
d09d201e9cc9cf5d28cebf214f87ec81f92a9d75
22751 F20101204_AACSIZ droll_m_Page_018.QC.jpg
08669a92bcce33ddd101f204dbd10835
c7fc1acaaa3d6ed677188963062a18f2a85c3043
109538 F20101204_AACRHA droll_m_Page_183.jp2
34b0b14f55235b53e87b2701c01f2b0d
3881463f465de6b2bef87d2025b9ca29b903ce7b
108695 F20101204_AACRGL droll_m_Page_165.jp2
cefcb32363d0328083891055647bbcd0
e6aefa9cd3a61ad569b6683e5d37b832cc203809
90560 F20101204_AACRFX droll_m_Page_148.jp2
52e2c4bdf91e7607cc9071b8e4b55ed2
5e135879a30abc40b6f8f2b31ecb65660558cb5c
6816 F20101204_AACSJO droll_m_Page_043thm.jpg
0c0f81a7917e2a4a31211ef5b4e7148f
f1a59ca86c2763700f59684ae246cf64e0addff7
103031 F20101204_AACRGM droll_m_Page_166.jp2
110f59293993a9686b2e093e8d433db5
07434ac53e0563442a10621fca7de35de1d4b7d1
111550 F20101204_AACRFY droll_m_Page_149.jp2
54971eef6d4e5a20acae9fea22099fdb
7444177cd00de2d9a4f89afdef65d46da328d437
23383 F20101204_AACSKD droll_m_Page_071.QC.jpg
5adcd1510c21f17195c53d72ea21e33a
dab65c713d40eb87401855d3c6cd65ed2759a57b
6238 F20101204_AACSJP droll_m_Page_044thm.jpg
e4feca0f39e2cb3e70a9878991eb8bca
6534e0d91b667471f7b999e9c3704c662272f2fe
129114 F20101204_AACRHB droll_m_Page_185.jp2
0b00508a60054826e782c66b17fd0df3
fa8e6178d35589d30eb8ace470ab1e44dd6fb494
113598 F20101204_AACRGN droll_m_Page_168.jp2
95d60ee2f8c86aa06bd39a270a4f7e9a
877cbbef97c669342d2c9e5e10cf96e350a07062
85970 F20101204_AACRFZ droll_m_Page_151.jp2
4501c0e32e3f604d5d4627e2af0d91f5
bbe69fc17c1ec67ddc37a6a8991bd861638272ae
6327 F20101204_AACSKE droll_m_Page_072thm.jpg
5bd0796757093570b6d23c8d6236af4e
b2555b41fd7b226427fe427b7bdf1d3d6f468d6c
23362 F20101204_AACSJQ droll_m_Page_047.QC.jpg
60ccf4439b8d85ad909177ba9a0f90d2
29bddc5245572d5b8285d75cf33a8e60c81416fe
110234 F20101204_AACRHC droll_m_Page_186.jp2
bc6b510ed0d555a4cef5a6c9fd5656ab
4d6688d990c9d1a901368d07182af8124fcfb311
119827 F20101204_AACRGO droll_m_Page_169.jp2
91478cb4f660a7df718a791f2139c510
0964dc6d047c310b9bb192ab3ad4ebdf9563ebf4
23519 F20101204_AACSKF droll_m_Page_074.QC.jpg
f45fe6f7ce754d2e3fd99fd9ca964a10
6fe4af0f6f183d76734eba043e8f5140c9f271f7
6221 F20101204_AACSJR droll_m_Page_049thm.jpg
a8fb6df6617c01b8d2eec5d84acc7df7
fad0232b5c6735bb8fc0131d8339599704495d15
130312 F20101204_AACRHD droll_m_Page_187.jp2
e76dbe557098cbe011bec8fcc40cef5e
ebf79893f9398be382155c703e936b2c54250474
106419 F20101204_AACRGP droll_m_Page_170.jp2
8c08e4fd4deb6fc8be778230060f762e
b02cf1a02e7bdd53b9e3ab963d0316ad37d01a78
23159 F20101204_AACSKG droll_m_Page_077.QC.jpg
5b7ce11146eca977090ce7641863feb6
43f2b1f73921cba57ad8dc4ae368e9eda7f84729
22641 F20101204_AACSJS droll_m_Page_052.QC.jpg
1a26ced18dc1fd01150307d5f268e1f7
30f03025c9e112ab37e5ec82f6f3994ab0600f5d
124349 F20101204_AACRHE droll_m_Page_188.jp2
7a3bc2ad73cf9f9fb666cdcf7e3ab11d
64e9c05df6bc3d0cee27b5661b3cec6d87a7bb6c
126811 F20101204_AACRGQ droll_m_Page_171.jp2
f1e7369d88f8559c7385f6b5dff98294
5c7d5afea6e7b62d35d658fdb37a928b9f9cd708
6681 F20101204_AACSKH droll_m_Page_077thm.jpg
bea6dbf8a7eec70ae87bd6beff6b5a72
1f2c07bd9bfb8dce630c8d42604f2f01b294c86d
22741 F20101204_AACSJT droll_m_Page_053.QC.jpg
4c2e82aad5b8479add64746a2a389a5e
29aa372425244633a136f070282bb26de9dee225
107735 F20101204_AACRHF droll_m_Page_189.jp2
aa3a275ccbe8f71acd463bc8cc53f01e
d262d0f2c43a4d45cdd4b64adc1f583b32601afb
87550 F20101204_AACRGR droll_m_Page_174.jp2
6acfce9498e9f0a1a6aff2a161ab8bd7
224ac376dc106d68121d8e018654f9f462a16a4f
22823 F20101204_AACSKI droll_m_Page_078.QC.jpg
440aa5f0ddefadc1d6834c5ec40f33d1
59a40d150ae04acaf1f1e9fab629c865739c9728
24271 F20101204_AACSJU droll_m_Page_056.QC.jpg
d60aed0da3966f3fe2c4b498da50bb66
cebb2dc1440479bd304421b73b0246135aa6379e
114911 F20101204_AACRHG droll_m_Page_190.jp2
96aa5c5c9da78a9e0ef46c06ea96e08d
94027b3dc441be5bd067dbebad78d6ecbd8e2ae6
120656 F20101204_AACRGS droll_m_Page_175.jp2
c24036756ba7f058eea52d48a9a76f85
baa7aa4b27ebcf65e388ed8b28132c08db9d8225
6457 F20101204_AACSKJ droll_m_Page_078thm.jpg
0762451b90aa82c3251e546fc39806b9
86b90ff082673f9e6eca71699f746f3276478b0f
6499 F20101204_AACSJV droll_m_Page_056thm.jpg
8a0c82d8a2f85e29bc6a81ae54e78343
95bd40e73cc7527f64f03e5ab0d1049f7eb5b805
104477 F20101204_AACRHH droll_m_Page_191.jp2
aeee955f90b60525a08950e1e90bfcd4
b16d4fa22fded3c6fba51c9c0fa8beaf68d3ebaf
99795 F20101204_AACRGT droll_m_Page_176.jp2
23b48734d59f76d06f1f8ddf8a6b249b
5d3b201e90633668ed887e34d430ed9f09562cf9
6594 F20101204_AACSKK droll_m_Page_079thm.jpg
8b1504eb485de1ff8fc6b4a839ce70a1
28243697ae639b6b4287a8f4581d7ef0e8627b0e
6446 F20101204_AACSJW droll_m_Page_061thm.jpg
92c43d4223cc2ac0679efb1ba133df51
9d0ddf83581b7ec3b88377cfb285b4dc932a4f6e
101421 F20101204_AACRHI droll_m_Page_192.jp2
16cfc6995a9d2615932f1f1e89b761a9
c32dfb15bde582c1657313d8aee38856c15909ca
135977 F20101204_AACRGU droll_m_Page_177.jp2
666e1c954d31bb4ce43db1e4e16a590d
1d35a3b12f96d4296c174b1b2d616f1d913b3203
16294 F20101204_AACSLA droll_m_Page_101.QC.jpg
73437079f70b48b67850433f8e559067
ec875b713b26b32b5172b5db8bebec8acaa07920
6525 F20101204_AACSKL droll_m_Page_080thm.jpg
e60b9a8fd11a14463e2764433f66e82b
e982a8a4befec80fa69989bacb0b766a67d0aeb7
6515 F20101204_AACSJX droll_m_Page_063thm.jpg
e7f785024aa1352211d43230ae201aa1
e87b8a4bcb934aded6e84331080f5d81c8321b48
120024 F20101204_AACRHJ droll_m_Page_194.jp2
031e73c01a01277668b3fe3fa248772f
3e0950fb89af7ffb4c0afcab7d0399683ae6e14a
120876 F20101204_AACRGV droll_m_Page_178.jp2
7234777bb137e459efb0edcfc25a2b75
8b685790f3672cd9f2aac336dfdf61945d7da428
5307 F20101204_AACSLB droll_m_Page_102thm.jpg
70c58df607feb7e3f1e60a300c207a1f
e6982075448724982adbf77e0c1848c991832905
6958 F20101204_AACSKM droll_m_Page_081thm.jpg
ccc06f59c5c0beb71b1237d6c73596db
d6a3e85a8229d9772fe13eec56cabece01223f15
6303 F20101204_AACSJY droll_m_Page_065thm.jpg
48b6b909cc56311da33956d7a903beea
ff84c8d398408e2c0c52a6babab6707459f7d0f7
125570 F20101204_AACRHK droll_m_Page_195.jp2
cef9b07a336e8dee541ee4d903ce91f4
6f3bff54373af78f7b26f1af160aa8b21d3f16eb
127422 F20101204_AACRGW droll_m_Page_179.jp2
b1baa60b9a5f883ce73c0822be956758
9f904bdbc08f8d0ade4ccc39142109a430735baa
18408 F20101204_AACSLC droll_m_Page_103.QC.jpg
85f13d9db71833cf86963ca55f27e9c8
7fdf3a8414fe3dc07cf756f07d0d7cc95b08a853
F20101204_AACSKN droll_m_Page_082.QC.jpg
bb5e2d1ff6f4e90fead03b4507c62f1e
58aef7eaec520cd64d35cc6fc34919bdbd4cf9da
24144 F20101204_AACSJZ droll_m_Page_066.QC.jpg
034fcbd2f437e99a8bdfa9efafdca6d0
c0f0098723bb09be60a98cfa9f5f507bb380468a
135043 F20101204_AACRHL droll_m_Page_196.jp2
bd394bc7cd26cb63c0744134ea74c8af
8847ac8cd63cd590b3b94d8c9334e5ea34b6e233
111285 F20101204_AACRGX droll_m_Page_180.jp2
9fa37d66332525234a501c2f2bf39625
22a2de9b35a16e69bdb6b14f15c78565fbe68c55
1053954 F20101204_AACRIA droll_m_Page_015.tif
c2f317116ba2403ddb4d4251edd56039
4064083c77967f4e2821d18db7353586a803f63e
16731 F20101204_AACSLD droll_m_Page_106.QC.jpg
5eec94d0ecc4acfd4280445c01aa43e1
36f5c4dfedd48bdf2a1bbdff945fa63b054f66ad
6626 F20101204_AACSKO droll_m_Page_082thm.jpg
9e198e9fbb0a6d2945fb6ff30f4bad03
b35a43432f001feb7c55bb20eb8b7babd8a76aa3
127828 F20101204_AACRHM droll_m_Page_197.jp2
ccf20a4d3a842ae420b3e8b33cc01f32
9064e2e5da7cbb9a5c9451d6b43e3231a3fd35a7
116973 F20101204_AACRGY droll_m_Page_181.jp2
f71b8b81b1ba62821d4cc1ddb5fdcd2d
fd9c2170139fbd4d88757359cc603d2f4e1ac323
F20101204_AACRIB droll_m_Page_019.tif
bad45e62583b6d94377574054662b772
9c6e5894b824ea83dca60b25150c278a33ab61e8
22492 F20101204_AACSKP droll_m_Page_086.QC.jpg
a695e8429486b59523c61ce9fc14617f
744a8e7e3038a530106c67d943ad8a9c7a092bcb
86234 F20101204_AACRHN droll_m_Page_198.jp2
5a9ba3c48c39be878176eacb32364426
53a7dd2e4beeb8015aa3db43ca15db95ccfcf2d5
117924 F20101204_AACRGZ droll_m_Page_182.jp2
bc2783552d6041a114d0fb847f05684b
6924f9aaf5e5509471153022932c46e5dc35b7f1
5313 F20101204_AACSLE droll_m_Page_108thm.jpg
e2e63005ec1dc5f4dc407bfc286843ac
f745232ed4aa703c1a4fb5b0cfebbb82dcba0793
6800 F20101204_AACSKQ droll_m_Page_087thm.jpg
546f7468f2a49a882b423d1ab735df17
c0a8624827d50362a10e96485c6404b27cc47837
98154 F20101204_AACRHO droll_m_Page_199.jp2
edbaefbac95a2f9f88f8525243765089
b50665f51c2b544194b4277f1a22bf046687925d
F20101204_AACRIC droll_m_Page_020.tif
8012509e7fa4cf4814f69513ccedb125
953b3434cd54beeb37b0857a9f4cde1ed5b88793
15327 F20101204_AACSLF droll_m_Page_110.QC.jpg
7bb55277fdba0214701d895c9512a307
7821125fb918509ec916459978d8864355051cb3
6357 F20101204_AACSKR droll_m_Page_088thm.jpg
68bb9f6eb31b2e312921df08d270dce3
eecda771a4d7a9db22bea9b0b2c9bb4780ca4f2f
F20101204_AACRHP droll_m_Page_001.tif
2f83eb2926d7a12bfd012151da370f9a
a8ead43ae02e6c52bc419428de9fc4ba245e668f
F20101204_AACRID droll_m_Page_022.tif
20dd4a0793be2c2605d9115945f8d0e7
99cfdbb197b70491ab5379bc78388e03402e7d53
4729 F20101204_AACSLG droll_m_Page_110thm.jpg
6fabdf3aae4429b939fd7e0a9464393a
6eaaac8e7f05ab97eabe7bf2cebd012b7d614b86
6879 F20101204_AACSKS droll_m_Page_090thm.jpg
f32a684f01c531fed0c6ccc4fe37be79
0d9eb1eb58426d1a7763f0e6bdf4ba9fc80090f9
F20101204_AACRHQ droll_m_Page_003.tif
0714e0186ff3ba933697eef4cb06b35d
5c8d48414be5dcb64912880beb59f052bb9a6968
F20101204_AACRIE droll_m_Page_023.tif
7a49816d90bfe213004983e98c50289c
d0642e81cc4b8c30e1a5a4be7138d3394d8a8f81
4744 F20101204_AACSLH droll_m_Page_112thm.jpg
693e40389651ec6b1840c78ce427f127
b70dfde3dcfbcd95998b386302e2c38b38f0253f
22929 F20101204_AACSKT droll_m_Page_092.QC.jpg
11d893063440215eb39c6adbbb56739d
1b3a75de1150750da5aa8fcb1e073283c4cada7d
25271604 F20101204_AACRHR droll_m_Page_004.tif
ad8e968add9da748ce1d662795cd2bfa
07fd8f6eee16a4839b97ad130661b49b65c8fa94
F20101204_AACRIF droll_m_Page_024.tif
2527a3d5b2b8dd531b417e6ce0667e7c
bd204cba6f58ffb87ac3dc5d50ee9786a5f19581
16280 F20101204_AACSLI droll_m_Page_114.QC.jpg
b1b8e84fcae4aef6cf3d5d18a503c8ac
cc35358b420749fc13fe3a6d0881ccbcb54f9364
6487 F20101204_AACSKU droll_m_Page_092thm.jpg
564e8669100bc7f7ee9e9a75bf2b47a0
c556a3c7b2df562496eb8e5468c880e170acf51e
F20101204_AACRHS droll_m_Page_005.tif
e7357977e7c7ea1ad99b32c97aac3a28
9d7ec176caf041b4ec37e9fc51a5877173409c24
F20101204_AACRIG droll_m_Page_025.tif
1fd32e9ffef0d55f30c4a33c5922ab77
ae8036c2a129a319ad07c75c59653306715e6b1b
16620 F20101204_AACSLJ droll_m_Page_116.QC.jpg
656fef01891f2813f5894b36024abbc5
16f23e441fba56ca7c0e6691e96e9847f909e940
6485 F20101204_AACSKV droll_m_Page_097thm.jpg
54f872685d20b962e8a0e71a494eea99
e02e556f97b72e9cd3424898eff119f6932d69a2
F20101204_AACRHT droll_m_Page_006.tif
9d590622c857f3b9891fe0ed599019a6
44bb1fc9d9c52402aa244b4cd56e7e88142a4e5e
F20101204_AACRIH droll_m_Page_027.tif
a46c0e01f04264965c0ce610bc1de328
5d4229424c7936718e2398e2fe862e72d5ca4592
15090 F20101204_AACSLK droll_m_Page_117.QC.jpg
3e05e7175068e3c899c44de729ff35a0
b2587f5be12336f10691acddb9652f6d784aca3c
4998 F20101204_AACSKW droll_m_Page_098thm.jpg
f1a4ca04bb9f69900a2d3a3363b1dce1
85f8ba5eb9a4da3bda8be9b0327a1d4db74873ec
F20101204_AACRHU droll_m_Page_007.tif
bb5563cfd996ff56e046510062df7ec9
87d9cc689428918c34359daca559b61c77a1ef5b
F20101204_AACRII droll_m_Page_028.tif
9d151b27e91dec6bd1568abe5e544060
e4e26190bf3cb9c1351ca7be8db9ca8a400a8ec7
17245 F20101204_AACSMA droll_m_Page_140.QC.jpg
227f88de0d3130a790c5529eaf9cff7b
f1b51e4c9b539c037b336cdf76effc1cc50aa87b
4980 F20101204_AACSLL droll_m_Page_118thm.jpg
076a4bfa0e582c3903fd73f8e27c8077
af1998f844030501c9a38aa30063f380b0e760d3
4656 F20101204_AACSKX droll_m_Page_099thm.jpg
6735c6776f11efd2516a99d5a0840620
6680c999b262c339ef250c5f0ec44dd2e2f9e946
F20101204_AACRHV droll_m_Page_008.tif
44169ddeac336d8785dd7ffb8d3cae1a
b5c9dcaaf39ef43e53bffa61c61f960d390d803b
F20101204_AACRIJ droll_m_Page_029.tif
71296331bbdbeb89cd0e7772bf20c9d3
a24612727f783f01d27d25feacf0be53ae20bdcc
6025 F20101204_AACSMB droll_m_Page_141thm.jpg
16968e02765b4b1435f9fe042a445ca2
879d7fb92c58cae178a025ac6a41aa3032a29fd7
18205 F20101204_AACSLM droll_m_Page_119.QC.jpg
6ac982d0404e4f1d3805112275d36077
1165c090cba638b55e79ddce71cc49d04c808cee
15840 F20101204_AACSKY droll_m_Page_100.QC.jpg
bb9e528bb2d510e8f6ba93cdb6140364
af1e8a074eee509ea61665dca0d7c835f2d7ee12
F20101204_AACRHW droll_m_Page_009.tif
96b42328c7447652cf84ae33c77510f3
612bc47518fefe65e844bf58f0b55a7f0801c787
F20101204_AACRIK droll_m_Page_030.tif
0b34d24704fa70bf5cb70944dac5ebb8
d63ba3c7d492b29411558b6dfbd5bbbf0c30aa41
6241 F20101204_AACSMC droll_m_Page_146thm.jpg
e4bcc3ea8b2725b92a56f069fea7bc86
8660257da0272dc8476cff484dfe6e094e5cfb0c
5744 F20101204_AACSLN droll_m_Page_120thm.jpg
e304a5af2305ca78d2c54a9fbb76011c
fdc6272e61d569dfa391215e0077815b645c565e
4755 F20101204_AACSKZ droll_m_Page_100thm.jpg
e7a31a8503ad1adcff7bf3d3129b70e3
df276e0d2825f738179451ac4896b0c0896f9962
F20101204_AACRHX droll_m_Page_010.tif
753348c390a655402e29be76076189ea
959c55ca632e75fefbfc1fbc23ab6067c938b499
F20101204_AACRJA droll_m_Page_046.tif
99ca0d7fe67a5dfb7518c73345e85af2
25477348e9a80b18ed7646508723400d58402d32
F20101204_AACRIL droll_m_Page_031.tif
10ebd75628cda0400c1cafe0300efd4d
5e982d80c75ce011b3933304c069aafc6da66948
6283 F20101204_AACSMD droll_m_Page_147thm.jpg
63f049cd1fc73c916eacdcf5d0f42a5c
5ec2941803c013dee2d96f05de18886b644e49ae
24616 F20101204_AACSLO droll_m_Page_121.QC.jpg
1e05f7b5b3b2c78b08e0d022a4095bcd
5a7909ef42c49772990e9c17be9f99a684b96033
F20101204_AACRHY droll_m_Page_013.tif
83d4a3108bf01f7808d8c1b891d6c5d1
5ce54c672a0e095bfb50dc58bb636c803549ab46
F20101204_AACRJB droll_m_Page_048.tif
0a84135fdc4a9fa3a00050cfb42d6273
5c3319bf5cc3caef5e4112ab4270abf0821735d4
F20101204_AACRIM droll_m_Page_032.tif
223eb0acebdb758a9314588ac7ec2ecc
5ab51dcaa2fe8ea5edf9d6aa2d262d0d1511fd84
19796 F20101204_AACSME droll_m_Page_148.QC.jpg
566cc0194d416e77e3ed33bd93227b5a
1689793cc3436af875532df20be00c373bbceeb2
6793 F20101204_AACSLP droll_m_Page_121thm.jpg
c653602e0d37407bbe50ab04c5d40b7d
2d763ffdaf371d8f07bdeb5f4cc49ba1316584da
F20101204_AACRHZ droll_m_Page_014.tif
644579e7a4faf4318231dbfeb533567b
3ca72d58ddff0e7857a723ff1750815983003466
F20101204_AACRJC droll_m_Page_050.tif
40a0a048d25308232c51ef9af16bb20b
3966b291703ff54579ce7277f60c4f96624880e1
F20101204_AACRIN droll_m_Page_033.tif
f52ce2d56142671d6de3ee4bc41e8372
66a59da048802a4e361fa6feb7b9823644923f20
19688 F20101204_AACSLQ droll_m_Page_125.QC.jpg
45108379ffda40dd545af5a8207ed03d
2c5170c5c0d4f4753f5382897395385e758f4116
F20101204_AACRIO droll_m_Page_034.tif
cea284d09d5d6677b008fbfaf31747a9
7654d8b670b4cbd2dcea2008d15209552fb72893
5650 F20101204_AACSMF droll_m_Page_148thm.jpg
582cc5cafc1ce844a7ef535e985f2923
cc3e42f6c8059ee26f62b1eabbb4ae823db68584
5665 F20101204_AACSLR droll_m_Page_125thm.jpg
8db433bb1d68412d5cb39f700afc2ab1
13a37d0e6a12423ff7257b6a0612f23663662b8e
F20101204_AACRJD droll_m_Page_051.tif
b64565e69f6b15911835793c650ab621
daac8b248aa137bdf56e9508d03d6ea894e2fd41
F20101204_AACRIP droll_m_Page_035.tif
3bd4ab9b21b8c9eb760d15cad57bccc4
30a21684313c3bb6a8b41ffa04829151773d3135
6011 F20101204_AACSMG droll_m_Page_150thm.jpg
4d7a9c9562309b842e2def514a88bab2
423d7ab0e82cf87ecf573abeee1c0326b18a8c08
18733 F20101204_AACSLS droll_m_Page_127.QC.jpg
11953f230c1becf4f7acbb7538cd21a3
c0d2011a38ab1d7523bb5c84c96e0456e9a64191
F20101204_AACRJE droll_m_Page_052.tif
64889e3f8e317c6e63971f44bead7ab5
c83deae848f50f462a130382ac275058747757e3
F20101204_AACRIQ droll_m_Page_036.tif
0f1db84d87cd9d537acb3611b0fea7a3
8a1dc8e29604f6c3d8e6d4c8cbd5cf5cb5b96d32
6428 F20101204_AACSMH droll_m_Page_152thm.jpg
e483a23f70d9fe933042442992e2d507
3e875207fab8c194cc96f6a19221af6c370c47ac
17526 F20101204_AACSLT droll_m_Page_129.QC.jpg
eed2861af15245dc96ea5f8602675c36
56277a246397b1322d01eb98c3bc14549d121e62
F20101204_AACRJF droll_m_Page_055.tif
de555e54111b86837ad00903b3d9a053
ed3f69ab15da7406462dd685d7282ef598ebee4a
F20101204_AACRIR droll_m_Page_037.tif
4104dbc2d879cf83f4a2d448df64996c
e6caa239a2dfc6b2aa62fb384bf0177b4bc67c07
24189 F20101204_AACSMI droll_m_Page_156.QC.jpg
cdf2aa837f87afe28ec906a9f773d8ce
359cdac880d07915b0cd6fe971ce7a8b69fcad03
6561 F20101204_AACSLU droll_m_Page_131thm.jpg
2239ec75ac264e056c5e5adb8f4aa46a
4e8976c283ddfc0059cd943085fb9e959e0e0678
F20101204_AACRJG droll_m_Page_056.tif
8696c104397027143174d4981efb8110
70275aeddfb2d760018e26aa1ccfaba5214e5089
F20101204_AACRIS droll_m_Page_038.tif
c5faad9dcd2d251fa605924443ddf338
612c3638840f8290aeeecedc24d470d3fd9fbbdd
6639 F20101204_AACSMJ droll_m_Page_156thm.jpg
f2710f94cb621fcdf79cabd9cf187b14
0e52d52dd15883cb19f69ba73c34cb0e52fbe4b7
22891 F20101204_AACSLV droll_m_Page_132.QC.jpg
58d9937593ee6e21d15665e6416de525
f3f95686dc6fa1f288c32607260b10f9bf68e59c
F20101204_AACRJH droll_m_Page_057.tif
5789de51dc4228f1d085aebb0d3f8172
f028176b08022b1600c319e4871df1242f4c4a73
F20101204_AACRIT droll_m_Page_039.tif
7fb2564753c4f52beb4003b10e5d4fbe
7022351722932f2d066ac9ee6b15c12a3d699d23
22705 F20101204_AACSMK droll_m_Page_157.QC.jpg
a0e229ae6c1dc496ca061d9e6bda1dbc
f4736397947f0c38e9b8c65a4a38ca99e9aa1472
16063 F20101204_AACSLW droll_m_Page_135.QC.jpg
75b8b9ccadb8a12d17d2aa0c4dd4ac9f
3de91abf794c1a2bdc05b8073a531670d4e8a328
F20101204_AACRJI droll_m_Page_059.tif
5a9b8f1ee1843c155137e6a813f4503a
4c6f79bbd51b1e8a79bbb52ca1c168b1c2b80e29
F20101204_AACRIU droll_m_Page_040.tif
4f17cee87c6c756852b1844fc432bfe7
e62a4a288a6601812309a34a31a70f35b016cc76
24242 F20101204_AACSNA droll_m_Page_178.QC.jpg
1d6f689ea38be2cedea365a66b3a3ad2
aed8fb93eafb56d0e2777775e26855d7bc567e13
6435 F20101204_AACSML droll_m_Page_157thm.jpg
b0eb0d418bcfbd51f6cb8bf8f3176aee
2cc10c8d297ae504b92c5054b1e0eee29140de0c
19511 F20101204_AACSLX droll_m_Page_136.QC.jpg
a5bd57765f634e652f195ff1dc691dc0
c3d220ea2e1be750069551e9e5749b3c3d59d1a0
F20101204_AACRJJ droll_m_Page_061.tif
c355b7d58d822943b89b11c9ffa6d441
32804680185fb8b27c2c5782394060b923ef3e5e
F20101204_AACRIV droll_m_Page_041.tif
36d3fb54fc54033192d32b8b2232aef1
83ed6716e235df0dbf4ee075025e8723c3aa71ca
24436 F20101204_AACSNB droll_m_Page_179.QC.jpg
503283f62b29a65c7476406ff0e9fd03
03aa9fd848de7adc0a6fdb551038de18fd6f318b
22674 F20101204_AACSMM droll_m_Page_159.QC.jpg
91694d29cc0bbd451b27cc372e025221
df0d18ce6faa6107fdce3d2e928e26824629462d
5704 F20101204_AACSLY droll_m_Page_136thm.jpg
ab907e962f75ce1d95c18c904f0f9821
575496d1ef6f8c3c3107d2f767c333522a1bd875
F20101204_AACRJK droll_m_Page_062.tif
83a0a98e161de65cb90422124d5963fb
15c0390d920555fea5677d2e2e703474e7f3e75f
F20101204_AACRIW droll_m_Page_042.tif
de53669bca9170729bb6790d8c036d24
963bab8352f8a1fef3f0ca5bc241f5f1fca1897f
23843 F20101204_AACSNC droll_m_Page_181.QC.jpg
26f4734f085549c4138064056dfa476a
a40b7ae2febbcf7662e6c91070705ddfc632b2cd
21017 F20101204_AACSMN droll_m_Page_160.QC.jpg
876418cc9a4f7f77995416015ed2f776
ab6f7d1648e5f2831ea06c111eabf7ed439e3e32
17876 F20101204_AACSLZ droll_m_Page_139.QC.jpg
a8183a34a920e42a2623cbf7eab3e128
ed95a5e084956d88ea2bae44c33f79500d2b3b82
F20101204_AACRKA droll_m_Page_082.tif
4a8a2805ca4426e723af1958fe8c111c
4947bc564bac2c6b63f6ecb54e1d421ccae27e56
F20101204_AACRJL droll_m_Page_063.tif
61da85f2b3b57e24261144fe529e9c79
d8dedea13e68b512f1d69629e200111580a403ee
F20101204_AACRIX droll_m_Page_043.tif
596c68126ee28de4c77b8837e07ada66
a8f6960bfd3bce3d547dae76e934296384419654
6449 F20101204_AACSND droll_m_Page_183thm.jpg
6742ab1eae5335a2d56f14de8d954d07
cbc6bc2365cc1dba0bf374299d009fc791b8bbf4
5978 F20101204_AACSMO droll_m_Page_160thm.jpg
86b94cacef5e403c05204f0a49ef9688
6631a472ea4e7583cd67b481cddcfae1e48e861c
F20101204_AACRKB droll_m_Page_083.tif
3a537902efde8bcc31fb3a2a854cc7c3
7ea65466c11f5248d69426b906d87716dc67ae4d
F20101204_AACRJM droll_m_Page_064.tif
858a28158b7e4c35b5444b6524b97941
710c880a7cab380be387b32ce0e9db1a0e42ef42
F20101204_AACRIY droll_m_Page_044.tif
3174cf972c7b0245f9656064fbeadbac
b0794fb2f3d39abd2190b85243c8623b39567d4e
24232 F20101204_AACSNE droll_m_Page_184.QC.jpg
738dbe42d76c3cf730d45a2ae8b25560
691b1a81f3c4d6143b60e03e25bde596403b83ec
23084 F20101204_AACSMP droll_m_Page_162.QC.jpg
bdb2f30e047a03e0879fade8699e7012
3b51b73e5effc91b94b8d9ccb3dad28345403f7a
F20101204_AACRKC droll_m_Page_084.tif
a9729f6954a594664ff3681bd5ac68c0
b4f104d2e0d9d4920bdc2c2e24c1c8dadc1755c6
F20101204_AACRJN droll_m_Page_065.tif
bb6373f5ac75a5e9747a4f566cf3b522
6a90baf29b1efeac7dbc259013b036dd4b73423d
F20101204_AACRIZ droll_m_Page_045.tif
30b8eaa760a0ce0746da42c50b8a26fd
d07d585d98a3bd5ddf5889853ea7f1190f9d912a
24739 F20101204_AACSNF droll_m_Page_185.QC.jpg
95d079798cd2a8af9382698c9c1fe73b
5aca90531d97f60d3efea190a7aedfd888beb094
F20101204_AACSMQ droll_m_Page_164thm.jpg
73d06f6caeaf9eab9d09f79bb73368c9
449431c76fc27882a62486625d532442ebab6aa0
F20101204_AACRKD droll_m_Page_085.tif
cdc9edcaa735a578ca313b998b13e14b
88dca0c15ab4c98a7c08bd5a1511e38191a764b3
F20101204_AACRJO droll_m_Page_066.tif
1bd735b1fd74267f5010df8d0f044a91
777bf6116516a089d9ecf6f071dd8a515742d085
22575 F20101204_AACSMR droll_m_Page_167.QC.jpg
e630fa9443f54fc238ab024a4c678af0
470b9baa75e1cfed8ac4fadfc7468b392314ad76
F20101204_AACRJP droll_m_Page_067.tif
4c6541ad2817f78d369db4c9afda1059
576e9615814c569429abd397e01ec061f0ee3106
6777 F20101204_AACSNG droll_m_Page_185thm.jpg
599e4d1a8e2f56964ea38da0a10001b3
bba2f6bc6cdc714b1e98d191b26afb11ae78ed1f
6637 F20101204_AACSMS droll_m_Page_168thm.jpg
8287d8818dbfa648ead3feb7b6390dbd
a9a5c69b63cf428f86361a2760b27185bbdbb9ab
F20101204_AACRKE droll_m_Page_086.tif
ff8b0224c079010e5c46a0db0faeeae8
1d1a920b0bd9a0cf27778e2948198bcfedfa62a9
F20101204_AACRJQ droll_m_Page_068.tif
00d2ae9ba517e9f98cf22ad4e58b23cc
10f04eef0e21fcb45dcfd7ae73fbdc909c107c53
6359 F20101204_AACSNH droll_m_Page_186thm.jpg
6201386b638a8beedce6c96414c269dd
c19dfdabb870e9adbcde45f27a4d901cd48ccc02
24856 F20101204_AACSMT droll_m_Page_169.QC.jpg
d216aca595172856299526272335f4b8
bedf333d3f3d7f143902841e45c58de141e127b0
F20101204_AACRKF droll_m_Page_087.tif
cdebf826db7963fb44eca442cc610c3f
cbd351d4f123213e1aac2ad3dea3b50bbab8ba2d
F20101204_AACRJR droll_m_Page_069.tif
7cf3b87871db0ff81faed7567eb94c34
3ab8a9df41e4b4376a6c7466aac292b7ebec81dc
6516 F20101204_AACSNI droll_m_Page_187thm.jpg
85ad03ea98238c8d626c43a9fc461d3f
2e08cbd062fd6b6e7d64dc54cd1436b5f522ea8b
6757 F20101204_AACSMU droll_m_Page_169thm.jpg
935b950f724809a3c1687c5de1ddb8af
7d0888f44fe38fc058937601f04da3df666d5490
F20101204_AACRKG droll_m_Page_088.tif
745e73273bcf93e5b110dea4371e656a
0466e71ce3fdf86bff240942566e816ec3049fe0
F20101204_AACRJS droll_m_Page_070.tif
d16627ad0c33eb2508ba5d00d412ad01
9454b98041373c512b069d70d027ed0eedd77ba0
21696 F20101204_AACSNJ droll_m_Page_189.QC.jpg
4bf7ef7afc4b43941e9d4616be52a53d
94189d605ce16b0e1171ce2c870bc299eb801548
22694 F20101204_AACSMV droll_m_Page_170.QC.jpg
50ff0ce2d8edfb75d0451c1bb3214712
898e9e360b80e8b99328a8f15aca2bb0690aa11c
F20101204_AACRKH droll_m_Page_090.tif
6021ab197bc9994488e99997fe45662b
232433ff70be17a37b65b5dc95935004d6410f40
F20101204_AACRJT droll_m_Page_073.tif
02c0858a7db0e506d844a31b7ae9071c
0f8588a01ca55c7b1585fa84529a3ddf829073db
6503 F20101204_AACSNK droll_m_Page_190thm.jpg
ff3cfebbfd4bbdcc69ed19a1bc70426b
59bfa64e243da1999176be27cc1bfde95c079928
25415 F20101204_AACSMW droll_m_Page_171.QC.jpg
709912dc0b3b3a418626066cc517e2e8
ef7f708e7a1a075f93676e42d2606cc2c07156df
F20101204_AACRKI droll_m_Page_091.tif
70182cad458e1c5f7270e2f75bef918a
6d9b785ed2fa138ed67d979b7ebc8e745551063d
F20101204_AACRJU droll_m_Page_074.tif
44c2979064ec42d5b0e768d142244f04
0eed8cb0da02bca8b1d613d6a4e6ecd5fb41a1cb
6149 F20101204_AACSNL droll_m_Page_191thm.jpg
4e69bbb07190211e4608086f0c00d1b8
0db4158c4112e46906313edf9cd4fd0de33c24f4
24168 F20101204_AACSMX droll_m_Page_172.QC.jpg
ef9fc6804b86fc080bdcb40a6250f1ab
537809984f15c9412b6951bdb988a4a4e646a1d3
F20101204_AACRKJ droll_m_Page_093.tif
9d8ba42c57ced62cc405c29112113793
3a30a9ddb27a883109045ddf7fe9020a032933e0
F20101204_AACRJV droll_m_Page_076.tif
85021a5acd0e6871c3458b5071b9208f
3af192214871d19a01f260e46d5ec0e3f390caee
21203 F20101204_AACSNM droll_m_Page_192.QC.jpg
23351b6ab4942d5584fcb7dd614acff0
a6f7c436ead0b3e807475f7fc8dda1bb8816eed3
23964 F20101204_AACSMY droll_m_Page_175.QC.jpg
04f4fb8b92319663b1f3d4aedf8a9e03
a884a3d15246be0c8c52b7366895be4d791a8448
F20101204_AACRKK droll_m_Page_094.tif
d67dffca5fb1deaf6bc81a65f70c4032
47da48c8cdedf8e372a74621bd7400d0cb8aed9d
F20101204_AACRJW droll_m_Page_077.tif
d4bd16191d51a845a19cc9d83bf72919
aae1e070c4936a850471a441945df2ba2da97276
6085 F20101204_AACSNN droll_m_Page_192thm.jpg
c817a8b358d6303b3bd4ff7b35d90a26
08ce31552ca6d458fb7eb95ac9a80528b24173da
20722 F20101204_AACSMZ droll_m_Page_176.QC.jpg
9a60b2219c14d7adcdbc56f03984cc8c
cb3e0c8347897b8ce7490dd94fea69b589110a01
F20101204_AACRKL droll_m_Page_095.tif
97073d022e02c060f86c3f16f1430dee
6b54469238f46294c234a051a61a7a280ffdfd0e
F20101204_AACRJX droll_m_Page_078.tif
ffbfee68daee0f78697c4d1f46c69319
914d8100b3242f4b349e50fbac1cf35525206aa9
F20101204_AACRLA droll_m_Page_114.tif
3034e3874729734c3cead187985e3bab
1641df61d5987633c26ba1694ffa4075ef5b365c
22341 F20101204_AACSNO droll_m_Page_194.QC.jpg
68dcdd669c1d9f18fae04e588d5c670c
31157b384107f80ef245b426cbfa284f1d8f98ae
F20101204_AACRKM droll_m_Page_096.tif
31a4ba9d6b05f33a6d42b58d334b9a33
f5d362b034a26bfb7eb1720d1eb447289bd7006c
F20101204_AACRJY droll_m_Page_079.tif
775c9112d7135b4c523a0667265e6658
5391c1571463c0515482a0c36c7c31f801e03407
F20101204_AACRLB droll_m_Page_116.tif
6cc950c1a612590562f1080d2737cbda
3661790b93bf60958b9bb7a41526a7212b01c254
F20101204_AACSNP droll_m_Page_195thm.jpg
f670af56eb171c293cd54579e9677c66
5458f042472c403dd5f70797a385aca1154caa8b
F20101204_AACRKN droll_m_Page_097.tif
480a084c7237d5144b63195e8c54d471
82440792583f086ac663c07d467a993c0ae50b2a
F20101204_AACRJZ droll_m_Page_080.tif
762b8f83074f28a3c9b4a160eca3865d
e60012e9e7b82f3c2b3d5e2a0cd04477c8ecb03c
F20101204_AACRLC droll_m_Page_117.tif
17bd7776bdec0b899b5696ca043d6ae1
99cda8aa876e8ad60587f5fbfaebefe76bd5fbb2
21526 F20101204_AACSNQ droll_m_Page_199.QC.jpg
d3a5d14145ae3a06c8c36940bf85b8f7
61b02bdef2ba413a7b45aa5732c28f62ecb50422
F20101204_AACRLD droll_m_Page_119.tif
f766597954e2a0caf6a8510e00bc4a72
5f080f25e28aceb7e092e691795bc732640f29cf
F20101204_AACRKO droll_m_Page_098.tif
2eefd2c3d930f277f2000dbe796be372
13400bbd2783fbbd71da752925e21fe128b9c476
F20101204_AACRLE droll_m_Page_120.tif
d1cb56ceba8b3a455b4b58fc4a1d1a72
b86eb462f101081f1c1e47c62c11b0f134bb9562
F20101204_AACRKP droll_m_Page_100.tif
a113da4b1d05bf6d8e674c0c3e960aff
1276c4267f0fcf5389bc3754a9cc99f998b75c80
F20101204_AACRKQ droll_m_Page_101.tif
9a268889257ad86f2dc35b30063bae54
076693e431c013de91697171d1ecdb2042daa9ae
F20101204_AACRLF droll_m_Page_121.tif
a5487be2c6f396e94b4f42397e52ddd0
9e76b6f7edfa72040e9e749f47b259196cb46947
F20101204_AACRKR droll_m_Page_102.tif
9d9d6c131c71a52afc629ecb425bc45f
ff7f1af63dcdd04cc7c78129628ff3abc24b67ce
F20101204_AACRLG droll_m_Page_122.tif
9c5350e3e327cc5f0eff0fb83f81673a
5648221765ed26294aad1f87fd9bb9fb2faecc00
F20101204_AACRKS droll_m_Page_103.tif
ad0462e3f3e965bfc5109fce45bc1323
6af6a6ed52b28521cc123d844c027733f4666c44
F20101204_AACRLH droll_m_Page_123.tif
4e45e90f6db5ee91d157ea385268456d
b2d9b515533e2effd13ba82b7d982e2328099014
F20101204_AACRKT droll_m_Page_104.tif
daae9664face937669481ceb3200d1f7
1c68de83b419f4924343a241012b56cdf2739351
F20101204_AACRLI droll_m_Page_124.tif
9e9b62ca8f3e19d73a4de9001ccd5f72
02ebe98c3f16757f5238f383426ef617e6e28867
F20101204_AACRKU droll_m_Page_107.tif
e830b0799fe66ca4bae438111a894d97
217a9406e75916708f2c54801cde8aeb7684da1c
F20101204_AACRLJ droll_m_Page_125.tif
56fd0e03ddf4e60df76be02326a44d70
1e8ec94ae817a07cb1e39234f9f9077520747dd9
F20101204_AACRKV droll_m_Page_108.tif
fcc1e63db0a031572ddf48a63e37150d
e7eac797401c65f0ca40839a21d0500d06954e8a
F20101204_AACRLK droll_m_Page_126.tif
a1033642d5cf49a99f7402b4a151a33d
ccc5bb13f6a86a8b50505ab5a7cf057c31512391
F20101204_AACRKW droll_m_Page_109.tif
440600685128500f00c52c38cc3cb7c7
871390efd1f81f27d246a3d22af780dffc44e555
F20101204_AACRMA droll_m_Page_147.tif
dacb0a5845bf455659409e65fe5dfa07
af1be01a97441f41b30601b587eafa7b8ded0c53
F20101204_AACRLL droll_m_Page_127.tif
f6b5fbe2089ad461fef59e2af24f5916
92116a13402bf50bf034060f2f1ee605f5bef8a4
F20101204_AACRKX droll_m_Page_110.tif
544d2a881df3224fc27793d5f9d945f9
f19a32618806365b74afd0be1de0d514cfa9723c
F20101204_AACRMB droll_m_Page_148.tif
c235462513883d2d64a2b86ff53fb3ea
c38d2f8e13e757aa9d5da0ef557c61bafe0e18c7
F20101204_AACRLM droll_m_Page_128.tif
ac00860db03541080648cc13acfde922
bb86fe100a33f66750b4db6153b7654e8e5873c6
F20101204_AACRKY droll_m_Page_111.tif
eba9a3e78854f6a4f8db85f939f88706
feea41f71c36345b4f4cf0c343691eef5b5f9846
F20101204_AACRMC droll_m_Page_150.tif
a58074e1bb712728c51a1ac87aa55d7c
446ed73bb0656795e409fb20faba0b865461bd10
F20101204_AACRLN droll_m_Page_129.tif
13a5a595197db882500d639ad27da459
78be3d4d5e34ab926adaea5c91078cf88b097223
F20101204_AACRKZ droll_m_Page_112.tif
4c8afdbee545fe1184ccd7caac5b0aa1
ff8cdc471b5edfc7b5f4f5e724b52e6a6af73e5c
F20101204_AACRMD droll_m_Page_151.tif
4f6f42fdf49f675c12158039bdfeb7e0
4e3fa380a3ce91950a0f1502a967cd1ffae45157
F20101204_AACRLO droll_m_Page_131.tif
bedfb6ced87229d44dae5bdda7523aca
a29e83f81b6bc1529dbb9430ad19ce25712051b0
F20101204_AACRME droll_m_Page_152.tif
e5741544bd7dac7f6d733e285585cec6
56aa81bccda4ca6d47538c4da1b2a8c1db90c6d5
F20101204_AACRLP droll_m_Page_132.tif
afef3adf540149a631d0ea6f03f82e16
3083e20cb9fa9db1fe307bd3ebeef5cbe4e8c811
F20101204_AACRMF droll_m_Page_153.tif
3fd862361ea26f9383c23fdc9639cb97
a52fe3952d4ff74c1745cbcc6f29018e651c03c7
F20101204_AACRLQ droll_m_Page_133.tif
1daf17badc45deeb009f8b1ab5fbc051
a9f1fe4ec2a98b987af345fb4488a407d183e0fa
F20101204_AACRLR droll_m_Page_135.tif
4627e79503607d4b7e93995397c8042a
039c157b65476b0e10d67245e0f32c5237074928
F20101204_AACRMG droll_m_Page_154.tif
1fd6a258ce00a1087c12491b76d21f00
b89d5107bcb41fba373a0861055046d62d4ae197
F20101204_AACRLS droll_m_Page_136.tif
799a0236768a9a95211391ab9d2b9843
dadcbfcd2f64dc88a90097b43e358f4ae726f574
F20101204_AACRMH droll_m_Page_156.tif
e734fda5a24a25e4e209a6edadc26609
b8c5a45b77f455ee8fa82017a9620e897beb15a5
F20101204_AACRLT droll_m_Page_137.tif
61cc40e9562530043f4e959dbea820d4
0769baf8a38d2208151392a05e8b07862ace814f
F20101204_AACRMI droll_m_Page_157.tif
3d38421b49ac51ea141d9eaf02120aa2
85068555b11bff1ed6cd43f9695350535170051a
F20101204_AACRLU droll_m_Page_138.tif
a3e97f95d2d57ffa6a6b138c9fc76322
e63f6a087a943dcf3ac187e0c260a1db5f9bfacc
F20101204_AACRMJ droll_m_Page_158.tif
ea246ac7fdf0a04aa22052aa892439b3
b3a06c4aee0c5f5c9b4e0b1095d518a95e1a6c93
F20101204_AACRLV droll_m_Page_139.tif
a4d50132825b417e7f4e16220f292e00
e306ae461969761e1290e746cff9c9d603120918
F20101204_AACRMK droll_m_Page_159.tif
b54dc373f04cec3f733160e2cf1a8ca8
2021fc2dada2e4da521c1359e698641c87138292
F20101204_AACRLW droll_m_Page_140.tif
448afe677be186d78a0d4a0bff48feb3
b0d6513cdbd36bc9c892ee235dc79373d573b644
F20101204_AACRML droll_m_Page_160.tif
9dc40dd68a53c729a6ec75dd15ff1c18
49fe39dc353fefa2e8f5bc8740eb0a4178c7f069
F20101204_AACRLX droll_m_Page_141.tif
fffc38c43bbd733503e4a0d2707f07ed
d4b882d3cd7ed37c7ee42a132976ff9237fca456
F20101204_AACRNA droll_m_Page_179.tif
4357ef0a591d57f684266b577c5f1ea5
b63a6cb8fa9b778d792bbb90e6bfb7443e262d55
F20101204_AACRMM droll_m_Page_161.tif
b138bf76cc3059644764c4a3e207d0ea
9cce20b67e77ee258e410d46152deac4c965291b
F20101204_AACRLY droll_m_Page_142.tif
7d7029eea4c0e56e7f82f37e47f8e601
a50733de67b942d56b6422ac1454c0c2f06369a8
F20101204_AACRNB droll_m_Page_180.tif
492eeb8973feafcf07a8f66a806736a0
c2a10f2e7b969d659f58cd8203e24bf540f75285
F20101204_AACRMN droll_m_Page_162.tif
fb6f2aff74563e26178b6ea9abb6b2a0
fa8968fcc5aeedd4bb305e6297d99ffedcacad99
F20101204_AACRLZ droll_m_Page_144.tif
333a62b4455c9429488bb75aca8b8c1f
44feb3cbb1af4adadef29df719fb45920abd51fd
F20101204_AACRNC droll_m_Page_181.tif
55145770230aee5a1cb84d87cc1a512c
2f7bbd67db284902d4db60134b5cd4b1937762fa
F20101204_AACRMO droll_m_Page_164.tif
e22d7a58e4588c49328ba4e23434f334
7dfb16a92c28aba49ba665142c02f1ff975b481d
F20101204_AACRND droll_m_Page_183.tif
963b743a21452a986b87da28ae73390a
892a958f6422323b06416b4b7661be498e3c5bf8
F20101204_AACRMP droll_m_Page_165.tif
d4e5b4ae9282e6ec37e426da84bab144
2279e2d83a979cc14d1bfe039358c05ba7531812
F20101204_AACRNE droll_m_Page_184.tif
be949e67dbb65a7ea3d0858cb44434c6
b0b360a3671845be573fd0cde7868e37d022ff12
F20101204_AACRMQ droll_m_Page_167.tif
14c901b950cff4d0dc9fb19eb0a6abb0
cfb92d77df30c4225f99717a67563d0fea6a891e
F20101204_AACRNF droll_m_Page_185.tif
575bee110abc5aaf008979201656e615
f2daad4f3520b10f5d7aa6ff6a5850cb673c6008
F20101204_AACRMR droll_m_Page_168.tif
b2222c6494d1622145b67f2cef32b309
1cfd4125650ab75908db90f7dcff42ae75ec0a67
F20101204_AACRNG droll_m_Page_186.tif
c0cabe84c4b3e9ae77e8e639a5fa2023
602b6a3fde1d22b448dfd2c277ace325eba6c526
F20101204_AACRMS droll_m_Page_169.tif
a18c86c58bef4e8bc39f7589ee06aaea
1302ad725b69b83e5d24798bbb6c8031bbbe8ff3
F20101204_AACRMT droll_m_Page_170.tif
c767d7f0f315d8b97bfdd55645e0ce06
ba4e28bf4c59c56162db9d5975c174d0bd5d5a6d
F20101204_AACRNH droll_m_Page_187.tif
b816f7d2bb2c16876d0e50e67ca2d310
93dece463523634a0da23c86f17dd953a05c844a
F20101204_AACRMU droll_m_Page_172.tif
c23db47c23321f538cec449b4559bc4a
61633c0e9062402078bbdb9d22f9ff731115fb44
F20101204_AACRNI droll_m_Page_188.tif
3ed6b059f69237706fae18a2621e8b2f
9bd32c203680374055bd4ae65d28ae1c67758f26
F20101204_AACRMV droll_m_Page_174.tif
e94738f4ecfd3b9573903766c6ebace8
eadb3b7c3e033af79b9d736796739f6a3a60752e
F20101204_AACRNJ droll_m_Page_189.tif
738600015b8e9bbe398b185ea8d52819
5737cb4b3282e9d449101c8475235cf7fd314bc7
F20101204_AACRMW droll_m_Page_175.tif
25359a0c4b88d635edb76eb09ab9fc6e
848b28b1c7bccb5780778f2fc0b0fb10539f2230
F20101204_AACRNK droll_m_Page_190.tif
1c6aa071ee3b008b8a5f9067289c9d34
2a31d984e9b886d72babd955e6e7386a66d3eeb3
F20101204_AACRMX droll_m_Page_176.tif
dd03ead433ca6be6dacb0ffed5db0018
372564011dabbd2c81ae0a0a4323ae3fc2136d0b
50046 F20101204_AACROA droll_m_Page_012.pro
eb2f4189a79bc48503380709a632bbd9
3cdec241b6cee4524fc9baab823f93bc6e9da1e7
F20101204_AACRNL droll_m_Page_191.tif
2103465dfd686659b0966c6da84b7f9e
645eccf77db15dfccded87b970f300a88a532801
F20101204_AACRMY droll_m_Page_177.tif
0868d996bfdbe7c94c43e94ec3f3bdc8
7011febe8a4a564199da523085d014232ee99009
48868 F20101204_AACROB droll_m_Page_013.pro
43b3f8ea97a21fb26e2ca8dd2cac6c90
f60f5aa13aab07ce08aafd418d809b2afdc6aa8c
F20101204_AACRNM droll_m_Page_192.tif
36cf194060d0c893bb69fcebcd4ba20b
2a89c45a59ad2428201b003bde28c415ed1d7b3b
F20101204_AACRMZ droll_m_Page_178.tif
ccf3ae9afa3fbc0ff54973d185384284
ba9429950614b71867ed95202e49f7bf3aa40431
53110 F20101204_AACROC droll_m_Page_014.pro
1ff65e41cc706eb7ea806551ed0a7fef
c99f5b6b496b1d1126d4005945e8fbec927e7ea6
F20101204_AACRNN droll_m_Page_193.tif
1cceb7026c929e51d99d145f0496642e
ba7e6d70f265ef6e764d75ba861e197b54c32b2b
46618 F20101204_AACROD droll_m_Page_015.pro
6d7594eb1edfb8445ff794d8e43390fc
22e434434f12e2066efec01771065a7216c26baf
F20101204_AACRNO droll_m_Page_194.tif
9ab059bac107e25c67091726a810478a
58b766dddb712725b0ceadfedfcafe8b1b6958f8
50278 F20101204_AACROE droll_m_Page_017.pro
399aef7000603d1e856558a4f4bda1da
803b5f3bcc902a115c9217abcee18c47f17fca0e
F20101204_AACRNP droll_m_Page_195.tif
82744c70d4d85c85448761fdeeaa6304
0bc3f1e27394133e3d0b60ecdb294eb2414ed5ea
48981 F20101204_AACROF droll_m_Page_018.pro
ca413f60705a348ac7340a96fd8ce438
00f13deddc00e6568690c1518cbb8777dbc9a32c
F20101204_AACRNQ droll_m_Page_196.tif
f86514de49ac7c6f48ab8a4427a367c3
4db1f9671499998a9ccfa75296a2886129f1b27f
49846 F20101204_AACROG droll_m_Page_019.pro
ff5c8dbbf65b580639d49615c42a6d36
03ef80581de9afd3dacd3d83aef094dd23e8060a
F20101204_AACRNR droll_m_Page_197.tif
1086e83832c97a7c059c2c30daedc255
34549dd2e9f24e529354f46b1b618efa4d40225e
48922 F20101204_AACROH droll_m_Page_021.pro
52e84d514f948dd83326aa9984d3cc2d
2075a93e0a9de52a34205daa2d6f8920c897f908
F20101204_AACRNS droll_m_Page_198.tif
4cc2892ef73f53d03c1a82cd9f21464f
d98b8466957dcf8a0ff4c34a539e8a1bb8011c5c
F20101204_AACRNT droll_m_Page_199.tif
d7ea176755ab49d7cf62a690e910cbeb
f9a7ef31bdee653a85de7732c766a1d481a0df34
44789 F20101204_AACROI droll_m_Page_022.pro
9f60d69b0253e34f7e2882c489c30f53
4dba20f8f89f9e84ff01b3201f044279b996ee05
64644 F20101204_AACRNU droll_m_Page_004.pro
76e93e3777d2b5d7ef029d1ac6b44b0b
9188bd0884183e4fff8be7311df2fcd44b66108f
50139 F20101204_AACROJ droll_m_Page_023.pro
a0a8ea33b330586db9a21a6fbe958063
a1bbd17dfcb79dea31e80682146e930a6c55403d
88572 F20101204_AACRNV droll_m_Page_005.pro
62fd46e2a6d4ade23fd8b5d5ce27d489
877df5a74437758e6a7a16eea63aa953dad74bc3
51237 F20101204_AACROK droll_m_Page_024.pro
2ff62011165fdea8b02eb7ff882de865
73dd8db49f33ae148c9733344b1937d2d670dec8
19141 F20101204_AACRNW droll_m_Page_008.pro
00c05e9a609b2b918debbb6cb9f0102f
d6b9bc4ac6056e4a032a1b95f01ad8228999b780
59083 F20101204_AACRPA droll_m_Page_043.pro
1f4fb0e2c4f10aa4ce2c159196f1c6be
e96af07c73f9ed966bf3cf1b5f5264754ac19c3b
42235 F20101204_AACROL droll_m_Page_025.pro
915ef3d1f27af5a30f19f1e25d3e186b
b891df4bcbb3ff49b37cf43d2daa1d6de2a0d4cb
37733 F20101204_AACRNX droll_m_Page_009.pro
26e60cf6661b2be94756cabcda804eee
bf4c42cfff5e548b2768f97912a3cdbe10a90125
47697 F20101204_AACRPB droll_m_Page_044.pro
5e313bf2a39f7ba2f38ac2a6ab3a40db
f6754f260b9e424127f561d077fee22904d7fe18
51205 F20101204_AACROM droll_m_Page_026.pro
a68d8745a99061a9ec34e5c31c4c9caf
92102a05285a029c97a27f8b05641d48da80d9e2
36822 F20101204_AACRNY droll_m_Page_010.pro
9d8b9378fbe651a77883faa82aa5dc83
139f0dd7b6d9cfac6febbb4e71b0815e12948e76
F20101204_AACQMA droll_m_Page_173.tif
8c597ee135ae9f14758e64f48f775cfd
f17f1b55e4137e89c40f0fd1b1be7d5337df5bbb
47505 F20101204_AACRPC droll_m_Page_045.pro
e135f0b8b79ca1071da58935601c8fd7
c439ef4ec41467476ba38698b91b89f5d955e6b7
43774 F20101204_AACRON droll_m_Page_027.pro
ed14b2b310ab2e871cf3730c8f8b2cdf
a9da390203dd217f81eb723fbd4d5ffa7ff1cf67
44013 F20101204_AACRNZ droll_m_Page_011.pro
c2564478233b19cb94a4364b4d77c9b0
454b178999a541f0ad5b0791c3171244a1521fff
49640 F20101204_AACRPD droll_m_Page_047.pro
be36cb296b741cc8c918b536962f3c5b
24197100d13bc16eadb77b335ee5876ab1106fbb
38774 F20101204_AACROO droll_m_Page_028.pro
76453703ab422ac8dbce82ead5e9b473
2a89ee440a875ca628d6bea5618dc23e26099446
24395 F20101204_AACQMB droll_m_Page_039.QC.jpg
fec316ce589fe712ea8d2f925f8c4ccb
9351f6a7d673dd6c336106289d7e98713c2e64f3
F20101204_AACRPE droll_m_Page_048.pro
4e2ed3b51e2e60190b84871bbc2ad4fb
f5462254ea9a7f64f78143b29781de0be4780035
54578 F20101204_AACROP droll_m_Page_029.pro
730ce59ab72619743e15d02c54320970
6260ab96aa6a30291712bd7f368dcd966578396b
48595 F20101204_AACQMC droll_m_Page_191.pro
a4ec44c0e05fdca51eface75795ebb62
5e13494dfef3ed7b85708881af21a09d4b49401c
46196 F20101204_AACRPF droll_m_Page_049.pro
436cb22353c5974953ab771aa8bda4b9
ad8f5517b1a2d9a75909cc7cce5013d5de661ba9
53087 F20101204_AACROQ droll_m_Page_031.pro
f2707f67e0cf41e2f08aa7c08e781eab
3666ff81e1ca8c9b262def24b81bfba55f6ae7ae
46143 F20101204_AACQMD droll_m_Page_054.pro
17ae7c46201be6b159d7b33e1cfdbe2a
dc2097e7a2d9e10ea87843513de9757a9457542a
58447 F20101204_AACRPG droll_m_Page_050.pro
b6a43ea1a7a7162cc35f31b3afc6c696
261afa2ef7f1f120a80143d6b54b16cadad677c5
50154 F20101204_AACROR droll_m_Page_032.pro
33cd18b02ae55b2b7837ed38a4250f8f
45e34d4539583ca6d04bfd50ead53b3d8aa08af6
108396 F20101204_AACQME droll_m_Page_063.jp2
3ce8ced5b624341e662aaa847504835f
dfa30cdd10834dff75fc92e113dc2f5002cdf56e
51303 F20101204_AACRPH droll_m_Page_051.pro
182ddd4fab44cdb7561cf2f1a3833acf
f4778fb92624129c99fa6addd0cb983e081e379c
55006 F20101204_AACROS droll_m_Page_033.pro
d29dca0b321c2236ece119395db85386
b5d307719dd97ddd5e5d6106d5b87db0cb25f49b
2348 F20101204_AACQMF droll_m_Page_043.txt
c1c1d6255595d58ece73e872d5c64eef
1b1d4ab6a66ef3dd4ad5999a2ab6d1ac27e45473
52616 F20101204_AACRPI droll_m_Page_052.pro
41cc47f82428164af406ff612c93a794
7ef39290458cdb605f32f9a28f29050f1a733593
53805 F20101204_AACROT droll_m_Page_035.pro
1d4fa43f03521ee3a078bbd18283730d
1814a5323a19acd5d4bfbf7a0bdadd6d2b516154
6472 F20101204_AACQMG droll_m_Page_029thm.jpg
0a132fd92d45b569c18092ba7fd9c688
91a057eca45b620eb7a7c7c9553f0deb639a6808
53921 F20101204_AACROU droll_m_Page_036.pro
39d11e4863a241119e6a2b0a3cded182
3e11001f901d95de841a81beb5e421ac7d5ae4ab
52127 F20101204_AACRPJ droll_m_Page_053.pro
27e1bc18a6c57c2775aa5a240cda38eb
de2a7d83508416013ba7c94bd6313169e4b8ecad
50670 F20101204_AACROV droll_m_Page_038.pro
57decb13431d0a9d2ce65102e866ebd7
72a1ede38384e775ebe88ff67a8d515d07f245bc
2033 F20101204_AACQMH droll_m_Page_039.txt
21649b9bccda0dfe469f16dede7a8a8c
a802491f703d3394cf6a82505c9bbc3ee0db7426
60972 F20101204_AACRPK droll_m_Page_055.pro
a0403dc24e20f5bac8bffa7c575670a3
bf29b1bf64dd494627f5074ff6cf0f05ace32034
51708 F20101204_AACROW droll_m_Page_039.pro
2a798539b87a5f4993143b286120b522
cc3f47bd1e422b1663d96f2898a39d8feefa6a61
41409 F20101204_AACQMI droll_m_Page_148.pro
617583b5262b1aa3f4a9b2c0f268ea56
c73adeeab68a7d96637c105086205c7965bcfd0c
46412 F20101204_AACROX droll_m_Page_040.pro
08789b2bd8f26da4559416d48c542e57
be4106602884401ca0fa77cfd08c47a2cbfae0ae
53768 F20101204_AACRQA droll_m_Page_079.pro
80927acfd0f025f09bc4f0b162f46421
84f901c30d32f5e766ea094b82d9eb9059582627
109049 F20101204_AACQMJ droll_m_Page_017.jp2
163e3d7151bd8ff2ba835945dc712e5c
ecc898ef7cb5ce38941b9d072c719afe10a204c8
13339 F20101204_AACRPL droll_m_Page_057.pro
533573af0eae634576d419ca849360b1
d3e8d903c289534ed0fe23761c04eb317b3f2ac7
46541 F20101204_AACROY droll_m_Page_041.pro
4e1308c8419fb700ed43456451d71005
0d2ec77ec5576b847acaebfe6da7db9317f8458a
51747 F20101204_AACRQB droll_m_Page_080.pro
a2dc1990e7b3610b73ab63e2b1bef373
9fcd97e8f1eeb92a41bdf459a07f723a608fc406
F20101204_AACQMK droll_m_Page_113.tif
bc37a64d3b052c9e084c008c538d1dca
e8d3798834a52c70e42b38d7661945bb08df8396
47586 F20101204_AACRPM droll_m_Page_059.pro
d868e4b370b239c57a8e73623bc62572
923f76f3811ce7d0724c18302dc88aaaaa38d411
64548 F20101204_AACQLX droll_m_Page_120.jpg
5b495415ca10fafb70f4511bc63442c8
d5a89412e46f9a09531cdccb26967aa9a57eb5ea
56866 F20101204_AACROZ droll_m_Page_042.pro
d4661b6b408bb7199798c90bad238ea0
c3a0f41064318dd47065e84fc4a5bf77178f9e4e
2058 F20101204_AACQNA droll_m_Page_172.txt
70d6e0999d8c50aaafc312986f0c0a25
a7de8e3e9dc90971b0b5011a78c8c9d6b62ce8d0
50062 F20101204_AACRQC droll_m_Page_081.pro
17e240476b1c9c06a600af98fc26a46a
94e7cf98d0e16183ea75ed8df931ee3443356468
F20101204_AACQML droll_m_Page_115.tif
6d221380426fd5baa8b1426297bad5c5
9e8aad018d97fa98ef193c7757a37a7de931cd0d
46385 F20101204_AACRPN droll_m_Page_060.pro
b22a2dcf5ec19da45b0debc1881757f6
734bd3f9d5bd05b5222336901decba8460b72424
51955 F20101204_AACRQD droll_m_Page_082.pro
777b43471c5a02107f9a5babff0e6413
2197c50567dcb2ecdb49e2753954859d1faee2d7
14817 F20101204_AACQLY droll_m_Page_113.QC.jpg
8660dc8c69c91de8a89c4c0fdafec8a7
f0f960a0ebe9f2a289d7185941f1eff2a5a8fc34
113619 F20101204_AACQNB droll_m_Page_087.jp2
37656e8520ca924c129a02d13ed77b33
6d49e4cd8547ddd9b69b6a6b5be9476ee1b794d6
50689 F20101204_AACQMM droll_m_Page_078.pro
b4fb3decf640a1431881347fc9197240
7eed5fb52de75af9fca80159e5c1245ad132e987
53128 F20101204_AACRPO droll_m_Page_061.pro
1ed1fb632acd812c476f4b49a12db086
4da62bc363b4453fb2f6eec7fba6c2bd036dd354
31492 F20101204_AACRQE droll_m_Page_083.pro
bc66303c17c73e2dcf51bd9adf1c1c8e
82587fd5990ed0e6b85eeb58f7cfaba6554f4a69
F20101204_AACQLZ droll_m_Page_021.tif
219ccf3712763262135651791be0dad0
223dbe8600781a68e0a51f0e16c6173769d56542
54706 F20101204_AACQNC droll_m_Page_173.jp2
981cff9d57cbc82da2575f5f199c5268
4db5f51ba06eaf0a471cd4f415dd277a347c9b66
106198 F20101204_AACQMN droll_m_Page_018.jp2
972abcab36674c0ffa4d723cdcf5f569
3473d4d65a201d476f84a79d93f0a6110cf392b5
47748 F20101204_AACRPP droll_m_Page_062.pro
f7ec6496bde4276530201cf755464d55
44ea9738b6db5bbf2cd265981445aa30cbca7083
48130 F20101204_AACRQF droll_m_Page_086.pro
ac1da3081ab6fce930373c678ac2df8f
55737c8d3960381bc76c27a27ee56ce9e279ae90
96026 F20101204_AACQND droll_m_Page_005.jpg
7a423348fd9ff881c3e406669f531a96
91ccde76d1222542a35bd68c2758709c37e1fca7
20931 F20101204_AACQMO droll_m_Page_150.QC.jpg
8159129956d7d7e4fef1b2b7be82dfcc
a356d028b17247c12487f60bd948d5a90f97e735
50134 F20101204_AACRPQ droll_m_Page_063.pro
c2acdcab57a6a0f1038ac77ec6433c37
2f8006629d69e132bd3b32bb67227fa633356db8
52057 F20101204_AACRQG droll_m_Page_087.pro
d4fbfff903c59afedcacf229a0a2063a
a39ba99544a0bf87bd26bd709fed7a7b4b76f20c
1051948 F20101204_AACQNE droll_m_Page_067.jp2
c011399174cc3307d7509e09f697b3a6
519f313915294aa144233e7d198af608bf60d521
94308 F20101204_AACQMP droll_m_Page_125.jp2
8a97c257bdd9628b3b1ab96d42daf2a6
c72c9ae797d3814b594ba3832f0138945edc2d47
53542 F20101204_AACRPR droll_m_Page_064.pro
83670a09fcba1bd539dd70f51a0cd154
a59071c194917f4fc4cc6d83b33e037a45f2e129
49838 F20101204_AACRQH droll_m_Page_088.pro
8e879e9fc5a128d7aa5c2c3c4b716860
83a01e0336727ab1dd465f17ed068a3e632de04d
21549 F20101204_AACQNF droll_m_Page_040.QC.jpg
6f51bc3d168c18b661e2554e6b250879
b7d592da9f4c59d2c2ddebeb907a3e7587eecdd5
50324 F20101204_AACQMQ droll_m_Page_076.pro
fa68f2c8d8f03e9c2b54bc45b96c3e26
ef1b90af736403f9b0954b00cb39c5fff8527d76
47773 F20101204_AACRPS droll_m_Page_065.pro
7e2c42f8f8a58ceddd37a78417ef8e33
920ac794e48c562f2caa4a251b5318a045fd30ba
48638 F20101204_AACRQI droll_m_Page_089.pro
0f055f7b609df6601d3db33a6a5a8dc3
04f85742fefe47a9c62f54f40fd93ede2b0cd628
F20101204_AACQNG droll_m_Page_012.tif
95974c5db1efb444df70964023cf1f6b
963d655d078b15a59dcd2b106b046be827cda39f
59265 F20101204_AACQMR droll_m_Page_009.jpg
832a6fed0f23e5f3c1efda2faf6b9b0c
8d662ae7e989a164a125213c7abd72215376f6ad
50548 F20101204_AACRPT droll_m_Page_066.pro
39e381578ecbe5d2d1d0d3a74811ee21
517508cb8be6835d34b912510c0f19f717507fea
53918 F20101204_AACRQJ droll_m_Page_090.pro
73d22edf02f423081fea6a13bc79560c
ebe8bfcc809182b9ba0ccb503b4c667b8d5dcb66
1999 F20101204_AACQNH droll_m_Page_023.txt
2d689c41b5ef4be3d5a627b1b61d1b72
2c4ae0ef43a01596140759c489abb8fd8c901425
F20101204_AACQMS droll_m_Page_026.tif
0222ac073bb0cbe5c5465e6bea95d03b
7311fa8293ac6524cffca1df9852f6944a192acc
47003 F20101204_AACRPU droll_m_Page_067.pro
b7231b168a8cfa12299a6da8c89dd489
8fb89c40e1d6795ae2a2308e7b75b3e8a430d5cd
17252 F20101204_AACQMT droll_m_Page_104.QC.jpg
0f0a0834446d9f832a6b341fe6681717
5f34ba0cb7245f715bf8ca91d1c07548105bb796
F20101204_AACRPV droll_m_Page_068.pro
d2eda1f4325343ccaf53e65b314a97ff
9bf3ff4061bcd8d7ff247b71413bd06a787abd00
52793 F20101204_AACRQK droll_m_Page_091.pro
2f53564eae4daf49187df7f2060530f0
31870740e8cd86ba2afb2ccefb1cc46824b0bc27
23214 F20101204_AACQNI droll_m_Page_131.QC.jpg
dfb4adc4db4419dbc536e3a4dde65536
38e57581124068bf8f98df8135914134dbf24ee1
5045 F20101204_AACQMU droll_m_Page_105thm.jpg
92deea2e71c74a2f7801b662729e451b
d9a48cbf3edb8efb4abfe823c7ea1ce83fc245d6
48702 F20101204_AACRPW droll_m_Page_069.pro
f6f938a5ef035ae3188d29778bf5d70f
891128ee9f787e8920683e43c7a44d6d26be95dd
37995 F20101204_AACRRA droll_m_Page_108.pro
7e734ff72b61155ad5b178cf1333272e
68357e3a4a627921505675399af8bb0973c48a2c
49143 F20101204_AACRQL droll_m_Page_092.pro
ca085356124f936cd92d355662d3463f
08b4ad18da05761a3d945866862b5f6d35f5c510
F20101204_AACQNJ droll_m_Page_054.tif
94e33085761ccaa24b351248d513d6ca
5a91aa396195e1fa455ea5ef49035e691e7c614a
82365 F20101204_AACQMV droll_m_Page_101.jp2
6722d7331e037257feb7bc8b24408adf
35665a9f34445c1b286400d6dd297b0d489b14b6
50098 F20101204_AACRPX droll_m_Page_071.pro
03406b8c5924a20534ad9e4fb5469b89
88d78a5ec3120a1841221f7f95f6dba2cffda8ac
34833 F20101204_AACRRB droll_m_Page_109.pro
abad272f701b4f25446dee6fd8eff7e8
ebe061db02804a93b9800b25508bd22b71452f7f
52814 F20101204_AACRQM droll_m_Page_093.pro
c65d7d3f0d005c5c78680de1a706ddfb
6d1c76dbd7e95129f0eb5747a7aa92ae96c84904
F20101204_AACQNK droll_m_Page_053.tif
74005a9d1b0a59ec98144dbbd0e70482
1a70a4c9f5d04893f7f072573dd8aa6f52e9db42
112066 F20101204_AACQMW droll_m_Page_172.jp2
7281ebf33593793d6c84588724522de1
7e02696b6bdad054ee26ebef80ee580bd6ac7700
48671 F20101204_AACRPY droll_m_Page_073.pro
f70cf2be817003b1245f561df729d032
384849ace02bb858cb4f91fdb69253de1e0df1e5
32492 F20101204_AACRRC droll_m_Page_110.pro
a00f379c014f1c8153875babf9091b8b
21a7f991ef101428f7f12a46cb5cd3fbc6d73be6
51103 F20101204_AACRQN droll_m_Page_094.pro
a75274d4016405657896c1b28598db3d
0d40daa2cbbe0813f54a4b468e451eae1928d978
46703 F20101204_AACQOA droll_m_Page_072.pro
4d75170541c852b34e61fa67523a3bcb
7a967d7013acbec08d848255e2f0e6b6b096b388
54540 F20101204_AACQNL droll_m_Page_104.jpg
7a68b8f3bdb04c4aabeaa170debe487e
55fcaa6cd2ce87480c4ddabac1ae655534a59729
65333 F20101204_AACQMX droll_m_Page_160.jpg
7c5689ba44638c70b25c0fbff06e5af8
2793ea8b5e4b85cee45a714a5b17900aa36d9ae9
50795 F20101204_AACRPZ droll_m_Page_074.pro
4f7a7b407d839bdb7c454afc4dc6e90c
9ce547f0c9fb567ffd86b190f150c2e192c08dd4
38905 F20101204_AACRRD droll_m_Page_111.pro
96731c0c38edb516b87f238fd71786a5
95810856ecea18f3baaba757fe423e6baa0fe509
53218 F20101204_AACRQO droll_m_Page_095.pro
2f2dbbb0992e642b6c95b85e5745722f
7dd901b4e7bc7bed3bf34fe73a22a5d7d0a45ad4
1975 F20101204_AACQOB droll_m_Page_012.txt
48c621bced2c4e4bd9deef8ad971a2b6
d9b6d4cd970770a754e5b8d5a31dbf2b3e862036
59186 F20101204_AACQNM droll_m_Page_130.jpg
7ffdec3ae94b78438f956fd5b9ccebcf
db8674ffe6362d79f1561873f91ec5b00057ae33
49348 F20101204_AACQMY droll_m_Page_170.pro
024cc07b04f07eebc3e497460e1101cc
1a4e9e8e96f9e8eaca9134f959cc0244069235f5
32938 F20101204_AACRRE droll_m_Page_112.pro
2f796cf3dc304f4c4f27a9bb9ca225ba
02d05045ad8b7ce492d92e61bbc01039f49635ae
51211 F20101204_AACRQP droll_m_Page_096.pro
cef243a41e8148b0d6f7557a3e185405
685202655b2cc1267398ee739db473afaff8a7bd
51765 F20101204_AACQOC droll_m_Page_037.pro
a6b5ba92b1f42ae3487c2c797c96824a
06554f4d39abc9a75ee19667a94588ad5eb09b36
1223 F20101204_AACQNN droll_m_Page_002.pro
dab7aa8067eff4f602b1c630c56f7dcd
4db20391331f8b1fa65360ee5b013160e467de31
23854 F20101204_AACQMZ droll_m_Page_182.QC.jpg
75cb9f3c58ec97c75ca195d7c478f11d
b9759556dad00db3eae48c380d03c17d08e6119b
28127 F20101204_AACRRF droll_m_Page_113.pro
42ce229b71f39c2c1f829532c23aeea2
9eff9f3354bb0892a1ff6a9b2e228fe72f1051a3
52547 F20101204_AACRQQ droll_m_Page_097.pro
4d63cc99ef56220f1cf2c7765ef88f39
6e0b095384a20285c2f44acee16614faa40103ad
90969 F20101204_AACQOD droll_m_Page_102.jp2
d8b6449bd4db7c89929006f3bec0063c
273fc5b179b320d74120f260708ee3d9f26cfdcf
57411 F20101204_AACQNO droll_m_Page_144.jpg
fa22d85fefb900e096728591236d8690
715f51770ee327548a0ef475534f109139b1ed05
31046 F20101204_AACRRG droll_m_Page_115.pro
0ce30dfa6083335f1a7618bc21ae4254
d74ce33385d8af2c27e612f1e19bac550f390e5c
36146 F20101204_AACRQR droll_m_Page_098.pro
5bf0129ec66168988efa84e972051cf4
b585041291f8dd230d4365aae2fa198638f92c00
6134 F20101204_AACQOE droll_m_Page_060thm.jpg
0e47e236a8999e7471e1b59bfc2bb080
5b91c58b4cf247b41ced1de820bd18ab73490d15
24620 F20101204_AACQNP droll_m_Page_095.QC.jpg
53e2863ee833e6aeda927f2318df3c33
75081b89e07b6dab7e8e40e340fefc1d8da25fac
35405 F20101204_AACRRH droll_m_Page_116.pro
cc6156a7c93a04ccd13ba203faccc5c0
62d1173ad10f8b659c98344e795c1e0e865502ae
30534 F20101204_AACRQS droll_m_Page_099.pro
f52c40883898a1faefc2550a5815159f
74da05c76b52649f9f758660277492dc05870dc2
1892 F20101204_AACQOF droll_m_Page_067.txt
abcaf785fd95814428f839b51011aa56
9b0e125f07c6b6719dc1b19126226e8b6db6faec
93216 F20101204_AACQNQ droll_m_Page_084.jp2
861d8fcd8cc732d194822ce1e8ccf3e3
29027b700b8a1c086b33fed8673299cf7f7da4ec
31258 F20101204_AACRRI droll_m_Page_117.pro
774ae4f5e7baeb6c4919e24c81d63c78
e7c68c21708cdadd968a92598c9bca60b2c68e14
32975 F20101204_AACRQT droll_m_Page_100.pro
3d72434d628f780faf7f4f716d970550
4056d692342141779844bd9a220c961939a8027f
71812 F20101204_AACQOG droll_m_Page_063.jpg
954370dab8e3a8cd48e5585584b5baed
31637b54ce26c29841661539aeaaba57524a96a2
111146 F20101204_AACQNR droll_m_Page_037.jp2
4995f0e2117016299a68ac01dab2a4b9
f71152118f27a214718b3dbc6bfb196089bceec8
34025 F20101204_AACRRJ droll_m_Page_118.pro
f1455251463a5f48dc248c2a18ff300c
b07b9e5e43ef7a7940d515455612055bb51424c2
34932 F20101204_AACRQU droll_m_Page_101.pro
b5a4198416e61dd48ce217fd66f3fdc4
f65cec1e5dc9796f198326d05afb21b7daa0cc86
126171 F20101204_AACQOH droll_m_Page_055.jp2
c520aadf833e0c2d104de93ae08cef6b
4bf4f549a95bdaa69f312984cf64ff6d324ba30f
110841 F20101204_AACQNS droll_m_Page_034.jp2
97d9ea2be966be597a254cfc5f81c0ae
09e212204f8cebf83154f18e563d54e072531615
41293 F20101204_AACRRK droll_m_Page_119.pro
8e33146c48447966430ee1d7b164f821
de09ba5f0a5e02187bf5dafed44bb2dee1806c5b
38352 F20101204_AACRQV droll_m_Page_102.pro
bdcef80a6cea615bc3e9d0871556e9fd
b0caf3e76c6b4a62f4356077e6deeb12a221c270
6450 F20101204_AACQOI droll_m_Page_181thm.jpg
a143370e436eba17087b54078be962c1
8160bd79ced984ae82b8dca9e2900b6bd166e998
55595 F20101204_AACQNT droll_m_Page_030.pro
891bddb11ca1dbf38abf11a953d102ae
0cee684b47a799f4fc2c9c10b73ba424b13d74da
40108 F20101204_AACRQW droll_m_Page_103.pro
747a474512eb1aaa6b7257a2666365d4
d53ab642c9550dfbe5fb992467e68df769a3fb22
79574 F20101204_AACQNU droll_m_Page_195.jpg
fa1331ecf6540563834a640a8d81fa92
1d6dad032a7d6c68d51236c4e95ba0e94264be71
32723 F20101204_AACRSA droll_m_Page_135.pro
85cdd01126db82f7f80a5bb5ad59e2b6
1f10a579fda7939b70cab02a42b97ad7d08b5606
40893 F20101204_AACRRL droll_m_Page_120.pro
1994e6696eb8cf7c8dfa37452ae079bf
cd4f3b8b406878fe9e752543d8841c2a4a4689da
32362 F20101204_AACRQX droll_m_Page_105.pro
90fcd242d78d0a23581dcd413ae5c44f
166ba9b22afc9dc8fd2764dbee3b193b9bed2081
101803 F20101204_AACQOJ droll_m_Page_054.jp2
8f1db0328c13ffb576f8ee1bec4c560a
45c03a86078fc14a796d848d6b72c5fb26204121
5963 F20101204_AACQNV droll_m_Page_199thm.jpg
21fed2b820473f772e0b7b36dbe85a5b
458e8e0e5945238e3c9a2f52124dbc2dabb2cb2f
41248 F20101204_AACRSB droll_m_Page_136.pro
3ffeb4d6a6595754d3b6020a929ae1ff
b37b8d764a005e59a75a0b1163a3149797515d8b
52477 F20101204_AACRRM droll_m_Page_121.pro
bcf11a83c16580c47f0ed17a8b173e67
d0d66635dda4fdfb1f70b8d9cf40a65ee3cc6b6a
35691 F20101204_AACRQY droll_m_Page_106.pro
08afe439d9ec2c548b22e962350353a3
9526302832c7081c8ea75dfd7d39870b248873a2
16974 F20101204_AACQOK droll_m_Page_198.QC.jpg
a48422a151095df7bb746db1e57c4159
74f691448f4f9a5b9c1be2c3c54d8eb2f277b690
6404 F20101204_AACQNW droll_m_Page_051thm.jpg
d3d6861d01e730d90fe85ac53f9fe582
5f1519a1bd9d28ab9924606f7ae158997aa6acd7
38712 F20101204_AACRSC droll_m_Page_137.pro
8f015d3b43f4a8973bb1f876f294dcba
32190eb7945c432900262e76bb1226a1373ddf49
49050 F20101204_AACRRN droll_m_Page_122.pro
655abe64df15d1eadfb51185fa939859
f904da8fb49507c91d1a0f6939af880c0e094e3b
29958 F20101204_AACRQZ droll_m_Page_107.pro
fe3333a6c9b092935d7cdecbfca1d0b5
f1f790fa8442bf92ddef413fd0f8909183e57206
20793 F20101204_AACQOL droll_m_Page_049.QC.jpg
56be219920090934582e3b308eb2ea03
3baa230e5c14482fc7af93c6a373f0e2fc162941
34374 F20101204_AACQNX droll_m_Page_104.pro
c3db340445d5e1ace90dc7c9830eed4e
f2728ca9aad58edd47bee14cb5577476cad3749b
1794 F20101204_AACQPA droll_m_Page_058.txt
a6fdcbf7fd5a80275a5df7c0c9469690
5ed0607fab90a169601f55cd575fc18b506d21ae
39096 F20101204_AACRSD droll_m_Page_138.pro
bef102509eee23baf216196f761e46aa
b47438ae2d74edbe48eb8c61810d55b812e16766
51969 F20101204_AACRRO droll_m_Page_123.pro
2c9857c6f46508ebfa1efd3358c24f52
efdf5e6861168da88b6d5447781157d726469f4f
1051885 F20101204_AACQOM droll_m_Page_006.jp2
11a5c8b2c7a3b4457135bb7a6b3a704e
42e9638ac35e1c369b1e1000f0baf1d448e0cb9b
51610 F20101204_AACQNY droll_m_Page_007.pro
57b2b323522b7636d651043615eaac51
2e77b38b53707ebaa6ed724f8dedd232070c7faf
F20101204_AACQPB droll_m_Page_149.tif
9837f4033fd20eb2efda00f635a61b02
cd4d45ac9e0825815176e44698df31d25dd74ce0
38103 F20101204_AACRSE droll_m_Page_139.pro
312a18c0a6e7eb2b182e642f10e58971
1b58be0051fd63be7662907c8c4184e7f4a77ff0
39070 F20101204_AACRRP droll_m_Page_124.pro
c98603329e0944f2e0f976c47090f23f
f412fe0f6b2faad047699d2526a1fb3954a2b956
110811 F20101204_AACQON droll_m_Page_038.jp2
469b0404f5877329488b92518bea039a
d4f7c00bc067b2029bcdb013d2db6a6a75413434
5659 F20101204_AACQNZ droll_m_Page_011thm.jpg
102100e2723ff53f28def101dab2f4a6
ef9f3db0683e131921a71f1082b3b1591740e449
4568 F20101204_AACQPC droll_m_Page_083thm.jpg
e0dfa44b61a9d31d53ae9121f2c13263
0abf469d2bae032edb5d2c3e11036f2b738b6e68
37264 F20101204_AACRSF droll_m_Page_140.pro
6d5af311a30b17d7992b9519705c94e5
2e12921ec86d01c5089a7bbfd7ff0b644e53cb43
42153 F20101204_AACRRQ droll_m_Page_125.pro
0c46301a4af1ab2bb7ed3237981c531f
dcc8b303383ea7499d92f992537590790b63c99f
F20101204_AACQOO droll_m_Page_145.tif
f8ff3b604df0732cbea41310e6e48196
712d37d2d4f0ae5f622075a96ec9b9e6bde7a61d
73384 F20101204_AACQPD droll_m_Page_082.jpg
9d3988a85f5fb88880ec7e0940a71471
968b83a070c1369869901bc4c07c8989978729de
42712 F20101204_AACRSG droll_m_Page_141.pro
783aa4eb92204bdb9ec9741bd8a3b8ac
e4f16e509879c50136267872beb5cf06e1495fd2
42600 F20101204_AACRRR droll_m_Page_126.pro
eb4ef4bfa187b8d9c2a909d38e3b0f84
b4c44e1b60dfa7b33c8f8e0cdb7bd942538ddfb3
6401 F20101204_AACQOP droll_m_Page_133thm.jpg
3c7660295662bcb044f47c11e696d8c6
74724bfe0274518f3d537a93c3d5cbde27df5e44
4504 F20101204_AACQPE droll_m_Page_145thm.jpg
9249c8ba74ee8d526f860c47ada0b3c1
fb92e14e2062ad4c4704ea4cdc27a932aaa27ca5
33902 F20101204_AACRSH droll_m_Page_142.pro
6a1ad050863b3447130663304ac5596c
d452dbc528203732a8edebdde9f487edf18fde41
40263 F20101204_AACRRS droll_m_Page_127.pro
8bd137a8126da5564dc6130af431ead4
9d68ee5a95bb3e060547f66f4ceae4053502f2a5
24009 F20101204_AACQOQ droll_m_Page_080.QC.jpg
900a021c2afadbed316905f121da7408
6fe2d7e9e45c3cb5cbcb630af8433796480214f2
F20101204_AACQPF droll_m_Page_018.txt
e69221bef19b1f1f4923d28dbf16b091
146c97ab896b4d1e3a18261430bf776c88abe5c8
28872 F20101204_AACRSI droll_m_Page_143.pro
142fc3994fa0f549d993e61890796acb
0b422b7792df5f69880ad92d1a9e362ab0ff623f
38951 F20101204_AACRRT droll_m_Page_128.pro
32c9ced17a0a358c97ab5e6ab437deb9
87be102f5054f9cb0ad5673d46137af1ca50a1e3
23870 F20101204_AACQOR droll_m_Page_006.QC.jpg
0a1681668c804be44ddd2d66b565e097
f0ce516c65b7fc576df48073b74e3b912719ff82
76204 F20101204_AACQPG droll_m_Page_097.jpg
945bbbbdc767dec21c2f5443bee148b9
ebeb2ce89adb4d1ff5947bd24eb66e51f07ca0fe
37815 F20101204_AACRSJ droll_m_Page_144.pro
70aa40850c743085e46d5000d10f2346
20804afe2948b45858cbb2a64ea70049e0c1deac
36661 F20101204_AACRRU droll_m_Page_129.pro
e7dd938fcd0e4fa6e8a459f35ac16b8e
9ad4d4f4f1f7aabd6895795f1b8c254f99969771
60708 F20101204_AACQOS droll_m_Page_128.jpg
260f9a2296564c45838c7aba7b7fffd8
ca9f91b7099a552cf89ef1d9a247684d0c392c33
42810 F20101204_AACQPH droll_m_Page_084.pro
2965ddfc33e8dfffddeff2bbbe61de7b
cbbdbf8dc24ec359a2632556f2f13dfa7968f564
31834 F20101204_AACRSK droll_m_Page_145.pro
8ee9f7aa70550bb1a849f61862610b21
ad1daa7b3eb350344978ea8fe4501e8d4a7e0289
38599 F20101204_AACRRV droll_m_Page_130.pro
bc7ec2b9a59666564cab4feeab6dba70
c3233bb720509308db899781fa951fea039ac51f
24166 F20101204_AACQOT droll_m_Page_123.QC.jpg
b7b0892b4380fac945f27309b607140a
56ddd381ce5d8963ed780b468e743071fa64d3aa
2076 F20101204_AACQPI droll_m_Page_121.txt
2d8fb51adb957964589f7ff87e6f307d
b75a5a8d678dbc907d9951d9ae85972a3f242dc9
48341 F20101204_AACRSL droll_m_Page_146.pro
370734db3a3d1e2fdd88e521a585f249
c90321f8e74b8e33e4118581575ba4aa3c527c1f
48606 F20101204_AACRRW droll_m_Page_131.pro
ed40ceb7074c83212f1f7afd0d022a99
059f82026e44e20e41240388fbccb95f8cedf40c
F20101204_AACQOU droll_m_Page_049.tif
55c26168c5ff6d6d230ca867f1007068
d5cb56641ce8a3062d1faa1ba8eb8cc222935a60
124433 F20101204_AACQPJ droll_m_Page_043.jp2
72d4fd1d02acd68d5098aa30849f9ce0
af10d0e528b97ea03f371638efa361ef0e9528c0
48711 F20101204_AACRTA droll_m_Page_162.pro
d53bbd7218d08b49c4a6e2f8ab62de83
9e6b75391332b9190fcc1b49d6ed521c4b6d8a7f
49070 F20101204_AACRRX droll_m_Page_132.pro
11cca200bf383fd2af258972b7f764e8
dccd56e4adec757ce18972fe78990653f3e0d1c2
F20101204_AACQOV droll_m_Page_072.tif
0ecd86e929d1e44fc2634ffbd04c7f2a
17ed3cb63028403f8fece9da055eaf3041ba531d
46955 F20101204_AACRTB droll_m_Page_163.pro
8d025fc1a7e0e5f86a0614d86fda8e08
d987e31e3703aa07949aa3a51a346e4b3bf1838c
49195 F20101204_AACRSM droll_m_Page_147.pro
c38433590a9c17746203c511aaf8b3d7
99920c3a9955bdc32ae524e7f9d5d5ef6772677d
46615 F20101204_AACRRY droll_m_Page_133.pro
0f448e86911c4fdae76ee0f3dd666e73
762d8a075056a745fc4723d87549eb13e0fa6195
51274 F20101204_AACQOW droll_m_Page_046.pro
95e329ce2a1d87f1faee2a6752219d65
7c95f31bfbec8fdcbd55f9441f343b291f104520
F20101204_AACQPK droll_m_Page_071.tif
79241fa675654d37eaf26fa00f1f1ac2
d8621fdd5780e6fc27d5cf188e5a7711c4376145
46763 F20101204_AACRTC droll_m_Page_164.pro
61d155d7265d434e1362cc5b5054499f
6e397ccc41f385c91f4c6d84197e369f6e4961ed
51012 F20101204_AACRSN droll_m_Page_149.pro
af5115ee7c4140aa21ab412a21829fdc
30c1af8662d98721ef9be025d3549ba6e026dbb0
36372 F20101204_AACRRZ droll_m_Page_134.pro
96008931e33a93a85632fa2e84aa1180
8d8b4a2e5d50c4b5aebc59b67e5737e2cd67fa1e
24366 F20101204_AACQOX droll_m_Page_034.QC.jpg
85c8e8ad4b523e869c80254345a0791f
12b8a3e4af000843fd5b9c058245b03eb6e1991f
1933 F20101204_AACQQA droll_m_Page_162.txt
46f687fd305b90ff02c70ba28578649c
860b6fd5b439233f58742f34a5ccfd8ef200719e
47294 F20101204_AACQPL droll_m_Page_085.pro
b28466a76a74db4da620590edb220f15
0f176c0602b19f29920a03594526c08569e6bf1e
49438 F20101204_AACRTD droll_m_Page_165.pro
c6e70967caf4ea7cf29d1ee8bc6e5ccc
a12c9b235894614ef5d1a26c56c48e0f5d16e94d
45541 F20101204_AACRSO droll_m_Page_150.pro
f6a963dd7360a8c549e6790e001e2093
46a7bf9bedd9004a78f4a6d1487d2d475706e2fb
74559 F20101204_AACQOY droll_m_Page_087.jpg
920f7eca7de8f187e88a68468341b29b
1544d3085500129efa6186bc11e1fb8c92dbe4ba
51735 F20101204_AACQQB droll_m_Page_034.pro
5bb571b5f83fb3405b6daa9644371987
3cae7def018c14f637e5479845152c9e2b1ece7f
23052 F20101204_AACQPM droll_m_Page_165.QC.jpg
f3a793153b6245e93141da9c45f028dc
b1852e6b5e791a4ebcd852bae30e3310b5822647
45848 F20101204_AACRTE droll_m_Page_166.pro
92c9b746c24b2410782d025487ba7ec3
f0e9fcca700b56bb558084f38485a52cfd39a33e
38303 F20101204_AACRSP droll_m_Page_151.pro
82bb34dc91a37762bf4b43e0dc72c3ac
cf3b763705eae92bcd04e65edcf9bc6ede527fca
F20101204_AACQQC droll_m_Page_134.tif
1c1e1a56be6013ccf902db2424d83617
4c5b2cee1941e7456937f44b417ba76955bca674
F20101204_AACQPN droll_m_Page_058.tif
c7041ee7c32db486edb3713f511c611a
bc45f5faa71e3ad5e5de2660d2b769afb391e703
45957 F20101204_AACQOZ droll_m_Page_070.pro
545d1a0465af2ed70834b3677390509e
8ef1da9e327f077ffea7a81a06a8d278af0b2105
48607 F20101204_AACRTF droll_m_Page_167.pro
fbe5557dd079b912aa8e791126e21544
79b0608caefa772f12d0c786584874b9eb4233d1
50810 F20101204_AACRSQ droll_m_Page_152.pro
a61383a2d797d3912e503657f05c7c1f
e76ef9acd1647765936b72aad6daeec82673b47a
F20101204_AACQQD droll_m_Page_011.tif
120da7491bd23acb7b9196ceb609fe41
95560676153f3c5f594c90a4ef316b60d3c323bf
59637 F20101204_AACQPO droll_m_Page_111.jpg
beef8825d089fd66a6328efbcf868bce
ca18f9d316e4adb4963a5ded77e01f0764553e13
52095 F20101204_AACRTG droll_m_Page_168.pro
8e78eafc82b1b8e7f094fd370b403051
1b4b5fbdb532262e7d15bbc0a92f098650571016
51697 F20101204_AACRSR droll_m_Page_153.pro
4c24e827168b49a40720bd5ba685bbac
6f00dafb7308e8c20dd09e7515b9c05c88712177
3810 F20101204_AACQQE droll_m_Page_005.txt
6625e51263b0b39fa6650beb96cec839
d74e15abdfa5136deb2ccd033b8a7ca70796fc12
47824 F20101204_AACQPP droll_m_Page_075.pro
7a85fc0bd295d82d1b56e34145b00f07
4066dff67c7162f01b5783912c4dd647a5bef3ea
55777 F20101204_AACRTH droll_m_Page_169.pro
ddc9adc0a9369d5b9356505acfecb7e0
66fe5385fa5a53a295a4dbd26659682f4c300016
54455 F20101204_AACRSS droll_m_Page_154.pro
b72fd54d0e654c2c9377394d0b5129e5
ec3f78df034adb18ea6041480c7f452550707d9e
F20101204_AACQQF droll_m_Page_017.tif
338a519f0976f9ead47fbcba3f3c1c18
fef71e99e21094722bc77591701b4d8be70d2a44
22264 F20101204_AACQPQ droll_m_Page_041.QC.jpg
c38460adb5f79cf754618c3d5eba0a38
794ac22702d6fa640bef0a8d8f4a3f04a308b426
60431 F20101204_AACRTI droll_m_Page_171.pro
6beb2e62b00d2275d5a2f23e57b6083d
ebb6cf1b143f73cfff6c79ebbb25fd5ff371c2fb
48707 F20101204_AACRST droll_m_Page_155.pro
f26fb5fb3de862b148460933b0afb29d
c6e7a3dab8d0218fb7e717fa88e79294a9a373f7
6790 F20101204_AACQQG droll_m_Page_196thm.jpg
d4b47fd4493200c86c4af67a6a152480
2a6ef4af3666f5ada3e26448e20eb4c24338b4c7
F20101204_AACQPR droll_m_Page_105.tif
356c7a3f684718558afccf6dd264319b
ab6585fbce5c4bbee9faa09f2bbed00fe02ad6bb
51465 F20101204_AACRTJ droll_m_Page_172.pro
118b64fc11ca161a2cae4b5ed179338c
9e4145beb66d109a59870a88b48b7fa2ce0184e0
52356 F20101204_AACRSU droll_m_Page_156.pro
da86b79c1c8af0aeb02a09ebb0a6d130
6291434b67295a540c5c95fd9d560a3636070d70
107131 F20101204_AACQQH droll_m_Page_167.jp2
daac15167d193b6c0d3736ac2380a5f4
10b031970a1b5e0586d88b62942f5a7fa6a77a7f
38506 F20101204_AACQPS droll_m_Page_193.jp2
721c0e46ac40ee6566c85436f462f1aa
78c923345d6a749beb8213da7946b6eccd34a1d5
23770 F20101204_AACRTK droll_m_Page_173.pro
c02332010026dfa0a382337444fa66c9
cff74136d817e1e9da2d11c33f08ccf6cf1246b1
48759 F20101204_AACRSV droll_m_Page_157.pro
007c1a61543e39af5c3cdbcdc451affc
d7e2b3238dff503dcbe03d65a1989b379437bfc4
114122 F20101204_AACQQI droll_m_Page_064.jp2
9edc48b5b3f1a058732d30e0aabc9c53
e6ef44d742846c6329869fecb881eb73b97d1aad
25620 F20101204_AACQPT droll_m_Page_177.QC.jpg
ad9c135c5a0c62f37652e2cc4f178e53
c953ab5db439aec2a5637900a1ff391882987d1b
39916 F20101204_AACRTL droll_m_Page_174.pro
8e1c27ea37868872364e70f3480ae625
c696fe476b771286bc5ab0e26252d166025cc438
49427 F20101204_AACRSW droll_m_Page_158.pro
9f1d5213fee7bb2e4d0fb561a9229ff5
62ec2f23ce7adf6acece4c01077bb86b9d1073e1
24097 F20101204_AACQQJ droll_m_Page_188.QC.jpg
57104a98e3edc5c8d77f33c9ca4859fb
92812f25c5159c281fdfd6b7890f0a77284c1a50
F20101204_AACQPU droll_m_Page_075.tif
88d50ce9461ffe08d42b0e76c325b9e3
8254a0ae3202d156d6513bfc11aa73f4e8700154
46725 F20101204_AACRUA droll_m_Page_192.pro
29340e336aa4e08b79a23649c48dcfdc
1926ebe1cb7134258744a00c1c21d7008e801e5f
59217 F20101204_AACRTM droll_m_Page_175.pro
3c1fc53fbe18539426aad303ac0183ea
1b5f69a3422651f839dd087b075873cef4a8d994
48672 F20101204_AACRSX droll_m_Page_159.pro
9ad73bd27e0f77a0887c37465e7ffeee
77e74e627f5f6e6601a391f909c7450764bd9589
F20101204_AACQQK droll_m_Page_081.tif
553e171937d34634ee7fd8ba076ee6d6
0b8d9eb28cff5201bf6d61d183d3e4d517cfae5c
6467 F20101204_AACQPV droll_m_Page_158thm.jpg
cbd8bd3d221503cdc099f819589e37d7
4e72eaa8b8e3e499dc3e22fc83b62e6d514dc941
16332 F20101204_AACRUB droll_m_Page_193.pro
5b0b39a3ec73d1b112bbc9acb84ff644
a2b5244ba9dce76f0f3e1bae3c608b8f3bcbee0f
45106 F20101204_AACRSY droll_m_Page_160.pro
a84241de5a23e77a0e4f2e309d7d3591
526595ec2784c4246ee198030e1d0047f1dbb3f6
51425 F20101204_AACQPW droll_m_Page_110.jpg
05fb1023345ec1abf4283bdcd43efcf3
d70a502806b344047fc342ceaae8d6b82324a0e3
54697 F20101204_AACRUC droll_m_Page_194.pro
2ebfdb6ba323d978f955f2f70d233d86
41da5bf7fa83101f9bef9a2fd90a76850b90e172
47618 F20101204_AACRTN droll_m_Page_176.pro
626650c0107ac452ade4336615274523
1549ef43f5f6263e8d1d2e127aa55002b6ee798d
47419 F20101204_AACRSZ droll_m_Page_161.pro
9d1067b3f1915693bbe45080e5f24fbd
bcf336dd867d498d8916f672fa5919645bd5361f
87661 F20101204_AACQQL droll_m_Page_196.jpg
567db51d163d3c2775f7e09000cf06c8
3bf3c685e7a5fffe143e23bb50067e77bb611dfa
F20101204_AACQPX droll_m_Page_143.tif
b9e4dab23b0c9e382ebeaae0164c84a7
93207d0a28480ac519b1d957e349370aa6932a2e
2189 F20101204_AACQRA droll_m_Page_154.txt
6a7c795e083859bb6e68a5479276a2f0
00026e1322dc70d65db67543eaa1c3539542d630
56518 F20101204_AACRUD droll_m_Page_195.pro
538c5e83966b327c622614650a16b74c
08286ab3eb8096b39d35a133abd4413307845e5c
67705 F20101204_AACRTO droll_m_Page_177.pro
ada611c510d616d5924514d5c1a055a5
3bee0310b7b0a0ba1d1bd935c2f0ad8d906a14c5
51713 F20101204_AACQQM droll_m_Page_020.pro
929890ba8c19a8949f98ae6674bdd4bd
3a5fbb83c01f33b52912ae7980408649c49a3f14
51855 F20101204_AACQPY droll_m_Page_056.pro
2c765e67ea0b49140eae012af2ac78b7
c2838a97d078319dc5e25093a3e6bca53af53796
5084 F20101204_AACQRB droll_m_Page_101thm.jpg
90ac40343a9df96b0b8baea46818c2f4
3185d9d152b8ea8dc5fdca1f7ca36042787f58ba
62646 F20101204_AACRUE droll_m_Page_196.pro
082422bb8b9a0a80f38aa0fb7fa19d40
a0cfcf5e92750c963029b11ca006f442ae470676
58001 F20101204_AACRTP droll_m_Page_178.pro
f85aee22dd1e7a1487cf31afc158359c
0bf0515304cf04d65709786784a0d24b9352e749
F20101204_AACQQN droll_m_Page_018.tif
c468914cf29754ca03f217ca4a94a4e8
3546e7a96c26a76b5280558aaa600b7665f70028
21981 F20101204_AACQPZ droll_m_Page_166.QC.jpg
ed93bad990af331a401c5989921e54a8
0b2d642a6f9c63e66e70e7c4a45ad226fee3bd10
1903 F20101204_AACQRC droll_m_Page_192.txt
f012219981b525ebe14277559d74b36a
353a613458dc7c4bde78dfb7b970256684d033a6
58355 F20101204_AACRUF droll_m_Page_197.pro
0f13b66dff6e420f2728268ddaac12ff
21d0d04882f52960da1c75aaf08e374a51415cd6
62855 F20101204_AACRTQ droll_m_Page_179.pro
cd9f5b2de255c2765dc2b0a79c24a145
548b3430afe418365e4528972a4c5c10e6146eb4
F20101204_AACQQO droll_m_Page_106.tif
7ae81f44f01e09eb273065c6343bd982
0727f4a7e53bd6234934fe0608b8437cd2c628b0
F20101204_AACQRD droll_m_Page_163.tif
da6f72a50482e76bb2b3e9311261a4e7
6c74d844fc726dc9d2680faed6ca01f99684c62f
2777 F20101204_AACSAA droll_m_Page_177.txt
2dd6cca80a6829a70b86cbf4b59dcecd
07017396702ccc2d54a2046f46578f9a48ee00f0
38521 F20101204_AACRUG droll_m_Page_198.pro
d705f7245ce87ddb3322eb205572a2ba
53229ed023661aa6752b8b185f997b72f33024ac
53036 F20101204_AACRTR droll_m_Page_180.pro
0cb3c980a9f04266f5b2eb65ac920541
2f189bdd0a802831d37fe3205693489190214358
446 F20101204_AACQQP droll_m_Page_001.txt
e49e2bd404882a7144aa597f60b31b4d
eb1b1f521a9668320d2683255a1acc79601ae3a6
74425 F20101204_AACQRE droll_m_Page_039.jpg
4751f3050698e004d862e9c2f1ac2cdb
906ee89c078c7fa30edab890f9d5c5d986e5d4c2
2354 F20101204_AACSAB droll_m_Page_178.txt
13cecf793cb721b8e7326d531f4cfbfa
f2d52553bbf36a6191225e458843630087a5bed6
44548 F20101204_AACRUH droll_m_Page_199.pro
125b51295e2688c063002088c7bae12b
9cd3c224d0b710d12bf006cd6db6fb33d9f2fac5
57202 F20101204_AACRTS droll_m_Page_181.pro
646b491a5e776bae5499b820aed21f56
b1171751458a8bc04b1d7f3a3cd5433f953523dd
43427 F20101204_AACQQQ droll_m_Page_003.pro
db28771168f0f6cc6f272c4b17c8eab8
c867059ec5e3c4cc13181251bfedcae239ba4651
21728 F20101204_AACQRF droll_m_Page_044.QC.jpg
21536e351d0c1f1094b6eef3c8687de9
5e26c0bb3788d0bd6feaff0d6a697ce1777577a6
2549 F20101204_AACSAC droll_m_Page_179.txt
0665065795c114ab390e205bc291000d
df272b17de5e8911c21245bb537522ae54d56dc6
113 F20101204_AACRUI droll_m_Page_002.txt
a94cadd0ea572eb2a049c395fb27bba7
339f553f231633bf6e56f7b8c8752ad7d3d24d2f
56259 F20101204_AACRTT droll_m_Page_182.pro
2058c25c11e3fb56b979ddf314bd0a52
cb1e88bab1b0ebb073c182dd1e990902c8e54fa0
72595 F20101204_AACQQR droll_m_Page_053.jpg
cdc99dc7f3e7d2293eaa1539f6c20026
7251677e205d42c4bc19641a73d9c6091e086555
72569 F20101204_AACQRG droll_m_Page_038.jpg
ca6a8fc205e76dd2fb189594569b5743
be284d0cfd8dc4b72624e97b96c6df255d772b45
2151 F20101204_AACSAD droll_m_Page_180.txt
9e4e241d710c85f8ef49984c83a6ea79
b3a7c1eb0e94584c6ab8969a2bca33b10463bffa
1755 F20101204_AACRUJ droll_m_Page_003.txt
896a0b28e5fb5eeb35efd86a1ec340be
c53c14dd33f10b56e8887f5e5d459d762ab77877
59260 F20101204_AACRTU droll_m_Page_184.pro
1000891adc669e7ce26e4f4949593391
70b65fd08e11830d8e8677b7f25def107d5ecfb5
F20101204_AACQQS droll_m_Page_058thm.jpg
b6dd6001722d55e681d6ff42903b0cff
28d5990eca21c486adb582b826f1fdf87088ff93
16103 F20101204_AACQRH droll_m_Page_118.QC.jpg
33795f26f19722dbfe4362f1aab2fc41
64540e183195b0bdb53f912d843b97861886f990
2323 F20101204_AACSAE droll_m_Page_181.txt
6894aa753a0edf317f091aed0f90fe87
4e39585b8863e04f9a0bec82902fccaf449aad03
2883 F20101204_AACRUK droll_m_Page_004.txt
9b6211a3b8b1a850d227cc5434ce105e
051691f24d9e2df0326a4ab33a30b64db9245bd5
63188 F20101204_AACRTV droll_m_Page_185.pro
33ae0787f8f34103072611b59a347131
42738bda2bb07e34d3cdf4ce90eda53f104e948f
21703 F20101204_AACQQT droll_m_Page_007.QC.jpg
42f5f3086945661d6384bc6867d85e18
72d78d1599e92705e320c6de380da7a493ec5e46
72903 F20101204_AACQRI droll_m_Page_024.jpg
e4f7cc44e8fc29da441a7ea6d20cddb8
81fce5baf654cf1b949834fed19b03caf1c7d03d
2284 F20101204_AACSAF droll_m_Page_182.txt
ec78941a2071e468c7d5e3a4fd4611c4
5b52e27301dafb3b07eb7086888ec43f97c69cdb
3466 F20101204_AACRUL droll_m_Page_006.txt
e66484c83891b0cb253a337a86eded60
d2d69de92d265729b564488c50e4f20f0946bf54
52709 F20101204_AACRTW droll_m_Page_186.pro
371d55910868371ad0e9b597673718a2
cda8344fdbdf879e904819f8e0f792a487023e85
35998 F20101204_AACQQU droll_m_Page_114.pro
74c3948293b4bc6f8af07a4f39a5f55a
1a2cb78edc214366dcbc8cf0f99a265be9489ca8
6673 F20101204_AACQRJ droll_m_Page_039thm.jpg
4b2fa89b33c9686425e1a7f0b73cda66
03c59eb5c4f463a306ad234549e8dc2cc1e58490
2110 F20101204_AACSAG droll_m_Page_183.txt
af746cc2e4d5bdcc310f4ad3dd676d09
8ca2a60f6dff2c1ce1eccb87189ec88bc5015ba5
1734 F20101204_AACRVA droll_m_Page_027.txt
527be3176cb387308f58bd8d462cf783
4f38c23a704743f251b7e1b552d55beab64e0c35
2100 F20101204_AACRUM droll_m_Page_007.txt
7fe7a4a0ba95d396c744a59162589d12
72a5097bab24d071517a1f89a198be4c91c20dd2
64177 F20101204_AACRTX droll_m_Page_187.pro
d7571faa050d65535b3e50e50a2ca1ed
94269a2914f03cca4d4b843b87c09a16a969e3d6
1654 F20101204_AACQQV droll_m_Page_134.txt
e3053e83ff9f8e83f3fa60dad85fa218
a345bd4247475f21edbafea90375ec89b22ce254
18546 F20101204_AACQRK droll_m_Page_128.QC.jpg
ce20c401159cd9636b916af0cd199916
d8c7d3c66ef2911a8f3ce1b886f9e6aa469c4987
2419 F20101204_AACSAH droll_m_Page_184.txt
4ee457b5eb838ffe6964e50c2ca6f334
15dd46b66e4d96fcd5be7c0260098e40ffa5dac9
1647 F20101204_AACRVB droll_m_Page_028.txt
1b33db9f583d9c9ed1a4605a494e1632
38ef1579da4b0692fa787396ee9f0e7b7b3b3b3e
1679 F20101204_AACRUN droll_m_Page_009.txt
c6e31ba065496d18a94c2add8b6979ba
f60c1e35a46d9f1355a9b7120af6a521416265e7
49970 F20101204_AACRTY droll_m_Page_189.pro
88fd056e0d028bcd69cdd487eaf6e2e1
2d8d436c5bc72c3f52322dabd8f93b687cc492ea
F20101204_AACQQW droll_m_Page_089.tif
8d4a21aa8eb8d60b642ebde50971a2a9
3550073b15d0258c98a988e3b89111550e67d446
F20101204_AACQRL droll_m_Page_099.tif
a6194369c0dcf6ac24eed15349938e51
e5b9fed600f80bb254f7573abb5905ff7bc76474
2562 F20101204_AACSAI droll_m_Page_185.txt
ae75892318986047f8efb54290f26561
c739f8deaa49d1a25554a57a373609e7d9c5ca7e
2194 F20101204_AACRVC droll_m_Page_029.txt
125d123ab56e263593d808fee8b9450e
c3c5a1b8e4b45eb5668293782f4befa9ee3ad2cb
54957 F20101204_AACRTZ droll_m_Page_190.pro
4fc44aed60f4b70b420c7c5af3c76cbc
cf0548f0e0e8bf3f80ba186fb2486e3b20cc0a75
F20101204_AACQQX droll_m_Page_182.tif
6c36c8183dfe1616d946bf1e6dd6976d
e0b36fea1e66544a77e052c85686d28973bfa0bc
6033 F20101204_AACQSA droll_m_Page_194thm.jpg
4f0947ca49a351b5cf3b8c2adda5108c
ce34391f17ddc1421cc82c7b124bdc9370d0c448
2615 F20101204_AACSAJ droll_m_Page_187.txt
b32b0c3ecb48e5f79254131b65bb6e07
83bbdd3b69ed8d4a37b53377a5285eba20c0d194
2227 F20101204_AACRVD droll_m_Page_030.txt
83497330e4fc0e028653d24d351dd6aa
f53d310c199ef18ad2c742269d9b8ead801c3e97
1480 F20101204_AACRUO droll_m_Page_010.txt
ab21eff0a642fa2a8f1c5934aa2655c7
03cf40289ef6484a87e59bab56165bdd0fc9a00a
23852 F20101204_AACQQY droll_m_Page_153.QC.jpg
44d430541df549fb8d062e114af5d752
4390c4481d85aeabd9563a13d1d067ed2f8a323e
F20101204_AACQSB droll_m_Page_002.tif
fd442f6f5125161566b8e38c359d1f46
32d815f39efed866881206a85528e443fd931621
105394 F20101204_AACQRM droll_m_Page_073.jp2
88ecd42695d64dd7364daa2f7143d7f1
221bcbc1f6988f732d1b6d125be6be91b6f2e109
2478 F20101204_AACSAK droll_m_Page_188.txt
00e655147b8860bfa65f9ed20aea339b
2ed20888898cb716d3c0fbf2cd7a11fab4511da6
2109 F20101204_AACRVE droll_m_Page_031.txt
04aa7f53d55a4fa2fae3b17605bc37e6
ae2abe3e284c42d892f01cf56d2e46ec033bb13b
1861 F20101204_AACRUP droll_m_Page_011.txt
4d5df852235e686beefecb8a71524164
9356bd24ff0fdc1a3eefdfe1fe5df63ad4011bca
17518 F20101204_AACQQZ droll_m_Page_111.QC.jpg
f12284eba5ae59087d419cd3531ef8a2
042c74ce938cd3fbbecb6a9e5e6a38aed4bae3f1
F20101204_AACQSC droll_m_Page_086thm.jpg
7e56282b68e37603d9b5062185b3bf6f
40dd1e5d35b8aeceed48c764769dd2990e6c9038
52113 F20101204_AACQRN droll_m_Page_183.pro
b58822cfadfea94649db271f72eb5e33
0fa135bca6d012d937582f2ae29664b195f39bd8
2024 F20101204_AACSAL droll_m_Page_189.txt
13c86fa325241e51633b62db16ef683a
26846cc95917080160d94ee44c25e321b70374c2
2001 F20101204_AACRVF droll_m_Page_032.txt
75967a5be0d5b4f6ff4d7724b9e58b05
d108f1493930b08cf5deb079451f9d32e06c32fc
1924 F20101204_AACRUQ droll_m_Page_013.txt
b3f5891c1715c1e1c46721de1e59850f
c2af28cefd18e92c4b71bfac9f22aa93abca7067
F20101204_AACQSD droll_m_Page_060.tif
59d12ff1eeda0870011e4fd3319075b7
64961703c0d18ee8d6d5879e979aab008bf7ae65
67178 F20101204_AACQRO droll_m_Page_054.jpg
6e32fa02f16e6b42e210e779f8047d0d
5663ed6462a61a10867b021a40f7e2ddcad2a5b3
4662 F20101204_AACSBA droll_m_Page_113thm.jpg
8f5f0fcd6d5ef7ba3d8b36a27392ef2e
6082123d87ed1cf4f148c36b9ea6aa5368d52552
2221 F20101204_AACSAM droll_m_Page_190.txt
10b79b816aaf4b5966e0972732c095e2
5b5f6b968aefb1fd8463ab51a1be85c48ca5d509
2219 F20101204_AACRVG droll_m_Page_033.txt
e7d4d9b0fba111fee8992310cdcc3f68
f34e27b764861673ab466db8a9039280e23acd3e
2107 F20101204_AACRUR droll_m_Page_014.txt
eea470ff4262f27a7338caef1217b5c4
e51650edbfb30102ccec6f8f72781e31b1813470
70917 F20101204_AACQSE droll_m_Page_158.jpg
3695c58ad549f335e0a3e6e0237f0a1a
08c42efb03a1c4f31e7568ab24807b9fa6095958
96496 F20101204_AACQRP droll_m_Page_120.jp2
67977212e4fdd63d58de62971c2cbe29
ba8a90ed27ae61ac342d5865172f44b54c3c148f
4672 F20101204_AACSBB droll_m_Page_142thm.jpg
532900a913ffa2323bdea351e1cb4f04
4deb23e5b3127badc9352df3ae656847043bd640
1969 F20101204_AACSAN droll_m_Page_191.txt
6028c00787de95ee540d076b999d8693
54edc190cf229463bc00b06b40d21ca15c3fadb5
2032 F20101204_AACRVH droll_m_Page_034.txt
25d31f26c12054591c196d252e620fab
a30e567c7b1b698544244847d3d1741494642a9b
1977 F20101204_AACRUS droll_m_Page_016.txt
47183ef6a681264da7e9e7bd25d78725
a14196fdb8fe0560c4b9553bb7b44fb53224be41
6471 F20101204_AACQSF droll_m_Page_162thm.jpg
30014fd0a27cdb474d4793f9d33c3dbb
754bfa5371f1c2ace2838f4485d8a7ba2bce1ad4
24266 F20101204_AACQRQ droll_m_Page_168.QC.jpg
2f475c6dbc6a3a1cf40d2b56e36b4d53
dc328b4137bff07dbdafd30347316db3c98d442b
6098 F20101204_AACSBC droll_m_Page_059thm.jpg
e93c307e54cc6f30ad692d2698d47114
b7da0324c0cf2663557c5d8d80b1c325199bccb8
703 F20101204_AACSAO droll_m_Page_193.txt
44e3608b25ffa92d42d22519426a36b0
65906b81df0d1ac0096e8c3b43e5d218bbbfa439
2147 F20101204_AACRVI droll_m_Page_035.txt
2f4d49e28f6e4b0c812d5ca8d703208a
18650a28e8290dce15bf3b31d521508481c4ddc6
2007 F20101204_AACRUT droll_m_Page_017.txt
18e46d2d8a8c341bc241426dfcfa79ec
0db9570c2dde2c8ac9c8a0a51ff6d5440fdb5575
18341 F20101204_AACQSG droll_m_Page_009.QC.jpg
201885d70a9ef5f38bbfb222452fb520
5e8fde683c41926db7203e01bc996462830bd063
122406 F20101204_AACQRR droll_m_Page_184.jp2
78561c5c9dd2909b016acd29a969de02
64fba0e3aa088f870c58e165a5108806c666e9ea
23722 F20101204_AACSBD droll_m_Page_017.QC.jpg
3b0beef4e15e7a2394de75f2dcde4624
2a679aeb4ef7ab79a9cfca97b671460344f4ec85
2231 F20101204_AACSAP droll_m_Page_194.txt
8751ee67b3589a1e97cba64f73298cd5
e76267a87c2e88cd38273b38d34c500751a5c2fc
2141 F20101204_AACRVJ droll_m_Page_036.txt
4fe40e29c25128fbb3b45a5f40b3bfd8
5f566a28ed7af2ff1ace160e84e8463ffb0ada74
2030 F20101204_AACRUU droll_m_Page_019.txt
d53548cd46295cc751898b6025fca71c
23ec66b4a49693263042b804ebebbc60782626b0
1870 F20101204_AACQSH droll_m_Page_041.txt
6d9f2feb72c40f101019bb17bca73f3e
e933ad75996f80f3ffe14e7a25aa46bfbdb33028
1623 F20101204_AACQRS droll_m_Page_139.txt
90b4fb530846d4aa85df6affd67efe52
51b4095a5f88718463b74f0ccef6667060414ae1
9952 F20101204_AACSBE droll_m_Page_008.QC.jpg
18a21459a9a8c43b64c2ead65c26cdf1
42b459c8aeb261c8078e24f95c592e3582dd4e5a
2298 F20101204_AACSAQ droll_m_Page_195.txt
948f1fa9423f665721bea54c1295ffce
bdfe074bc6ac3c8e3cf518e316156a41c8714306
2089 F20101204_AACRVK droll_m_Page_037.txt
ab260e5ac7d5cbe509c3dd6ed7c2719e
efcfc0ffc1338899894a75fdc09c42f95be368ad
2086 F20101204_AACRUV droll_m_Page_020.txt
a3d2f044901070ed60264a16b9231e06
adda42523083a79a83b3cce887bf339f8d3c0b07
24450 F20101204_AACQSI droll_m_Page_064.QC.jpg
9c47cd617076e01563f7931b1ecfed30
9d5cda2940aea5eaad0cd462e9b5ff8e1c663642
1819 F20101204_AACQRT droll_m_Page_166.txt
0f5314275fe02270a3c1a3e7da101b49
4b1789a13fa97133dd6e6d5c34d090a69ff14bbf
23921 F20101204_AACSBF droll_m_Page_020.QC.jpg
1cda870a2ea3864fc87ee51430b2f6dd
d8f38c03e9d42d3ee595a100d7efaea874faeda2
2524 F20101204_AACSAR droll_m_Page_196.txt
e28979f1c3b0c2c5353cddfd5ea065eb
d564de5c5b3364672d15b815b65c3d82053c79ed
1995 F20101204_AACRVL droll_m_Page_038.txt
1df7ba307c2fb09f6c8d573fe48cd5cb
e2ce093d7679ad823f26e81d68578fc7991939d7
1927 F20101204_AACRUW droll_m_Page_021.txt
41f6e93084f2d7b250fdd1306b61913b
a1500b5afa8482bb192d1cf5e4485f8605251830
6413 F20101204_AACQSJ droll_m_Page_161thm.jpg
92e1a148abcc973b1ed62d4573130f6f
b5331f8fca2144364803fbfe04eb0b6ab66e5c43
116281 F20101204_AACQRU droll_m_Page_035.jp2
458febf774bd67d14824d88035fccc97
a061c8959180f5c64c83fba9a207516b09c6a9b1
6542 F20101204_AACSBG droll_m_Page_182thm.jpg
22ea15178c32b01b631d880f9703534d
3ed852dadaca9d82215f7cc0472fae2cae431ab7
2371 F20101204_AACSAS droll_m_Page_197.txt
0556d02a972f48f6ba141f9182f1eac3
83fea64a79a18b93c4e44d5ea467d31cbf4a3131
572 F20101204_AACRWA droll_m_Page_057.txt
f3f61f5c952209500723f04d2fd04165
d517d7abd2f246e6eff0f466af3be7369a73016a
1840 F20101204_AACRVM droll_m_Page_040.txt
b44af0c1e4eaf3e946dcc39712b5ccbc
6ce5bcc40a6547b88679f741a3a3d8b9b76072f1
2017 F20101204_AACRUX droll_m_Page_024.txt
dde6143db1679401721afcec2848c65e
dc6be736534e45c5cbff66df828a915727b6d21c
6376 F20101204_AACQSK droll_m_Page_045thm.jpg
9be05b5d5d5ae10a66a237a6cb9ff978
7e559bd0d6af137d8b27ce957a68bb40f1490175
F20101204_AACQRV droll_m_Page_166.tif
35736f3aa12c745d85aea4036517b1a9
30bf98cf0740701e27303f6b5c3584662b4c500e
6464 F20101204_AACSBH droll_m_Page_055thm.jpg
fc6bf4466f2fc55d9e9838d374ea029d
4500849ef1fda50224771be49b5a62feadb8adf8
1588 F20101204_AACSAT droll_m_Page_198.txt
c933648ab8ec468cabd0d96af294f78b
c93143400decc0fc16e5b7aeb46bd50b11393d1d
1899 F20101204_AACRWB droll_m_Page_059.txt
68c6fc546620791cdc1a478a93a12ae6
264f89a708a5fd7ffb09de001141233450ee8ab2
2304 F20101204_AACRVN droll_m_Page_042.txt
29417c03096894700d2dfb1c677fd312
5b40069e565f672441b4e4d382de1a38f0808c6a
1701 F20101204_AACRUY droll_m_Page_025.txt
5fcf0abafaff0da2dcebe1cea71a6477
0a08ba3221586ac35ec3b79b5e58ef3764f29049
83565 F20101204_AACQSL droll_m_Page_179.jpg
e1c00227f2c219d858b03bfd2c0c1bc4
107f59af49ef71b36fdce14512b7d425797f4476
119323 F20101204_AACQRW droll_m_Page_033.jp2
b95e77f7783fcf364ec2f2a701cd0eb1
063e01afa01bf153dfc38c8252ff9f27f7fdcaed
5114 F20101204_AACSBI droll_m_Page_106thm.jpg
2801fb06c3bc22107df96d4ccda9ab85
a1c2b58ea883c73ede94ad653a02a4cd11a8dc40
1797 F20101204_AACSAU droll_m_Page_199.txt
742d7723ae0b39068e99f91349e31204
755418fc58323c8f6f6d60cdbc7986e40f51c58e
1856 F20101204_AACRWC droll_m_Page_060.txt
605c3ff19597645b35041ef8925ac34b
ed1a34a0f50cb9c03bbc8277c8ad3e931dcdb1b2
1887 F20101204_AACRVO droll_m_Page_044.txt
1254aa561ce51d86e7916eaf6eabcfca
b7599257b2f18b5e77abeee370285adbe0e2817d
2047 F20101204_AACRUZ droll_m_Page_026.txt
6e54799bfb9b10aaf9cd9414984f8270
13adcd4eed3c585c9f004fb9642eac499b128f90
2124 F20101204_AACQTA droll_m_Page_186.txt
b37aaebd91e52bcedb2e0b445532f6df
e44f0a97adcd886ba7b930fa0dc99b178753fc2c
60277 F20101204_AACQSM droll_m_Page_188.pro
702a61596189d068fbb98dfae36eafc6
238c1c656751246694f30f38123e0ba647969cce
55005 F20101204_AACQRX droll_m_Page_134.jpg
7df91ac55ef010470246b54eef7d097f
5cf1f7a6e6e9dc06f72853a1ba013ba6b447c022
24587 F20101204_AACSBJ droll_m_Page_093.QC.jpg
bd5e572003bccce64b6a77d989779b33
b03e61db7a023c0d9f5bbc357bb8f77d2f619b19
2350 F20101204_AACSAV droll_m_Page_001thm.jpg
aed2ce004426543451955134ea28a15a
530224524f58d2cf2854d278a969d16c626c4427
2128 F20101204_AACRWD droll_m_Page_061.txt
605861787481c77e9fc51d88777ffaf0
46f1a5143423334988e5b96ff4e12714831e99c5
65833 F20101204_AACQTB droll_m_Page_070.jpg
bc84a7bb2ce6572e340cffdb48f5ec3b
d2f3744732495744457e431fb12023e87bbe80e7
106375 F20101204_AACQRY droll_m_Page_089.jp2
e9ae8b2d29caec36192e2676b3157139
d9ae6345904324c0f7a6faf3e1c4712427e1d122
22021 F20101204_AACSBK droll_m_Page_122.QC.jpg
86ab3309e20a3ea8dae7af39c977c8f8
701c3fa2e1531ad8b1c6080ab91f4ab715bd6265
803310 F20101204_AACSAW droll_m.pdf
9ec452221f79e264f37ef99194bb16a0
dda9accede9bd57298640ce2f14a243428061ab8
F20101204_AACRWE droll_m_Page_062.txt
348c5818728b65c3ff1491baae30f457
f5ef1842e6d20c55410f9cbac74b286cd68b091a
1883 F20101204_AACRVP droll_m_Page_045.txt
3e17d8de36dc92b362769efb961862e5
dc7d23e3f13710539b93a7dd079e09cc277986e3
8434 F20101204_AACQTC droll_m_Page_001.pro
b77110fad6790a9dd97046d3d7edaa54
cf5aa2b8c819012e6d9237d1b9e7d7cb770c908d
42442 F20101204_AACQSN droll_m_Page_058.pro
e95dd0ff54ff2022e7c9c4e22327e1ba
e49f1588edecb12fae554e82e1148ae2f63cca67
6250 F20101204_AACQRZ droll_m_Page_155thm.jpg
d909b0e283550082f15839862cf7ed6f
4f745229837edfcf8a1b9923afe7936b38340980
F20101204_AACSBL droll_m_Page_013thm.jpg
f10f8803b4b29bd183fab80c7f4c33c6
943a1616a55ec562636d8d12781132a7746ded77
6432 F20101204_AACSAX droll_m_Page_096thm.jpg
afbcf78265644c9d07497a7a3a78cd07
3eb73f5d1b1531fa6952acc0434511ae82ee0060
1974 F20101204_AACRWF droll_m_Page_063.txt
2ee7d71071d4102d7a4c0bc4076e139f
6d943b85e14c63b5323997d625d0030d5ced3cee
2020 F20101204_AACRVQ droll_m_Page_046.txt
fca8642e547d67ea14f233f5ceade6c4
6d3223d69b564a1a94de90bd740d26cb82ada18a
74794 F20101204_AACQTD droll_m_Page_142.jp2
5ec9ca76c4f32cff68d47308ba354619
16f0bea41eca3b7d3e7cc03afed74487cf5abc48
6495 F20101204_AACQSO droll_m_Page_089thm.jpg
cadc6c58dfdf8d0b7bfc50ee8dd02f79
3c30d5145f38c5ebdca51bc16bacbd77cf1f781a
22776 F20101204_AACSCA droll_m_Page_096.QC.jpg
0f02ba02e11e5b2e2b85bf8bf19d8f4e
30b80ac4baa17eb8a699ddfa9543f84d583b2851
5014 F20101204_AACSBM droll_m_Page_109thm.jpg
3f5d20fd2e99045b4356f6e8cfc1c465
51d38c10099701f9614af6c7a53686d9930a8ef2
6188 F20101204_AACSAY droll_m_Page_163thm.jpg
df93d179f37c691f5efc043fb04b1c14
f5bcd03af9ec72d261173a733e25df1751dc6731
2129 F20101204_AACRWG droll_m_Page_064.txt
2959ed703fd7c56a7ec39e3a156e2f1c
b96017837ef9e7b190c1ae030561f3bf2eb1ad52
1966 F20101204_AACRVR droll_m_Page_047.txt
6a822ca078b4d0a4ddcb9e6c4d211dd4
6f024505e1dd2e2f74fd22f55cbd99f3e391a5c8
F20101204_AACQTE droll_m_Page_118.tif
000442341ea7fd850ead18c873c92833
19f0fa8026ca6eaf4ed142e8a9d08f254aa9ad40
66259 F20101204_AACQSP droll_m_Page_040.jpg
2e94509fee40a0cf7a97bda433c8648a
d515596e9768d0529367a8e9ed4b5e6fe60f17c0
5858 F20101204_AACSCB droll_m_Page_003thm.jpg
e9c4bfc889d44e1a462942d91c325aa7
93b2be0f2c2474e7c61faecb6f5476e479adfadb
22082 F20101204_AACSBN droll_m_Page_054.QC.jpg
50e2f189faf6b39be46106874eedda47
a5236cff1eb381232ea03d41969a324a45cd433e
1898 F20101204_AACRWH droll_m_Page_065.txt
3acce6b06b29ef8cedf1a241fbe06305
a0a36922e6922aabf5eb0fc0ad93843773648c32
2035 F20101204_AACRVS droll_m_Page_048.txt
f61e29a168aa10689d1c92058a98cb29
55c36a42f26eda584d458ec76e572c39afa70b25
56365 F20101204_AACQTF droll_m_Page_129.jpg
8e31fd0237a03fd9fa90626163f85297
f118476383ecf86b0793e9e87f0cbc504e03a538
6554 F20101204_AACQSQ droll_m_Page_020thm.jpg
a0f05e0d0af83c342d39faa0ae915ab7
e9cbc1b081315176d3f468d603a7731a225fca78
5199 F20101204_AACSCC droll_m_Page_129thm.jpg
41d41c9a6a203787de7c4434ccfc34b2
beb683f179ad663c907f890cf270e1d5d6be6fb2
23314 F20101204_AACSBO droll_m_Page_063.QC.jpg
92b92c5b67d6af2c561b1a850feb827a
92062151308c3c87843be9eee44831abb5795a27
6509 F20101204_AACSAZ droll_m_Page_094thm.jpg
7e60dc3c34be5abfc3114722653f308f
2d1ec1cd896721836763b2c237d6e9133b27c93c
2028 F20101204_AACRWI droll_m_Page_066.txt
fdb7cace6ca6b2baa55f419c2ac8c1fb
40e126450106ca24615f31c1d3449f94445b6eb2
1873 F20101204_AACRVT droll_m_Page_049.txt
9e7e31eee426e00da67e485c8743b246
89106a01e1a018aaed6b76ce7699e9a99b9d3bdd
5702 F20101204_AACQTG droll_m_Page_006thm.jpg
a0d8887f2e387f678f37acfc1bc758e7
eb572e44d201cc03d6b8b2b5566abb4c3bfcd4f1
56221 F20101204_AACQSR droll_m_Page_098.jpg
6d924f1ea300143a4ef6ff1c0fb2f15d
99b3c24d3bf72a630f6d8a413dd4b832629c4788
24119 F20101204_AACSCD droll_m_Page_097.QC.jpg
75805c0a49de8c33963fa9328131ee17
dff1e38ec54722823a519ace2120bc50f1c2297a
24208 F20101204_AACSBP droll_m_Page_154.QC.jpg
35c252e5e8a2b906286ae477a82c3274
e13962e5b807af2a6e441950b4cd2599ab2302a5
1964 F20101204_AACRWJ droll_m_Page_068.txt
d8fb21ae644b7319438b37539acb4de6
d207a87df171ea1972ad40f6cce3bcebd05d96dd
2372 F20101204_AACRVU droll_m_Page_050.txt
bb6e44689a2bfd6c1115b612094862dd
ba372a34a2c08a329cb514c0ef611a18b56f64ce
F20101204_AACQTH droll_m_Page_130.tif
763eb0ed0a948642b016ff9898f249dd
19425d06a83b16a97d001fdbc1b706620214959f
F20101204_AACQSS droll_m_Page_161.jp2
868af0ef2e768bb71197a2f0d0893b1d
fc90fc0b2c558983f2a524a57862de250af1c66e
6743 F20101204_AACSCE droll_m_Page_149thm.jpg
e31d0a8df305505d97e2436ddbe6d9b6
4b874efe0d4a0ad6eb36866c716a62563d04ba10
15812 F20101204_AACSBQ droll_m_Page_083.QC.jpg
045c18601c707948d3902cb28d09f5cf
2ce09ac132bd8e6443d8556942a4ae9dc6831c27
1930 F20101204_AACRWK droll_m_Page_069.txt
e378ba9283459b108a724f0854a79f98
fb2c98513b35002f835441cd5b007ed5e167f310
2056 F20101204_AACRVV droll_m_Page_051.txt
2d55fc746eaf2399ca8620adc416b5d4
70a45160155b5f92f8aa59390fed1a84e17e5ccc
6701 F20101204_AACQTI droll_m_Page_172thm.jpg
fa048e618d80df341405ff1b97037a02
3cc21708c6bf72c155d2536e1b28a62cc190a0b3
5068 F20101204_AACQST droll_m_Page_140thm.jpg
1e8441ed46b8f76844ebfa7ace1ed673
b9455dc209f228b1493694cf2b2ca833691e90d5
5506 F20101204_AACSCF droll_m_Page_127thm.jpg
2f43303279cf2844e659a0d51dcf4bb7
dc282c928b4ab32563e650b02e7434ef84fec93b
22382 F20101204_AACSBR droll_m_Page_186.QC.jpg
0f7245aa47806bb36d52ee9a2025fbd4
24740b3c35eafe845e8cf2212a30167770a26b67
1817 F20101204_AACRWL droll_m_Page_070.txt
481d2602bc791c9cb12384785c7b3c7f
d9b1a5e777389ceb35f7ebe050b6310f72d777ec
2126 F20101204_AACRVW droll_m_Page_052.txt
6629c051f298d037ab627246a2671e05
ae1ab46c94db681183c28cea5ac2305e9c9f60d1
20528 F20101204_AACQTJ droll_m_Page_141.QC.jpg
a062bec5e5e7edc6e9616d7f3300c9cf
5c832a94ca5ea5a743b3b7ae2e1175f4bfac49aa
2095 F20101204_AACQSU droll_m_Page_053.txt
5ea247087d536196fb6ed972dc5ee968
ea1c5da7690f5bfa1cc79f21a24245a35c6523ee
6399 F20101204_AACSCG droll_m_Page_073thm.jpg
9e137c2cbe062ec197ae43066dc59db1
bb0686e2bf6c297ab2256f5c590b39aa806b80b5
23518 F20101204_AACSBS droll_m_Page_076.QC.jpg
87ea354deec1db4cc3f828c0269f75fd
af27e70f968754002f5fc29ff466c12844bc6480
2068 F20101204_AACRXA droll_m_Page_087.txt
57848bf60ec42f1107da0215118ce507
d9f9024f3f73aabe0627bda705ed7031fbee8f4d
F20101204_AACRWM droll_m_Page_071.txt
e66f025b3133889c45569fe5a58beee9
a9cd1bd85032d4a054006c1a4243701e275d0ee6
1838 F20101204_AACRVX droll_m_Page_054.txt
82bf1ebb1fd38b41f01c36aef4a2f67a
c5519e25ef6da4b3cc0fee7ffd3b53db4e2f5a73
1651 F20101204_AACQTK droll_m_Page_099.txt
6226cfc85d53a44c5547018e7c1a6925
900fb0126cd6cc7f5c1cec62e82d0caa43c67610
5205 F20101204_AACQSV droll_m_Page_134thm.jpg
2172e82c23edb4bdaa458d7074e91e3e
4c5e3659da8a231472c081fe744767262b7fd8de
15469 F20101204_AACSCH droll_m_Page_099.QC.jpg
627ac64b4790a81dee4905690c013208
62b88aa5f19263e1180447c4a5d627c97aa894e7
6346 F20101204_AACSBT droll_m_Page_085thm.jpg
8e63d7204bb48ce3df65445c37d91f92
2635c99083825a1811770f086df157c1cadb72eb
1957 F20101204_AACRXB droll_m_Page_088.txt
5eefe53f48921099bb16ffbed0d07281
b10115865adc90bea684ccd1d81eb55aaf3719d0
1853 F20101204_AACRWN droll_m_Page_072.txt
4e382ee107b1a52948dd4f8a565b3fda
24de37d45057e458b7da96857429fc3009ebd3e6
2503 F20101204_AACRVY droll_m_Page_055.txt
40e2ffe4f8ba0db376ac2a85b44ecbe4
3e445cdd2c1a9ce0afb19e4eede1f10c2b7b94f7
71859 F20101204_AACQTL droll_m_Page_019.jpg
8bbdcbb7ce7fc19209c8794a050a75ec
0884b3b8d418018517b3e3a284e5bfa9172cccc7
F20101204_AACQSW droll_m_Page_197.QC.jpg
8694aa4c8e2f16d463d8dffc96084b50
184b46e46c9442b261a4c892d24723acc5735e80
24554 F20101204_AACSCI droll_m_Page_087.QC.jpg
cabc93992e08b10cfaf9f726ec181da8
7512688b1f593cfb1b00ee872ee2e90172475d6e
6628 F20101204_AACSBU droll_m_Page_066thm.jpg
a7b5a636d0199efac7b94b5f3f3b002d
1699057fcc12774cc97e53bb840c4a748eb53490
1941 F20101204_AACRXC droll_m_Page_089.txt
9d8904e98ace66a470394d99a095a5c8
12af4c3637a371ab201616c5ca0cda9239e32e3e
1922 F20101204_AACRWO droll_m_Page_073.txt
992563cd9492194c571aa91924db995c
21ba364f58c7933b9e9b08a4b418c970fb4e713e
2070 F20101204_AACRVZ droll_m_Page_056.txt
edc403bde2a936bcf410ebd86230a36f
c284573ed4cfb2edf3bbc925fc86be3a52351b10
F20101204_AACQTM droll_m_Page_171.tif
3648817b5a63ad3681bd07a2a2f01616
e063c7367e473fe1f9679498a601ec4b0a2f5a3e
108630 F20101204_AACQSX droll_m_Page_071.jp2
315678e009537079f648e196699a1e77
736dc79a87f3c45f77212932317fd4c70c4e24f8
67662 F20101204_AACQUA droll_m_Page_041.jpg
531c5a8af63dd4371a51d3c2473e5762
24a4f2a57762ee5536df6340e8950eeec4be34e9
23840 F20101204_AACSCJ droll_m_Page_067.QC.jpg
469e4cb99d14b83bae6f409c07d9ec30
8cd0d096d8bda473c28257120ef38503ff8cc974
6159 F20101204_AACSBV droll_m_Page_022thm.jpg
c9150d175dbd155006ccc2bd1d6c7074
78b53c23c6cf7120f87b5fea7721fb2dd281fe22
2134 F20101204_AACRXD droll_m_Page_090.txt
242c54951cfc162bf12fdfde24f5dc69
8a80a4e23eb25ea1c062afeb570bd1c6ce1924a3
2002 F20101204_AACRWP droll_m_Page_074.txt
053bf1d8ceddd7ecaa5cbde277117d0d
8af3d7e983aa47c50654055f7d46fdeae7ea6969
106097 F20101204_AACQTN droll_m_Page_159.jp2
decec8b9a33e43d77150d3ce11e0df6f
9d38aee193ab6f95b808a3028359ddb17f73af8e
F20101204_AACQSY droll_m_Page_155.tif
bc87d0517b36c017074ed77794c220e8
c52f5ec8fe40118baf3f757e8d4b1e6dc2a2a12c
31391 F20101204_AACQUB droll_m_Page_057.jp2
89919b40fc8bd7de46270f752d85782f
d6e82b3121919fbc20af0fb555fe7e66a51367a8
18423 F20101204_AACSCK droll_m_Page_010.QC.jpg
480c56f290e2aa5a7035ce272f0744c6
fd327eff07fde0ccba3f9e7bbee57ab7bd6e2060
24755 F20101204_AACSBW droll_m_Page_090.QC.jpg
64fab3867171e039ff48f7fb67cc1ad1
e77a89aba603415ddeb21306b316e758c5e5ccbb
2103 F20101204_AACRXE droll_m_Page_091.txt
9e96dd3384190e6902549b62ac6d87a0
b5af6a153fa3c5fcb1f06a38c834feb6d2bbe4a0
F20101204_AACQSZ droll_m_Page_016.tif
32e535f4bce08ea1b21dc66877f68734
9436cbf562a498d5be30148b733772bfda4de09a
765 F20101204_AACQUC droll_m_Page_008.txt
b6c7f668671b15f049c947e761651af0
6c592422beda892216f059fedef23f11fd37d6ae
4963 F20101204_AACSCL droll_m_Page_135thm.jpg
8471a2e2747670ec7a556a18b8408211
a760b9aea9582f6053024d13f9e2a261689eb984
12894 F20101204_AACSBX droll_m_Page_173.QC.jpg
6712e7fbebecdc19b5c18348baa35c21
c9a66eac804d757c53c560db864fb35b5f8e8563
1929 F20101204_AACRXF droll_m_Page_092.txt
a3970caf74d81f5d0fd6f5ec8363e9df
4f3a302e28e86e2ea7e9f3d414c3fcd467512bcc
1989 F20101204_AACRWQ droll_m_Page_076.txt
83950a0f2c48e997aad3b88edb7635f8
20fd82c1254536c1f6940acf08f5aa71e65d7ec6
5403 F20101204_AACQTO droll_m_Page_138thm.jpg
290d33626c95435e6f6f9db4d332e47a
aff79f033eb6ad79c3690edddd52b8b8c4bb65c3
5435 F20101204_AACQUD droll_m_Page_139thm.jpg
e44a7b7652cdd0f6bd9bf587d7fea9a2
4dbb29b717ee3e45b54ad9d72a34c9b4ffce5c49
6275 F20101204_AACSDA droll_m_Page_166thm.jpg
d9e1deb8dde989a7259a95beb7a62128
4bafa06f191f786524c7367d73c4b8c529c388af
7446 F20101204_AACSCM droll_m_Page_001.QC.jpg
7e85be6f9591e9264396a4b34a6d67ed
6f3c278490d40e549eb40f677b535ef2845f852b
22203 F20101204_AACSBY droll_m_Page_147.QC.jpg
586c2111b3520286e2c652d1448be6b8
4be464122675857fd878d2bc4f4ba8936da9dd05
2102 F20101204_AACRXG droll_m_Page_093.txt
01530a470f5b9df39886aa1c4a69ac4e
3ca6d41b5f190dd76d4f9bbca01ecd1f72904981
1875 F20101204_AACRWR droll_m_Page_077.txt
b60a2b3fedb7800e1a51e17b183976fe
0f1073ee75fef872e1a39dbda74da198e9955670
50210 F20101204_AACQTP droll_m_Page_016.pro
3bea7d71eb265fae2922b16d3050ecad
a80ca1795d5c5a908c75af39c2adf0793978ea5d
2041 F20101204_AACQUE droll_m_Page_080.txt
ad5c2b4a15137115c4330779c7095111
afdec374ef6b52cb01212204c06284157aa8b588
22794 F20101204_AACSDB droll_m_Page_051.QC.jpg
afdc40406e2eb80e2a3033e15fae1f5b
5382d4066b748d648ccaf226a8d08921b6eec121
6407 F20101204_AACSCN droll_m_Page_167thm.jpg
7016f149b310454aede355369a349fa8
7f596d3277b7ff03f3d19b440773431afee62a94
5250 F20101204_AACSBZ droll_m_Page_111thm.jpg
09f780ccccbc82e94e746b913166f36a
6efcc487a3fc6e829b59d00ac256e467b2562e57
2037 F20101204_AACRXH droll_m_Page_094.txt
5202c5f9a16da9ac124b423c66e5e62e
784b8747a2162cf2b370e53e44c696fcdad175f8
F20101204_AACRWS droll_m_Page_078.txt
296687019ac122e59045d8eea94eb77e
bfc562d5941ec2f1b4eaa3a1659e9a6d960f9ce6
1716 F20101204_AACQTQ droll_m_Page_144.txt
231d730045ef20cff6af1e497a405d9a
2345a30e7a41ed07dded9e5e395391a985429d0b
9313 F20101204_AACQUF droll_m_Page_193.QC.jpg
20d9ac2adfbd843c3582719e8d6cc8df
dc364587a5ecb26ba2ee94c6fb0c783daa7ad22d
6154 F20101204_AACSDC droll_m_Page_041thm.jpg
e46ae9d993e34f03343e531b6cbca502
63bf8c1d7ab1a9dc89594b88c9a633e6cd57808e
15096 F20101204_AACSCO droll_m_Page_115.QC.jpg
22810356d037e9f5a951bed7da65eb05
4ee27499e819dfb10277aebc3368fd5d02cb4dcb
F20101204_AACRXI droll_m_Page_095.txt
9030acbd9aa7cefd96814f6236aff3f6
8b6253184b159b4963102f83c50257c9569ce7eb
F20101204_AACRWT droll_m_Page_079.txt
2a55030e8b87e1cf0c53781a40d6b476
dc1c98b4e7bd15135b3aabc63223d03eee0372dd
2090 F20101204_AACQTR droll_m_Page_111.txt
3c718702d73b88a2216411c87233c33c
a0a5aed8db4d7c3a6b45a6d420adc2c1b7b19dfe
69516 F20101204_AACRAA droll_m_Page_157.jpg
bdbb0caf3845d0a169666aed7dbc9047
b0aea5b6d769810621576f5afabc45af49444ea5
F20101204_AACQUG droll_m_Page_092.tif
ecf12467b25f87e490f698eb1880a2a8
aff477660938e378a5e4bb0d36000d880cf6eb26
14141 F20101204_AACSDD droll_m_Page_143.QC.jpg
2c6583ebc2cc0a68cfa3c330a30fcd9a
26217d2b0c7da1e7e898e69f802674263031332d
2654 F20101204_AACSCP droll_m_Page_057thm.jpg
753d5fb191bf91fbaebccc7153f96730
ec27539df3d486aa96a7ba38e2c8d498771f133f
2054 F20101204_AACRXJ droll_m_Page_096.txt
53c9fc86a7e65d60e7ae013f5ee240a8
dd4288e366b6e1e8b81448e0ef7db315449294f1
F20101204_AACRWU droll_m_Page_081.txt
0157ae00ea5c25441b81ac9fb2d3ea38
871d5b38a8d362ad7e88ebd828340e94e6265d16
107027 F20101204_AACQTS droll_m_Page_013.jp2
e2604d856244f5111885ab33a54192c0
6eb41a1c0423542825eae5a93db56357e23c2993
68748 F20101204_AACRAB droll_m_Page_159.jpg
04a47bf8ccd8e054295007643d3fb6e3
f1e3e1168f3c9e91d631ec05d2646f0e5c953503
1959 F20101204_AACQUH droll_m_Page_104.txt
68eb494cc9d22bc04de6b6fb1bc7910c
5bae7e96ad9064e3e46f4d38a1d4f235f12ef0bc
25246 F20101204_AACSDE droll_m_Page_033.QC.jpg
f81586c175c080fa238bf9c74e3ea672
5318be8501bfc1612b2df9a5f5f6d14a7d0c2c45
6731 F20101204_AACSCQ droll_m_Page_179thm.jpg
ad5eea486cd33c5c488eb7e70bca301b
25384f1b601cc9de19d2e1469eef2d491ed067b0
2081 F20101204_AACRXK droll_m_Page_097.txt
1ab2e0c8cad62fa52b37a65aec6d1cc7
1b26befca30e200096728738c345e5af9e6ef2e4
F20101204_AACRWV droll_m_Page_082.txt
9937a80aa6b6bc238e601409da7cfbd5
407184a02f80e30417ac6e8f3fea7ca1ecbb1fa3
99030 F20101204_AACQTT droll_m_Page_150.jp2
a0f1ff9428ef109f86f664ca10507adc
3bc64b8b0322ac6f276b8f1c6365b3d0595b2354
69295 F20101204_AACRAC droll_m_Page_161.jpg
4a35d43f9aca805721819d812291c9ea
1bb181984bad168616386cb83b77f2a478bd1ae7
47444 F20101204_AACQUI droll_m_Page_077.pro
ba7ed4b954cf5e59a642b79bd38e908b
1e5d8fd7c8947ea382ebe18664b9995cd150e5d4
22211 F20101204_AACSDF droll_m_Page_164.QC.jpg
c3c185ae1c84ac3f29ba3f4ec45eafbb
8137e20be0d56b5c2696c7a530449cfdfc2dd346
21869 F20101204_AACSCR droll_m_Page_161.QC.jpg
8df0f4eaae5b9d9234abbd728ebe5fcf
622d478d6de933f3edc1fb52ac4658c036413e5d
1996 F20101204_AACRXL droll_m_Page_098.txt
20852ccf678dde759fcbd4a282ab77ed
c882e3349b928714539c5bf205e3fbd38f11dbf6
1249 F20101204_AACRWW droll_m_Page_083.txt
15cfe51609e5e0bda3781629f2a40038
1483049908823f343283312c513b0229e339ef9b
F20101204_AACQTU droll_m_Page_047.tif
726738e9bbc435a3eb16880895b5c84f
b5d1b999c74e6249feb99124fe75532f8cc3f06d
70462 F20101204_AACRAD droll_m_Page_162.jpg
1ab4960c6c55e7635a2c69d8a9f7d7ae
b39721440b60fb4210fc7d570b8e97e9488d4de9
67664 F20101204_AACQUJ droll_m_Page_164.jpg
d8040e3818347a4241766ec129b5be44
d3208147a3adb841ccbcc0115b989a2bad5f892c
20869 F20101204_AACSDG droll_m_Page_003.QC.jpg
848a334d82265d1e128d5c1eac4f0ca4
32d88080bea7952fe578478d6f3dc2a9da700f3c
22063 F20101204_AACSCS droll_m_Page_085.QC.jpg
720f08c26e852ab5c9d212f77627804c
9368fc9526f6785cfecdc6ce11eb99d58a25dd89
F20101204_AACRYA droll_m_Page_116.txt
f461e3bd661760fefdb4f586a1fda1ad
f290e1511a7016c9f41ed7c87939218b94091755
1827 F20101204_AACRXM droll_m_Page_100.txt
a27bde1fb57aa946349863da47ef1c79
4e26ca50fca61c5ac87225931280b75b1287c6dc
1785 F20101204_AACRWX droll_m_Page_084.txt
db8e2f43095bfb023c4475705c7c06cb
d32a7b86c178305794796a83b4b878264522633d
99840 F20101204_AACQTV droll_m_Page_049.jp2
41618b07123fa214c30b11d83739576d
aaf0e4cf14dc0e047a6d52f9e9930a2cffec1225
68134 F20101204_AACRAE droll_m_Page_163.jpg
dbf6cac31f42bd60bae6069d124db6a6
ba85fc7669c00d34b0584bf9991a9e96d6088d4a
1917 F20101204_AACQUK droll_m_Page_015.txt
f21b962509d504c940f940585709ff58
a3e396c2690ee0f1f4b98637f9275edefe42f611
23365 F20101204_AACSDH droll_m_Page_152.QC.jpg
5355dcdcb0a8e2a56b3990ff22f39b5d
a16f9cde72e16397c04ca194ffe706b8492b7930
21224 F20101204_AACSCT droll_m_Page_022.QC.jpg
c199ceaa9599a0c054be083ff6c01c53
2556bbc497aa5ea4d4202425da1c1f20b82fcbf9
1804 F20101204_AACRYB droll_m_Page_117.txt
abb51ad9b8c8cb9ac22d40fbcc4d69a3
7d8def98d12f811e609051b6f0aacf99735b1389
1981 F20101204_AACRXN droll_m_Page_101.txt
c2ab158e1ceb28563e2a9b4602332b70
edc668ca8155be39374ec8f31936506cde32ef8d
1932 F20101204_AACRWY droll_m_Page_085.txt
c3713f64824cf35fc18ee8c503299712
91ee082141187cc2ca89dfe8f473b0f4ca3c3ad0
71024 F20101204_AACRAF droll_m_Page_165.jpg
d4760e1ab93eb8bb1413bb1c6e5476db
33f72ee96eb07f51b26c0f2c3276cb397d0d713e
6689 F20101204_AACQUL droll_m_Page_095thm.jpg
70bda8637c5b5ad6429846ece2c63ea6
2551645b4f26626e7160f6f696caa00791d85f1d
20565 F20101204_AACQTW droll_m_Page_126.QC.jpg
e9da72e6c2414a282d2c927f9ace5a0a
0b1044644ef0f6981e20150980fe485bd3d058aa
22892 F20101204_AACSDI droll_m_Page_068.QC.jpg
8b7b5e64fab9ff650abe1b7826f8b75d
116375092e3eee01da07e79d95ee244920a5b5bd
18773 F20101204_AACSCU droll_m_Page_138.QC.jpg
0b6882f33c2868b48d55b0200b1cda32
dcc3e1f488fc5f5b938d0976a2a4bfb0e268c85a
1912 F20101204_AACRYC droll_m_Page_118.txt
52812c76ac9409d3f5ba30df04c209c8
f3d8f3c261a10681a6e2934d3a4eb0b839c1ca86
F20101204_AACRXO droll_m_Page_102.txt
1ad79b8c5c98600e4b6c59ccf261942c
40c778dc512df3d51c313bb1e3c97781e7b8910d
1900 F20101204_AACRWZ droll_m_Page_086.txt
2250e47ce621ab1ee439f68b09bd8506
cb0f1e80f1a886456ac64475a60eb9c4ac0731d4
F20101204_AACRAG droll_m_Page_166.jpg
3a7b271f3e68b0d0c1cbce966473411b
7bfc9f9fe0b4269b82c824304796a605bad6a59c
227592 F20101204_AACQVA UFE0010111_00001.mets
5a979d6facfe62e7af5aa431f2194190
fe9197ed97d2f155ddfb96f2d022264a72beb790
5026 F20101204_AACQUM droll_m_Page_116thm.jpg
31ae602ae924d4724c95158c576fadde
93247507d6429e9ca905896327419db74cd5eff4
75670 F20101204_AACQTX droll_m_Page_079.jpg
e6f994d4524308b40ccc511502b30d49
ba7453fd60fa9726dfced1a6ad8fef66eec77c52
16145 F20101204_AACSDJ droll_m_Page_105.QC.jpg
ba8824f464f0c769b577d72692c03535
e8ef8bd6427040e01bfdc005aec107f91acb44a6
23196 F20101204_AACSCV droll_m_Page_195.QC.jpg
98b0b792a47f58511d1a0b3f8919384b
539b00e8937bb399c31818dcd32e0a418db80646
2197 F20101204_AACRYD droll_m_Page_119.txt
401e847e2cdfcd00b9f673d94702e1de
bd9a403f6c0dde7244d1b6dbe720cd705c55ee8a
F20101204_AACRXP droll_m_Page_103.txt
7034431f9b763b6e8af2c2462890c825
e8871691785e7fdfdb7e8a5c62c40c36f70c693d
69992 F20101204_AACRAH droll_m_Page_167.jpg
d44c834b9f5f0b531e19bbbbb7e865dd
b2c9f368e2cc3594abd10cc88fc52f80fe80ac9f
110432 F20101204_AACQUN droll_m_Page_024.jp2
696dfac0fb8853f2ccf9bb70199c748f
2233ba9b6d921c34ed56235ca3406ce3951b3a01
24704 F20101204_AACQTY droll_m_Page_057.jpg
fe796da2800a2a8ad0b20fd8ad906316
a1a1c478495a60b3206088eb0d3fd02ab87fb2c6
6448 F20101204_AACSDK droll_m_Page_074thm.jpg
dac482f28c7c2aa1273810d5c2c02fdf
19ec13a4938eb39147c98ebce96a975e2feccc7d
6549 F20101204_AACSCW droll_m_Page_046thm.jpg
f0083af7958ab63339a9f6c86f881f62
d1d58fc27c6725de4a1f19178377a1dd9769dd97
1671 F20101204_AACRYE droll_m_Page_120.txt
e2fd7dfd9e097efbf3a7f70b61f0f63d
73123310d43eab80fe287bd0ef75d4a4b405cf2d
1763 F20101204_AACRXQ droll_m_Page_105.txt
e275afd53ec4f9979533473004a1f35c
a033cf88c28767e6876a63704452ca4429426641
74330 F20101204_AACRAI droll_m_Page_168.jpg
a6989e9fcda0f8428cf2693937c331d8
c41f9589ece73d8f1ff9413d9c6f86598987d914
17519 F20101204_AACQUO droll_m_Page_108.QC.jpg
54ad317ea344a4ea9b85c3749564456a
b4a6660f478917bf2cd16a06cb02de288e1e443c
81838 F20101204_AACQTZ droll_m_Page_006.pro
f9799961aff3d3d278036e1de74c2ef3
fc143254b12abe8a1bf560ab1491254e9de0f292
23802 F20101204_AACSDL droll_m_Page_031.QC.jpg
af946d6a8d76730939163b9dde307450
072abefe384e982654df9c63588782a22ec820be
4915 F20101204_AACSCX droll_m_Page_114thm.jpg
82d70502d1e8d8b489c757daf94a177c
6c3b424a75526ea84ff0de02c169db49983a5ad7
1962 F20101204_AACRYF droll_m_Page_122.txt
1f4ebceec3880207a6d2ae84fed6bd8b
997a3584395ff35750dbfa1131a0622fa5883de6
78911 F20101204_AACRAJ droll_m_Page_169.jpg
0ef6231caa00a17ec539eea94deecfd7
4da3db1519c5e483f5dcd875ee583616828c122e
24092 F20101204_AACQVD droll_m_Page_001.jpg
82a1fee01b2dbaabfa481886c112953d
f6ef40e0b54095bae4f23164ded2d5dc32fab8cd
6288 F20101204_AACSEA droll_m_Page_189thm.jpg
95e2244164384e9e9828badf70d13aa2
a84fbea648fc149073093c5cd9c83d7fa012fc15
8311 F20101204_AACSDM droll_m_Page_057.QC.jpg
2ede5270c0ed927b247d3c73c6b01051
d06d73e87180ce25bfa76cd99aa53f4fa29c37b5
6599 F20101204_AACSCY droll_m_Page_042thm.jpg
73ed6fd071cd90647f4cb916c9a0e868
3a22a7178507493aeaa640d360588eddc7fb8363
2060 F20101204_AACRYG droll_m_Page_123.txt
6a683fee5847fec0603e393f7246ef8c
8cdb89da186eb47bf37c7cd6ac3861a22b0cb3bf
1799 F20101204_AACRXR droll_m_Page_106.txt
fe12ae16384df136d037f542f8261a3f
5f13d04e91fa507ab3720c5fa3480cbcab520ea0
70544 F20101204_AACRAK droll_m_Page_170.jpg
d2cd53ba9580d80cd80fd45cad9de529
02265e4c4694e44fd46a5e47738588d263b15bab
10385 F20101204_AACQVE droll_m_Page_002.jpg
6b038652425f41479a9c8b85a5a77050
b0a7dd709a896119396dd7c25f3b156e07bbe80a
49309 F20101204_AACQUP droll_m_Page_145.jpg
56dede381104c9966816ae2392f5fddd
bb81403171256203aadeba5315f3c9f534e4d4ca
24385 F20101204_AACSEB droll_m_Page_050.QC.jpg
64f7707a8b8c101c88bd46cd04411285
7ed456ab0a3b8c63ed44d4e929feb1b0c4d42aa9
17528 F20101204_AACSDN droll_m_Page_134.QC.jpg
1fd8cc06efa3b23441f7107ed9ffc4cd
10dc7b6e9bc8499e74bc094c3545ece11af5162f
6227 F20101204_AACSCZ droll_m_Page_180thm.jpg
8f64e89489ae68a291603eebf33aabee
afaf3161ac3b2a0a4d81d30d27e3a12400b3f1af
F20101204_AACRYH droll_m_Page_124.txt
23c0b4f12273cc5a170f59b28544cb69
84c3a61b0033578d6548ed32b2eae45765674852
1748 F20101204_AACRXS droll_m_Page_107.txt
f5ff9d2f8fdc7fe2804a762c00a748bc
e3bcc99904ce5cd895ee2784073fe3366bede074
83482 F20101204_AACRAL droll_m_Page_171.jpg
c0fd77cfe4cf0ba8e1f327516c77cb6f
c1074f85412bea5658ba6bc0342a75adb84f26e0
63928 F20101204_AACQVF droll_m_Page_003.jpg
57714c5132a30090eb024dbf3321a261
64c07f5148c9c07edbb77fcc3928cb55bbc30e25
69057 F20101204_AACQUQ droll_m_Page_085.jpg
e781cc598af4529dc2b132f8987f1415
17a65d06b5e514aa79bc7a668bbf36443996ea63
6992 F20101204_AACSEC droll_m_Page_012thm.jpg
8f6ddd99448dfcca9ef6785b38a5f14a
7521cdada007cc8e2181019bf879bcd20d9e6667
24099 F20101204_AACSDO droll_m_Page_149.QC.jpg
f3a54978510b0944df1221e39095d8fb
bacd295168d374dfe9e4cdee33f6dee5b5804b44
1788 F20101204_AACRYI droll_m_Page_125.txt
85f17f26598e4823705409a0e820cdf0
1502da524cac64498762f358a039ddd1c15f58d2
F20101204_AACRXT droll_m_Page_108.txt
cd90c01ee7eafb951728c530f186c19e
68aa21620a0133b31a16aee1f61d42e07deaf6c0
73486 F20101204_AACRAM droll_m_Page_172.jpg
26307e87f8ae6611053c086e05bd0447
336b4c080fd70da33634b030a7ec1aac92927e80
71121 F20101204_AACQVG droll_m_Page_004.jpg
4b299014d5723cbc58f3ef2275e8966f
c4802ac4342784e7a1a6f14593ef3653b35f1f59
80022 F20101204_AACQUR droll_m_Page_182.jpg
a0ec50300c3a1eea4bb8053785438037
28c4247710e816269d4f13dc509b5e2ae83507e1
85171 F20101204_AACRBA droll_m_Page_188.jpg
2c5807ceb5d963e55587fa3047165c08
9d7a25c274e83546268dee17d3ce51cd9722dc9d
5415 F20101204_AACSED droll_m_Page_174thm.jpg
ec7b2a051e078617b9db0febcf79e2ad
57ef8347f85c93d6575cdca246d669bd56be4f8c
6483 F20101204_AACSDP droll_m_Page_075thm.jpg
378e0501d12a1cae4fd48478153119cd
bf8977428feeb7cbdd29fef8c52e62126917ef72
1812 F20101204_AACRYJ droll_m_Page_126.txt
75ad4a94960a293feb656469afe47788
c60f6f177ab79e1524d83b85cda099f39453926d
F20101204_AACRXU droll_m_Page_109.txt
aa22f3fe80724795eb55429d4c982518
f20355a2c11e347015c4d2d5c7dc67d8b2abbeca
39022 F20101204_AACRAN droll_m_Page_173.jpg
b3eb0df247eee688f62e645eef7c67be
d4be9802a2880bf62ab263eb11aeccb508e295d7
96828 F20101204_AACQVH droll_m_Page_006.jpg
666e5c7812f0b2a04d2f09d6b5ef1d02
e8bfee8e84b87ad1fbb6a78f5456e5c3700fb0d0
109812 F20101204_AACQUS droll_m_Page_032.jp2
ecf709575df550b7237794317e392892
f1f0ed8487d82f776f5ab7d19941ead4d69d324e
72226 F20101204_AACRBB droll_m_Page_189.jpg
1ebb319153d1b01995e6dd92ad73ae87
73d8b590c6c81311acf57e17f6137ce6d2db7190
21733 F20101204_AACSEE droll_m_Page_146.QC.jpg
0bc75381c5521a98a6a657be9605f71c
5deee2d29494035afc133a26249831004b49f649
6492 F20101204_AACSDQ droll_m_Page_188thm.jpg
5e4134dfb577c186c47f3b61a044994b
6d8e6ad8c9b2001ed83f97d665408a950de90d53
1780 F20101204_AACRYK droll_m_Page_127.txt
856de4d53b2649087336cc5bcc260bb7
889fdf34cd3b484e10b54e7523c1a84b32e9f02a
F20101204_AACRXV droll_m_Page_110.txt
8fbf9a4ba6f012c366da22fb5ebcf0b2
3cb65abee740c045eb470593b5cfda5f83d9f314
58629 F20101204_AACRAO droll_m_Page_174.jpg
c689a222fd0e86bc20550478b6e4813d
471c3d4419f4b7ab9ec96051a6aae51e75190240
73670 F20101204_AACQVI droll_m_Page_007.jpg
e805925724e9f3991772f1b8af909cf6
122f4e4a12a6fc2f4ef2f85a6fc47c9861f85ffc
F20101204_AACQUT droll_m_Page_146.tif
8c5f3e8d522e3a1ddf288e74ef205d77
2d622c3a05190d85235b1c18fe34cdc01403fcb3
74912 F20101204_AACRBC droll_m_Page_190.jpg
c7f6b970642603542e3238a61c261f10
89d825b6a2b9f03464125b7bdbe189a043178d18
15679 F20101204_AACSEF droll_m_Page_112.QC.jpg
e5d58903254c3b188771983078a3c47c
0e55c362f8aec248a5e6376fb6c5c7effa629f92
6505 F20101204_AACSDR droll_m_Page_068thm.jpg
10ab03476d52b22e5dff1300cb575512
494addc664330c348c904ebd11813d24fafcedd0
1810 F20101204_AACRYL droll_m_Page_128.txt
72218b6bfaf237e0bc9867c007449385
7f6ec4ca753d3b7625bd485663a83ca77b89cfee
1897 F20101204_AACRXW droll_m_Page_112.txt
ee41d04a3c310d3c28455acdb5203289
2209c21727a64e633fc613cb528b277db997a1f3
81173 F20101204_AACRAP droll_m_Page_175.jpg
35e8b456a0bedd7561930b120bd22e2d
158c675696a582ca19b31d728458e2aff30c4048
33135 F20101204_AACQVJ droll_m_Page_008.jpg
913832c1c092e4a3a41d88b836e90fbb
1efab4d3528da615377617bf4eac2cabab72d83b
1650 F20101204_AACQUU droll_m_Page_137.txt
32883c403fc02fbb1c33c3c39709bf19
4ed240515c758408fc75886b20642e10c6746d69
71178 F20101204_AACRBD droll_m_Page_191.jpg
c650e089e0bd249b456c2c730439dad4
2677d57a3da8cb505d9f5c6530bf4c2e0bd61c74
3310 F20101204_AACSEG droll_m_Page_002.QC.jpg
9ca5f56a3891fc39fa51d9485e5de4b8
32453ee590fd210350d4f325cdd4fe98579033c1
24997 F20101204_AACSDS droll_m_Page_187.QC.jpg
dc5a3fd9acfd320b8a26718970e03020
607bb7ae9e0e37902e455b2ef45dc96f76e99160
1987 F20101204_AACRZA droll_m_Page_147.txt
3226e48edde7ce8b30231c9990435df1
e71c7860b07cc32039c0eed37ea1edf3a20b89bd
1584 F20101204_AACRYM droll_m_Page_129.txt
6d63b8ea45f0a231c23a19adf0447e9d
dd7497a24a4afebb57ac7aa2dc381e4f89271f7a
1536 F20101204_AACRXX droll_m_Page_113.txt
a7d7cbcb15c1a719331fdcb7311e0466
d36a5e704e8de21f666a32649c00d5f1de0569dd
68856 F20101204_AACRAQ droll_m_Page_176.jpg
5aeec6b874ca3675bd5fe4ba02ecc36a
c3b68902c22d5f61ed690faadd0f69d010453b0d
56135 F20101204_AACQVK droll_m_Page_010.jpg
8a2b4f1d80913d766e49f51c7313c203
10366fa755d579f23a67c444daf5a344fd332e73
6898 F20101204_AACQUV droll_m_Page_062thm.jpg
13762af10ac1ee2fe32b9f7c655d2df3
ef5426df8d78247b5e8fb5659963bcb36dd39eda
66977 F20101204_AACRBE droll_m_Page_192.jpg
b4e93da5614c68b9441e5d5174155ab4
68eed7c10e25b820aad0f3533a9383380f14b69e
23424 F20101204_AACSEH droll_m_Page_023.QC.jpg
ac4247d61af8d6055b24370703e80b40
592afff70af92e3fa8a15def41d38b9a17339bfd
22863 F20101204_AACSDT droll_m_Page_013.QC.jpg
94011a7d8b34e545b895a7d053904b11
3791e7393f6ac553393b90fefa5933d035469b98
1667 F20101204_AACRZB droll_m_Page_148.txt
e0d04f3695d4b716912f9d9ef86350ed
6f3cfa2e2bd856df7a3a4effd2ff87368a929b67
1649 F20101204_AACRYN droll_m_Page_130.txt
e2ed97a507c69c48252c55e003f86a05
8e044f2591c619bf56fcfdb4973d5e7d3e3803ea
2016 F20101204_AACRXY droll_m_Page_114.txt
f4a6e13507849fccf85c5e6dba9c76f0
4d9b30ca01db6960ea76c0d10aa583828b108cb3
91418 F20101204_AACRAR droll_m_Page_177.jpg
fb5db4b6e26752acfd19c09c60cd199b
9e807d4a94a66b9639785a2541eec701a23441e2
66136 F20101204_AACQVL droll_m_Page_011.jpg
0a2d218dd5ca0d888d93ba6d7d539caa
28c9e2b9014f4cfb4e27d850e39be754043ab210
1807 F20101204_AACQUW droll_m_Page_022.txt
e9e2468dc7152e2a6eacc250e67b419e
f73174fec0a2f2279de6825d73968f8d5973b9d1
28924 F20101204_AACRBF droll_m_Page_193.jpg
65cd2146a6995a61d60c1a89ef4d9711
cebb6b18cb3a6cd107508058645ce0cf22648b39
23322 F20101204_AACSEI droll_m_Page_158.QC.jpg
a6cc816817cbb1a9589d996eecbf97be
298e0b8d6b7c8abf6ec9f69b2d5fdcadb98a5acb
6444 F20101204_AACSDU droll_m_Page_050thm.jpg
ee1d85d7cefdb7fee2ceb4483b558a18
db61b41f5662b03147d5b65c8c6221ce81a81013
F20101204_AACRZC droll_m_Page_149.txt
f31c7c89aa6141e3ccb0cd8021ac583a
4cb6d4555ad2b07c3924f133846e359e70dfa444
F20101204_AACRYO droll_m_Page_131.txt
d8666fefb4d2a1906e1acf97cd2e2757
e5c4a2ab32514e11ba9ead3fb67e8600f03f9e6e
F20101204_AACRXZ droll_m_Page_115.txt
017c3bfba399d5b2390d4c258d70ce3d
8bc0618509438a356851c4fe9add4324002bd0b0
81699 F20101204_AACRAS droll_m_Page_178.jpg
017f4727213c92c70023eb05d90779c6
76ffd9506b0847eaaa2b43ac88e518d806fa0353
79502 F20101204_AACQVM droll_m_Page_012.jpg
edd7d02cbbcddc2af2a7b3a13af8b218
f668f9c11d09ead852dd92aa917fdee30b888544
6584 F20101204_AACQUX droll_m_Page_048thm.jpg
77beaf1edf664bf5fd128314dedc0a7d
ebd9a1f2aa7ff21009c83772c863cd876582556b
75806 F20101204_AACRBG droll_m_Page_194.jpg
d4630c45272dbb14a6def5623b951b80
1f44bbf4a91ee3218b0c214c3ac4531fe32e811f
58717 F20101204_AACQWA droll_m_Page_028.jpg
904ffaee0695cf614b2373b160b43672
e591e63eb747af599f2ef803b47936434af53105
22386 F20101204_AACSEJ droll_m_Page_059.QC.jpg
3b46af2273d0a9ab1139b656a1dd005e
508bb43ad27fbab602c07a348753ef76478b1d32
17770 F20101204_AACSDV droll_m_Page_102.QC.jpg
39692f01225527ee3cc6a93553253dec
146f6f36eb7a26ea86a39dd6376d80f52c9e7816
F20101204_AACRZD droll_m_Page_150.txt
9c1926219f4ee4e47915bd3f9472b3c7
2c9e23bc6a61ded4b8e2f4b1fac32e4ff46e8ef0
1934 F20101204_AACRYP droll_m_Page_132.txt
708f476318c29629ed8c4daf732facdc
44a4a28333ed77ce96621ab7599c221cac53575f
71650 F20101204_AACRAT droll_m_Page_180.jpg
b799d183843981836e5e67ab3376f54e
8513ce64b75ecff94c0d89ab2192867b0202508b
70916 F20101204_AACQVN droll_m_Page_013.jpg
333ed31d28bff961b5bc3fe4a65a1184
838d9a659028f3052e8ae7f7e55f311b5d646b60
1936 F20101204_AACQUY droll_m_Page_075.txt
ed5fcd1428cf3eb5815dc549fc20393d
a4979c03b0ffdf8aa59b6c299b602f34bb03eb47
82707 F20101204_AACRBH droll_m_Page_197.jpg
2b5b81d600436b81409811d307536046
56051ff2159e9b31194aa62934e09b320819b365
76785 F20101204_AACQWB droll_m_Page_029.jpg
77ede7a7bb0fcdc8e142d3607c83bae4
5aca3c6055a77c85b6ed6c204799656b53c8e1b9
3108 F20101204_AACSEK droll_m_Page_008thm.jpg
2f73b493c034e358c430a02541f8d27b
02041c403c70eab599f1d5d33bdf6b68cf9e8c9c
6343 F20101204_AACSDW droll_m_Page_021thm.jpg
f0b9e639d0079952d1a8a239e653e4c7
e76c986389c85ea75d36a3c63a6f0fe0a03b334f
1622 F20101204_AACRZE droll_m_Page_151.txt
a8cd6d0eba36074d95ea7b9676c78444
c4cc7033d7e92ac6eb33c5fc2586a7ca98c1199e
1971 F20101204_AACRYQ droll_m_Page_133.txt
581e0b62bc788bb2de68838cfa73a4ac
5e87bf94d6b89fd4c4a5d20127b2e1fadedc1c92
74917 F20101204_AACQVO droll_m_Page_014.jpg
4bc1dd397a88f8497d27b42837795c0d
35c01730c62bb915f5941abae692c43dce160b26
85381 F20101204_AACQUZ droll_m_Page_028.jp2
094182ea17deeab0f13cab85e2db73b1
d6d4b558a503c0b34ba64d884d0d24e8d212b010
58580 F20101204_AACRBI droll_m_Page_198.jpg
baf5db8687559d81cc63f8bc17db2c42
ea718f23c53cfe7f6e51871677e5571456b538b6
77412 F20101204_AACQWC droll_m_Page_030.jpg
4ce50e60f1642cdfed1fc236ed1c0f5b
b6c2ed4b56b72bbb1a9b41e8e71bce20f92ddf3d
76480 F20101204_AACRAU droll_m_Page_181.jpg
91871b2af7df18b395ffe77aad68eb84
5208c9eb2ac7ad08281dc4529418ce6cb2f9e995
18703 F20101204_AACSEL droll_m_Page_124.QC.jpg
57f416fffee7dbcd504c40c3b4175a5c
35877370d9fc3829355496b66dba5f852057e2cf
6417 F20101204_AACSDX droll_m_Page_170thm.jpg
4209d73b3c652763421944aa78000d33
11a2454c1afe04c8a6767b7aa29673712b1ad2a9
F20101204_AACRZF droll_m_Page_152.txt
6820ab9e29ddcde86ef1260c0d58f73e
69f72cdedae27100e0ae62d1bd7f32583095e81e
1475 F20101204_AACRYR droll_m_Page_135.txt
91198d11db4a22f8af1997b6c1fa8c54
02c80b93a2cfebc5cec848678e0898291e0e70f5
68121 F20101204_AACQVP droll_m_Page_015.jpg
c9978f81ebd62ac24de6877e2f8388da
edc56073d924b1891ffa94693190fd0285619380
66076 F20101204_AACRBJ droll_m_Page_199.jpg
e3d844696caef099c5bbb9980864b617
cb97e952cd2fdc2ccc567a91d0644fa35a980b85
73799 F20101204_AACQWD droll_m_Page_031.jpg
9a3453d992585d3419867ed46769de61
5c3b21528c7b54595933d61dfa5df702dbb053ab
74130 F20101204_AACRAV droll_m_Page_183.jpg
6ef1a6e1186b8ed5f7480f261236ece6
de047ec2018ccf45f94ed55102e9ab48af86c4b8
24283 F20101204_AACSFA droll_m_Page_094.QC.jpg
ecd78ca7f882aa57418c1a6f86aa9bfc
5495312a9e617a8f44c8e308fe4b93e4c8f4288f
20231 F20101204_AACSEM droll_m_Page_025.QC.jpg
1328ea6abfe54da6a1dfd8ca05278f0c
8811efca897967b69dcab8159952a4230be33e8c
5538 F20101204_AACSDY droll_m_Page_130thm.jpg
652d88ae70c3818285e6dfc3129fb2e5
ffcb22526c15605725369a3e468efd48fc1cff37
2029 F20101204_AACRZG droll_m_Page_153.txt
bbe155e95fa89f3f40fc40e25713dd9e
5a475e13a2a15fa353822acecb41686e075f0352
23807 F20101204_AACRBK droll_m_Page_001.jp2
543dd3be59179bc83a5609f699524551
23dc4eec871e0ff8dab315c9cc00df20b2e22b0b
73064 F20101204_AACQWE droll_m_Page_032.jpg
5f25a0b593f9f544994d5945cc97c95b
d16b89a1cf6d02a70e98cf62033625495b9e152f
81826 F20101204_AACRAW droll_m_Page_184.jpg
4b5104af93861c3654cbcb8711b779e4
48bb198db80dfd2634564f93dd625db93b3b6666
6633 F20101204_AACSFB droll_m_Page_153thm.jpg
78c3be746765e7bd0a56ef6915e8e5b2
235dc2706c9d43b7c7533f69050b6b5d33a0dbb9
23287 F20101204_AACSEN droll_m_Page_089.QC.jpg
a0a65c448f525867f4897baa7bd443b8
d33a04448ca23ef23d30aca06b4d062c42206e52
6501 F20101204_AACSDZ droll_m_Page_132thm.jpg
8b84006b293f3bbf6090a85c999e5d08
7fbdd6965c098a466b0c2c7aafe7b5585a8a40c9
F20101204_AACRZH droll_m_Page_155.txt
6fed69266b84c6b23eab536e07b1bcea
7b04a16105a7916795853d919ee13f9c0ac4f8ec
F20101204_AACRYS droll_m_Page_136.txt
eedc736eab450a5a0a95570c404083f1
b27a36db27ccbf1c00ed0e1ffcb354bb3aa71289
71462 F20101204_AACQVQ droll_m_Page_016.jpg
7c439661e7bafe29bfa558217d347774
0dd343847c1d1d3d09e41b93922ed393b70b8dce
5614 F20101204_AACRBL droll_m_Page_002.jp2
3c3c7372f85536df3a7e03779201f45f
6feda71b89cfd486ade48ba66e69b32a9d7964d5
78137 F20101204_AACQWF droll_m_Page_033.jpg
baaba5e1f8de82b4b9731a01558dfc1e
68f52dade21f93de9f38adcfb5067854bed06498
88500 F20101204_AACRAX droll_m_Page_185.jpg
532f25d424d68043bb198b8c456cdbaa
0a7086ca7a7882ff352795779853ef0c9474dd33
21434 F20101204_AACSFC droll_m_Page_191.QC.jpg
5b7f37126b9ec2ec844595f7e6f68bdb
4ee4fc0bbfdc876cae326a5c50507b0091427b37
24142 F20101204_AACSEO droll_m_Page_062.QC.jpg
c8b9c62ff146d8511af09fbe4767c2c5
21c0e571e24c521df4b2dcc09dea621b2d5fb140
F20101204_AACRZI droll_m_Page_156.txt
096254e174150292b2c80c31a057c6b7
ad15d4f58e5b1a3c8673652374a8365f866b73c5
1689 F20101204_AACRYT droll_m_Page_138.txt
b6e83e3fe070e85eda8025f6edb2dc9f
648adfe4b90446c53384a6d97788ce0772ea2220
71305 F20101204_AACQVR droll_m_Page_017.jpg
ba953a899370cf7cdaeccbaac5a51d02
17ba18a71c865d7fc671f6cb888b5a64f606a941
107874 F20101204_AACRCA droll_m_Page_021.jp2
36d087ed66e975b5c5be533c5e3282e6
45b8de53c833b2b49c720ff57fdaa8327a3e37e0
92928 F20101204_AACRBM droll_m_Page_003.jp2
918f0837fd942d41debfa107deebe4ae
66533b7c126b2f08f068b29824a11a5def027d37
72809 F20101204_AACQWG droll_m_Page_034.jpg
38a3f86613effc5360095646cb98124f
5f9472ba49908318e779ec39e52907a80c6c8415
74092 F20101204_AACRAY droll_m_Page_186.jpg
0dbfb48f88c5bbd20f05432500c97d75
06bac224d48817d96a2bda978873d1008922ded0
6344 F20101204_AACSFD droll_m_Page_122thm.jpg
95df00297ab190e3cf76c264e7380702
80f3fe5a8b2addcb8fb7af428d32e49e5186501f
16286 F20101204_AACSEP droll_m_Page_109.QC.jpg
1720dde3c318cc20ed67d4b86a6a975f
911df89cc40a861326b80df1e24f7cbcf8a443e7
F20101204_AACRZJ droll_m_Page_157.txt
73f6ae542c8d2f0dd35859b8e7fe47df
3ed2aec10395691f972e04543a3710c611b300dd
1776 F20101204_AACRYU droll_m_Page_140.txt
1dc38265d61ee7ce2bc77b508c4597d7
2e6d28f53727c080b79774102be811963d4877f9
71069 F20101204_AACQVS droll_m_Page_018.jpg
7cfb98d3686dd857163f34687fa068c1
14631bdce5da816f1d66d45e6854980bc5079064
98289 F20101204_AACRCB droll_m_Page_022.jp2
dc841fe12986b46ecd8583e6cec6ffaa
b20ec1d9fd282a36122ce7f8ab89e33b65b6893f
1051969 F20101204_AACRBN droll_m_Page_004.jp2
c2db6341eaa8aa2a55be22855833569c
433d24f9e1075da6292bcdd3ed5eb46a5241f2f7
75490 F20101204_AACQWH droll_m_Page_035.jpg
07114945462274da359d88e81691eb25
f24248c9531170890e0250c52d866882622dcd1e
88041 F20101204_AACRAZ droll_m_Page_187.jpg
74b8242740483502a2858c83286de7c7
3e296d68ece3a765449298578cf6bf44c56be975
F20101204_AACSFE droll_m_Page_165thm.jpg
197d45e4d586ee7520c31913decd244f
58e25cf680344189420850f658d50d21a9930786
6648 F20101204_AACSEQ droll_m_Page_091thm.jpg
060c61a230e0d8db707805f8c0c2a4dc
5e518a4d00cada59e93825db410043c005a313ee
F20101204_AACRZK droll_m_Page_158.txt
bc765ea7cfbb80a6cdf6bb41015c2b05
e4210053d14b129f9a7e939d5e9b37e61564636f
1824 F20101204_AACRYV droll_m_Page_141.txt
072f262388ac8b8bac8d874c9efbac9f
edd2585491e63ae816ef9fda7acf9c16160a56eb
79950 F20101204_AACQVT droll_m_Page_020.jpg
bf4af00adbe72be2247a7b7cd46039dd
64494dea9639838f48a3b44ebcc807bc785a3aac
109427 F20101204_AACRCC droll_m_Page_023.jp2
99e008824baaeb5a56c608b1a6c78866
4d69f54473e553f366b93c09c7f59c8945c4700a
1051972 F20101204_AACRBO droll_m_Page_005.jp2
86e6d57999a58a80fb9c41cd5521bf31
f67aed1b12c365ff355c2529e2672a6234654883
77053 F20101204_AACQWI droll_m_Page_036.jpg
24b8e0f75af3c5c32ac97e6266c1f0d0
5039ad1136819853b288fc5fd22dc15e7acf1a7f
6623 F20101204_AACSFF droll_m_Page_067thm.jpg
6cb267bbd0064413db90c4cb5e189cfc
4e841f086a1ddc0c2bae017a805cd8dae86cc7c1
6091 F20101204_AACSER droll_m_Page_015thm.jpg
1ce5e30e96d4dec9aab2f21f17f964cc
b415fa7ced62bae609949f7bab3e472afd9126df
1921 F20101204_AACRZL droll_m_Page_159.txt
460341bf63bed9a82ec826f7a2aea132
bbe417eb8f96467662faec0d399276509d347686
1624 F20101204_AACRYW droll_m_Page_142.txt
95e5844e4507665fc1826df802ab2234
a1afc5a001b5320163326680df495c678f642fba
71031 F20101204_AACQVU droll_m_Page_021.jpg
44ff5c96e0d821bb1d993fe9ca47f050
b86d1b6f6db9177bb05667c21969f6aa7ad6b514
92970 F20101204_AACRCD droll_m_Page_025.jp2
ccf2e98028b86f1e1911591e0bf33ae9
a920051dfff04325367555d158161f448fce57c6
1051980 F20101204_AACRBP droll_m_Page_007.jp2
678c0a594b2e4602ad0e1d7187ee74e8
218bfc50bc7e3025cdb1fbef22c73a545eb63f63
73096 F20101204_AACQWJ droll_m_Page_037.jpg
a31cd6fa70d206ecdd432787d838ec72
9b8f535005079ca8a901883262752498891a2c00
23391 F20101204_AACSFG droll_m_Page_021.QC.jpg
ef8fcf457c9f6e8ede8209d1fa87939e
f72b515cf7bba86829fd7eef7198ce722e58668e
22406 F20101204_AACSES droll_m_Page_019.QC.jpg
ffa0a012cab9993f10b56047d08189ee
5bc4ca607bc21757ed9f6c5ed70f444b080cdd31
F20101204_AACRZM droll_m_Page_160.txt
4bb0d0a86cbf44e47f70e336ea5457ef
92b6920876929798560e644856010f9fd612e950
1276 F20101204_AACRYX droll_m_Page_143.txt
8e948d64a81d57c6152f27c1c9f12004
5905b40a8140a55ddac3e111dfc465fdf8425fa4
65703 F20101204_AACQVV droll_m_Page_022.jpg
fed8e149fa54b82845665755facecd85
a9f0d54f79a4fcc4c15176c2d2f1afd5bc538217
109748 F20101204_AACRCE droll_m_Page_026.jp2
0eb2ac6c80292ba92b911800e44ff38b
5f7a122681f0e847cbaa036646c814acec928ca3
828463 F20101204_AACRBQ droll_m_Page_008.jp2
ff01a8d2b9fa1c20bf7cfea91d76a398
f278d4fe82c6c9cc16f5c1b293b00ec929cfeabf
79830 F20101204_AACQWK droll_m_Page_042.jpg
794a11fab7dea0ce52c9863921a9e1ef
320beaf144bf4f9409e3e4e48e82b7cab6780b44
20094 F20101204_AACSFH droll_m_Page_011.QC.jpg
64a60aaf38a474124fe392808ece3ac9
0296f2c32d0a8246b42e6eb1e3b73e55b00d6e96
22995 F20101204_AACSET droll_m_Page_061.QC.jpg
b8c95a2b86803747778a31c64972f7b6
0a6c0e9c86e3e41d12797b47c3e0262c8b3cbb31
1874 F20101204_AACRZN droll_m_Page_161.txt
e1e8ecba7f6904ace4efdc69b3ec53b4
e3a9c2f985473acc583fdad3032c38281f7be3be
1382 F20101204_AACRYY droll_m_Page_145.txt
954d869f10b7f6928397848b133bbbe1
72c4993420b4860f4cda0a78d5f5b6ca9adc938a
71530 F20101204_AACQVW droll_m_Page_023.jpg
a8a39284a0d11607b3626591f56c8649
ae15105af5559a0540915ef8f8495cf3a38f05ec
96099 F20101204_AACRCF droll_m_Page_027.jp2
6bc681e319ceeea5fa18ba22b409a44b
fc87516ef892592bddd42861604cec583c9f1c27
83540 F20101204_AACRBR droll_m_Page_009.jp2
938e97e38885c4aa71e1ab64e0c6dd1f
2af21fd09bdd59f7d60c26a2ebdc5ef17645f2e1
82079 F20101204_AACQWL droll_m_Page_043.jpg
4974bb86cd1af414e82adf759c5c7c52
904c7f5a4a18710db858e7f86a0ef47bb76cc54e
22500 F20101204_AACSFI droll_m_Page_190.QC.jpg
2fea4d081a36ba595ef3986260ac19a0
799ee622d7e553cec4fc1472321b92719acfe3a7
19047 F20101204_AACSEU droll_m_Page_028.QC.jpg
02ba0e3c23c362f3b26507438cf364cc
28eaa9f345bc05c790fe9a501f815eb8d3c07017
F20101204_AACRZO droll_m_Page_163.txt
808fff2c46aa99c27980fd6b70abb8a8
b9830328f92f9c3b7414a06a76c78d98eed1857d
1967 F20101204_AACRYZ droll_m_Page_146.txt
af5f2d17d98c71b050e4df5b6191b2e6
0b6d8fd34d952eaea493ce09508da5c795737ab3
60795 F20101204_AACQVX droll_m_Page_025.jpg
debc3f87f87cabf7984c878dbaf0c969
d5b36c7efe9b7f1b92be963ff11e80e5ad08b2a1
116580 F20101204_AACRCG droll_m_Page_029.jp2
86abb6f3f4dfd789b78f7466abb380b2
a65aa7726b69a04b36c663d7bb77307002d6004e
74840 F20101204_AACQXA droll_m_Page_061.jpg
c27156d5fb945b752f81811973369382
4f4eba1a77b034cc8f002bcf3cdc2a8bf7427cf2
83354 F20101204_AACRBS droll_m_Page_010.jp2
b00c0d4e78a14bc2e375a0017dcc6a39
b1e45b7d76e1b5d6d5290ef128d5ad237ae1bf8f
68303 F20101204_AACQWM droll_m_Page_044.jpg
f0eff1e575a0005909fa0ff33b5b731c
3fa35cf851accf09a87252825864f69fc816ca66
21370 F20101204_AACSFJ droll_m_Page_070.QC.jpg
4b03b2ceff597c21c9347019ccf1ad51
de82ab1248a41d85cf43604f844c2c576ed365f7
20002 F20101204_AACSEV droll_m_Page_120.QC.jpg
c994e52089653ec9951324c319265d84
e591f416b5ee8bb4856d821f80882064b343fa5f
1858 F20101204_AACRZP droll_m_Page_164.txt
d80415ea7223092cb2559a3491bc9b15
fcdcf8cc4c17ca2515a61c72ddaa85b62fb3b93f
72826 F20101204_AACQVY droll_m_Page_026.jpg
c94880131a940d500955c42152e57231
3cf4738a6999b2a8f44b2cf9fb49faa189edd7c6
117715 F20101204_AACRCH droll_m_Page_030.jp2
a9e02f63821fe9bf65790cc265e83cb3
2359176c141ab9f796bd06f7bbdc0f087e70ab9d
75566 F20101204_AACQXB droll_m_Page_062.jpg
f7c454123ad3c3de248b94b8f9329922
7972e34d48d2ae25f3a609c999c19588bf814361
96478 F20101204_AACRBT droll_m_Page_011.jp2
45cc1dbff7632d7cbcdbfa35d52b41c6
39812bb320358452ff38d715c8889898f413a1d2
68236 F20101204_AACQWN droll_m_Page_045.jpg
211279e9b7021a0226f45b6363c8f610
43ede5e6ab08e44cb4556c22767e9863e250218f
14779 F20101204_AACSFK droll_m_Page_145.QC.jpg
20387c7d9559b02d5841fb59167cfdb9
74cee9cbf70f5cf422617ef10b3cf4419ad56c26
22052 F20101204_AACSEW droll_m_Page_163.QC.jpg
2975655de4d24d0565a84026a9a0cbf8
7f13935fdba5436cc3a0af5a19cdd10bc2682184
1947 F20101204_AACRZQ droll_m_Page_165.txt
a315ea10ed749d476527df234d9c9f50
29f9e1946d2341dc39e15c3ca5b76fba3e7ead0f
64527 F20101204_AACQVZ droll_m_Page_027.jpg
c4f5fce4c21cb983742326e9af3c79ca
6d32a146e159e12af4689b656d6f20a92d211f24
112630 F20101204_AACRCI droll_m_Page_031.jp2
eb8afb4354b9b0da95d149fbf2c6f15f
1327521081c9b67ad5741d8d2bed646003a540cc
75163 F20101204_AACQXC droll_m_Page_064.jpg
feaf8d9eb24cb6929ece2735d8347fa7
ae828b17c0d1038151886b48427d0479ab791570
1051984 F20101204_AACRBU droll_m_Page_012.jp2
e8e77fa82165a6473b8d17fbac3c9bb4
44dd122a9c25d45d00962e632ce4b96dc73ab51e
72892 F20101204_AACQWO droll_m_Page_046.jpg
1bd45d9e693da270be16252dc4c38c22
b1f241d9c654c15e23e437052f809af53665b495
18295 F20101204_AACSFL droll_m_Page_130.QC.jpg
81ed3d3a6cdb5fab88ca88bcd1f87baf
fba43470a1a065ed2c5b14ebabdcb26f510baf14
F20101204_AACSEX droll_m_Page_040thm.jpg
115a2f51a4d0720ec8ddb9a5936c3a2b
ea9e12cad61e08bb7399ac8d2fa05d9141d5bef7
1938 F20101204_AACRZR droll_m_Page_167.txt
a1e952197d60addf2c8d9413931fb3a2
947c349d5379b9333c9ee9f0626655421f6b76fd
114475 F20101204_AACRCJ droll_m_Page_036.jp2
528179e1030279ff4bb14097c6416b29
f5d837706254ffc75fb82a29d8e369cb8e9e045e
68789 F20101204_AACQXD droll_m_Page_065.jpg
889e1bf25c59d5fe8baa91917dc69fce
a36b702801e2263c02b0ba25154c37492596c2e5
114589 F20101204_AACRBV droll_m_Page_014.jp2
dd2cd7e0096ee4d8f043967e90714c54
a04bd8de6550aa45ccabb4f70097614c0bffa179
71168 F20101204_AACQWP droll_m_Page_047.jpg
c1b092f14bbb395ea2c75de832bbd39f
118076ecfcc045896772f3563b217c8b4a80ebfa
22780 F20101204_AACSGA droll_m_Page_073.QC.jpg
9ea7e5cf8b56421abef4830cd8a639f4
0738ed4b785e40d5394baa837b79f3376067f88c
23447 F20101204_AACSFM droll_m_Page_016.QC.jpg
8a7066a20e1da6d9e7c2a6a3057f43a3
10e3d420cd3384e6b0772c48052105bd94340db2
5657 F20101204_AACSEY droll_m_Page_084thm.jpg
63c6494d8cb1605542d16c77da58e31d
ca1657e8230e52dceb935a33594667d11d95aa36
2043 F20101204_AACRZS droll_m_Page_168.txt
cb1c16ae00f6d2762431d3ffadaca0ed
ed9e842062ae0511e9c293b26db8427db38556cc
113755 F20101204_AACRCK droll_m_Page_039.jp2
a8a6ebea0e062ef0d4cf86a65a8d0a75
4cded449bd65beabbe983907a65959077b75bee8
72867 F20101204_AACQXE droll_m_Page_066.jpg
7b138dea327334199b9653ef7171a0c4
5df170ef215ef9c157e010384647d10cc9743aa1
102776 F20101204_AACRBW droll_m_Page_015.jp2
b0655cf5ddec1813396caddb25717c8c
5a04214ea6e3ac642c3e94ac5088745518605a88
72765 F20101204_AACQWQ droll_m_Page_048.jpg
584209774de9c0ec874f113b69218004
99e6958ee6f1f6bd4c8e6dcb559f43aa36ba026d
23638 F20101204_AACSGB droll_m_Page_026.QC.jpg
19dc1777b1aa1b162eeab72e686bcd16
40c736c00354a68f7ff08c08a8c430289c023bc0
4667 F20101204_AACSFN droll_m_Page_107thm.jpg
4e1489018d9279ed1b99cce8e5a815a8
b4bb1772333fba2968b29a008406c206d67e355d
6248 F20101204_AACSEZ droll_m_Page_052thm.jpg
c66b263deb5f9b38b577f027aab83add
6c3e30890a94c6de9d18fba4a303a83de26ca01a
101387 F20101204_AACRCL droll_m_Page_040.jp2
03317feb97377eab2e9b8134d433ccfa
bb24558c6cdd5bc91ed33fe74679717d954706e7
75497 F20101204_AACQXF droll_m_Page_067.jpg
93935202046f4e373c0eb17027456cf1
726316cc5ccf330f972eb2e6f6fdd2be342bca51
110308 F20101204_AACRBX droll_m_Page_016.jp2
70ccac69f49d5f77f440b49f20b2bb87
d9046b18dfb725c6e14563ba70151a532df246eb
2914 F20101204_AACSGC droll_m_Page_193thm.jpg
9d9fb242e622c3ee39be710dbbd6f368
2bfed8a58b59a690c132aa288b904992fe8263cb
6506 F20101204_AACSFO droll_m_Page_047thm.jpg
08e0d1ed0b5dc39089a55d45cfb6f3fe
57e52f44fb831fb468369434139d3f5e910d9a87
2211 F20101204_AACRZT droll_m_Page_169.txt
57e587a807bd6210ae1e8c658a049d97
9dd46ca15fce81ec7b9708d071c619eb0a994a7f
101998 F20101204_AACRCM droll_m_Page_041.jp2
48414454358b3bd12ec3aec6a3d4692c
408042ecbdf3812d3c917d34a092c9ad847825bb
71288 F20101204_AACQXG droll_m_Page_068.jpg
8fb8e1188ce2b487999c19f172326bb9
9958ca61f284018b64eed7dcad90c4b20c1cf1a4
108088 F20101204_AACRBY droll_m_Page_019.jp2
19e8b6a098cf5aa53ae173203abf9695
15fe778cf092aefa48091288a9f4aebb9e2148ae
67676 F20101204_AACQWR droll_m_Page_049.jpg
903df82ab0ea75abeb76fdd8eb95e55d
d3dc16f2884ddf50d1c8575c4d9291d1c27d0b07
101164 F20101204_AACRDA droll_m_Page_060.jp2
47036e9ebea26117e372789a3b15bd27
7be8197300023774300410e78598c9b305e364e7
25388 F20101204_AACSGD droll_m_Page_196.QC.jpg
5a483f8aea2a1b5d0ae276247bc13882
be18ce2fceb6f34283a5b532a29780a251520d27
4481 F20101204_AACSFP droll_m_Page_143thm.jpg
0cf3c0f640b2a634b7c7de5bc12a210e
1ccc32b894315057875041850dd4810ddefb6fc5
1973 F20101204_AACRZU droll_m_Page_170.txt
2a6d6b0cd4205ba507679ec1ed20c707
e9acc943b5b2bf34e82c9e3288ad9d4d2edaa31b
120474 F20101204_AACRCN droll_m_Page_042.jp2
2c868585468ea14e433dcf842a1ec0d7
dba586aa8cee485fe11a408aae00f2ced4950059
69688 F20101204_AACQXH droll_m_Page_069.jpg
37618ddaa64d2827f1beb28d5837c1dc
92e6bca7ccdf49dd7d919f1fb9c10483cbce28c2
115649 F20101204_AACRBZ droll_m_Page_020.jp2
54b67608658dec49bc8b4207e45a6938
1888429933d16b7e387bcfc86fe6a20e9502b036
81054 F20101204_AACQWS droll_m_Page_050.jpg
adbe16d11dde81203b235add7aa10665
e22ae350070f294a99570f652f0b836a073419ca
112762 F20101204_AACRDB droll_m_Page_061.jp2
dd62337cad570c41c0634cb21a2bd216
1f18b2bc59424417344326420e5c61f5862f4cbc
24182 F20101204_AACSGE droll_m_Page_055.QC.jpg
163e9af578ae54215bffc318954cdf60
8cfa2401254e4309e9924b46a73a7eb35a6ddc44
21999 F20101204_AACSFQ droll_m_Page_065.QC.jpg
48249a56bd7d43dbe54ecebecafd1923
7373c1bce3b281dd71f13e7b99ac354e38a8572d
2452 F20101204_AACRZV droll_m_Page_171.txt
553a5b94cb9a3d43cc93821f5197129c
fae6b2b9dc7ec1deed0201d76adb19f40fe6b05c
103649 F20101204_AACRCO droll_m_Page_044.jp2
b71f8f40bd79515e2edba1469751cbbe
4964e553ef2c6f88e2e94fecad2388b0c5fd81f2
71825 F20101204_AACQXI droll_m_Page_071.jpg
c99321dbf4edabc4f2e04c78b60161d7
d039afc5dcfe74f53d007cb9efe68bc703df9db5
72645 F20101204_AACQWT droll_m_Page_051.jpg
2b0514e7ff09538d8fd92c2c952cf552
102bbd549be338d129923df497f768c379c5a147
1051956 F20101204_AACRDC droll_m_Page_062.jp2
35f074c19c1a15ab9d0c84e2f4b17abf
277b7d69e9e8815587be3fb06be3fcf53fd1c7c2
21836 F20101204_AACSGF droll_m_Page_133.QC.jpg
fae10ce1c90822e4be1f0c1b245da8a0
74d2bab21f1664824c6a5e3e6eeca4d9f885eaf2
6118 F20101204_AACSFR droll_m_Page_037thm.jpg
4462285d465c38b4d3044351a0ee52c5
a68d55e3af653a36916556ed90f317aaedf26e95
955 F20101204_AACRZW droll_m_Page_173.txt
69022d8c0314edf108f372783283b53d
536713f20a352442a5d14bb50d595afb58048ffd
106308 F20101204_AACRCP droll_m_Page_045.jp2
f14cce93a97d600bbc40028747b9f052
368edb5c8197b10876b64c00c7e9137ac4c44508
67261 F20101204_AACQXJ droll_m_Page_072.jpg
435976609e6840d0deca91f879704a92
a8bc009c1d88ac0017af6bb00b3bc6b5f704f45a
74285 F20101204_AACQWU droll_m_Page_052.jpg
5a3350e5e15f504a1461aa5942e556ea
e916d561f9a1849df184a8c2c65bb43ef346a5de
104345 F20101204_AACRDD droll_m_Page_065.jp2
0fce0243b129805fe94783165b673124
142e864fb045c1359c3fe6f344d94a63ed7eff23
24863 F20101204_AACSGG droll_m_Page_091.QC.jpg
35ce3bdba661c36b2f9b869f6403117c
f98a4c6dac3336b807dd76d180eec86a3e9f3878
6497 F20101204_AACSFS droll_m_Page_019thm.jpg
f0017e062cd341e3eabb9d3793f1da14
45cb4e7c6d0def8720b8cbea8e9866f598625fd4
1669 F20101204_AACRZX droll_m_Page_174.txt
1054d62b7bbcc32136a54baa06126457
198c783f8edc41dc11a99c90fd1caced60b0a8fc
111532 F20101204_AACRCQ droll_m_Page_046.jp2
d75b3d6fc822b2ac4481c9579a3e4e74
c51e4744974685be0cff15197eeb184a2d53753c
70190 F20101204_AACQXK droll_m_Page_073.jpg
31b870face7b984f1d6d187651082648
dd537f98817ce06868111c8ca19eb5c64b5bdaf1
84058 F20101204_AACQWV droll_m_Page_055.jpg
cd46e306b91721351465f861a3990ad5
878fb055cf857cb8c1b486414e4137edb852cc36
110814 F20101204_AACRDE droll_m_Page_066.jp2
661710c8c49a44932f8ee7d6e6539e47
d291d2fc72ccd6d1ebf54c5ededd82f787e3e400
F20101204_AACSGH droll_m_Page_036.QC.jpg
e26da40ac9a3e9115e3bcd2c813e6b21
9b5a05c97416c1639343bb6e68c57c183032c54f
5311 F20101204_AACSFT droll_m_Page_144thm.jpg
7225eeaaa3e9475ef20a544f99e3cece
1b09ee425548d726fbae9365c9097f7445cea292
2420 F20101204_AACRZY droll_m_Page_175.txt
c085ac7435c6e14fa7a6c4746743c730
78a02d381b0dd0810159c931c4710f0cc4c4e2ae
108401 F20101204_AACRCR droll_m_Page_047.jp2
ff1e3e37babe8f08d58694043390c2c6
3d83c568f421e5e4d2ce8a16728a4217cafd618d
72164 F20101204_AACQXL droll_m_Page_074.jpg
f4de4e13dfc8a5a25a4f5e2a9729cb1e
24b7e4419415065ca21fe59eea84d0eeb947f767
73574 F20101204_AACQWW droll_m_Page_056.jpg
c50a434f6ccc7cdd7ba524f78b3ed82d
7950aeb41d6ef4992f082cd5961888a51dbd6ddf
107346 F20101204_AACRDF droll_m_Page_068.jp2
b19e614a7e62040c360e2c905c4ef8f0
801512d2ca0ad3b18d9ab122e848baa23f1a305f
24508 F20101204_AACSGI droll_m_Page_042.QC.jpg
b1e418ca4b3f0595ff3f6316e9713427
a992b2646ef35439cd91629d55fb3f9892520d2d
F20101204_AACSFU droll_m_Page_177thm.jpg
2c65767bc36f158a63f8826fe807b0c4
6955572418eefa132a606ad5bf357215fe73439c
1923 F20101204_AACRZZ droll_m_Page_176.txt
8ad6af4dc91c7dad600ba34bf85fb832
8c1daa66bcbacf115f12b4130020c69f131abee9
75660 F20101204_AACQYA droll_m_Page_093.jpg
07035a63bc588e3e8c57457d1ae33b9c
4c13cca2819a9f98ae5b399f124aed92cca069fb
111605 F20101204_AACRCS droll_m_Page_048.jp2
948cfb57459a3e71cd6e597365da2c6f
5c05bddedb81f58a7724d3e931abb19ad896804a
68444 F20101204_AACQXM droll_m_Page_075.jpg
bc1a0802877d677f43732e887b50018e
24e8bd6ebbb6ac1cd6233256acc56c65d1b49df4
62992 F20101204_AACQWX droll_m_Page_058.jpg
2078a06ec3cb46a502fe73e40c4052cd
8cb92ff0a68d2e557bf92fe47f00e97b7686b52b
105950 F20101204_AACRDG droll_m_Page_069.jp2
252cbfe47820e47bddaab2aecb0e4151
b88fc1972f048a4523903d5de670fa50ed2344e8
4639 F20101204_AACSGJ droll_m_Page_115thm.jpg
d4b592e62b1e63aa02bb2cd81b8e809b
1b553e93f82ba45a7a6b695b756954df0c3f4260
6265 F20101204_AACSFV droll_m_Page_018thm.jpg
ee098513ce2a182fd98fb78f1db905d5
b89bc41e6cb32b8baba25193bfe60bfd658a85a2
73344 F20101204_AACQYB droll_m_Page_094.jpg
9b62fb92933392aae238a61e8bdc619f
a19b250d0ef5beabfcd23dd668b078b93d368e74
124796 F20101204_AACRCT droll_m_Page_050.jp2
eddc5cee557196a78026674902fa2cd1
937f0f854c1d9f01c5cc6445fe8d61ff3771a5ed
72998 F20101204_AACQXN droll_m_Page_076.jpg
5135c033981a041d7e56334298586f4e
d2a768d53f95d23f1dc54d1bc2a48fb030af723c
68687 F20101204_AACQWY droll_m_Page_059.jpg
05158b62313a3248a229bbb93fe51d3d
7546c08ebbeb40abc32e155105e67b3505e2a105
101536 F20101204_AACRDH droll_m_Page_070.jp2
3c845283f27c53e68134e74cc8dc0e93
51b2664deed87c0cc4399cdddd206d4bb9fe499e
22728 F20101204_AACSGK droll_m_Page_075.QC.jpg
041f18c22f06812bff0c6557dbe831ad
265674e580ba5d8a2fe8de419c8860e2f4fb3495
23948 F20101204_AACSFW droll_m_Page_030.QC.jpg
8b9eb5258c8a7644794dc6513ee88d7e
bd4fe6b22e87f803c71afa3f92254f8237f55286
75302 F20101204_AACQYC droll_m_Page_095.jpg
080955cc2c2fef1353ecaeaae34801f4
3eb5748e3dd236da69e0d92e76ccc17db42ad775
110354 F20101204_AACRCU droll_m_Page_051.jp2
9f49ebea45d5704e41a60dbc318446e4
9ba40c4c13e7f40638fdee11cdf6394897c39197
74918 F20101204_AACQXO droll_m_Page_077.jpg
75b67bd12dbfcda5baa73c30fce51410
712ec62cbb226799841244faf0a5637e6c84b1b1
66868 F20101204_AACQWZ droll_m_Page_060.jpg
32f05f49db428b85ab4f7d3c2dc8450a
2e959c8fa26b3a13865ad82a9ead08e568d11a99
102059 F20101204_AACRDI droll_m_Page_072.jp2
6756887235fdaf5344ac33b6befc0eb7
859bf3ee20f6b5995f2518702e552ed2c286e900
22861 F20101204_AACSGL droll_m_Page_045.QC.jpg
cfe686fbc6028b9dd0a562a18ea6b251
a9f90ecf188e6142c8c98ea1c9dd43c9b759b8d0
5290 F20101204_AACSFX droll_m_Page_009thm.jpg
1a6e49e27c501abc3476d8576255a87a
8a2fccce117f1f65ace99a026b77a9346cb04892
74179 F20101204_AACQYD droll_m_Page_096.jpg
4dea95fb1e7b3463aea67a984b240b0f
26aed8a83eb48cef70472e82be828193d55fdcb6
111818 F20101204_AACRCV droll_m_Page_052.jp2
4445148440caabe2fe105d7104bd5675
98cee65ef7205066d49c8eb6e5560847de76e8b5
71547 F20101204_AACQXP droll_m_Page_078.jpg
29d389504fd0d5226fcbd61de54035fd
3c01aa690918f1c2893b8d37eabcdd04fd604007
109499 F20101204_AACRDJ droll_m_Page_074.jp2
0c76a95f5ac9321779e67cd59271d6e1
115ddcf22b6c5f8adeac8b58b4409688c6f63927
6228 F20101204_AACSHA droll_m_Page_054thm.jpg
e2090d3247d29d58e3a0526323adab11
be2a2523e1350585ac17b3a47acaff44d0b97959
14940 F20101204_AACSGM droll_m_Page_107.QC.jpg
1a7028e177b38b8dfe37e1ca66d8d6a7
1584225f5d99d0a2599a5bd134015182b6f2a72a
6393 F20101204_AACSFY droll_m_Page_053thm.jpg
aa18c554f30e8579eb2be3b9afea1e9d
f1c57db630df409df28f68e6258753b233aa461a
51495 F20101204_AACQYE droll_m_Page_099.jpg
f803ec410418f49984b147b999453890
747fc16347fc1a18d7e2c5dc93195bd560c0017e
111227 F20101204_AACRCW droll_m_Page_053.jp2
88b399542057ee1f2c8b8cd2f09a250f
3daf10d633db323c8fab9d24fdeb8364d4eafdea
72126 F20101204_AACQXQ droll_m_Page_080.jpg
5e99bc7bd86302a2da723f2a1b0cf9b9
78db57d5191e91f3ee14f2aead46d1a8bb20e268
105635 F20101204_AACRDK droll_m_Page_075.jp2
aed10ff71acfde6893107f3cc213ae49
b54b1fab4fad40943b36301b8b3d18e4443b903f
18400 F20101204_AACSHB droll_m_Page_137.QC.jpg
efc1ab0d0f3784f421d15f44cd2a989f
489327f57765e566bf86ffd4b754f53820a605c4
6569 F20101204_AACSGN droll_m_Page_175thm.jpg
49e58e27b4df1d233fd6949f2f33e895
630cd203fa88f9f2044280db0bfbe9c3440fd7c2
5409 F20101204_AACSFZ droll_m_Page_137thm.jpg
25114e9feb200e64ad70b31f6fbc7b74
c439712073fcf062e3ff5fff405c3bc466a63a96
52311 F20101204_AACQYF droll_m_Page_100.jpg
f1dbef8ada950b7b54a1340b248feaa8
ebe8e92559dc06860a04ad5773aca77a19c8ff46
111324 F20101204_AACRCX droll_m_Page_056.jp2
04d049236800dd6133e3742c290b2b57
4618699857ea80f138938ea3f50be42eb93918ac
79528 F20101204_AACQXR droll_m_Page_081.jpg
bdc3a1e64693b9d7f3a644a152875b51
5fe905ec59c652d740e6e9d0e678ce51ee8531bf
109738 F20101204_AACRDL droll_m_Page_076.jp2
2036568ac5b6b52030216b14f95e0372
ddea4d0b2c17a151b20de8d489bf44261eb50601
6658 F20101204_AACSHC droll_m_Page_064thm.jpg
353a44e50a48afe8c8b374aff835f741
562159e76e0c1ea98e7d4c705df0aeac1f724147
6677 F20101204_AACSGO droll_m_Page_076thm.jpg
7270ae5d92a73603c92c0a95ae093f08
7ceaada23d030e501346ff83f50fe3c08fdb38ec
54366 F20101204_AACQYG droll_m_Page_101.jpg
de08199d454529ac4d651eed1f355f7d
63c47276ad6eddb7ac7f4bdf1c633afa82e6c0b5
92802 F20101204_AACRCY droll_m_Page_058.jp2
416b38f3522fbf20d8b24e298e825641
4a89887d21eca8a5d8dc2d40a28c9e9f9e9e9ee3
110452 F20101204_AACREA droll_m_Page_094.jp2
f0d313857ecef692a2fd4f65de87cac8
bc47876d70fc97b7405dd58af91ae779cf244aaf
1050073 F20101204_AACRDM droll_m_Page_077.jp2
d2d10addc585e3a091bcbc832a144288
8918e201b967dc62ede117065d038acb858ec38e
23527 F20101204_AACSHD droll_m_Page_005.QC.jpg
50f2642f40987a7d21bb0efa7d537852
6ef30daedfb60e183903464ce57cea13ab05a72b
6469 F20101204_AACSGP droll_m_Page_026thm.jpg
7d60a134662a61022e407f501b5b3446
a87f5139ef303f1b9ae473ada1f9217b8e9e02cd
58873 F20101204_AACQYH droll_m_Page_102.jpg
86c9877376dcaf1639b25cbdbe736ae7
f40221e5132ab7f0ae5ef7d5d6abd00e54374c6c
104571 F20101204_AACRCZ droll_m_Page_059.jp2
5ea30ff729746d4b339b503f5e115174
763328c3646b61923407af94ae3c7d1dbb2f5a35
48458 F20101204_AACQXS droll_m_Page_083.jpg
277a89959124a887c1390efa044d20e9
a0e83289506ea287588529f0395b2d583b026727
115430 F20101204_AACREB droll_m_Page_095.jp2
22591d3f499ae63f7aaccc633355d663
6118ddd171da03691b25f47e43f7260bc4d6cfff
108269 F20101204_AACRDN droll_m_Page_078.jp2
c00cdb6127c6d527d9bf3819373cf32f
cc22ab99ea97a7e7d72ee41e22cec4e488753134
6908 F20101204_AACSHE droll_m_Page_171thm.jpg
fa59b8c03da4a4c143eb83d6363907d8
3bd87ced703f1ecaa457be7951d91fcef654445a
23882 F20101204_AACSGQ droll_m_Page_046.QC.jpg
33899adbaf28ae22b7c4197cc0ad0824
b49df4fb098f59869984bfe0eed44021bb144a6a
59918 F20101204_AACQYI droll_m_Page_103.jpg
456494fe780e923df2f03ab40e1b464f
e989cb0baaafe05afc05e47677a07d3c8996abdc
63520 F20101204_AACQXT droll_m_Page_084.jpg
304e51ad4a3a6be77c261fb46f4fee5f
3628a6511e6cc39b208477f24158a370f49806db
107330 F20101204_AACREC droll_m_Page_096.jp2
0c14ef44f4db7c0922b9e6993192f67c
9fbe315d2a67ee59039dd2dc2889af9e668dff23
113899 F20101204_AACRDO droll_m_Page_079.jp2
d4a92b6ff80fa9eafb014dbe4e85b100
3ede74449e5dc929959dc50b7c9bcebe08b0d83a
F20101204_AACSHF droll_m_Page_071thm.jpg
ca5e3fb04811ad234d4de904ee2b81b6
d1afe9b81f1977423ffbde7e7c56d19af27443a1
6529 F20101204_AACSGR droll_m_Page_024thm.jpg
b7768cfa2aaab41784df77e047d1abad
9a9285d83eab08f18bf930bb92b6b833a96ea354
53881 F20101204_AACQYJ droll_m_Page_105.jpg
88ddf5983ab4af36e6d2a1daf9986882
af1ef8320cbc1e51be08167e49ebe9b29e0b3374
69307 F20101204_AACQXU droll_m_Page_086.jpg
877b9237462e7d5d0236b38a2314892c
2129c7a89cd00b1a5bf3965e246720870c16a474
113884 F20101204_AACRED droll_m_Page_097.jp2
c58fe183f47e2ece08654a74a03392f3
c2de0ba7bdfca140556f42a84e2b8701e21fa78c
110942 F20101204_AACRDP droll_m_Page_080.jp2
91a996958c5fd5ec60a78a6f824cd2b3
2d98adee04d016266aefd7b06b7b144dbd7dc348
6635 F20101204_AACSHG droll_m_Page_197thm.jpg
e9ac195b2544bb1f71bd4fbc73578877
cef8dc0f8a28ee72451067289b54fd3d7492d5aa
22193 F20101204_AACSGS droll_m_Page_155.QC.jpg
ae2cc6f98bbe449371232bfb0488ea66
00fbd194a0aabd346d68cb42542b4cda04143702
55646 F20101204_AACQYK droll_m_Page_106.jpg
fbb58d10bb6fa7630be2e0505b9f3156
83681c4c73d3c09a53626de8c37735b937f2abcf
71392 F20101204_AACQXV droll_m_Page_088.jpg
38a1bda67ccfe1924f1465ff0cc50823
b0b02ff6776dd63f788a280090dcebb08a7f07bf
86116 F20101204_AACREE droll_m_Page_098.jp2
72910eb22fd051ed5b6fd105e0842064
821d0d79c532d11a02ec96f402d1d378baa19b7b
1051930 F20101204_AACRDQ droll_m_Page_081.jp2
4989736bbca706ace3cc35df2f2b2ebe
a851c4e7f739d7d587e5fc4414a8fc03dc4672de
6694 F20101204_AACSHH droll_m_Page_034thm.jpg
ff3cc02501fa112232625849876b8fcf
45b46687ab6a86283da17697a649f10c17f9478e
16754 F20101204_AACSGT droll_m_Page_098.QC.jpg
2b8a314d28e7b0ccee9bf14e3d93a9f5
727eb98435e46f85fbe52528c3fd68d0b2a510ee
48694 F20101204_AACQYL droll_m_Page_107.jpg
b4bb2bd095f99e72f3e9f82b0c7010ba
0844fdc9b6f015ea0ce50db27c46b5214ca0db35
70668 F20101204_AACQXW droll_m_Page_089.jpg
82c57de97aad25ec189d68f94185c649
0ad16cec7780acd1e68b86ec76a4f63577932857
78032 F20101204_AACREF droll_m_Page_099.jp2
39401824d6f52b081f058fd90b8c1c41
6430719e1e05905c37ffcab11519c94080108134
112536 F20101204_AACRDR droll_m_Page_082.jp2
22a8bfe992be945fccb6413e92a61ccb
2f202993a16d2e5e5d7c680cc2431582406b54ce
6814 F20101204_AACSHI droll_m_Page_093thm.jpg
73563da8ebd8b0eae83d56e0d1c7c93c
d4aaefcce69ea9a4efcd853c1c9bc0632b56bbb5
6160 F20101204_AACSGU droll_m_Page_159thm.jpg
eac31a8e559423df540dcfe297008dda
840738ab9d1c1f6654e13c8df6268d693badd9bf
58052 F20101204_AACQYM droll_m_Page_108.jpg
6a7923c9f822cd3a5a28c9c4116e1668
2cebe4869f002c8212d97e5461adfd8d3761537b
76669 F20101204_AACQXX droll_m_Page_090.jpg
72b4e43b144144fb9308c1f0149e4dad
63d9a391de6adc901d5ba051c6c4dcd85ad479fd
80425 F20101204_AACREG droll_m_Page_100.jp2
52c67213447718f24f46eae1d7f6c7d0
890c564495dc13308f03554bec915fcd51b3eadc
63458 F20101204_AACQZA droll_m_Page_125.jpg
152ea34f8af676ddaa98b6b28f8f6735
eaf568c0960edf05e1082b7935cfce71367460cb
70291 F20101204_AACRDS droll_m_Page_083.jp2
bed37d59fee9cfc643f0353d6e7c08e9
53eee3c7f11132af773ac2304e39c7c70cc4389c
17907 F20101204_AACSHJ droll_m_Page_144.QC.jpg
ba2ea7167d594587291e1e1ecb7f149a
fcf0baba1a98e0af2d1d87f48da835fd9d8c1bff
5578 F20101204_AACSGV droll_m_Page_124thm.jpg
c1fadcf6bcf4a109d0ddcec8aa76e543
bd188fdb8f742f3feee5da93e56a4fa8584f8785
54359 F20101204_AACQYN droll_m_Page_109.jpg
2ccc86aa8ec8cd7e22a8cb8eb1d2064b
e5762d6241c59a99fb336a412089eb681c32016e
74872 F20101204_AACQXY droll_m_Page_091.jpg
4088aabecc45a7fbe4a6e9b10c2194b7
b7386bbc304b9fb0948c86c142895a17cbf76b76
95085 F20101204_AACREH droll_m_Page_103.jp2
f41c27fe2a20be196e14902be20b7c89
cba04b2b991750a4dff4e28f1ce4a77a7e3fe8e3
64043 F20101204_AACQZB droll_m_Page_126.jpg
a5b2a1f238a667597d30ec76b59e4da4
716eefb48cc844162185c6b7d826a1a05c96b5ca
103070 F20101204_AACRDT droll_m_Page_085.jp2
b994890665c27469df6e61dd0da8f3dc
a6cc73676c6dbfea8719651ef3f168f53da89e6b
24056 F20101204_AACSHK droll_m_Page_048.QC.jpg
9fb604c5903aadf4a9b2e087d3d8cc7c
a459509fa6e38a058c54b990bbcb7b37c4f87a66
20103 F20101204_AACSGW droll_m_Page_058.QC.jpg
db498c6fade6cfbcd1469cfa95568a92
30b51e3939cb5bfc63cbe2c20cd0500c12efca2e
52184 F20101204_AACQYO droll_m_Page_112.jpg
580ac1583feac52d51547d99840ebc73
6a327266eb1698cb3f0b1c3e21d14e24753c9a37
71454 F20101204_AACQXZ droll_m_Page_092.jpg
b4f7655930e46295ee6af6d2426675e5
fe502cd4da657fd0b23d952c91e1c2e7fb870b1b
82841 F20101204_AACREI droll_m_Page_104.jp2
67e5e3037e5b01d9a8ca8b0ef36566a4
6498c0ed6984a3c59f032ee540224cb0c0cd334c
60060 F20101204_AACQZC droll_m_Page_127.jpg
e688606d451c57eed71a6604e13c9e8e
9deb5170df30d7507cf238dd800333ca741fa1c2
105287 F20101204_AACRDU droll_m_Page_086.jp2
5308ae6d703a67775ad048eb865b8d40
688b40bba48c8ed79271deec469f451e7c55cdb2
21387 F20101204_AACSIA droll_m_Page_072.QC.jpg
f7c577e903744445b0991f2b1cde97c8
2cf43fbf2fd093ad6de37e964e5d2e77f841e603
5156 F20101204_AACSHL droll_m_Page_104thm.jpg
f4c24dada1bc3a0ade5846d9449ad3bc
3392e7765cc1dfd3547560489acdfc8a60761fe6
22133 F20101204_AACSGX droll_m_Page_180.QC.jpg
2f53d60ef661bbe06003d5b1452a6958
3b1d5508c24489f01652db8a896435574f3d8ef1
48023 F20101204_AACQYP droll_m_Page_113.jpg
88f7ac7dacfd4b070927daee2c20705c
08e5b62e6601765760bbb96aede8454068e18238
80740 F20101204_AACREJ droll_m_Page_105.jp2
d8439c54871ea04158959d828e208737
4d290e7fabd393537bda8dd54e1ce4c9e3e1e406
69003 F20101204_AACQZD droll_m_Page_131.jpg
29072de9feaf07ba6bab420d30df8723
211493d76e358fe3225366ac9f81d03b8881f3c0
110276 F20101204_AACRDV droll_m_Page_088.jp2
9b02a0c3c82d49c05ea4755920bce341
1ee9237d2790592c00c086c56e3416a4aec90881
22975 F20101204_AACSHM droll_m_Page_032.QC.jpg
e8b0703aa78ff05f89d1de4225dbafeb
8e798db38359fe5980884271a78cbaf0bdde5e72
6318 F20101204_AACSGY droll_m_Page_032thm.jpg
134ffd27f5c0e6d37066e7121bdcd607
8fa395aece6617d927d2902c5e527c3d2be06a14
54632 F20101204_AACQYQ droll_m_Page_114.jpg
876028838f77b3930db8b900e88febd4
2309e58816a31a7ad24c56f429c57f0cd7a9b66b
85947 F20101204_AACREK droll_m_Page_106.jp2
843f8d98755ff073142d9aa8834b4032
fb66e046f7816fb2357b1f6ca619ae3ed703c575
70489 F20101204_AACQZE droll_m_Page_132.jpg
c3afd6db58960b8ad9da9b097901df07
fb94814959f63d9af97776140fd6c1a1aeec178e
116131 F20101204_AACRDW droll_m_Page_090.jp2
8a2a87e7671d1d53e7894b3a9943e70b
88018b73c8c488bb0fe42ce565dc5c6262d1f1bd
F20101204_AACSIB droll_m_Page_023thm.jpg
f58d9504955baaf9919c55f3cfb55c9f
f0baac25955195062a521c3ea5cedaee67137009
21473 F20101204_AACSHN droll_m_Page_060.QC.jpg
fc3df7ee5feb359d7a65927847ca7d01
0b0d24e394b8e14641abe44f370fddab3be8d9a4
21227 F20101204_AACSGZ droll_m_Page_015.QC.jpg
6eff9604d0ed0a234d031dd0b4295c04
566ecc467959541453880c79d6407c1b93dad549
49557 F20101204_AACQYR droll_m_Page_115.jpg
fd2dc38ee8e4561c881c116831553efc
ca746b401d45705a20dd356f81fba843cc21320e
74370 F20101204_AACREL droll_m_Page_107.jp2
dfb084debb52cc033e6dabb40b509c97
10fae8b7a25b427bc247311c5e616a79c68c8481
69772 F20101204_AACQZF droll_m_Page_133.jpg
4a2b341a692971e1930520fbc624f34e
35b27171cfb45ba3ef273ee8676b7ae72db8b671
113576 F20101204_AACRDX droll_m_Page_091.jp2
4f7a93ece3c4b198b7155279b1b2ba2c
57ade377a55d5130e3b17ed6785f59eeb5ccd445
24172 F20101204_AACSIC droll_m_Page_079.QC.jpg
c174270c55fa3f6993d62d10922bab8b
4ac0bb2cb0b46c5b36754ca9bea6cb54fe63b256
15402 F20101204_AACSHO droll_m_Page_142.QC.jpg
4c4f02d16e72283d53f71b858d4aa4e9
28a74360597fe814e41519edb1ddd6538215cc34
55816 F20101204_AACQYS droll_m_Page_116.jpg
af907e6364954ed668e98775c8937085
548630b6bb029b8253a7b3d395a69118ea8a72d4
110769 F20101204_AACRFA droll_m_Page_123.jp2
f7b94693be7466fd330ef46584200687
2e27a974714b061f20028999f6a901ba21741bb2
88618 F20101204_AACREM droll_m_Page_108.jp2
9ab78912afc2176100021ad8fb8d78dc
a9c41e109664304fbaec6f1260c0fe4527db4800
50765 F20101204_AACQZG droll_m_Page_135.jpg
3ddb03cc63f689eebc271e67b4d121eb
f4ea27ff91d9889ad689f308a724253fbf4f0f7c
107242 F20101204_AACRDY droll_m_Page_092.jp2
3e3a4ba68350368276cd2b56b07adbbc
5984a4853bcfe1b55051f427ffa09cbc6723a795
4632 F20101204_AACSID droll_m_Page_117thm.jpg
cc27b91f2394d37c0f86941e8f7f473a
340702a34017592b10f958c870f9805a8ebecedf
6724 F20101204_AACSHP droll_m_Page_036thm.jpg
a21dc0e94b7356db706007f745d45a19
10c6d876028412b89016c7ab384f00e6fcad0865
88825 F20101204_AACRFB droll_m_Page_124.jp2
f042a2d65b93aa99fb1be17c9e3e9931
170c7a04990ddc8c95220a14285f91f8d2bb0c04
83061 F20101204_AACREN droll_m_Page_109.jp2
d498335676ffe8094cbd16ae747f07f8
7cfd32dd146aa3e5f565a5f5df4ad974f667e84b
62271 F20101204_AACQZH droll_m_Page_136.jpg
7168fc2cdfa46849f64ea0c95c3acc25
278d78189d08f4d0c8cc3e9ede656e185cd26322
116128 F20101204_AACRDZ droll_m_Page_093.jp2
a4f8fca0d352cb7617c35c4de59c1a91
98eb5a8b02a5411b3b3c7cd3de63ee2a799d221f
19206 F20101204_AACSIE droll_m_Page_151.QC.jpg
d5d97d45d7bcfd381c3719de81800061
91b7f77165b3a87d8156d0f6d774de77d4cb3e13
22358 F20101204_AACSHQ droll_m_Page_183.QC.jpg
48fdda08474e44392f8d986d9ad6f22c
0e7f8527c0afaa79b5c6ef7b7c03f08c13b16076
94683 F20101204_AACRFC droll_m_Page_126.jp2
d2451a50bb4664c380021e7aa441ade3
67b0c2143c607edfcaa9756483483ef861d3b3e1
78509 F20101204_AACREO droll_m_Page_110.jp2
795df33cb962b22138627ab13835ac04
c41a34eaa9e7874857bc56ee43f256599911625d
58964 F20101204_AACQZI droll_m_Page_137.jpg
c8a3f03b8ab877c3c0cfa989c27f5a88
aff7d06bf48064294dd26273e0b0eb6b632863af
50260 F20101204_AACQYT droll_m_Page_117.jpg
50c9e1881a659d9ad50ed8a64f13758d
5e0683df747c64fb29b6888820da1c921f96c7f1
20667 F20101204_AACSIF droll_m_Page_084.QC.jpg
54e90e591c2633338ad5fc138c6ba3de
072e6b49d1068ede6c8b7fde88a206da166bbc3d
5695 F20101204_AACSHR droll_m_Page_128thm.jpg
d7905bb944f49409440af1c460bc8b48
50ef1c7b4ff66476df1fa0624cfa6a211956fc0d
90732 F20101204_AACRFD droll_m_Page_127.jp2
446fba58fda2471e703e7abc3c4b0602
30eb526c7226156ba7c782648e3fdf22dbaf848e
91020 F20101204_AACREP droll_m_Page_111.jp2
bf9d0876710ca3a614775c703b747add
80284c8d6aee161471a064e1c2fa7628761431f6
59513 F20101204_AACQZJ droll_m_Page_138.jpg
c08b7a825ba668f9e37705e90ff8a85d
f302dea6d2f6985cf6b2da9739c1fc0b46233149
F20101204_AACQYU droll_m_Page_118.jpg
15ac5e0ea914ea431f7f3202ff231f99
f629a6720ce7bb66dc89b8c1444a111916e735ae
6783 F20101204_AACSIG droll_m_Page_123thm.jpg
9dd2ec25a167f179c2ec9ff6c28f9421
3f1be55b4c429d6d504fa5797b37736c4fc81170
5484 F20101204_AACSHS droll_m_Page_119thm.jpg
9ef71968b64cf70e045be0a701454f91
fde82743eeb1e16e5db41ba29fb3540bf92d3916
89526 F20101204_AACRFE droll_m_Page_128.jp2
57fe2c22383a7ef7738734f1c2d9283b
4506596f0f871046a048871ffe84463c2a2e5328
77802 F20101204_AACREQ droll_m_Page_112.jp2
7f8a2c6f227dcfda85de3293944457aa
70c7d8a045fec6e530641d59fbaf760c06802714
57624 F20101204_AACQZK droll_m_Page_139.jpg
3f7858714add7e5a208ee5eec8e0de80
674cc0456575442383025296beb0c1eebaa89f22
60889 F20101204_AACQYV droll_m_Page_119.jpg
830d307419f042d08d52bec2f48b237c
3cf7a4badef2af90e2d22ac145c50f331f7d0ec2


Permanent Link: http://ufdc.ufl.edu/UFE0010111/00001

Material Information

Title: Use of the Delphi technique to derive a common definition for work-related education
Physical Description: x, 189 p.
Language: English
Creator: Droll, Michael Lee ( Dissertant )
Honeyman, David S. ( Thesis advisor )
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2005
Copyright Date: 2005

Subjects

Subjects / Keywords: Educational Leadership, Policy and Foundations thesis, Ph.D   ( local )
Dissertations, Academic -- UF -- Educational Leadership, Policy and Foundations   ( local )

Notes

Abstract: This study was designed to (a) test if work-related education conformed to O'Banion's six principles for the learning college and (b) if the principles could be supplemented with other components of work-related education. The central point was to further the knowledge of these relationships, which in total could be modeled to derive a common definition for work-related education at community colleges. A three-round Delphi technique was conducted to seek levels of agreement and consensus on O'Banion's principles and seven components associated with work-related education. The Round 1 sample (n=20) attrited by Round 3 (n=15) to six CEOs and nine administrators from community colleges whose CEOs were represented on the League for Innovation's board of directors. Content validity was essentially built-in by the development of the content of the scale matching the content domain, as conveyed by the participants' responses and what they considered to be the constructs of interest. Internal validity claims were met by following the established procedures for the Delphi technique to answer inferential questions about the scores and to develop well-founded conclusions from the data. A Duncan's multiple-range test confirmed significance between Round 1 and Round 3, which combined with the research procedures and study attributes, validated reaching a superior group view. Review of the quantitative and qualitative data over the three rounds revealed levels of agreement and consensus on the principles and the components. Correlation coefficients were organized to facilitate comparisons between the principles and the components for significance. Significant correlations were found between the principles and six of the seven components. There was specific commonality found among the funding component and the coordination and planning component across the majority of the principles. The correlations were modeled to derive a common definition for work-related education. Results of this study suggest that such a prototype model and further research could facilitate a consolidated position and common definition for work-related education. The study's conclusions have implications for why community college leaders and policymakers should pursue a common definition for work-related education within a national context to uphold the community colleges' role as "preparers of the nation's workforce."
Subject: career, college, common, community, coordination, definition, Delphi, education, finance, funding, leadership, occupational, planning, policymaking, related, vocational, work
General Note: Title from title page of source document.
General Note: Document formatted into pages; contains 199 pages.
General Note: Includes vita.
Thesis: Thesis (Ph.D.)--University of Florida, 2005.
Bibliography: Includes bibliographical references.
General Note: Text (Electronic thesis) in PDF format.

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0010111:00001

Permanent Link: http://ufdc.ufl.edu/UFE0010111/00001

Material Information

Title: Use of the Delphi technique to derive a common definition for work-related education
Physical Description: x, 189 p.
Language: English
Creator: Droll, Michael Lee ( Dissertant )
Honeyman, David S. ( Thesis advisor )
Publisher: University of Florida
Place of Publication: Gainesville, Fla.
Publication Date: 2005
Copyright Date: 2005

Subjects

Subjects / Keywords: Educational Leadership, Policy and Foundations thesis, Ph.D   ( local )
Dissertations, Academic -- UF -- Educational Leadership, Policy and Foundations   ( local )

Notes

Abstract: This study was designed to (a) test if work-related education conformed to O'Banion's six principles for the learning college and (b) if the principles could be supplemented with other components of work-related education. The central point was to further the knowledge of these relationships, which in total could be modeled to derive a common definition for work-related education at community colleges. A three-round Delphi technique was conducted to seek levels of agreement and consensus on O'Banion's principles and seven components associated with work-related education. The Round 1 sample (n=20) attrited by Round 3 (n=15) to six CEOs and nine administrators from community colleges whose CEOs were represented on the League for Innovation's board of directors. Content validity was essentially built-in by the development of the content of the scale matching the content domain, as conveyed by the participants' responses and what they considered to be the constructs of interest. Internal validity claims were met by following the established procedures for the Delphi technique to answer inferential questions about the scores and to develop well-founded conclusions from the data. A Duncan's multiple-range test confirmed significance between Round 1 and Round 3, which combined with the research procedures and study attributes, validated reaching a superior group view. Review of the quantitative and qualitative data over the three rounds revealed levels of agreement and consensus on the principles and the components. Correlation coefficients were organized to facilitate comparisons between the principles and the components for significance. Significant correlations were found between the principles and six of the seven components. There was specific commonality found among the funding component and the coordination and planning component across the majority of the principles. The correlations were modeled to derive a common definition for work-related education. Results of this study suggest that such a prototype model and further research could facilitate a consolidated position and common definition for work-related education. The study's conclusions have implications for why community college leaders and policymakers should pursue a common definition for work-related education within a national context to uphold the community colleges' role as "preparers of the nation's workforce."
Subject: career, college, common, community, coordination, definition, Delphi, education, finance, funding, leadership, occupational, planning, policymaking, related, vocational, work
General Note: Title from title page of source document.
General Note: Document formatted into pages; contains 199 pages.
General Note: Includes vita.
Thesis: Thesis (Ph.D.)--University of Florida, 2005.
Bibliography: Includes bibliographical references.
General Note: Text (Electronic thesis) in PDF format.

Record Information

Source Institution: University of Florida
Holding Location: University of Florida
Rights Management: All rights reserved by the source institution and holding location.
System ID: UFE0010111:00001


This item has the following downloads:


Full Text











USE OF THE DELPHI TECHNIQUE TO DERIVE
A COMMON DEFINITION FOR WORK-RELATED EDUCATION
















By

MICHAEL LEE DROLL


A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA


2005

































Copyright 2005

by

Michael Lee Droll
















ACKNOWLEDGMENTS

I could not have completed this maj or life' s proj ect without the genuine concern

and support of many people. I thank my wife, Molly, for her stick-to-itiveness by

example and undiminished motivation mixed with lots of love that spurred me on to

"mission accomplished." To my parents, Dot and Chuck, my sister, Kim, my sons, Karl

and Mark, and all my other family members--the sky has no limits when one is the

recipient of "unconditional love" such as I have received from my family.

Dr. Larry W. Tyree is my true mentor and colleague, and I will be forever grateful

for his wisdom, calming demeanor, and mutual admiration. Dr. Dave Honeyman was,

without a doubt, my man of the hour. My sincere appreciation goes to him as I would not

have achieved so much so soon without his encouragement. I especially thank Drs. Lynn

Leverty, Dale Campbell, Jim Doud, and, in particular, my suite mates, Drs. Art Sandeen

and Phil Clark, for all their support during my doctoral program journey and for making

me a part of the college's extended family.

Finally, I could not have completed this long journey without financial support

beyond what the G.I. Bill provided. I am especially grateful to Dr. Tyree for funding my

graduate assistantship. I give heartfelt thanks to Dr. Harry T. Albertson for the once-in-a-

lifetime, paid internship as recorder to the Council of Presidents by way of the Florida

Association of Community Colleges. Lastly, I would like to acknowledge Dr. Doug

Olson, my dear colleague in institutional research, who reciprocated with paid consulting.

I plan to stay in contact with everyone!






















TABLE OF CONTENTS


page

ACKNOWLEDGMENT S .............. .................... iii


LI ST OF T ABLE S ................. ................. vii...___....


AB STRAC T ................ .............. ix


CHAPTER


1 INTRODUCTION ................. ...............1.......... ......


Background for the Study ................. ...............1................
Statement of the Problem ................. ...............5................
Theoretical Framework............... ...............7

Purpose of the Study ................. ...............8.......... .....
Primary Questions .............. ...............8.....
Secondary Questions .............. ...............9.....
Definition of Terms .............. ...............9.....

Significance of the Study ................. ...............12.......... .....
Overview of the Research Process ................. ...............13........... ...
Assumptions, Delimitations, and Limitations .............. ...............15....
Assum options .............. ...............15....
D elim stations .............. ...............15....
Lim stations ................ .. .. ....... .... .. ......... .............1

Organization of the Remainder of the Study ................. ...............16........... ..

2 LITERATURE REVIEW ................. ...............18................


Introduction and Scope ................. ............ ...... ......... .........1
The Concept and Identity of Community Colleges ................. .........................19
Work-related Education................ ...... ............2
A Purpose of the Community College ................. ...............24...............
Terminology of Work-related Education ................ ............... ..............29
Learning College Theory and Work-related Education ................. ......................31
Recent Attempts at Educational Reform ................ ..............................31
Learning College Theory............... ...............32.
Six Principles of the Learning College ................. ...............35...............
Components of Work-related Education ................. ...............38................












Mission and Organization............... ..............3
Funding .................. .. ... .._._ .... .. .. ..............39
Needs Assessment and Documenting College Success............... .................4
Instruction, Programs, and Delivery Systems .............. ...............42....
Staffing .............. .... ...............43..
Coordination and Pl anning ........._...............___ .....__ ............4
National Proclamation and National Database ........._.._........_.. ..............45
Summary ............._. ......_.. ...............46.....


3 METHODS .............. ...............48....


The Setting............... ...............48
Primary Questions .............. ...............49....
Secondary Questions .............. ...............49....
First Stage ............._... ...............50......._ ......
Second Stage .............. ...............50....
Third Stage .............. ...............51....
Fourth Stage ............._... ...............52......._ ......
The Participants .............. ...............52....
T asks and Material s .............. ...............54....

General Operational Design .............. ...............56....
Data Collection .............. ...............57....
Round One ............._... ...............57......._ ......
Round Two .........._...._ ...............60..._..._ ......
Round Three .............. ...............63....
Communication Process .................. .. ...............65.

Data Management and Statistical Procedures............... ...............6
Reliability of the Instrument ................. ...............66................
Validity of the Instrument ........._._.. ....__........ ......_.. ...........67
Analytical Procedures............... ...............6
Descriptive Statistics .............. ...............70....
Parametric Statistical Tests............... ...............71.

Summary ........._ ....... .__ ...............72....


4 PRESENTATION AND ANALYSIS OF DATA ....._._._ ........___ ..............74


Introducti on .........._...._ ......... ...............74......
Primary Questions .............. ...............75....
Secondary Questions .............. ...............75....
Results of the Delphi Technique. ............._.. ......._ ......_.. ...........7
Selection and Confirmation of Participants ................. ............_ ......_.........77
Response Rates to Delphi Surveys ................ .. ...............79.
Verification of the Accuracy of the Delphi Technique .........._...._ .........._. ........80
Round One Results ........._..... ...............8 1....___. ....
Round Two Results............... ...............83
Round Three Results................... ....... .. .......8

Differences in Responses by Subgroups and Rounds............... ...............85.












Confidence in the Accuracy of the Delphi Technique ................. ......................1 10
Data Relationships to the Research Questions .........__.. ..... ._ ........_._.....111
Primary Questions ........._.___..... .__. ...............112....
Secondary Questions ........._._... ..._._.. ......___ ...... ..........1
Research Question Pertaining to Principles of the Learning College..............._._. ...1 13
Principle I ........._.___..... .__ ...............114....
Principle II ........._.___..... ._ __ ...............115....
Principle III ........._.___..... ._ __ ...............116....
Principle IV ........._.___..... .__ ...............117....
Principle V ........._.___..... ._ __ ...............118....
Principle V I .............. ... .. ...... .... ...... ...... .. ........ 2
Research Question Pertaining to Components of Work-related Education. .............121
Mission and Organization............... .............12
Funding .................. .. .......... .. ......... ............12
Needs Assessment and Documenting Success ................ ................. ...._125
Instruction, programs, and delivery systems ................. ....__ ...............128
Staffing .............. .... ...............128..
Coordination and Planning ................... ........ ..... .......... ......... .... ...........3
National Proclamation and National Database for Work-related Education.....131
Research Question Pertaining to Strongest Advocated Principles and Componentsl31
Strongest Advocated Principles ......... ........_____ ......... .............3
Strongest Advocated Components .............. ...............134....
Consensus Reached by the Panel of Experts ....._____ ..... ... ._ ................ ..136
Relationships between the Principles and Components .............. .....................3
Relationship of Components to Principles .............. .....................137
Sum m ary ................. ...............139..............

5 CONCLUSIONS AND RECOMMENDATIONS ............__.. ...................._..141


Introducti on ........._..... ...._... ...............141....
Primary Questions .............. ...............142....
Secondary Questions ........._.._.. ...._... ..............._ 142..
Model of Work-related Education ........._...... ......._. ....._.._..........14
Commonality of Components across Principles ........._.._.. ....._.._ ................1 57
Suggestions for Further Research ........._.._.. ....._._.._ ...._.._ .............6
Implications for Community College Leadership .............. ...............161....
Implications for Policymakers .....__.......____............. ............16

APPENDIX QUALITATIVE RESPONSES FOR ALL THREE ROUND S...............1 64

LI ST OF REFERENCE S ............_........... ..............1 4....

BIOGRAPHICAL SKETCH ............ ............ ...............189...

















LIST OF TABLES


Table pg

1-1 Percent Breakdown of General Operating Funds for 1998-1999 for States
Represented on the League for Innovation Board of Directors .............. ..... ........._.5

1-2 The 20 Colleges Represented on the Board of Directors to the League for
Innovation in the Community College ................. ...............10........... ...

4-1 T-test of Average Total, Principles, and Components Scores for CEOs and
Administrators during Round One ................. ...............86................

4-2 T-test of Average Total, Principles, and Components Scores for CEOs and
Administrators during Round Two ................ ...............86................

4-3 T-test of Average Total, Principles, and Components Scores for CEOs and
Administrators during Round Three .............. ...............86....

4-4 Duncan's Multiple Range Test of Scores between All Three Rounds ..................87

4-5 Descriptive Statistics of Statements Compared Across All Three Rounds ...........87

4-6 Principle I Statements in Round Three ....__ ......_____ ..... ......__........1

4-7 Principle II Statements by Exception in Round Three. .............. ............._..115

4-8 Principle III Statements by Exception in Round Three .............. .............. .116

4-9 Principle IV Statements in Round Three ..........._ ..... ..__ ............ ....11

4-10 Principle V Statements by Exception in Round Three .............. ............._..118

4-11 Principle VI Statements by Exception in Round Three ............... .............. .120

4-12 Mission and Organization Statements by Exception in Round Three ................123

4-13 Funding Statements in Round Three............... ...............124.

4-14 Needs Assessment Subcomponent Statements in Round Three............_..._... .......125

4-15 Documenting Success Subcomponent Statements in Round Three ....................127










4-16 Instruction, Programming, and Delivery Systems Statements in Round Three ..128

4-17 Staffing Statements in Round Three ........._.__........._. ...._.._ .........12

4-18 Coordination and Planning Statements in Round Three ............_. ........._.......130

4-19 National Proclamation and National Database Statements in Round Three........131

4-20 Strongest Advocated Principles in Round Three .........._._....... ._._............132

4-21 Strongest Advocated Components in Round Three ................. ............. .......134

4-22 Correlation Matrix of Principles and Components in Round Three ................... .137

4-23 Significance between Principles and Components in Round Three ...........__......138
















Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy

USE OF THE DELPHI TECHNIQUE TO DERIVE
A COMMON DEFINITION FOR WORK-RELATED EDUCATION

By

Michael Lee Droll

May 2005

Chair: David S. Honeyman
Major Department: Educational Leadership, Policy and Foundations

This study was designed to (a) test if work-related education conformed to

O'Banion' s six principles for the learning college and (b) if the principles could be

supplemented with other components of work-related education. The central point was to

further the knowledge of these relationships, which in total could be modeled to derive a

common definition for work-related education at community colleges.

A three-round Delphi technique was conducted to seek levels of agreement and

consensus on O'Banion's principles and seven components associated with work-related

education. The Round 1 sample (n=20) attrited by Round 3 (n=15) to six CEOs and nine

administrators from community colleges whose CEOs were represented on the League

for Innovation's board of directors.

Content validity was essentially built-in by the development of the content of the

scale matching the content domain, as conveyed by the participants' responses and what

they considered to be the constructs of interest. Internal validity claims were met by










following the established procedures for the Delphi technique to answer inferential

questions about the scores and to develop well-founded conclusions from the data. A

Duncan's multiple-range test confirmed significance between Round 1 and Round 3,

which combined with the research procedures and study attributes, validated reaching a

superior group view.

Review of the quantitative and qualitative data over the three rounds revealed levels

of agreement and consensus on the principles and the components. Correlation

coefficients were organized to facilitate comparisons between the principles and the

components for significance. Significant correlations were found between the principles

and six of the seven components. There was specific commonality found among the

funding component and the coordination and planning component across the maj ority of

the principles. The correlations were modeled to derive a common definition for work-

related education.

Results of this study suggest that such a prototype model and further research could

facilitate a consolidated position and common definition for work-related education. The

study's conclusions have implications for why community college leaders and

policymakers should pursue a common definition for work-related education within a

national context to uphold the community colleges' role as "preparers of the nation' s

workforce."















CHAPTER 1
INTTRODUCTION

Background for the Study

The central and comprehensive role that community colleges fulfilled in preparing

America' s workforce this past century and continue to accomplish into the new

millennium has been most apparent. Indeed, work-related education was a key aspect of

the community college mission from its beginnings. In the early 1900s, Witt,

Wattenbarger, Gollattscheck, and Suppiger (1994) pointed out in America 's Community

Colleges: The First Century:

Whereas universities molded students to fit their classical curricula, two-year
colleges adapted to meet the needs of a changing nation. They created vocational,
technical, and preprofessional programs to train skilled workers, from nurses to
keep people healthy, to mechanics to keep people mobile. (p. 3)

In recent years, the work-related education mission of community colleges evolved,

as substantiated by Bragg (2001): "Modern community colleges have a maj or

responsibility for preparing the nation's current and future midskilled workforce, which

accounts for three-fourths of all employees in the United States" (p. 5). As reported by

the Department of Labor, occupations requiring a postsecondary vocational award or an

academic degree accounted for 29 percent of all job growth in 2000 and will account for

42 percent of total j ob growth from 2000 to 2010 (Bohlen, 2004, p. 4). The National

Center for Education Statistics (NCES), and the Education Commission of the States

(ESC), and the Center for Community College Policy (CCCP) reported that about half of

the 565,000 associate degrees (National Center for Education Statistics [NCES], 2001)









and the greater part of the 165,000 advanced certificates (Center for Community College

Policy [CCCP], 2002) awarded annually were in work-related Hields, including nursing,

business, and engineering technology.

However, defining work-related education as a national model has been difficult

because of the lack of agreement about what was salient to all or most of work-related

education, including how policy was made and funding was channeled. In addressing the

Annual Economic Forum in Scranton, Pennsylvania, Dr. Patricia Donohue, president of

Luzerne Community College, advised panel members that "this country doesn't have a

definition for what workforce development is, and we don't have a workforce policy. We

don't have it nationally. We don't have it at the state [level]. We don't have it locally"

(Donohue, 2004, p. 9). Contributing to the lack of a common definition was the wide

range of individuals who sought work-related education from diverse backgrounds with

varying purposes and outcomes. This range included high school and technical center

graduates, current and transitioning employees, immigrants, as well as adults in

continuing education. Donohue emphasized the difficulty in trying to define the broad

mission of work-related education noting that "[it] could mean anything .. from how do

we help a welfare person get basic skills to how do we help a double Ph.D. get a new

skill for the highest tech thing and everything in between" (p. 9). Another contributing

factor during the past 20 years has been the attempt of legislators "to address the issue by

funding work-related education through 382 pieces of federal legislation, not to add on all

the state legislation" (Donohue, p. 9).

Community colleges, individually and collectively, have never been static

institutions, and they have demonstrated great flexibility and creativity in response to









student needs, work-related education expectations, and governmental requirements. At

the same time, the varying and vast descriptions and labels, as well as the categorical

funding and the multiplicity of funding streams, have made it difficult for community

colleges to maintain consistency in their work-related education programs and to

establish new programs based on emerging workforce needs.

Part of the disconnection between community colleges and their stakeholders

perpetuates itself with the seemingly ever-changing terminology used for work-related

education. In the case of "vocational education," it would seem the terminology for

work-related education has come full circle from the early 1900s to the new millennium.

In 1918, Floyd McDowell published the "first known dissertation" of the "first national

study of junior colleges," and it was noted "that 17 percent of the work offered by public

junior colleges was vocational" (Witt et al., 1994, p. 41). The phrase "career studies"

was the in vogue terminology for work-related education in Cohen and Brawer' s (1996)

The American Community College (third edition). By the fourth edition, however,

"vocation-technical" or simply "vocational education" was again the preferred

terminology (Cohen & Brawer, 2003, pp. 222). In fact, during the past century many

different terms have been used as labels for work-related education, such as

"Occupational, career, technical, or technological, semiprofessional, subbaccalaureate,

and terminal with each of these labels having a slightly different but admittedly related

intent" (Bragg, 2001, p. 6). Consistency in describing, labeling, and defining work-

related education had yet to be achieved during the 100-plus years that community

colleges have existed. Bragg referred to another set of terminology associated with work-









related education in the 1990s, including "workforce preparation, workforce

development, human resource development, and economic development" (p. 6).

Communicating a common definition for work-related education to the highest

levels has been one of the more complex problems for community college leadership and

government officials. However well-meaning and practical, the community college's

natural tendency to incorporate the local community's situation and state-specific funding

in its delivery of work-related education has resulted in many forms of organization,

multiple sources of funding, and varied perceptions on the part of governing agencies.

In fact, there was considerable variation in the support patterns and organization of

work-related education among the states. According to the Education Commission for

the States, state policy was a key factor to how effectively a community college

supported work-related education in its area. In a survey conducted by the Center for

Community College Policy (2002) at the Education Commission of the States, 17 of the

45 state agencies, which were responsible for oversight of community colleges, indicated

the level of funding for workforce development was a policy issue that had been debated

during the past two years (p. 5). In addition, 25 respondents stated that coordination of

workforce funding was also a policy issue which was debated. The Center for

Community College Policy stated that "to support workforce-training programs,

community colleges often need to cobble together funding from a variety of sources" (p.

7). Moreover, according to a survey conducted by the Center for Community College

Policy:

Too often, community colleges are seen as "junior colleges" or "remedial
institutions." This widespread lack of appreciation for their contributions to local
economic development is surely one reason that community colleges are often
poorly funded, particularly in comparison to four-year institutions. (p. 9)










In a 50-state survey conducted by the Education Commission of the States (ECS,

2000), significant variations were noted about how community colleges are funded across

the country, as shown by example in Table 1-1 for the states that comprise the League for

Innovation in the Community College Board of Directors. For example, Illinois

community colleges receive only 0.08 percent of their funding from federal sources

whereas Texas community colleges receive 14.4 percent of their overall general operating

funds from the federal government. From a state-to-state perspective, Arizona's

community colleges collect only 21 percent of their operating funds from the state,

compared to North Carolina' s community colleges, which collect 75 percent of their

operating funds from the state (ECS, 2000).

Table 1-1 Percent Breakdown of General Operating Funds for 1998-1999 for States
Represented on the League for Innovation Board of Directors
Tuition &
State Federala State Local Fees Otherb
AZ 1.00 21.00 57.00 20.00 1.00
CA 3.80 50.90 44.50 0.80 0.00
FL 0.25 68.51 0.02 23.06 8.00
HI 2.70 61.80 0.00 16.80 18.70
IA 3.21 45.66 5.89 38.97 6.27
IL 0.08 25.77 43.24 26.93 3.97
KS 2.00 24.00 40.00 16.00 18.00
MI 0.30 26.50 25.00 23.20 25.00
MO 2.00 41.00 26.00 24.00 7.00
NC 3.20 75.20 12.90 8.20 0.50
NY 5.70 29.00 31.30 34.00 0.00
OH 2.71 45.29 16.73 32.21 3.05
OR 11.50 39.90 19.90 16.20 12.50
TX 14.40 37.90 17.90 19.90 9.80
WA 5.00 59.00 0.00 17.00 19.00
alncludes all Perkins funds. blncludes federal financial aid and restricted funds other than Perkins.

Statement of the Problem

Numerous authors and practitioners have described and discussed how work-

related education has been symptomatically developed, established in policy, and funded.









However, the root cause of many work-related education issues that community colleges

face underscores policy and funding challenges, specifically the lack of a common

definition for work-related education beyond the local community, state, and regional

perspectives to a national level.

While community college leaders collectively possessed an appreciation and

understanding of work-related education, local governments, state legislatures, and the

federal government did not necessarily acknowledge or hold in the same regard the

comprehensive nature of work-related education in the role of educating and training the

nation's workforce. It was not apparent in the literature that a common definition existed

for work-related education, which was universally recognized by community colleges let

alone all those governmental agencies. On the other hand, it also was not apparent that

community college leaders have collectively conveyed the vision and mission of work-

related education beyond the communities that they serve. This lack of emphasis in

vision and mission may have subsequently impacted the ability to present a common

front for policymaking and funding at higher levels of government.

Throughout the past century, work-related education has had its opponents and

advocates who have argued on political and ideological premises. Bragg (2001),

however, states that "rarely have they been based on empirical results" (p. 13). A large

part of the issues surrounding work-related education deals with "its changing focus and

evolving goals" and how research may address questions about "new ideas, models, and

approaches" (Bragg, pp. 13-14). The situation was summarized as "unfortunate because

it suggests that community colleges have missed opportunities to steer vocational

education in directions that would provide the greatest benefit" (Bragg, p. 13). Therefore










the challenge ahead would be to derive a common definition for work-related education

and thus add to the body of knowledge pertaining to work-related education at

community colleges.

Theoretical Framework

Many community college practitioners agreed about the concepts of humanistic

education, learning communities, and, in particular, "the learning college." However,

there were no direct references in the literature that focused specifically on the

applicability of the six principles outlined in O'Banion' s model of "the learning college"

to work-related education. "The learning college" was a term used by O'Banion (1997)

as a generic reference for all educational institutions (p. 47). The theoretical framework

of this study was based on the applicability of the learning college' s six principles to the

processes and structure of learning in the work-related education setting. This theoretical

application and the research study method offered community college practitioners an

opportunity to carry out a dialogue about reframing work-related education in the

theoretical context of the learning college on an equal footing with credit-based, degree-

granting programs of study. However, the six principles could not be considered all-

inclusive to the research by O'Banion's own admission: "Content, funding, and

governance are examples of key issues that must be addressed and for which principles

must be designed" (p. 61). For the purposes of this research study, O'Banion's theory

was augmented by additional components specific to work-related education, as were

identified in the literature. The learning college and its six principles thus served as the

theoretical framework to develop, evaluate, and reach consensus on how student learning

should take place in work-related education settings as part of deriving a common

definition for work-related education.










Purpose of the Study

The primary purpose of this study was to test if work-related education conformed

to the six principles of the learning college (O'Banion, 1997, p. 47). Furthermore, this

study would supplement O'Banion's principles by examining additional complementary

components of work-related education. The overall purpose would be to determine if in

total the principles and components could be identified, categorized, and ranked to derive

a common definition for work-related education at community colleges. In addition, this

study tested the application of a Delphi technique to determine if a Delphi technique

could be effectively applied to an educational forum for leaders of community colleges.

The point would be to achieve consensus and levels of agreement to support the rationale

for establishing a consolidated position on and a holistic approach to what constitutes

work-related education. Such a position could facilitate clarity and consistency in

policymaking, particularly funding decisions, at the federal, state, and local levels. By

participating in this study, the community college leaders acted as a panel of experts

assisting in the research to derive a common definition for work-related education at

community colleges. This researcher anticipated that through the collective focus of the

panel of experts, useful insights might be gained into how work-related education could

pursue an increased presence and improved levels of support for community colleges in

the United States. The following research questions were developed to guide this study:

Primary Questions

1. Which, if not all, of O'Banion' s six principles of "the learning college" could be
associated with work-related education?

2. What other components could be identified for the work-related education function
at community colleges?










3. What were the most strongly advocated principles and components supporting
work-related education?

Secondary Questions

Additionally, secondary questions were identified that could be answered as a

result of this study. These questions were addressed through a compilation of answers to

the primary research questions.

1. Could a selected group of community college leaders reach consensus, using a
Delphi technique, on what principles and components could be identified to derive
a common definition for work-related education?

2. Could meaningful relationships be confirmed between the six principles and the
identified components to derive a common definition of work-related education?

The research questions were first addressed through an extensive search of relevant

literature on work-related education in the Hield of higher education with a focus on

community colleges. This literature review examined classical and current literature in

the field of vocational and occupational education, as it related to the postsecondary

educational organization. The need for this research was to establish a basis for testing

the theoretical framework of O'Banion' s six principles of the learning college and to

identify components of work-related education. The principles and identified components

were subsequently used in the development of the initial survey instrument. The purpose

of the survey was to collectively present a basis of potential principles and components

which could generate expert feedback. This feedback would contribute to deriving a

common definition for work-related education.

Definition of Terms

Community colleges were defined as any public, regionally accredited,

comprehensive, two-year institution. This study was limited to those 20 community

colleges and districts listed in Table 1-2 whose chief executive officers (CEOs) served on









the League for Innovation in the Community College Board of Directors. Community

colleges were not necessarily defined as a single college. They may have been part of a

consortium or multi-campus facilities.

Table 1-2 The 20 Colleges Represented on the Board of Directors to the League for
Innovation in the Community College
State/
College City Province
Anne Arundel Community College Arnold MD
Central Piedmont Community College Charlotte NC
Cuyahoga Community College Cleveland OH
Dallas County Community College District Dallas TX
Delta College University Center MI
Foothill-De Anza Community College District Los Altos Hills CA
Humber Institute of Technology & Advanced Learning Toronto Ontario
Johnson County Community College Overland Park KS
Kirkwood Community College Cedar Rapids IA
Lane Community College Eugene OR
Maricopa Community College District Phoenix AZ
Miami-Dade College Miami FL
Monroe Community College Rochester NY
Moraine Valley Community College Palos Hills &L
San Diego Community College District San Diego CA
Santa Fe Community College Gainesville FL
Seattle Community College District Seattle WA
Sinclair Community College Dayton OH
St. Louis Community College St. Louis MO
University of Hawaii Community Colleges Honolulu HI

Community college leaders were defined as the experts on work-related education

within public community colleges. These leaders were identified as the colleges' "chief

executive officers (CEOs), academic affairs officers, business/industry liaison officers,

continuing education officers, or occupational education officers" (American Association

of Community Colleges [AACC], 2004, p. 3) at the 20 colleges listed in Table 1-2 whose

CEOS comprised the League for Innovation in the Community College Board of

Directors. The League for Innovation "is an international organization dedicated to

catalyzing the community college movement" (League for Innovation in the Community









College [League], 2004, About the League, preface). Twenty (19 in the United States

and 1 in Canada) "CEOs from some of the most influential, resourceful, and dynamic

community colleges and districts in the world comprise the League's board of directors"

(League, para. 3) The League has more than 700 member institutions from 10 different

countries and has partnerships with more than 100 corporations. With this innovative

core of directors, members, partners, and collaborators, the League leads proj ects and

initiatives including the expansion and improvement of workforce training programs in

the United States and Canada.

Components were developed from a self-study guide published in a National

Council for Occupational Education (NCOE) monograph (Hamm & Mundhenk, 1995,

pp. 4-9). The components were defined as: (1) mission and organization; (2) funding;

(3) needs assessment and documenting college success; (4) instruction, programs, and

delivery systems; (5) staffing; (6) coordination and planning; and (7) national

proclamation and national database.

The Delphi technique was defined as a methodology that utilized the expertise of

current community college and district CEOs and their designated representatives. This

methodology was used to reach levels of agreement and consensus on principles and

components. The purpose was to derive a common definition of work-related education

at community colleges within the realm of higher education in the 21st century. The

methodology is based on a series of questionnaires or surveys with each being more

structured and requiring more focused reflection on the part of the participating experts.

This technique is a preferred methodology in the measurement of subj ective judgments

when the problem or study does not lend itself to other precise analytical methodologies.









The Delphi technique is an iterative process that is recognized as an inductive-based

approach to examining multiple issues and extracting specific answers to questions in a

variety of disciplines.

Higher education institutions were defined as all institutions of higher education,

inclusive of two-year, four-year, private, and public institutions that grant undergraduate

or graduate degrees.

A panel of experts was defined as those individuals who were selected to

participate in the Delphi technique study. The panel of experts was chosen based on the

participants' knowledge, familiarity with the problem, and skill with written

communication.

Significance of the Study

The point of this study was to depict the ongoing plight for work-related education

at community colleges, as well as the potential in pursuing a common definition model.

To illustrate the dilemma, Donohue (2004) framed her remarks to the 2004 Economic

Forum by stating: "this country doesn't have a definition for what workforce

development is, and we don't have a workforce policy" (p. 6). This study was significant

to community colleges and community college leadership to present a consolidated

position on how work-related education could be defined for students, policymakers,

business and industry--all stakeholders in a comprehensive sense. If the research

demonstrates that a common definition for work-related education is desirable,

community college leaders could more effectively present a unified position for work-

related education for policymaking and funding decisions at the federal, state, and local

government levels. Communication of a common definition to bridge the gaps among










practice, policy, and funding could be critical for work-related education to achieve a

level of parity with other programs.

This study sought to confirm levels of agreement and consensus on O'Banion's six

principles of the learning college and other identified components of work-related

education, which in total supported a common definition. Although the results of the

study could not be generalized to another group or nationally, the concepts and

framework were readily transferable for use by other researchers. Policymakers and

community college associations may choose to use this study as a guide for further

research and policy development. They can give due consideration to the complexity of

work-related education and how identifying the principles and the components significant

to a common definition may simplify the concept. Developers of future work-related

education programs may find the research helpful in determining the priority, focus, and

applicability of new programs, which are based on the levels of agreement and consensus

reached in this study.

Overview of the Research Process

This investigation was based on a constructionist epistemology and a

phenomenological perspective. It used a Delphi technique methodology for the research

and for any generalization of the results (Crotty, 1998, p. 42). The research method used

to gather and analyze data was based on a mixed survey of the Likert scale and open-

ended items. Research participants had considerable influence over the development of

the survey instrument related to the research questions or statements. This methodology

was identified as the Delphi technique, which uses a series of questionnaires or surveys to

aggregate the knowledge, judgments, or opinions of experts in order to address complex

questions (Moore, 1987, pp. 50-51). This Delphi technique was conducted in three









rounds of surveys, in which the instrument was adapted with each subsequent round. The

Delphi technique was recognized as an appropriate study design and assessment to make

important decisions about educational policy (Clayton, 1997, p. 386).

In addition to the Delphi technique, the theoretical framework offered by this

research provided an evaluation frame to better compare, interpret, and support work-

related education with other programs. The panel evaluated work-related education in

the context of O'Banion' s six principles of the learning college. The panel also identified

other components that could contribute to a common definition for work-related

education to establish a consolidated position. The panel of experts was surveyed to

assess their perspectives regarding work-related education at their college, in their state

and region, as well as nationally. The philosophical stance of this study was based on

research of numerous works published during the past century. These works defined

work-related education only to the extent that others addressed the problem to satisfy

self-interests or the focus of a certain period in its history. Even more recent studies and

research in the 1990s did not offer a solid context for today's policymakers to better

determine the "who, what, why, and how" of work-related education programs. Several

community college leaders and researchers had alluded to the problem and its impact on

policymaking, funding, and delivering quality work-related education, but the dialogue

fell short of deriving a common definition.

The use of qualitative data in educational research is recognized as important to the

study and the understanding of educational phenomena, as well as providing a natural

basis for interpretation with explanations emerging from intensive examination of the

data (Tuckman, 1999, p. 400). Validity was essentially "built-in" after each phase by









virtue of the Delphi technique' s development of the content of the scale matching the

content domain. This was conveyed by the responses from the panel of experts and what

they considered to be the constructs of interest. Internal validity claims were met by

following established procedures for the Delphi technique to develop well-founded

conclusions. External validity was dependent on the selection of the panel of experts as a

representative body. Their scores or ratings may or may not be generalized for all

community colleges in a particular sample, group, or population.

Assumptions, Delimitations, and Limitations

Assumptions

This researcher made the assumption that participants would answer the survey

honestly and they would return each of the three phases in a timely manner. This

researcher also assumed that the information collected would be usable as part of this

research and that the participants were representative of the opinions of all their peers.

Finally, this researcher assumed that those participants who chose to respond

electronically would read, understand, and respond to each phase of the survey within a

specified time frame without delegating this responsibility to a subordinate.

Delimitations

The research was delimited to public community colleges and districts whose

CEOs served on the League's Board of Directors. Twenty CEOs were contacted to

participate in the research study. In addition, each CEO was asked to identify another

participant for the research study who carried out duties pertaining to work-related

education at their institution.









Limitations

The limitations of this study included the use of a sample group from a specific

organization. Only current chief executive officers and their designated representatives

who were actively employed at community colleges were asked to take part in the study.

While the theoretical focus was on O'Banion' s six principles of "the learning college," it

could not be construed that the participants understood the paradigm shift offered by this

framework, nor directly endorsed the six principles, nor implemented "the learning

college" vision in its totality at their respective institutions. Although there is no

guarantee, this study may provide valuable information to other organizations or regions

of the United States which were not directly represented by the sample group.

Only two-year public community college leaders were included in this study.

Universities and private colleges were not included. This study did not attempt to

compare two-year community colleges to four-year colleges and universities.

Additionally, these data were self-reported rather than observed by an impartial

third party. It was vital that these data were the direct reflection of the community

college leaders and others who were chosen to participate--and not the opinions of

subordinates. This limitation was discussed in initial contacts with the participants. Each

participant received and submitted materials electronically through the use of electronic

mail systems and web-based systems at their respective community colleges.

Organization of the Remainder of the Study

Chapter 2 represents a review of literature pertinent to work-related education in

the community college, in particular, the concept of the learning college as a theoretical

framework for the study, and also the identification of other components pertinent to

work-related education. Chapter 3 describes the research methodology, including the









development of the initial survey instrument, the three consecutive rounds of the Delphi

technique, and the statistical procedures and tests employed for data analyses. Chapter 4

identifies the stratified sample of the population group within the public community

college arena as chief executive officers and administrative leaders who comprised the

panel of experts. Chapter 4 also contains detailed reporting of the data results and

analyses from the three-round Delphi technique. The Delphi technique focused on

evaluating the applicability of O'Banion' s six principles of the learning college to work-

related education, as well as other components identified from the literature and

developed through the three-round survey process. Additionally, qualitative comments,

opinions, and responses were encouraged from the participants to enrich the study, and

their inputs were reported in the appendices. The use of qualitative data in educational

research was recognized as important to the study, as well as and understanding of

educational phenomena and providing a natural basis for interpretation with explanations

emerging from intensive examination of the data (Tuckman, 1999). Chapter 5 presents

conclusions based on the results from the data that were compiled and the relationships

which were identified to expand the existing body of knowledge pertaining to work-

related education. A prototype model of a common definition for work-related education

is outlined, the commonality of certain components is identified, recommendations for

further research are offered, and implications for community college leaders and

policymakers are presented.















CHAPTER 2
LITERATURE REVIEW

Introduction and Scope

This chapter consists of a review of the relevant literature for the study. The

research focused on the concept of work-related education in community colleges and on

the principles and components that could be identified to derive a common definition for

work-related education beyond community, state, and regional perspectives to a national

level. Models of higher education were researched to seek a learner-centered innovation

as a theoretical basis for framing a common definition for work-related education.

Specific theory principles identified in the literature established the framework of the

study and the three-round Delphi technique to which participants could respond to the

application of learner-centeredness in work-related education.

To complement the theoretical framework, components of work-related education

identified in the literature were included as touch points. These touch points were used

for an expanded agenda to discuss the research topic in an effort to seek potential

agreement of what constitutes work-related education. The research considered the

evolution of work-related education in higher education and primarily focused on

community colleges. The results of the literature review were arranged in this chapter by

topics that evolved from the beginnings of community colleges and work-related

education to the theoretical framework and component identification.









The Concept and Identity of Community Colleges

The comprehensive literature on community college evolution (Koos, 1924, 1925;

Witt et al., 1994; Baker, Dudziak, & Tyler, 1994; Cohen & Brawer, 1996, 2003) begins

with historical, yet perceptive, analyses of how community colleges evolved and then

culminated with trends, challenges, and forecasts for the future. From a historical

perspective, the initial reactions of many to what the function of the community (junior)

college was could be summed up as: "to look upon this new unit in the school system

solely as a sort of isthmus connecting the mainland of elementary and secondary

education with the peninsula of professional and advanced academic training" (Koos,

1925, p. 16). To the contrary, Koos (1925) purported that he and others like him had

higher expectations for the junior college as "an institution affecting much larger

proportions of the population and influencing profoundly the organization of education

on levels above and below" (p. 16). Witt et al. offered the following concept of two-year

colleges:

Whereas universities fought to remain exclusive, junior colleges measured their
success by inclusion. Whereas universities molded students to fit their classical
curricula, two-year colleges adapted to meet the needs of a changing nation. They
created vocational, technical, and preprofessional programs to train skilled workers,
from nurses to keep people healthy, to mechanics to keep people mobile .. truly
becomes the university of the common man. (p. 3)

Gleazer in his foreword to America 's Community Colleges: Thze First Century

emphasized that community colleges have continually pursued a search for institutional

identity "for recognition and public understanding in terms of a mission different from

and yet in some respects similar to the missions of both its progenitors, the secondary

school and the college" (Witt et al., 1994, p. vi). By building on Koos's (1925) views,

Gleazer made the point that a long- standing complaint of community colleges was the









lack of understanding and misunderstandings that evolved from the "mixed parentage" of

the earlier days. Gleazer noted in those days that "the junior college could be another

two years of secondary school--an extension of the high school .. or it could be the

first two years of college" (p. vii). Lucas (1994) shared the perspective that early two-

year schools viewed themselves as "a preparatory step to university life and a

professional career. by the late 1920s and early 1930s the trend was .. as terminal

institutions where students of limited means might prepare themselves for skilled trades

and semiprofessions" (p. 221).

Cohen and Brawer (2003) stated:

The easily accessible, publicly supported school became an article of American
faith, first in the nineteenth century, when responsibility for educating the
individual began shifting to the school, then in the twentieth, when the schools
were unwarrantedly expected to relieve society's ills. (p. 3)

Campbell, Leverty, and Sayles (1996) found that community colleges assumed

these responsibilities readily as a new concept in higher education based on their

"demonstrated flexibility in adapting to social and economic challenges facing

communities, states, regions, and the nation" (p. 172). Lucas (1994) found that the

"somewhat ambiguous and paradoxical role" of two-year institutions "was to satisfy the

precept that in a democracy everyone is entitled to access to higher education" (p. 221).

Cohen and Brawer described the beginning evolution of the concept and identity as based

on community colleges setting a new precedence in higher education:

The community colleges thrived on the new responsibilities because they had no
traditions to defend, no alumni to question their role, no autonomous professional
staff to be moved aside, no statements of philosophy that would militate against
their taking on responsibilities for everything. (p. 3)

However, along with flexibility, adaptation, and educational innovation,

community colleges have experienced an evolutionary mix, which contributed to









confusion about the community college mission, definition, and nomenclature (Baker et

al., 1994, p. 4). The Morrill Act (Land Grant College Act of 1862) provided basic

expectations for higher education within the states. However, the manner in which states

carried out implementation produced many differences in organization, structure, and

control of these institutions (Witt et al., 1994, p. 226). Ratcliff (1994) in A Handbook on

the Community College in America (Baker et al., 1994) provided a historical context of

this dilemma as:

Many things are meant by the terms "community college," junior college,"
"technical college," and "technical institute." The lack of definition of these terms
is attributable in part to the wide variation in mission, governance, finance, and
structure of two-year colleges in the United States. (p. 4)

The use and definition of such terms as secondary education, vocational education,

colleges, universities, and even higher education were peculiar to each state. These terms

compounded the identity predicament with essentially 50 different bureaucratic

implementations by states of the federal vocational acts that funded the states (Witt et al.,

1994, p. 226). Gleazer (Witt et al.) found that while legislative language and accrediting

manuals define and describe community colleges for their particular needs, there still

"persists, most notably at the federal and state levels, less than full appreciation of the

community college as an institution with an identity of its own" (p. viii). At the federal

level, the community colleges' functionality was driven by the wars in the 20th century

and, particularly, in the 1960s as educational legislation grew. Gleazer (Witt, et al.) also

noted that "it was often necessary to examine congressional intent in order to determine

whether community colleges were included in legislative measures affecting 'colleges.'

." (p. viii). Campbell et al. (1996) found that "state funding for higher education reflects









each state's preference for higher education among other services funded by the states"

(p. 174). Cohen and Brawer (2003) stated:

Nevertheless, other writers in education, and certainly the maj ority of those who
comment on the role of the community colleges, suggest that education is an
essential expenditure for economic growth, a common good, and is not merely a
nonproductive sector of the economy, a form of consumption." (p. 242)

Witt et al. (1994) found that the Carnegie Commission on Higher Education

recommended that more students be channeled into two-year colleges, particularly into

vocational programs, which in turn helped mold America's emerging statewide

community and junior college systems (p. 216). Honeyman and Bruh (1996) noted an

observation in the preface of the Seventh Annual Yearbook of the American Education

Finance Association made by Mary P. McKeown in the 1980s. She stated that higher

education funding issues were tied to "sources of funding, levels of funding, and the very

existence of the institutions" with education being equated to economic growth in the

United States and the world (p. vii).

Transitioning to the 1990s, Honeyman and Bruh (1996) endorsed "the public

expectation that colleges and universities contribute to the economic well-being of our

nation by producing a highly trained and skilled workforce" while noting the change

from McKeown's reference point to "two themes of increased attention in American

higher education: institutional accountability and educational quality .. with new

performance-based accountability demands" (pp. 9-10). Again, the impetus of many

aspects of the community college movement can be tied to national policy and significant

pieces of legislation, which formed the concept and identity of community colleges over

time. Witt et al. (1994) identified the following two examples of maj or national










legislation, which expanded American higher education. These examples were

significant to the establishment and expansion of community colleges as well:

The Land Grant College Act of 1862 and the GI Bill of 1945 have represented great
steps in moving American higher education toward the universal educational
opportunity envisioned by Thomas Jefferson and by the Ordinance of 1787. The
agricultural and mechanical colleges were often referred to as "people's colleges"
as were early junior colleges. (p. 275)

Cohen and Brawer (2003) stated that community colleges would continue to appeal

to recent high school graduates because of "easy access, low cost, and part-time

attendance possibilities," as well as to "j ob seekers because of the high demand for

people in occupations for which some postsecondary training but not a bachelor' s degree

is expected" (p. 407). Witt et al. (1994) found from the beginning of the community and

junior college movement that these colleges advocated a mission with three basic tenants:

"preparing students for transfer to a four-year college, providing vocational training, and

serving as a source of continuing education for the community" (p. 235). Although all

three tenants have been stressed throughout the history of community colleges, the work-

related education tenant in particular has not always received equal attention.

Work-related Education

In between the origins and the outlook for the future, community college

researchers and authors of the past century often identified and addressed work-related

education as one of the purposes of the two-year colleges, as well as making an early

connection of community and college within their works. Cohen and Brawer (2003)

described the early days of vocational programming as a force where "the community

colleges grew in part because some of their earlier proponents recognized the coming

need for semiprofessionals and despaired of the universities' adjusting rapidly enough to

provide this less-than-baccalaureate education" (p. 220). In fact, at the American









Association of Junior Colleges (AAJC) "organizational meeting in 1920 and at nearly

every meeting throughout the 1920s and 1930s, occupational education was on the

agenda" (p. 221). The discussions at these early AAJC meetings centered around

arguments on behalf of work-related education.

Cohen and Brawer found that "the thesis of Brint and Karabel's book The Diverted

Dreamn (1989) is that the AAJC was the prime force in effecting a change in community

college emphasis from prebaccalaureate to terminal-occupational education" (p. 221). In

1922, the AAJC revised its statement of purpose in its constitution to better reflect the

ties to community and work-related education: "The junior college may, and is likely to,

develop a different type of curriculum, suited to the larger and ever-changing civic,

social, and vocational needs of the entire community in which the college is located"

(Witt, et al., 1994, p. 40). Parnell (1985) stated that work-related education at community

colleges "brought vocational and technical education into the halls of ivy-covered

institutions" and noted that community colleges communicated that there was dignity and

worth in all honest labor (p. 87).

A Purpose of the Community College

Cohen and Brawer (2003) found that "vocational-technical education" was one of

the "curricular functions" written in the plans for public colleges from the beginning and

identified in most states' legislatures. In 1900, William Rainey Harper suggested that

many students were likely to terminate their education after completing junior college in

order to seek positions as teachers or to go into business (Cohen & Brawer, 2003, p. 220).

In 1915, Angell noted the interest of some junior colleges to "specialize particularly in

industrial, engineering, and vocational directions, with its main interest centered on

young people who will not go beyond the instruction it offers" (Witt et al., 1994, p. 39).









The 1920s found junior college leaders, in particular, Leonard V. Koos, focusing

more and more on vocational education. In 1924, Koos was referred to as "the greatest

booster of vocational programs" by some college researchers and authors (Witt et al.

1994, p. 48). He sought out university deans to confirm which professions could be

taught at two-year colleges. Koos (1924) confirmed that in the field of engineering, as an

example, the university deans he contacted identified 43 occupations that could be moved

to a junior college level (p. 155). These "semiprofessions," as Koos referred to the

occupations, became touch points on a national effort to expand work-related programs at

junior colleges. Koos (1925) found "an appreciable beginning and partial awareness of

the large need represented," as he sought out work-related education (semiprofessional

training) at 300 institutions or organizations (p. 133). His overall investigation identified

20 purposes for the junior college, which were organized into five groupings. While the

first purpose pertained to transferability, it was more conscionable when Koos (1925)

made the point that:

Purposes (2 and 3) are among those which would make it possible for the junior
college to serve the interests of those who are "not going on." The former urges for
such students the provision of opportunities for "rounding out their general
education, opportunities which are not given if the work offered is only that
regarded as preliminary to some form of advanced training. (pp. 19-20)

Koos (1925) was seeking specific status for work-related education when he coined

the phrase "semiprofessional" training, a descriptor for the third purpose. This referred to

preparation for occupations, such as teaching as a "sole occupation" for which states

granted "certificates to teach upon the completion of some or all of the work of the two

years if the candidate includes courses in education" (p. 20). Gleazer (1968) made a

point in the 1960s regarding national studies:









But actually two-thirds of those enrolling will not transfer to a four-year college ..
.They will require organized educational experiences other than those leading to
the bachelor's degree. .. A substantial part of the two-thirds will prepare for
employment. (pp. 66-67)

Parnell (1985) advocated a concept for improving work-related education in the

mid-1980s to address the same concern about the education system as "the lack of a

rigorous, constructive, and focused program of study to prepare the sixty to seventy

percent of our high school students who will not likely be pursuing a baccalaureate-

degree program" (p. xi). Bryant (1996) noted that ten years after Parnell's concept was

introduced, "the initial 'tech prep' steam, however, has begun to slow down" (p. 414).

He attributed the slowdown, not to any viability issue with the concept, but rather with

the inability to adequately define the concept--a system which was "as fluid as the wind."

Bryant advocated that "several of the problems disappear with the proper definition of the

articulation system" and offered a model based on "required, suggested, and

postsecondary components" (pp. 418-421).

Ratcliff (1994) presented a different perspective of the community college (Baker

et al., 1994) and sought to cast a social context by describing the community college

evolution as "seven streams of educational innovation" of which one stream was

identified as "the vocational education movement" (p. 4). Ratcliff also referred back to

the influences of William Rainey Harper who brought about the establishment of two-

year colleges, including the Lewis Institute of Chicago in 1896 and the Bradley

Polytechnical Institute (now Bradley University) in Peoria, Illinois, in 1897 (Baker et al.,

1994, pp. 11-12). Witt et al. (1994) documented that following the classic works of Koos

in the 1920s, work-related education took on a central role as a solution to the Great

Depression era. In 1934, Doak Campbell, the secretary of the American Association of









Junior Colleges declared "that education was the strongest and cheapest social insurance

that could be employed, and the nation that neglected it was inviting disaster" (p. 104).

Lucas (1994) noted that two-year institutions were "lauded as instruments of social utility

and efficiency .. [and that] junior colleges continued to flourish throughout the

Depression years, even when larger public universities languished for lack of adequate

funding from state legislatures" (p. 221). Witt et al. noted that "Campbell foresaw a time

after the current disaster when three-quarters of all junior college graduates would be in

vocational and terminal programs" (p. 104).

Cohen and Brawer (2003) found that vocational education enrollments began

growing at a rate greater than liberal arts enrollments in the 1960s and continued to do so

for 20 years. They attributed the increases in enrollment for work-related education to

several causes:

This rise is attributable to many causes: the legacy left by early leaders of the
junior college movement and the importunities, goadings and sometimes barbs of
later leaders; the Vocational Education Act of 1963 and later amendments; the
increase in the size of public two-year colleges; the increase in part-time, women,
disadvantaged, disabled, and older students; the community colleges' absorption of
adult education programs and postsecondary occupational programs formerly
operated by the secondary schools; and the changing shape of the labor market. (pp.
226-227)

Indeed, as found by Witt et al. (1994), "Vocational programs were a boon to local

industry," and junior colleges could quickly adapt to the needs of employers by

retrofitting existing programs and developing new ones which satisfied industries' needs

(p. 49). Consequently, it was those towns with a job training function that attracted local

business, and industry reciprocated by becoming leading supporters of their local junior

college.









Witt et al. (1994) found that community colleges had traditionally been

shortchanged in federal funding whereas Congress directed vocational funds to local high

schools and technical centers. Community college needs often fell through the cracks of

federal funding, as well when higher education programs, such as those funded by the

National Defense Education Act, went primarily to universities (p. 209). Witt et al. found

that the American Association of Junior Colleges (AAJC) Commission on Legislation

under chair Kenneth Skaggs was the leader in developing and publishing "principles for

legislative action that gained wide use" (p. 226). The commission's model for state

legislation drew from the research carried out by subsequent commission chair Dr. James

L. Wattenbarger and others, and the model was used as a guide by a number of states

(Witt et al., p. 227).

Two maj or successes for work-related education came in the form of two landmark

education spending bills signed by President Lyndon Johnson in 1963. Witt et al. (1994)

noted that the Higher Education Facilities Act of 1963 provided $1.2 billion for

postsecondary construction proj ects, of which $690 million was authorized for matching

grants for undergraduate institutions (p. 209). With community colleges guaranteed 22

percent of the Facilities Act funds, $15 1 million brought premier attention to the

significance of community colleges. Congress soon became aware that there was a

community college or junior college in almost every congressional district (Witt et al., p.

209). The Vocational Education Act of 1963 provided $450 million in new funds for

construction and operation of vocational education schools. The act created departments

or divisions of junior colleges that were associated with work-related education entities

and thus eligible for the funds. Witt et al. noted that this act was amended in 1968 to









fund equipment grants, exemplary programs, consumer and homemaking education, and

curriculum development (p. 210).

Witt et al. (1994) found that the Education Amendment Act of 1972 provided $707

million for postsecondary vocational programs. Those funds were supplemented with

Congress appropriating an additional $981 million. Two years later, they found that "the

number of vocational graduates doubled, and by the end of the decade, 62.5 percent of all

two-year college graduates had received occupational degrees" (pp. 251-252). Besides

federal appropriations and legislation, states were also committed and, in some cases,

more supportive of work-related education. The state legislators noted that while

workforce reform was traditionally centered on federally funded programs, state

expenditures exceeded those of the federal government (Cohen & Brawer, 2003, p. 228).

Terminology of Work-related Education

Cohen and Brawer (2003) found that in its earliest beginnings, work-related

education in community colleges was based on teaching skills that were beyond the level

of high schools. These skills were "originally conceived as an essential component of

terminal study--education for students who would not go on to further studies. ." (p.

22). Bragg (2001) details the beginnings of work-related education labeling, such as

"occupational, career, technical or technological, semiprofessional, subbaccalaureate, and

terminal." This education labeling was in addition to the newer terminology of the

1990s, which included "workforce preparation, workforce development, human resource

development, and economic development." The different usages were based on the

historical significance, intent, or focus, for example, technology, and how broadly

defined the activities for the referenced label (p. 6). Koos (1925) again coined the term

"semiprofessional" for that final training, which required more than secondary school










years and was classified as "trades" and contrasted to "professions," which required four

or more years of work beyond high school (p. 20).

Wattenbarger (1950) noted that "terminal education may be thought of as being

general and vocational" (p. 60). Wattenbarger was first referring to the ties to high

schools and the liberal arts college as general education. He referred to "vocational

education," as defined by Arthur B. Mays, as "the unprecedented developments in the

physical sciences, in medicine, and in the social science since the turn of the century

[that] have greatly expanded the vocational area for which careful training is required" (p.

62).

Cohen and Brawer (1996) found that semiprofessionalss typically referred to

engineering technicians, general assistants, laboratory technicians, and other people in

manufacturing, business, and service occupations" (p. 224). Cohen and Brawer (1996)

referred throughout their third edition of The American Community College to the term

"career education" and chose this term to represent "a collective term for all

occupational, career, and technical studies." "Career Education" was the term

popularized by the U. S. Office of Education in the 1970s but "was coined in the 1950s to

connote lower-school efforts at orienting young people towards the workplace" (pp. 216).

Lombardi (1992) stated that "the term 'career education' has no more potency than the

old names--occupational, semiprofessional, technical, vocational, trade" (p.79). By their

fourth edition, Cohen and Brawer (2003) chose to change their point of reference back to
"vocational education" as~ th e descrri pntor for woir k-r elatedI educarti on suggesting that


career education never quite caught on (pp. 224).










Bragg (2001) noted the historical significance of the many terms that have been

used to describe work-related education, but she chose to focus on the term "vocational

education because of its longevity and historical significance" (p. 7). Both Bragg and

Cohen and Brawer (2003) agree that many terms have been used to describe work-related

education and that the terminology "has never been exact" (Cohen & Brawer, p. 222).

The overall interest and focus of the published works was not necessarily on the

particular term to be used. Instead, a term was used consistently to establish "a logical

benchmark for assessing change from the past to the present and to the future" (Bragg, p.

7). This is essentially why the researcher chose to refer to an original term "work-related

education" within the boundaries of this study to be inclusive, not limiting, and without

any bias of past terminology.

Learning College Theory and Work-related Education

Cohen and Brawer (2003) indicated that community colleges today and in the near

future will continue to realize sufficient, if not increased, enrollments. They noted that

"the absolute number of 18 year olds in the United States peaked at 4.3 million in 1979,

bottomed at 3.3 million in 1992, and is proj ected to regain the 1979 level by 2009" (p.

405). However, two major reform reports in 1983 and 1993 gave reasonable concern as

to whether or not community colleges, as part of the national higher education system,

were up to the task to serve the students of those decades and in the future.

Recent Attempts at Educational Reform

The National Commission on Excellence in Education (1983) presented its

viewpoint of American education in A Nation at Risk: Thze Imperative for Educational

Reform as:









We report to the American people that while we can take justifiable pride in what
our schools and colleges have historically accomplished and contributed to the
United States and the well-being of its people, the educational foundations of our
society are presently being eroded by a rising tide of mediocrity that threatens our
very future as a nation and as a people. (para. 2)

In the next decade, the Wingspread Group on Higher Education (1993) reiterated

the general alarm ofA Nation at Risk in its report, An American Imperative, which noted:

A disturbing and dangerous mismatch exists between what American society needs
of higher education and what it is receiving. Nowhere is the mismatch more
dangerous than in the quality of undergraduate preparation provided on many
campuses. The American imperative for the twenty-first century is that society
must hold higher education to much higher expectations or risk national decline.
(Higher Expectations for Higher Education, para. 1)

As a precept to his concept of the learning college, O'Banion (1997) addressed

the reform efforts of the past noting that "a great deal of reform effort .. focused on the

traditional architecture of education. .."

The traditional education system is based, as has been noted, on an architecture that

is time-bound, place-bound, efficiency-bound, and role-bound, undergirded by a grading

system that assigns only 5 of 26 possible letters in the alphabet to designate amount and

kind of learning achieved. (p. 63)

Learning College Theory

In response to an education system bound and restricted by traditions, O'Banion, a

self-described "zealot for Humanistic Education," proposed a new way of thinking. His

view would place the learner first and thus upset the established system and replace it

with a new model. He based this new model on the writings of Dewey, Rodgers, Combs,

and others pertaining to the natural educative process, client-centered therapy, and the

humanistic education movement, respectively (pp. 42-44). Other authors and researchers

also recognized that a new era for community colleges was necessary to unbind the ropes









of failed reform and forces of resistance. How community colleges would have to

respond to increasing enrollment and satisfy student needs in a new era of community

colleges was noted in the late 1990s by Alfred and Carter (2000):

By today's standards, however, first generation institutions could not survive. Their
strategy was to "develop and deliver" and factors of demand, competition and
quality were relatively insignificant as part of this strategy. .. Growth was their
focus and a comprehensive institution with many offerings was a natural, though
inefficient, organizational form. To be fair, they did plan and were sensitive to
market dynamics and student needs, but nowhere near the extent to which today's
high performing organizations do these things. (para. 1)

How community colleges respond and ensure they are indeed "high performing"

required concentrated focus on what Alfred and Carter (2000) referred to as "market

dynamics and student needs" (para. 1). As one scholar rhetorically asked, "Is education

to be organized around institutions, credit, and credentials .. or is education to be

organized around learners as an optimal system for distributing knowledge and

encouraging its utilization?" (p. 11). One widely recognized theoretical response was

O'Banion' s (1997) model for community colleges of the future titled A Learning College

for the 21st Century, a new concept in the late 1990s. He described this concept as being

"built on the long-established values in the community college, values that place a

premium on quality teaching. ." (p. xvi). In the foreword to O'Banion's modern

classic, Patricia Cross described how O'Banion offered "a compelling rationale for

focusing the attention of higher education on student learning, i.e., on creating the

'learning college.' .. Community colleges will be bellwether institutions if they adopt

O'Banion's vision for the learning college" (O'Banion, 1997, p. x). In his book,

O'Banion provided "a framework of the reform movements of the past decade and the

emerging focus on learning" (p. xiv). He presented "a new model for education designed

to help students make passionate connections to learning," and he identified six key










principles that formed the emerging definition and character of the learning college .. ."

(p. xiv).

Again, O'Banion (1997) referred to many other critics besides himself (for

example, Cross, Marchese, Daggett, Leonard, Gerstner) who criticized the reform efforts

of the past two decades as "mismatched," "falling short," and even "detrimental."

O'Banion summed up the reform effort prompted by A Nation at Risk in 1983 as the

"spectacular failure" (p. 6). O'Banion concluded that reforms were too focused on "add-

ons or modifications to the current system," and did not address core issues that alluded

to the root cause that the institution itself was the problem. He further described the

reform efforts in the 1980s as "trimming the branches of a dying tree" (O'Banion, p. 7).

In direct response, O'Banion proposed a focused theory for the community college

"based on the assumption that .. the learning college places learning first and provides

educational experiences for learners anyway, anyplace, anytime (p. 47).

O'Banion' s (1997) theory was used as the theoretical framework for this study and

was based "on the assumption that educational experiences are designed for the

convenience of the learners rather than for the convenience of institutions and their

staffs" (p. 47). The focus of this study, that is, the establishment of a consolidated

position on and a holistic approach to what constitutes work-related education, was

anticipated to reveal how a common definition or lack thereof was integrally related to

policy development and funding decisions. The theory of the learning college and its six

principles served as a foundation to understand the process. The principles provided a

framework for developing, evaluating, and synthesizing the complex dynamics that










influence how work-related education could be viewed in higher education and at the

federal, state, and local levels of government.

Six Principles of the Learning College

Again, O'Banion proposed a focused theory for the community college "based on

the assumption that .. the learning college places learning first and provides

educational experiences for learners anyway, anyplace, anytime "'. The framework of the

learning college is based on six key principles (O'Banion, 1997, p. 47):

Principle I: The learning college creates substantive change in individual

learners. This principle was described by O'Banion as "self-evident" and "an embedded

value undergirding all other principles" (p. 48). This principle was integrated into the

research study to determine if work-related education should create substantive change in

its learners. This principle was used to "kindle" (stimulate) new ways of seeing, thinking,

and doing--in dramatic "first" events and new discoveries, and also to "kindle"

(stimulate) new ways of seeing, thinking, and doing--incrementally in day-to-day

experiences.

Principle II: The learning college engages learners as fullpartners in the

learning process, with learners assuminzgprimary responsibility for their own choices.

This principle was described by O'Banion: "A series of services will be initiated to

prepare the learner for the experiences and opportunities to come" (p. 49). This principle

was integrated into the research study to determine if work-related education should

communicate that students are full partners in the creation and implementation of their

learning experiences. This principle was used to determine: if students will assume

primary responsibility for making their own choices about goals and options; if students









should be required to participate in a structured induction/orientation process; and if a

personal learning plan or negotiated contract should be required for students (pp. 49-51).

Principle III: The learning college creates and offers as many options for

learning as possible. This principle was described by O'Banion as "options regarding

time, place, structure, and methods of delivery (p. 52). This principle was integrated into

the research study to determine if work-related education should offer a full array of

options to accommodate individual differences in learning styles, rates, aptitudes, and

prior knowledge and what options should be offered (p. 52). In addition, the research

was augmented by other statements which complemented O'Banion's descriptors. This

research was to determine if work-related educational options should be: "seamless," that

is, not operated in isolation, so students can make reasonable changes in their programs;

"trackless," that is, same beginning courses for several programs so students can explore

before committing to a single track; and/or "classless," that is, similar skills within same

programs at other institutions, thereby providing mobility for students who may change

institutions (Hamm & Mundhenk, 1995, p. 13).

Principle IV: The learning college assists learners to form and participate in

collaborative learning activities. This principle was described by O'Banion as

"transforming the traditional institution ideal of a "community of scholars" into a new

ideal of "community of learners." This principle was integrated into the research study to

determine if work-related education should focus on creating communities among all

participants (students, faculty, and other learning specialists) to: support individual

learning; form and support learning communities in the workplace; establish learning

communities and provide assessment services in the workplace.









Principle V: The learning college defines the roles of learntinzgfacilitators by the

needs of the learners. This principle was framed by O'Banion to submit that "if learners

have varied and individual needs that require special attention, then it follows that the

personnel employed in this enterprise must be selected on the basis of what learners

need" (p. 57). This principle was integrated into the research study to determine if work-

related education personnel should be hired on the basis of department or course needs or

hired based on what learners need. This principle was used to determine if work-related

education students should also participate as learning facilitators, that is, "to capitalize on

the resources students bring, to free professional staff for other roles, and to reduce

personnel costs" (O'Banion, p. 60).

Principle VI: The learning college and its learninzgfacilitators succeed only

when improved and expanded learning can be documented for its learners. This

principle was described by O'Banion as the "framework for documenting outcomes, both

for the learner and for the learning facilitators" (p. 60). This principle was integrated into

the research study to determine: if work-related education should require work-related

educational competencies for entrance or exit; if portfolio assessment should be the

primary means by which work-related learning is documented--should national or state

standards not be available; and if community colleges should employ specialists or

contract to develop "industry-based" standards (similar to health care occupational

programs).

While O'Banion' s (1997) six principles "refer primarily to process and structure,"

they were developed with the "basic philosophy that the student is central in all activities

within the scope of the educational enterprise" (p. 61). Nora (2000) referred to









O'Banion's notes about community colleges providing the "ideal forum" for the learning

college with a caveat that "different practices work differently on different student

populations at different two-year colleges" (A Blueprint of Priorities for Action for

Community Colleges, para. 3). Nora thus acknowledged the potential of this research

direction of whether or not the theoretical framework of the learning college theory and

its six principles could be identified and associated with work-related education.

O'Banion also recognized that "other principles .. must be considered. .. Content,

funding, and governance are examples of key issues that must be addressed and for which

principles must be designed" (p. 61).

This study sought to supplement O'Banion's (1997) principles by examining

additional complementary components of work-related education. Accordingly, it was

anticipated that such a combination of specific, theoretical principles and relevant,

practical components could be identified, categorized, and ranked to derive a common

terminology and definition for work-related education at community colleges. In

addition to O'Banion's six stated principles, components were identified as: mission and

organization; funding; needs assessment and documenting college success; instruction,

programs, and delivery systems; coordination and planning; and national proclamation

and national database for work-related education.

Components of Work-related Education

This section consisted of a review of the relevant literature to identify components

of work-related education which could be considered complementary to the principles of

the learning college (O'Banion, 1997, p. 61). These components were included as touch

points for an expanded agenda to survey participants on the research topic. They were

also to seek convergence of agreement on what principles and components, in total, could









be identified, categorized, and ranked to derive a common definition for work-related

education.

Mission and Organization

Mission and organization were described by Hamm and Mundhenk (1995) in the

form of three questions (p. 4):

1. Does the college's mission statement focus in a significant way on workforce
development?

2. Are those parts of the college that deliver in-service upgrade workforce training and
retraining, as well as noncredit pre-training, explicitly a significant part of the
mission?

3. Are they politically an important part of the organization?

This component was integrated into the research study to determine: if the mission

statement should clearly claim the role of work-related education equal to other mission

tenants; and if work-related education should be politically centrally planned and funded

as an important part of the organization. Gleazer (1968) advised that "when a community

college commits itself to occupational education .. it is affirming an institutional

viewpoint which affects every aspect of its operations (p. 79). Cohen and Brawer (2003)

noted the following issue of merging work-related education on an equal basis with the

collegiate function:

The full effects of vocational education as a primary function have yet to be
discerned. The public' s view of community colleges as agents of upward mobility
for individuals seems to be shifting toward a view of the institutions as
occupational training centers. This narrowing of the colleges' comprehensiveness
could lead to a shift in the pattern of support. (p. 251)

Funding

Funding was described by Hamm and Mundhenk (1995) in terms of community

colleges assessing the viability of workforce development funding by developing









"strategies to modify the priorities of those who do control them" (p. 5), such as the

criteria for funding. They posed two questions (pp. 5-6):

1. Do funding mechanisms acknowledge the centrality of workforce development?

2. Does the college make any efforts to influence funding formulas in order to include
the needs of the emerging workforce as well as instructional innovation?

This component was integrated into the research study to determine if funding formulas

should be influenced to include the needs of the emerging workforce on state, regional,

national, and global basis and/or to include the needs of instructional innovation in work-

related education.

Cohen and Brawer (2003) found that "funds are often secured through priorities

established by state and federal agencies" (p. 233). However, according to Merisotis and

Wolanin (2000), trends identified in 1995-1996 for total institutional revenues for

community colleges indicated significant shifts since 1980 toward external revenue

sources and away from core state and local funding for basic operations.

Merisotis and Wolanin (2000) found:

Since 1980 the fastest growing revenue categories for community colleges have
been government grants and contracts--federal, state, and local programs for
training and research--and private gifts from corporations and individuals. In fact,
as a share of total revenues, these four categories grew from 2 percent in 1980 to 20
percent in 1996, a tenfold increase in less than two decades. At the same time, state
and local appropriations for basic operations fell from 70 percent of total revenues
in 1980 to 50 percent in 1996 (U.S. Department of Education 1980 and 1996).
These revenue trends suggested that the process of financing community colleges
has migrated toward a more private, workforce-oriented education model. As the
focus of community colleges has broadened to include more focused worker
training, resources to pay for this training have increased substantially. (Revenue
Trends, para. 1)

Needs Assessment and Documenting College Success

Needs assessment and documenting college success were described by Hamm and

Mundhenk (1995) in the form of two questions (p. 5):









1. How effectively does the college assess labor market needs or use available local
labor market data?

2. Does the college explicitly measure its success in terms of its contribution to the
development of local and regional economy?


This component was initially integrated into the research study to determine if

community colleges should be the experts at monitoring local labor markets and

collecting data for program planning and needs assessment. The integration of this

component was also to see if success should be measured by work-related education

program, certificate, or degree completion rates, or if success should be measured in

terms of job generation, upgrades, retraining, and economic development. In addition to

completion rates, student enrollments could be interpreted as an element of work-related

education success (Cohen & Brawer, 2003):

The number of students who are already employed and enter vocational programs
only to get additional skills must be factored in, just as the students who obtain job
certifications but find no jobs available to them should be tallied. Students who
leave before completing the programs and enter employment in the field for which
they are prepared must be considered program successes. (p. 235)

Cohen and Brawer (2003) found that "the college staff presumably initiates

programs by perusing employment trends in the local area and surveying employers" (p.

233). In addition, they found that "career program success can also be measured by the

number of students who obtain employment in the field for which they were prepared"

(p. 234). With obtaining employment as the key endpoint of work-related education, it is

a higher risk pathway compared to a liberal arts education. "The costs in tuition and

foregone earnings may be the same for both, but occupational training is almost entirely

wasted if there is no job at the end" (Cohen & Brawer, p. 248). Overall, when









considering needs assessment and documenting college success, Cohen and Brawer

(2003) found:

Because vocational education has several purposes, the measures of success that
can be applied to it vary. It prepares people for specific jobs. How much do
business and industry gain when their workers are trained at public expense? It
assists the disadvantaged and people with disabilities to become self-sufficient.
How much is that worth to society? It aids economic development. How much
does a locality or region gain thereby? It enhances individual income generation
and career mobility. What value has been added, person by person? Indicators of
success and, indirectly, legislation and funding depend on which purpose is being
reviewed. (p. 239)

Instruction, Programs, and Delivery Systems

Instruction, programs, and delivery systems were described by Hamm and

Mundhenk (1995) in terms of whether community colleges were providing only entry-

level job skills or rather "leaming to leamn" skills and an orientation to lifelong learning.

They posed two questions (p. 7):

1. Do programs train for discrete j obs or for j ob clusters?

2. Are the business development and workforce training centers part of the regular
college programming?

This component was initially integrated into the research study to determine if work-

related education should be on entry-level skills or on leaming-to-leam skills.

Orientation to lifelong learning is an example: if work-related education should focus on

preparation for a single job requiring a focused set of skills or on learning skills with

broad application to several similar occupations; or if work-related education should be

on an equal footing with regular college programming or considered as add-on revenue

centers (separate from traditional credit programs).

Cohen and Brawer (2003) viewed the future for work-related education as positive

in that "vocational education will remain prominent. There can be no reversing the










perception that one of the colleges' prime functions is to train workers" (p. 420).

Merisotis and Wolanin (2000) found that:

The rapid evolution of the workforce means that employers are increasingly turning
to community colleges as essential centers of worker training. .. The key question
for community colleges is how to strike a balance between these direct worker
training efforts and general education programs that provide students with broader
skills, such as critical thinking. (Evolving Workforce Needs and Employer
Relationships, para. 1-2)

Staffing

Staffing was described by Hamm and Mundhenk (1995) in terms of whether

community colleges were providing only entry-level job skills or rather "learning to

learn" skills and an orientation to lifelong learning. They posed three questions (p. 8):

1. How do staffing patterns within workforce development areas match patterns in
traditional credit programs?

2. What workplace experience do current instructors, counselors, and administrators
have?

3. What percentage of the college staff serves both traditional and nontraditional
students?

This component was initially integrated into the research study to determine: if

community colleges should establish a staffing pattern of both hard-money and full-time

positions to ensure that work-related education programs receive their fair share

(compared to credit programs); if work-related education faculty and staff need real

workplace experience to communicate effectively with students; if work-related

education advisory committees provide sufficient "real world of work" input to faculty

and staff; if student services and advising should be the same for work-related education

students as it is for traditional credit students; if the student placement office should

primarily focus on identifying career openings and pathways; or if the student placement

office should primarily focus on short-term training for immediate employment.









Cohen and Brawer (2003) found that college staffs established work-related

education programs "by perusing employment trends in the local area and surveying

employers." They also noted that "program coordinators are appointed and advisory

committees composed of trade and employer representatives established" (p. 233).

Coordination and Planning

Coordination and planning were introduced by Hamm and Mundhenk (1995) with

two questions (p. 9):

1. Is the thinking and planning of the college primarily 'local' or 'regional'?

2. Can the college provide data that support regional or national claims about
workforce development?

This component was initially integrated into the research study to determine: if work-

related education programming and planning should take into account regional, national,

and global trends; if community colleges should cooperate with one another to

accomplish regional and national work-related education planning collectively; and if

community colleges should seek to build coalitions and partnerships with other colleges,

organizations, and business to define roles and a vision for work-related education. In

addition, this component engendered the political overtones as suggested by Hamm and

Mundhenk' s statement that "colleges are expected to be sensitive to satisfying the

demands of the district and their governing boards (p. 9).

Vaughn (1994) stated that, to be effective, community college leaders must focus

on establishing political leadership and that "the president ensures that the college's

mission moves in concert with the goals of the community, the state, and when

appropriate, the nation (p. 73). Eaton (1994) made four suggestions about the

presidential role in public policy:









First, presidents must be pivotal in the definition of the issue at hand. Second,
presidents must be strategically positioned to influence both locally based and
state-based constituencies: they have a responsibility to the local community, local
legislators, and the local press, as well as to their colleagues on the same level.
Third, presidents must strengthen their state-level organizations to augment their
individual efforts and build a sense of equity among institutions. Fourth, presidents
need to be willing to take some risks to redesign and restructure the community
college as part of the states' higher education enterprise--when and where
appropriate. (p.124)

Furthermore, Eaton noted that "community college presidents have a great responsibility

in influencing public policy .. presidents need to take risks in planning for the future in

the areas of public policy and college governance" (p. 136).

Finlay, Niven, and Young (1998) found that vocational education and training

(VET) systems were in fact prominent in viewing work-related education on a global

basis:

Many developed and developing nations are looking to their VET systems to
provide a response to changes in the global economy. Our earlier research (Finlay
and Niven 1996) indicated that some countries are proactive with respect to these
changes, adopting long-term strategies that should benefit their economies. (p. 3)

However, Cohen and Brawer (2003) indicated that other industrialized nations

offered few insights of how to reform or improve work-related education in the United

States:

Some countries depend on postsecondary institutions to carry the main burden,
some on schools in the compulsory sector, and others on adult education that is
provided by other than formal educational institutions. .. The greatest proportions
of students in vocational programs in formal postsecondary structures are in Japan,
Germany, France, and Italy (National Center for Education Statistics, 1994f). (p.
241)

National Proclamation and National Database

A national proclamation and a national database for work-related education were

described by Hamm and Mundhenk (1995) in terms of "preparedness" at the local level

and its relationship and visibility at the national level so that community colleges could









"have appropriate influence on policy, funding, legislation, and rule-making" (pp. 15-16)

This component was initially integrated into the research study as statements seeking to

determine: if a national proclamation should be created and promoted which defines the

role of the community college in work-related education; and if a national database

reflecting community college potential and achievement should be created to assist in

identifying limitations and areas of growth and improvement. Brand (Finley et al., 1998)

described the process of change for work-related education in the United States as

"complex, difficult, slow .. possible," which was impacted on a national basis due to the

"decentralized nature" of work-related education in the 50 states. Brand also noted that

"despite federal legislation impasses, governors and state and local officials are taking the

lead in changing their programmes [sic] and simplifying, consolidating and improving

them" (p. 153).

Summary

Work-related education continues to be a topic of interest, as well as a topic of

complexity and/or perplexity. Research about community colleges has produced a

myriad of works on the broad topic of work-related education throughout the history of

higher education in the United States. This topic continues to be viable research today.

The lack of a pointed study and demonstrated efforts to achieve a common definition for

work-related education reflects a gap in current research and practice. This is evident by

the varying terminology and individuality of work-related education programs in the

United States. Specifically, little evidence was found in the literature that directly

addressed the need for or value of a common definition for work-related education. Yet

the numerous and repeated deficiencies have surfaced that appeal to the value of pursuing

a common definition. The literature review offered a basis for this study and future






47


research, which could provide meaningful, replicable information to shape and promote a

common definition for work-related education in community colleges. More specifically,

the literature review served as the foundation upon which the six learning college

principles and other components identified in the literature could be used as a catalyst for

this research study to take place. This research could ultimately contribute to efforts to

derive a common definition for work-related education.















CHAPTER 3
IVETHOD S

The Setting

This study sought to determine if the Delphi technique could be effectively

employed in an educational forum for leaders of community colleges. The purpose

would be to reach levels of agreement and consensus on what principles and components

could be identified to derive a common definition for work-related education.

Epistemologically, the Delphi technique could be derived from constructionism where

the research results and conclusions essentially represent a "shared meaning" based on

the interactive process of the Delphi technique (Stewart, 2001, p. 923). The research

included the use of the Delphi technique to collect data from a panel of experts to

determine if work-related education conformed to the six principles of the learning

college (O'Banion, 1997). Nora (2000) referred to O'Banion's notes about community

colleges providing the "ideal forum" for the learning college with a caveat: "Different

practices work differently on different student populations at different two-year colleges"

(A Blueprint of Priorities for Action for Community Colleges, para. 3). This

acknowledges the potential of this research direction of whether or not the theoretical

framework of the learning college theory and its six principles could be identified and

associated with work-related education.

Furthermore, this study sought to supplement O'Banion' s (1997) principles by

examining additional, complementary components of work-related education. This study

would determine if, in total, qualitative principles and components could be identified,










categorized, and ranked to derive a common terminology and definition for work-related

education at community colleges. The use of qualitative data in educational research was

recognized as important to the study for an understanding of educational phenomena and

testing hypotheses. Qualitative data also provided a natural basis for interpretation with

explanations emerging from intensive examination of the data (Tuckman, 1999).

Linstone and Turoff (1975) summarized that the "Delphi may be characterized as a

method for structuring a group communication process, so that the process is effective in

allowing a group of individuals, as a whole, to deal with complex problems" (p. 3). By

participating in this study, the community college leaders acted as a panel of experts

assisting in the research to derive a common definition for work-related education at

community colleges. Primary and secondary research questions were developed to guide

this study and execute the methods.

Primary Questions

1. Which, if not all, of O'Banion' s six principles of "the learning college" could be
associated with work-related education?

2. What other components could be identified for the work-related education function
at community colleges?

3. What were the most strongly advocated principles and components supporting
work-related education?

Secondary Questions

Additionally, secondary research questions were identified that could be answered

as a result of this study. These questions were addressed based on the compilation of

answers to the primary research questions.

1. Could a selected group of community college leaders reach consensus, using a
Delphi technique, on what principles and components could be identified to derive
a common definition for work-related education?









2. Could meaningful relationships be confirmed between the six principles and the
identified components to derive a common definition of work-related education?

This research study evolved into four stages:

First Stage

The first stage consisted of a review of the literature pertaining to community

colleges and the role they fulfill in preparing the workforce. In conjunction with the

literature review, a theoretical framework was identified in past literature which focused

on the six principles outlined in O'Banion' s model of the learning college (O'Banion,

1997). Additionally, specific work-related education components were identified based

on work contained in a National Council for Occupational Education monograph (Hamm

& Mundhenk, 1995, pp. 4-9). These components were included in the survey to

complement O'Banion's theory for those areas not addressed: O'Banion advised that

these components were "key issues that must be addressed and for which principles must

be designed (O'Banion, p. 61).

Second Stage

This stage consisted of developing an initial mixed methods Delphi technique

survey of Likert scale items, open-ended items, and open-ended comment blocks to

assimilate data from the panel of experts. These data were based on the principles and

components initially identified. This initial identification process by the researcher

ensured that the three-round "limit" of the Delphi technique was maintained by

complying with the following three procedures (Linstone & Turoff, 1975, p. 88):

1. The monitor team [researcher] devoting a considerable amount of time to carefully
pre-formulating the obvious issues;

2. Seeding the list with an initial range of options but allowing the respondents to add
to the lists;









3. Asking for positions on an item and underlying assumptions in the first round.

This initial survey design ensured that all the "obvious" statements and issues had

been included to the extent possible and that participants were "being asked to supply the

more subtle aspects" pertaining to each topic (p. 88). The initial survey was juried by a

community college vice president with responsibilities for work-related education, a

psychometric analyst, and two institutional researchers.

The open-ended comment blocks allowed participants to suggest additions,

deletions, or changes in the wording of statements, which were then introduced as new

items. These new items were developed by the researcher after a careful distillation of all

the qualitative responses, as presented in the appendix. Linstone and Turoff (1975) also

found that the ratings on items were sensitive to wording, and because of this property,

the material can mushroom in size after the first round:

If the respondents feel strongly about the issues, and this should be the case, they
will generate a large amount of written material. If they are provided a certain
number of items to deal with on the first round then each of them will make
approximately the same number of written comments or additions in response.
These must be abstracted carefully and duplications among the respondents
eliminated. (pp. 92-93)

Third Stage

The Delphi technique was conducted in a series of three rounds to facilitate a

detailed critical examination and structured communication process to focus attention on

the problem. Both frequencies and the group's optional qualitative responses were

shared with the participants in Rounds Two and Three. They supported the convergence

of agreement on which principles and components could be identified to derive a

common definition for work-related education. Surveys were offered in either an online

electronic format or hardcopy format. The online web format was the first choice by 100










percent or 20 out of 20 total participants. Participants self-identified so the researcher

could verify receipt of each participant' s survey and facilitate any follow-up as required.

Fourth Stage

The fourth stage consisted of the final compilation and reporting of the data, the

analysis of the data, and the presentation of the results. Both quantitative and qualitative

analyses were provided as assimilated from the panel of experts. The researcher used the

results and analyses to draw conclusions and bring forward recommendations for future

research.

The Participants

A specific group of community college leaders were identified to comprise the

panel of experts. These leaders were identified as the colleges' "chief executive officers

(CEOs), academic affairs officers, business/industry liaison officers, continuing

education officers, or occupational education officers" (American Association of

Community Colleges [AACC], 2004, p. 3) at the 20 colleges listed in Table 1-2 whose

CEOS comprised the League for Innovation in the Community College Board of

Directors. The League for Innovation "is an international organization dedicated to

catalyzing the community college movement" (League for Innovation in the Community

College [League], 2004, About the League, preface).

Twenty (19 in the United States and 1 in Canada) "CEOs from some of the most

influential, resourceful, and dynamic community colleges and districts in the world

comprise the League's board of directors" (League, para. 3) The League has more than

700 member institutions from 10 different countries and has partnerships with more than

100 corporations. The list of the League' s Board of Directors was posted at:

http://www.1eague. org/l eague/ab out/b oard_of directors. htm. Some institutions were









organized with a chancellor as CEO with multiple presidents; other institutions had a

president as the CEO. By requesting the personal participation from each CEO plus his

primary administrator responsible for work-related education, the total possible sample

size consisted of 40 possible participants of which 20 community college leaders agreed

to participate in the study.

The participants were recruited by a personal communication signed by the

president and a president emeritus, respectively, of one of the colleges which was

represented as one of the members of the League' s Board of Directors. The personal

communication explained the Delphi technique and specifically requested a commitment

from all the CEOs to personally participate in the study along with the primary

administrators who were responsible for work-related education at their colleges. The

topic of study and the time involved to participate were communicated to the invitees.

All CEOs received a postcard to return confirming their interest to participate in this

research study plus requesting their electronic mail addresses for communication. The

postcard also provided: space to confirm contact information (name, position, electronic

mail address) for all the administrators who would participate in the research study; the

choice of completing the surveys using the online web format; and the choice of

receiving an electronic copy of the finished research study.

The researcher obtained informed consent by way of a follow-up letter once interest

was confirmed by the return postcard. The informed consent document was mailed to all

voluntary participants. All participants were informed that they would not have to

answer any question they did not wish to answer. After the primary investigator received

a signed copy of the informed consent by mail, the first-round survey was announced via









electronic mail with the survey's corresponding web link. The participants were

informed that only the researcher would have access to the data and the self-identifiers

were to be removed during final analysis. The participants were also advised that their

identities would be kept confidential to the extent provided by law and their identity

would not be revealed in the final dissertation. All participants were advised that there

were no anticipated risks, compensation, or other direct benefits as a participant in this

interview. The participants were advised that they were free to withdraw their consent to

participate and could discontinue their participation at any time without consequence.

The gender proportion was 65 percent male and 35 percent female. Nine or 45 percent

were CEOs, and the other 11 or 55 percent were primary administrators responsible for

work-related education at their colleges. Three of the participants were between 46 and

50 years old. Nine were between 51 and 55 years old. Five were between 56 and 60

years old. Two were between 61 and 65 years old, and one was 66 years old or older.

Tasks and Materials

The Delphi technique was the method chosen by which the panel of experts

provided levels of agreement, opinions, and beliefs regarding a specific set of statements

in three successive rounds of surveys. The most successful studies are the result of three

rounds of data collection as "three rounds proved sufficient to attain stability in the

responses; further rounds tended to show very little change and excessive repetition was

unacceptable to participants (Linstone & Turoff, 1975, p. 229). The researcher compiled

the data, tested for validity, and reported back to the panel between each round. The

panel was asked to respond to the statements in each round, as well as provide optional,

qualitative comments. The data were then collected and analyzed with additions,

deletions, and refinements, as deemed appropriate from the responses and quantitative










analyses. The survey content was the task because all participants received the same

instructions and experienced the same activities. The content and task were therefore

constant for all participants (Tuckman, 1999).

The Delphi technique followed the procedures of using self-reported data. The

procedure used for establishing the Delphi technique was to jury the proposed instrument.

The instrument was juried by a community college vice president with responsibilities for

work-related education, a psychometric analyst, and two institutional researchers. The

instrument was juried on the basis of their reading comprehension of the statements, ease

of completion, length of time required to complete the survey, and the manageability of

the web-based technology.

The sample group was offered a choice of a mailed hardcopy or an electronic copy

of the surveys. All the participants chose to respond by accessing an electronic copy of

the survey, and they were provided links via electronic mail to the web-based electronic

surveys. No one chose to participate using a hardcopy survey.

The sample group responded to the initial survey, Round One, and the survey data

were compiled with the item frequencies/percentages redistributed to the sample group.

All responses to the open-ended questions in Round One were compiled and provided

anonymously in an attached file. New and revised statements were developed from the

responses and added to Round Two.

In the second survey, Round Two, with frequencies posted for the items,

participants were asked to consider the frequencies and to rate each of the Likert scale

survey items in an effort to seek convergence of agreement on the items. In addition,

participants were advised to provide their underlying reasons for any statements) with









which they may have taken exception with the converging group view. Open-ended

blocks were provided after each set of statements, which pertained to each specific

principle and component for participants, to give their reasons for any exceptions.

The data were assimilated a second time, making note of changes that occurred

since Round One. The item data were compiled with the item frequencies/percentages

redistributed to the sample group. As in Round One, the responses to the open-ended

questions in Round Two were compiled and provided anonymously in an attached Eile

and once again distributed to the participants as a Einal survey, Round Three, via

electronic means. Participants were given a Einal opportunity to validate or revise the

data previously submitted. Participants were asked to justify their responses, particularly

in cases where their responses differed from the maj ority of the participants. At this

point, the data were then organized according to the statistical assumptions. This last

group of data were compiled and analyzed. The data between Round One and Round

Two were analyzed, as were the data between Round Two and Round Three, to

determine the effects of the responses on the statements in the subsequent surveys.

The iterative process of the Delphi technique allowed for data collection to be

implemented in a non-threatening manner. The group was afforded opportunities to

change its ratings in subsequent rounds. The group could reach further levels of

agreement on specific principles and components, which could be modeled to derive a

common definition for work-related education.

General Operational Design

The study employed Internet-based survey research to examine if and how

community college leaders could reach agreement using a series of surveys to aggregate

their knowledge, judgments, and opinions as a panel of experts. The panel of experts was










comprised of community college CEOs and their respective administrators responsible

for work-related education in order to address the complex question of how to define

work-related education. The panel of experts communicated their knowledge and

experience through a three-round iterative process. They used the Delphi technique to

reach agreement on which principles, components, and other aspects could be identified,

prioritized, and applied to a common terminology and definition for work-related

education. Differential outcomes were considered based on the participants'

position/capacity at their colleges and characteristics of their colleges.

The operational design represented multi-factor repeated measures and longitudinal

research with three levels labeled as Round One, Round Two, and Round Three (e.g.,

repeated measures: time-1, time-2, time-3). The design included the total score, the

principles' score, and the components' score factors whereby the participants (CEOs and

administrators) responded to the research statements. They then reached levels of

agreement through the iterative process of the Delphi technique.

Data Collection

Round One

Once interest was confirmed by the voluntary participants, the informed consent

document was mailed to all participants advising them that the web link to Round One

would be electronically mailed upon receipt of the signed informed consent letter. The

web link to Round One was electronically mailed to all participants. The introductory

electronic mail to the first round survey welcomed the participants to the study and

explained the general procedures to follow. All participants were asked to read the

rationale, directions, and instructions before attempting to complete their surveys. Upon










opening the web link, each participant was asked to self-identify with the first Hyve letters

of the last name for the researcher to verify receipt of survey results.

The Round One survey consisted of three sections. The first two sections

contained 46 statements, which served as a task to be completed by each participant. The

third section contained demographic survey items. The first section of 21 statements

pertained to work-related education and O'Banion' s (1997) six principles of the learning

college. The second section of 25 statements contained other key issues, areas of focus,

and components of work-related education, as were initially identified in the literature

review. A Likert scale was placed immediately below each statement. The Likert scale

was self-explanatory. Participants were asked to rate each statement by checking the

perceived agreement with each statement. The rating scale had as the lower anchor,

"Strongly Disagree," and was assigned a negative three (-3) value, and the higher anchor,

"Strongly Agree," was assigned a positive three (+ 3). The options were coded as:

"Strongly Disagree" = -3; "Disagree" = -2; "Slightly Disagree" = -1; "Slightly Agree" =

+1; "Agree" = +2; and "Strongly Agree" = +3. A neutral option or response choice of

"undecided," "no opinion," "uncertain," or "don't know" was intentionally left out. It was

feasible to use a response scale with an even number of responses and no middle, neutral,

or undecided choice because most participants, as experts, had an opinion and

corresponding level of agreement for the items. Otherwise, as per the informed consent,

it was clear to the participants that they did not have to respond to every statement. But

they could take a "no judgment" view, which is a practice commonly applied to Delphi

studies (Linstone & Turoff, 1975, p. 90; Adler & Sainsbury, 1996, p. 188).









After each set of statements pertaining to a specific principle or component

subsection, participants had an opportunity to write optional statements of opinion or

comment, if they so desired. The second section was followed by a final, open-ended

question as an opportunity to list any other items that the participants believed would

contribute to a common definition for work-related education. The intent of this final

question was to encourage reflection beyond the initial survey with an additional purpose

to provide points of reference. These points of reference would assist with the data

analysis where lack of agreement or consensus existed. The third section of Round One

consisted of three demographic questions pertaining to current work capacity, gender,

and age.

Six days after the initial notification was sent via electronic mail, self-identified

survey results were reconciled to the list of confirmed participants. A generic reminder

request was sent via blind copy electronic mail to those who had not yet completed

Round One. Verification was made on an every-other-day basis until all expected

responses had been received up to nine days from the original notification. On the ninth

day, individual personalized electronic mails were sent to the remaining six participants

who had yet to complete Round One. Six of the seven participants responded by the 13th

day from initial notification. The 20th participant made contact and advised technical

difficulty in accessing the web survey. That participant was advised and subsequently

responded to Round One.

Survey results for the 20 participants were downloaded into spreadsheet and text

files. Likert scale items and demographic information were analyzed statistically using

Excel, SPSS, and SAS programs. Open-ended comments and opinions were interpreted










as subj ective information, which had characteristics relevant to the research questions.

This subj ective information was developed and aggregated as revised with additional

survey items for new statements in Round Two. Six statements were revised and

replaced, 16 new statements added, and in total the number of statements increased from

46 to 62.

Round Two

The Round Two survey was electronically mailed following the analysis of Round

One, posting of frequencies, and aggregation of revised and new statements. The

electronic mail for the second round survey thanked the participants for their support of

the study and explained the general procedures to follow for Round Two. All

participants were asked to read the rationale, directions, and instructions before

attempting to complete their surveys. Upon opening the web link, all participants were

asked to self-identify with the first five letters of the last name for the researcher to verify

receipt of survey results. Round Two consisted primarily of 40 out of 46 of the same

items in Round One plus the open-ended comments and opinions, which were developed

and aggregated as 22 revised and additional survey items. A Likert scale was placed

immediately below each statement. All the anonymous open-ended responses and

comments were contained in a Microsoft Word document file, which was also provided

via an attachment to the electronic mailing. For those statements which did not change,

the frequencies/percentages were placed in front of the respective Likert scale options.

The frequencies identified the relative position of the group consensus in relation to the

Likert scale options for ease of understanding and reevaluation for convergence of

agreement.










During Round Two, each participant was given an opportunity to re-rate each of

the 40 original statements with knowledge of the group's convergence of agreement. It

was explained that although consensus was desirable, they should not have felt compelled

to re-rate according to the group's ratings. However, participants were advised that if

they differed markedly to the group's ratings, they should have given careful reappraisal

to those particular statements. As in Round One, the Likert scale was self-explanatory.

Participants were asked to rate each statement by annotating their perceived agreement

with each statement. The rating scale had as the lower anchor, "Strongly Disagree," and

was assigned a negative three (-3) value, and the higher anchor, "Strongly Agree," was

assigned a positive three (+ 3). The options were coded as: "Strongly Disagree" = -3;

"Disagree" = -2; "Slightly Disagree" = -1; "Slightly Agree" = +1; "Agree" = +2; and

"Strongly Agree" = +3. A neutral option or response choice of "undecided," "no

opinion," "uncertain," or "don't know" was intentionally left out and deemed reasonable

as virtually all participants had an opinion and corresponding level of agreement for the

items. Otherwise, as per the informed consent, it was clear to the participants that they

did not have to respond to every question. But the participants could take a "no

judgment" view, which is a practice commonly applied to Delphi studies (Linstone &

Turoff, 1975, p. 90; Adler & Sainsbury, 1996, p. 188). Again, participants were advised

to provide their underlying reasons for any statements) with which they may have taken

exception with the converging group view. An open-ended block was provided after the

two individual principle and component subsections for participants to provide their

reasons for any exceptions. The third section of Round Two consisted of: three










classification questions pertaining to the organization; size; location of the participant' s

college or district; and one demographic question pertaining to current work capacity.

One week after the initial notification was sent via electronic mail, self-identified

survey results were reconciled to the list of confirmed participants and a generic reminder

request was sent via blind copy electronic mail to those who had not yet completed

Round Two. Verification was made on an every-other-day basis until all expected

responses had been received up to 11 days from the original notification. On the 11Ith

day, individual personalized electronic mails were sent to the remaining six participants

who had yet to complete Round Two. Four of the six participants responded by the 22nd

day, and one participant responded by the 29th day from the initial notification of Round

Two.

Survey results for the 19 participants were downloaded into spreadsheet and text

files. Likert scale items and demographic information were analyzed statistically using

Excel, SPSS, and SAS programs. Open-ended comments and opinions were interpreted

as subj ective information, which had characteristics relevant to the research questions.

This subj ective information was developed and aggregated as revised and additional

survey items for new statements in Round Three. The data obtained from Round Two

were analyzed using descriptive statistics. A criterion was set so that any statement not

scoring an overall positive mean was excluded from Round Three. Six statements not

achieving the criterion were eliminated. In addition, one statement was revised into two

statements, and these two new statements were added. In total, the number of statements

decreased from 62 to 59.









Round Three

The Round Three survey was electronically mailed following the analysis of Round

Two. The electronic mail for the Round Three survey thanked the participants for their

support of the study and explained the general procedures to follow for Round Three.

Each participant was asked to read the rationale, directions, and instructions before

attempting to complete their survey. Upon opening the web link, each participant was

asked to self-identify with the first five letters of the last name for the researcher to verify

receipt of survey results. Round Three contained the statements from Round Two--less

the six statements not scoring an overall positive mean--plus the new and revised

statements. A Likert scale was placed immediately below each statement. Frequencies

for all statements in the form of percentages or levels of agreement were placed in front

of the respective Likert scale options. The frequencies highlighted the relative position of

the group consensus in relation to the Likert scale options for ease of understanding and

reevaluation for convergence of agreement. A Microsoft Word document file containing

all the anonymous open-ended responses and comments was also provided via an

attachment to the electronic mailing.

With Round Three, each participant was given a final opportunity to re-rate each

statement with knowledge of the group's decision. It was explained that although

consensus was desirable, the participants should not have felt compelled to rate according

to the group's rating. However, participants were advised that if they differed markedly

to the mean rating, they should have given careful reappraisal to that statement. In

addition, participants were given the opportunity to explain their reasons for their ratings

on Round Three, but were not compelled to do this.










As in Round One and Round Two, the Likert scale was self-explanatory.

Participants were asked to rate each statement by annotating their perceived agreement

with each statement. The rating scale had as the lower anchor, "Strongly Disagree," and

was assigned a negative three (-3) value, and the higher anchor, "Strongly Agree," was

assigned a positive three (+ 3). The options were coded as: "Strongly Disagree" = -3;

"Disagree" = -2; "Slightly Disagree" = -1; "Slightly Agree" = +1; "Agree" = +2; and

"Strongly Agree" = +3. A neutral option or response choice of "undecided," "no

opinion," "uncertain," or "don't know" was intentionally left out and deemed reasonable

because virtually all participants had an opinion and corresponding level of agreement for

the items. Otherwise, as per the informed consent, it was clear to the participants that

they did not have to respond to every question, but they could take a "no judgment" view,

which is a practice commonly applied to Delphi studies (Linstone & Turoff, 1975, p. 90;

Adler & Sainsbury, 1996, p. 188). Again, participants were advised to provide their

underlying reasons for any statements) with which they may have taken exception with

the converging group view. An open-ended block was provided after each of the two

sections for participants to provide their reasons for any exceptions. The third section of

Round Three consisted of one demographic question pertaining to the current work

capacity of the participant.

One week after the initial notification was sent via electronic mail, the self-

identified survey results were reconciled to the list of confirmed participants, and a

generic reminder request was sent via blind copy electronic mail to those who had not yet

completed Round Three. Verifieation was made on a daily basis until all expected

responses had been received up to two weeks from the original notification. Individual,










personalized electronic mails were sent to the remaining five participants who had yet to

complete Round Two. Verification was made on a daily basis until all expected

responses had been received. Two of the remaining five participants responded by the

17th day from initial notification of Round Three.

Communication Process

During the course of the study, telephone contacts were initiated, and numerous

mailings and electronic mail messages were received and sent. Only work phone

numbers, work addresses, and electronic mail accounts were accessed. The researcher

offered both work and mobile phone numbers along with an electronic mail address to

facilitate frequent and cordial contact with the participants to achieve the maximum

response rates for the three Delphi rounds. Personal communications in the form of

thank-you cards were sent to every participant after the closure of Round Three. Those

participants who requested an electronic copy of the final study were advised that copies

would be forthcoming after the successful defense of the dissertation and its subsequent

publication.

Data Management and Statistical Procedures

The data from the surveys were analyzed after each round was downloaded. Data

were managed and entered into a spreadsheet denoting individual responses to each

statement and open-ended questions. Specific comments and responses to the open-

ended questions, which were downloaded into a spreadsheet, were subsequently

transferred to a Word document for qualitative analysis. The use of qualitative data in

educational research is recognized as important to the study and understanding of

educational phenomena, testing hypotheses, and providing a natural basis for

interpretation with explanations emerging from intensive examination of the data










(Tuckman, 1999). Sample characteristics included: the number of participants (n) and

demographics, which included each participant's position at the college; his gender; his

age; and classification data on his respective college, which included the organization

(number of campuses), size (number of students), and location of the college/district

(urban, suburban, or rural).

The data analysis and statistical procedures that were employed by the researcher

focused on the primary purpose of this study: to test if work-related education conformed

to the six principles of the learning college (O'Banion, 1997). The other purposes of this

were to see if other complementary components of work-related education--in total,

principles and components--could be identified, categorized, and ranked to derive a

common terminology and definition for work-related education at community colleges.

This study tested the application of the Delphi technique survey method to determine if

the Delphi technique could be effectively applied to an educational forum for leaders of

community colleges. The purpose would be to achieve consensus and support rationale

for establishing a consolidated position on and a holistic approach to what constitutes

work-related education. As such, by focusing on the purpose of reaching consensus,

levels of agreement, and further identifying issues pertaining to work-related education,

this study did not address whether or not the "extreme" answer was the "correct" answer.

Based on the research questions guiding the study, the right answer would instead be

determined by whether or not the experts reached consensus and levels of agreement on

the various constructs.

Reliability of the Instrument

Reliability refers to the consistency of such measurements when the testing

procedure is repeated on a population of individuals or groups (American Psychological










Association, American Educational Research Association, & National Council on

Measurement in Education [APA, AERA, NCME], 1999). Reliability also refers to the

extent to which the responses are free of measurement error. As such, the responses

should be the same every time the measurement is repeated on the same group, sample,

or population. To achieve reliable results, the scale and instrument were constructed so

as to minimize random error in responses. The study focused on the proportion of the

experts who responded to item stems (statements) according to the scale scores. That

Rounds Two and Three of the Delphi afforded the experts an opportunity to change their

initial ratings in light of the new information further ensured that the results could be

used for well-founded conclusions.

Validity of the Instrument

"Validity" refers to the appropriateness of use and the proposed interpretation of

the scores for a given purpose under a prescribed set of conditions. Validity is the most

fundamental consideration in developing and evaluating the extent to which an

instrument is doing what it is supposed to do. Crocker and Algina (1986) refer to

Cronbach's description of "validation as the process by which a test developer or test user

collects evidence to support the types of inferences that are to be drawn from test scores"

(p. 217). Validation begins with an explicit statement about the proposed interpretation

of the scores. There is no single all-inclusive form of validity. Validity is instead a

matter of degree with types of evidence adding weight to validity, described as content,

criterion-related, or construct validity. These three types of evidence are only

conceptually independent, and rarely is just one of them important in a particular

situation.









The types of evidence describe the extent to which the data obtained are

systematically representative of the true state of affairs, and they describe if the

assessment items give information about what the items were intended to provide

(Penfield, 2003). Content validity describes how well the content of the scale matches

the content domain intended to be measured by the scale. In other words, it makes

human judgments about whether or not the content of the items covers the maj or facets

related to the knowledge areas. Content validity addresses features of the test, not the

scores. In fact, content validation often occurs before scores are even obtained. Crocker

and Algina (1986, p. 218) outlined the following steps for content validation:

1. Defining the performance domain of interest;

2. Selecting a panel of qualified experts in the content domain;

3. Providing a structured framework for the process of matching items to the
performance domain; and

4. Collecting and summarizing the data from the matching process.

Content validity was essentially "built-in" with the juried "expert review" survey

and also with each round of the Delphi technique. This was by virtue of the development

of the content of the scale matching the content domain, as conveyed by the experts'

responses and what they considered to be the constructs of interest. Other types of

evidence related to content validity existed, such as face validity, which is really not as

antidotal as perceived by some (that is, does it look professional, serious, worth taking,

and so forth). Face validity was appropriate in this case as a type of evidence in which

items appeared to measure a construct that was meaningful to laypersons and may have

served to motivate participants "to perform their best since the instrument appears to

measure a meaningful construct" (Crocker & Algina, 1986, p. 223). Criterion-related









validity pertains to the accuracy of decisions linked to the validity of the scores. For the

purposes of this study, these two secondary research questions were asked:

1. Could a selected group of community college leaders reach consensus, using a
Delphi technique, on what principles and components could be identified to derive
a common definition for work-related education?

2. Could meaningful relationships be confirmed between the six principles and the
identified components to derive a common definition of work-related education?

Construct validity was used to determine whether or not the items of the scale

measure the constructs they are supposed to measure. Construct validity addresses the

degree to which scores represent the unobservable trait operationalized through the items.

Internal validity claims were met by following established procedure for the Delphi

technique to answer inferential questions about the scores and further define and distill

the data to well-founded conclusions. It would have been most difficult, if not

impossible, to incorporate a comparison group into the Delphi research design to

establish certainty of the instrument. External validity was dependent on the selection of

the experts as a representative body, whose scores may or may not be generalized to all

community colleges in a particular sample, group, or the population.

Analytical Procedures

Descriptive statistics, t-tests, the Duncan's multiple-range test, and the parametric

correlation were all statistical procedures and tests used to examine the data in this study.

The data in each phase were analyzed in terms of the statements' means, standard

deviations, and confidence intervals, in addition to the participants' responses to open-

ended questions for each item. The iterative structure of the Delphi technique developed

what the panel of experts identified as the content domain and what they considered to be

the constructs of interest. Again, the participants' responses to open-ended questions









were given considerable weight for revising, deleting, and adding statements in the

subsequent rounds. Negative means gave an indication of how individual statements did

not support agreement of specific principles or components and their lack of association

with work-related education. After Round Two, those statements, which did not meet the

criterion of an overall positive mean, were excluded from Round Three. This criterion

acknowledged that participants did not endorse specific statements as descriptive of the

research questions and constructs of interest.

Descriptive Statistics

Descriptive statistics, which are those statistics used to best answer questions about

or describe the parameter of interest, were analyzed for every item over the participants,

which included the measures of central tendency-- mean, median, mode--plus confidence

intervals, standard deviation, frequencies, including cumulative frequencies.

Penfield (2003) defined the mean as "a common method of obtaining a

representative value for the average of a group of scores" (p. 72). Tuckman (1999)

defined the median as "the score in the middle of a distribution: 50 percent of the scores

fall above it, and 50 percent fall below it:" (p. 288). Furthermore, "The median defines

the middle of the distribution and is not as sensitive to extreme scores as is the mean"

(Tuckman, 1999, p. 289). The mode is a measure of central tendency, which describes

the most frequently occurring observation. No calculation is involved to identify the

mode. It may not exist or there could be more than one modal value, which suggests a

bi-modal observation, that is, the indication of two separate distributions.

Penfield (2003) described a confidence interval as an interval that has a certain

level of confidence of containing the value of interest (p. 1 10). The lower and upper

bounds of the confidence interval are useful in specifying with a certain degree of










confidence, such as 95 percent confident, that the interval around the sample mean is

expected to contain the population mean (Crocker & Algina, 1986, p. 433). The measure

of dispersion of the participants' responses used in this study was the standard deviation.

Penfield (2003) defined the standard deviation as "a measure of the amount of spread in a

group of scores, which equals the typical distance that each score in the group lies from

the group mean" (p. 72). Frequencies and cumulative frequencies were used to verify

and analyze the Likert scale data. The frequencies for the statements in the form of

percentages were provided to the participants for comparability and convergence of

agreement between Round One and Round Two and between Round Two and Round

Three .

Parametric Statistical Tests

A large amount of the quantitative portion was descriptive due to the nature of the

Delphi technique and driven by the small number of participants (n). An independent t-

test was used as an investigation of any differences between the two means for CEOs and

other administrators. This was to ascertain the probability that any difference between

them reflected a real difference between the groups of participants rather than a chance

variation in the data (Tuckman, 1999, p. 300). The t-test was an effective tool for

predicting any statistical differences between the means for the scores (total score and the

individual scores for the principles and the components) as the dependent variables from

the independent variables (CEOs and administrators). An analysis of variance could not

be used for factor analysis since the participant sample size (n) in the study was too small

to permit a meaningful factor analysis. Three primary questions were posed:

1. Did the average total score depend upon the j ob capacity of participant (CEOs
versus other administrators)?









2. Did the average score for the principles of the learning college subsection depend
upon the j ob capacity of participant (CEOs versus other administrators)?

3. Did the average score for the other identified components subsection depend upon
the j ob capacity of participant (CEOs versus other administrators)?

As a follow-up to the independent t-tests, a Duncan's test (Duncan's multiple range

test) was employed as a type of multiple comparison test used to make pair-wise

comparisons of means that are not significantly different among themselves. The

Duncan's test provided output that was essentially a "picture" of which pairs of means

were significantly different as a post hoc test. The Duncan's test was employed to

determine if statistical differences existed among the average total scores with any

statistical significance between Round One and Round Two, between Round Two and

Round Three, and between Round One and Round Three. Correlation coefficients were

used to confirm statistical significance for the principles of the learning college and the

components identified, as pertaining to work-related education. A correlation matrix was

constructed in which the correlation between every pair of variables was computed. Then

the variables were organized into a matrix to facilitate inspection and comparison of each

for significance.

Summary

The aggregate data from all three rounds were collected, compiled, and the

participants' comments were noted after each round. The aggregate data and comments

were shared with the participants for review and reflection when they received each

subsequent round. Participants had the opportunity to change their minds at any time

during Round Two or Round Three to allow time to reflect on an issue. After the Einal

data were compiled, the quantitative analysis of data was completed using a statistical

program (SAS). The qualitative analysis took into consideration the compilation of









comments made throughout the collection process. The Delphi technique was

recognized as an appropriate study design and assessment to make important decisions

about educational policy (Clayton, 1997, p. 373). Internal validity was a function of

developing an appropriate survey instrument and administering the surveys over a three-

round iterative process, while compiling and redistributing the aggregate data to the

participants after each round. External validity was a function of determining if the

results obtained answered the research questions and further the process of deriving a

common definition for work-related education. External validity was also viewed in

terms of whether or not a common definition would extend beyond the sample and apply

to a larger sample of the population. When designing a research project, the two

principle types of validity, internal and external, must be balanced to obtain conclusive

results (internally), but they must still represent a more global reality (externally) for the

results to be generalized to other groups, a larger sample, or the population (Tuckman,

1999, p.10). Detailed reporting of the data analysis is contained in Chapter 4.















CHAPTER 4
PRESENTATION AND ANALYSIS OF DATA

Introduction

The primary purpose of this study was to test if work-related education conformed

to the six principles of "the learning college" theory (O'Banion, 1997, pp. 47-61).

Furthermore, this study would supplement O'Banion's learning college principles by

examining additional complementary components of work-related education. The

purpose would be to determine if in total the principles and components could be

identified, categorized, and ranked to derive a common definition for work-related

education at community colleges. O'Banion recognized that "there are certainly other

principles that must be considered. .. Content, funding, and governance are examples of

key issues that must be addressed and for which principles must be designed" (p. 61).

Included in the study were the learning college principles, components identified in the

existing literature, and other related components, which were developed by participants

through the iterative process of the Delphi technique. This study tested the application of

the Delphi technique to determine if it could be effectively applied to an educational

forum for leaders of community colleges to achieve consensus and levels of agreement.

This in turn could support a rationale for establishing a consolidated position on and a

holistic approach to what constitutes work-related education. Such a consolidated

position could facilitate clarity and consistency in policymaking at the federal, state, and

local levels. By participating in this study, the community college leaders served as a










panel of experts assisting in the research to derive a common definition for work-related

education at community colleges.

Specifically, this study sought answers to primary and secondary research

questions, which were developed to guide this study and execute the methods:

Primary Questions

1. Which, if not all, of O'Banion' s six principles of "the learning college" could be
associated with work-related education?

2. What other components could be identified for the work-related education function
at community colleges?

3. What were the most strongly advocated principles and components supporting
work-related education?

Secondary Questions

Additionally, secondary research questions were identified that could be answered

as a result of this study. These questions were answered through a compilation of the

answers to the primary research questions.

1. Could a selected group of community college leaders reach consensus, using a
Delphi technique, on what principles and components could be identified to derive
a common definition for work-related education?

2. Could meaningful relationships be confirmed between the six principles and the
identified components to derive a common definition of work-related education?

Results of the Delphi Technique

This primarily qualitative, mixed-methods study was designed to be experimental.

The Delphi technique was used to survey the perceptions of community college CEOs

and administrators responsible for work-related education. They served as a panel of

experts to confirm levels of agreement on those principles and components from which a

common definition for work-related education could be derived. The initial survey was

juried by a community college vice president with responsibilities for work-related









education, a psychometric analyst, and two institutional researchers. Revisions to the

format and content were made based on the recommendations of the jury. One-hundred

percent of the participants chose to do the survey electronically, and they were provided

the web-based electronic surveys. No one chose to participate using hardcopy surveys.

The participants' results from the three-round Delphi technique were collected through an

online service based on an open-source proj ect, which originated at Virginia Tech

(http://www. opensource.isc.vt. edu/products/survey/) The online service was titled:

Survey A web-ba~sed survey tool, and was available via the College of Education website

at the University of Florida. When each round was completed, the researcher

downloaded the survey responses data into a pop-up window and saved the raw data as a

text file. The survey data in the text file were exported to Microsoft Excel via the

"import external data" function for the initial analysis and data coding.

Quantitative statistical analysis tools were applied to each statement in each round,

including the measures of central tendency--mean, median, mode--plus confidence

intervals, standard deviation, frequencies, including cumulative frequencies. Goldstein

(1975) identified "the means, standard deviations, percentage distributions" as

appropriate descriptive statistics used in Delphi research studies and these statistics for

each statement in each round were included in this report (Linstone & Turoff, 1975, p.

222). Each statement was analyzed using Excel and SAS programs to provide empirical

data to alleviate concerns regarding the use of the Delphi technique. Each statement also

addressed concerns about inconclusive evidence, which was subsequently supported by

statistical data analysis.










The qualitative responses were collected through the use of open-ended comment

boxes on the web-based survey tool. The qualitative responses data were simultaneously

downloaded into a pop-up window along with the survey raw data and saved as a text

file. The qualitative data in the text file were exported from Microsoft Excel to Microsoft

Word for aggregation, analysis, and presentation in the Appendix. Additionally in Round

One, one participant electronically mailed expanded reflections regarding the principles,

components, and perceptions. These comments were included in the qualitative data

response results in the Appendix.

Selection and Confirmation of Participants

The participants in this study were all current employees of public community

colleges. Twenty community college CEOs and 20 of their administrators, respectively,

for a total of 40 community college leaders, were invited to participate in the study.

These community college leaders were defined as the experts on work-related education

within a population of public community colleges. The sample included the colleges'

"chief executive officers (CEOs), academic affairs officers, business/industry liaison

officers, continuing education officers, or occupational education officers" (American

Association of Community Colleges [AACC], 2004, p. 3) at the 20 colleges listed in

Table 1-2 whose CEOS comprised the League for Innovation in the Community College

Board of Directors. Out of the 40 potential participants, 20 agreed to participate.

According to Clayton (1997), the group size may vary with "15-30 people for a

homogeneous .. and 5-10 people for a heterogeneous population," which is an

acceptable number of participants for using a Delphi technique in educational studies (p.

8). Ziglio (1996) noted that "good results can be obtained even with small panels of 10-

15 individuals" (p. 14). The final group of 20 participants provided a stratified sample of









CEOs and administrators from varied community college classifications of organization,

size, and location (Community College Survey of Student Engagement [CCSSE], 2005).

Eighteen participants were from the United States and two were from Canada. The

American participants' colleges represented a geographically diverse sample. The

geographic diversity was demonstrated by membership in five out of the six regional

accrediting organizations, as recognized by the Council for Higher Education

Accreditation (CHEA) and the U.S. Department of Education (USDE). The 18

participants at American colleges included:

* Seven from the North Central Association of Colleges and Schools
* Five from the Southern Association of Colleges and Schools
* Three from the Middle States Commission on Higher Education
* Two from the Western Association of Schools and Colleges
* One from the Northwest Commission on Colleges and Universities

The participants represented colleges which were 42 percent urban and 58 percent

suburban; no colleges were considered rural. Eighty-nine percent of the participants

were employed at colleges with extra large enrollments (15,000 or more students); only

11 percent of the participants were employed at colleges with a large enrollment (8,000 to

14,999 students); no colleges were considered medium or small, e.g. 4,500 to 7,999 or

fewer than 4,999 students). In terms of organizational size, 58 percent were from a multi-

campus organization, 37 percent were from a single campus, and the remaining 5 percent

portrayed a single member college in a multi-college system. The gender proportion was

65 percent male and 35 percent female. Nine or 45 percent were CEOs, and the other 11

or 55 percent were primary administrators responsible for work-related education at their

colleges. Three of the participants were between 46 and 50 years old. Nine were









between 51 and 55 years old. Five were between 56 and 60 years old. Two were

between 61 and 65 years old, and one was 66 years old or older.

Response Rates to Delphi Surveys

The response rate for Round One was 100 percent. All 20 of the community

college leaders (nine CEOs and 11 administrators) who agreed to participate, actually

completed Round One. An initial response rate of more than two-thirds was considered

high for a Delphi study and showed significant interest on the part of the panel of experts

(Jillson, 1975, p. 132). Considering the intensity of the schedules of these top community

college leaders, this researcher anticipated that not all participants would complete all

rounds, and this was the case in this study. The response rate for Round Two was 85

percent. Seventeen out of the 20 who agreed to participate, (7 CEOs and 10

administrators) actually completed Round Two. The response rate for Round Three was

75 percent. Fifteen out of the 20 who agreed to participate (six CEOs and nine

administrators) actually completed Round Three. In all three rounds, all or 100 percent

were able to respond through the web-based survey format. The aggregate response rate

for the three rounds was acceptable considering that Round Three's sample size was 15

with "15-30 people for a homogeneous population" as an acceptable sample size for

using a Delphi technique in educational studies (Clayton, 1997, p. 8). Ziglio (1996)

noted that "good results can be obtained even with small panels of 10-15 individuals" (p.

14). A review of other Delphi research (Linstone & Turoff, 1975; Jillson, 1975) and

Delphi-based dissertations (Smith, 1975; Nemr, 1977; Lewis, 1984) revealed a reduction

in response rates from the first to the final round, "particularly those involving voluntary

participation" (Jillson, p. 132).









Verification of the Accuracy of the Delphi Technique

The Delphi technique was used as "a communications structure aimed at producing

a detailed critical examination and discussion" with certain quantification of the

participants' viewpoints (Turoff & Hiltz, 1996, p. 56-57). This communication process

required the participants, serving as a panel of experts, to rate statements in a series of

iterative surveys to quickly identify levels of agreement and disagreement. The groups'

levels of agreement for each round were shared in subsequent surveys (Round Two and

Round Three). The response rating frequencies or percentages for each statement, which

were integrated in the subsequent surveys, focused on which principles and components

could be identified, categorized, and ranked to derive a common definition for work-

related education. To verify the accuracy in using the Delphi technique for this research

study and to present the results of the three-round process in a comparative depiction,

response ratings from all three iterative rounds were presented in a round-by-round

evaluation of the data with the appropriate descriptive statistics. These response ratings

frequencies, means, and standard deviations by survey statement were reported for each

of the six principles of the learning college and for each of the seven identified

components of work-related education, as summarized and presented in Table 4-5.

The researcher shared the groups' viewpoints after each round with the participants

for self-comparison and as a point of reference for considering the other participants'

views. The frequencies reported were based on the aggregate of the total number of

responses and their corresponding ratings in each particular round. In addition to

providing the frequencies, the qualitative responses were collected through use of open-

ended comment boxes on the web-based survey tool. These qualitative responses were

also shared in the subsequent rounds (Round Two and Round Three), as summarized and










presented in the Appendix. The Delphi technique and these procedures offered each

participant an opportunity to reconsider his position in light of the groups' views in

addition to considering new items that were introduced. That the second and third rounds

of the Delphi survey technique afforded the panel of experts these opportunities to

change their ratings in light of "new information" further ensured that the results could be

used for well-founded conclusions. Content validity was essentially "built-in" with each

round of the Delphi technique. Content validity was verified by virtue of the

development of the content of the scale matching the content domain, as conveyed by the

panel's expert responses and what the participants considered to be the constructs of

interest. The merger of a Delphi technique, the research procedures, and the web-based

process verified and promoted reaching a superior group view of the task at hand through

the phenomenon "collective intelligence" (Turoff & Hiltz, 1996, p. 80).

Round One Results

Twenty participants rated each of 46 statements according to the following scale:

"Strongly Disagree," "Disagree," "Slightly Disagree," "Slightly Agree," "Agree," or

"Strongly Agree." Any absence of a rating for a statement was classified as "no

judgment." As in each round of the Delphi, participants were asked to complete three

sections. Instructions and statements were provided for each section as a task to be

completed. After each subsection for a principle or component, the participants were

asked to list their reactions, initial thoughts, comments, and/or recommended changes to

any statements pertaining to the specific principle or component. The participants who

rated the eighth statement either "Agree" or "Strongly Agree" for "work-related

education should offer a full array of options to accommodate individual differences in

learning styles, rates, aptitudes, and prior knowledge" were asked to respond to an open-









ended question, "what options should be offered." At the end of survey's first and

second sections (before demographics data section), the participants were asked to list

any other principles, components, changes in service delivery, and innovative ways of

thinking which they believed would contribute to a common definition for work-related

education. Finally, the participants were asked to respond to a third section which

requested demographic data, as detailed earlier in this chapter. The percentages, means,

and standard deviations reported in Table 4-5 were based on an aggregate of the total

number of participants who responded by round and the corresponding group ratings.

Not all percentages equated to 100 percent due to fractional rounding or in those cases

where participants chose not to respond (classified as "no judgment") to specific

statements. Open-ended comments and opinions were interpreted as subjective

information, which had characteristics relevant to the research questions. This subj ective

information was developed and aggregated as revised with additional survey items for

new statements in Round Two. Six statements were revised and replaced, 16 new

statements added, and in total the number of statements increased from 46 to 62. At the

conclusion of Round One, the participants were electronically mailed the web link to the

follow-up Round Two survey. The follow-up survey contained revisions based on the

panel of experts' recommendations along with the qualitative data and comments

compilation from Round One. The information in the Appendix reflects the raw

qualitative responses and comments from Round One which were analyzed to refine the

survey instrument for Round Three. The information in Table 4-5 reflects the levels of

agreement for each statement in each round, as rated by the panel of experts.









Round Two Results

Seventeen participants rated each of the 62 statements in Round Two according to

the following scale: "Strongly Disagree," "Disagree," "Slightly Disagree," "Slightly

Agree," "Agree," or "Strongly Agree." Round Two consisted primarily of 40 out of 46

of the same items in Round One, which were presented with the participants' aggregate

frequencies or percentages from Round One for the Likert scale responses. In addition,

the open-ended comments and opinions were analyzed from which 22 revised and

additional survey items were developed and aggregated into Round Two. Additionally,

an absence of any rating for a statement was classified as "no judgment." As in each

round of the Delphi, participants were asked to complete three sections. Instructions and

statements were provided for each section as a task to be completed. After each section

for the principles and the components, the participants were asked to provide their

underlying reasons for any statements with which they may have taken exception with

the converging group view. These statements may have pertained to a specific principle

or component, as reported in the Appendix. Finally, the participants responded to a third

section, which requested demographic and college classification data, as detailed earlier

in this chapter. The percentages reported in Table 4-5 were based on an aggregate of the

total number of participants who responded by round and the corresponding group

ratings. Not all percentages equated to 100 percent due to fractional rounding or in those

cases where participants chose not to respond (classified as "no judgment") to specific

statements. Open-ended comments and opinions were interpreted as subjective

information, which had characteristics relevant to the research questions. This subj ective

information formed the basis for revised and additional survey items which were

aggregated as new statements in Round Three. The data obtained from Round Two were










analyzed using descriptive statistics. A criterion was set so that any statement not scoring

an overall positive mean was excluded from Round Three. Six statements not achieving

the criterion were eliminated. In addition, one statement was revised into two statements,

and these two new statements were added. In total, the number of statements decreased

from 62 to 59. At the conclusion of Round Two, the participants were electronically

mailed the web link to the follow-up Round Three survey. The follow-up survey

contained revisions based on the panel of experts' recommendations along with the

qualitative data and comments compilation from Round Two. The information in the

Appendix reflects the raw qualitative responses and comments from Round Two, which

were analyzed to refine the survey instrument for Round Three. The information in

Table 4-5 reflects the levels of agreement for each statement in each round, as rated by

the panel of experts.

Round Three Results

Fifteen participants rated a possible 59 statements according to the following scale:

"Strongly Disagree," "Disagree," "Slightly Disagree," "Slightly Agree," "Agree," or

"Strongly Agree." Each round of the Delphi technique requested the participants to

complete three sections. Instructions and statements were provided for each section as a

task to be completed. Round Three contained the statements from Round Two less the

six statements not scoring an overall positive mean. Round Three also contained revised

and new statements based on qualitative inputs. Thirty-five statements carried over from

Rounds One and Two. As in the two previous rounds, open-ended comments and

opinions were interpreted as subj ective information, which had characteristics relevant to

the research questions. For Round Three, a criterion was set so that any statement not

scoring an overall positive mean in Round Two was excluded from Round Three. Six









statements not achieving the criterion were eliminated. Again, this subjective

information was developed and aggregated as revised and new statements in Round

Three. From the subjective information, one statement was revised into two, and two

new statements were added. In total, the number of statements decreased from 62 to 59.

The data obtained from Round Three were analyzed using descriptive statistics.

Additionally, an absence of any rating for a statement was classified as "no judgment."

An open-ended comment block was provided after each of the two maj or sections for the

principles and the components. Participants were given an opportunity to provide their

underlying reasons for any statements with which they may have taken exception with

the converging group view. These statements pertained to any specific principles or

components. The percentages reported were based on an aggregate of the total number of

participants who responded by round and the corresponding group ratings. Not all

percentages equated to 100 percent due to fractional rounding or in those cases where

participants chose not to respond (classified as "no judgment") to specific statements.

The information in the Appendix from Round Three reflects the participants' reasons for

taking exception with the groups' views and further general comment about the work-

related education topic. The information in Table 4-5 reflects the levels of agreement for

each statement in each round as rated by the panel of experts.

Differences in Responses by Subgroups and Rounds

An independent t-test was used as an investigation of any differences between the

two means for CEOs and other administrators. The t-test was to ascertain the probability

that any difference between them reflected a real difference between the groups of

participants rather than a chance variation in the data (Tuckman, 1999, p. 300). The t-test

was an effective tool for predicting any statistical differences between the means for the










scores (total score and the individual scores for the principles and the components) as the

dependent variables from the independent variables (CEOs and administrators). An

analysis of variance could not be used for factor analysis since the participant sample size

(n) in the study was too small to permit a meaningful factor analysis. Three primary

questions were posed:

1. Did the average total score depend upon the j ob capacity of participant (CEOs
versus other administrators)?

2. Did the average score for the principles of the learning college subsection depend
upon the j ob capacity of participant (CEOs versus other administrators)?

3. Did the average score for the other identified components subsection depend upon
the j ob capacity of participant (CEOs versus other administrators)?

As noted in the Tables 4-1, 4-2, 4-3, statistical differences between the CEO scores and

other administrator scores were not present.

Table 4-1 T-test of Average Total, Principles, and Components Scores for CEOs and
Administrators during Round One
Average Scores CEO Admini strators
Total Score 71.3 64.9
Principles Score 37.6 31.2
Components Score 33.8 33.7
* Statistical differences between CEO scores and other administrator scores were not present at p< 0.05.

Table 4-2 T-test of Average Total, Principles, and Components Scores for CEOs and
Administrators during Round Two
Average Scores CEO Admini strators
Total Score 103.4 100.9
Principles Score 55.4 52.1
Components Score 48.0 48.8
* Statistical differences between CEO scores and other administrator scores were not present at p< 0.05.

Table 4-3 T-test of Average Total, Principles, and Components Scores for CEOs and
Administrators during Round Three
Average Scores CEO Admini strators
Total Score 131.0 120.6
Principles Score 69.8 63.3
Components Score 61.2 57.4
* Statistical differences between CEO scores and other administrator scores were not present at p< 0.05.










Given the absence of statistical significant differences between CEO scores and

other administrator scores, as presented in Tables 4-1, 4-2, 4-3, another means test was

run as a follow-up on the entire group between Rounds One, Two, and Three. The means

test conducted on the entire group was the Duncan's test (Duncan's multiple range test),

which, as a type of multiple comparison tests, was used to make pair-wise comparisons of

means that were not significantly different between each other. The Duncan's test, as

presented in Table 4-4, provided output that was essentially a "picture" of which pairs of

means were significantly different as a post hoc test.

Table 4-4 Duncan's Multiple Range Test of Scores between All Three Rounds
Round One Round Two Round Three
Total Score 60.7* 66.6 72.7*
Principles Score 26.4* 29.6 32.9*
Components Score 34.3 36.9 39.8
* Denotes statistical significance at p< 0.05.

The Duncan's test was employed to determine if statistical differences existed among the

average total scores with any statistical significance between Round One and Round

Two, Round Two and Round Three, and Round One and Round Three. Statistical

differences were not evident between Round One and Round Two or between Round

Two and Round Three. However, statistical significance was evident in average total

scores between the first and third rounds. This significance indicated that measurable

progress was made in reaching consensus from the first round to the final results.

Table 4-5 Descriptive Statistics of Statements Compared Across All Three Rounds

Section A: Work-related Education and the Learning College

According to O'Banion (1997), "The learning college places learning first and provides
educational experiences for learners anyway, anyplace, anytime."
Please consider if work-related education has a place in the learning college and how
each of the following six principles may or may not apply to work-related educational
experiences.









Table 4-5 Continued

1. Work-related education should create substantive change in its learners.


Round One
5%
0%
0%
0%
50%
45%
2.200
1.323


Round Two
0%
0%
6%
0%
59%
35%
2.176
0.951


Round Three
0%
0%
0%
13%
47%
40%
2.267
0.704


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation


2. Work-related education should "kindle" (stimulate) new ways of seeing, thinking, and
doing--in dramatic "first" events and new discoveries.


Round One
5%
0%
5%
5%
40%
45%
1.522
0.667


Round Two
0%
0%
0%
6%
41%
53%
2.471
0.624


Round Three
0%
0%
0%
7%
20%
73%
2.667
0.617


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation


3. Work-related education should "kindle" (stimulate) new ways of seeing, thinking, and
doing--incrementally in day-to-day experiences.


Round One
0%
0%
0%
5%
60%
35%
2.300
0.571


Round Two
0%
0%
0%
6%
71%
24%
2.176
0.529


Round Three
0%
0%
0%
7%
60%
33%
2.267
0.594


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation


Principle II: According to O'Banion, "The learning college engages learners as full
partners in the learning process, with learners assuming primary responsibility for their
own choices."









Table 4-5 Continued

4. Work-related education should communicate that students are full (and active) partners
in the creation and implementation of their learning experiences.


Round One
0%
5%
5%
0%
30%
60%
2.250
1.372


Round Two
0%
0%
0%
6%
18%
76%
2.706
0.588


Round Three
0%
0%
0%
0%
20%
80%
2.800
0.414


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation


5. Work-related education should communicate that students will assume primary
responsibility for making their own choices about goals and options.


Round One
0%
5%
5%
5%
40%
45%
2.050
1.356


Round Two
0%
0%
0%
6%
35%
59%
2.529
0.624


Round Three
0%
0%
0%
6%
33%
67%
2.667
0.448


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation


6. Work-related education should require students to participate in a structured
induction/orientation process.


Round One
10%
5%
15%
15%
30%
25%
0.950
2.038


Round Two


Round Three


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation










Table 4-5 Continued

6.1. Work-related education orientation should be tailored to the individual learner--some
begin after a single point of engagement, while others may continue orientation for a few
days or a few weeks.


Round One
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a


Round Two
0%
6%
6%
18%
47%
24%
1.647
1.367


Round Three
0%
0%
13%
7%
47%
20%
1.600
1.298


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation


6.2. Work-related education orientation should offer many formats (flexible times, on-
site/workplace, group, one-on-one, self-guided, mentoring, on-line, etc.).


Round One


Round Two
0%
0%
0%
18%
29%
53%
2.353
0.786


Round Three
0%
0%
0%
7%
13%
73%
2.533
0.915


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation


7. Work-related education should require a personal learning plan or negotiated contract
for students.


Round One
5%
5%
5%
10%
60%
15%
-0.950
0.999


Round Two
0%
0%
0%
12%
82%
6%
1.941
0.429


Round Three
0%
0%
0%
0%
100%
0%
2.000
0.000


Strongly Disagree
Disagree
Slightly Disagree
Slightly Agree
Agree
Strongly Agree
Mean
Standard Deviation