Facilitated Peer Review Committee Report, 2012-2013

MISSING IMAGE

Material Information

Title:
Facilitated Peer Review Committee Report, 2012-2013
Abbreviated Title:
Facilitated Peer Review Committee Documents from August 2013
Physical Description:
Documentation
Language:
English
Creator:
Facilitated Peer Review Committee
Acord, Sophia
Dorsey, Alan
Jefferson, Rebecca
Simpson, Betsy
Taylor, Laurie N.
Telg, Ricky
Publisher:
George A. Smathers Libraries, University of Florida
Place of Publication:
Gainesville, FL
Publication Date:

Subjects

Subjects / Keywords:
Peer Review
Digital Scholarship
Digital Publishing
Scholarly Communications

Notes

Abstract:
Committee, description of responsibilities from the charge: The Facilitated Peer Review Committee has been established with the approval of the Dean of the Libraries to support the peer review process for innovative and/or nontraditional forms of scholarship and scholarly work, which is particularly needed for new works of digital scholarship. The need for this process arose with a digital scholarship project created by two faculty members in the Libraries, and this project will serve as the pilot project for the Committee. The Committee will establish the requirements for the facilitated peer review process – with the committee acting akin to an editorial board that would facilitate, but not conduct, peer review – specifically for innovative forms of scholarship. This process is critically needed across many fields to support scholarly efforts in multimedia publication, data curation, publicly-engaged scholarship, collaborative work, and other forms of alternative scholarly products that do not fit within established disciplinary traditions. The Committee will establish the norms, standards, and documentation for the facilitated peer review process. The Committee will refine these materials and apply them to the approved digital scholarship project which will serve as a pilot project. The Committee will document the application of the process to the pilot project for peer review of that project and for that project to serve as a model for facilitated peer review for other scholarly works as well as being a model for possible application within other Colleges after the process is established for the pilot within the Libraries. At this time, establishing the process with the facilitating committee, working through the full pilot project, and documenting the entire process is expected to require up to 5 hours per month for each committee member, primarily for reviewing and refining draft documentation on this process.
General Note:
PDF versions of Report with and without documentation.

Record Information

Source Institution:
University of Florida
Holding Location:
University of Florida
Rights Management:

The author dedicated the work to the Commons by waiving all of his or her rights to the work worldwide under copyright law and all related or neighboring legal rights he or she had in the work, to the extent allowable by law.
System ID:
AA00016553:00006


This item is only available as the following downloads:


Full Text

PAGE 1

Page 1 of 11

PAGE 2

Page 2 of 11 With Gratitude: The F acilitated P eer R eview C ommittee members thank the pilot project scholars Brian Keith and Bonnie Smith, and expert peer reviewers Emily Backe and Kim Burhop, for their review s upporting both peer review of the pilot project and improving the whole of the facilitated peer review process. Thanks go to the c ommittee members for the time, insight, thoughtful reflection, and engaged discussion in making possible th e pilot and supporting future processes Facilitated Peer Review C ommittee Members : Sophia Acord, Associate Director for the Center for Humanities and the Public Sphere, College of Liberal Arts & S ciences Alan Dorsey, Professor of Physics and Associate Dean, College of Liberal Arts & Sciences (1/2012 4/2012) Rebecca Jefferson, Librarian for the Isser and Rae Price Library of Judaica, George A. Smathers Libraries Betsy Simpson, Chair of the Catalog ing and Metadata Department, George A. Smathers Libraries Laurie Taylor, Digital Humanities Librarian, George A. Smathers Libraries (Chair) Ricky Telg, Professor of Agricultural Education and Communication, IFAS

PAGE 3

Page 3 of 11 Table of Contents Page Executive Summary 4 Overview and Committee Activities 6 Findings and Recommendations 8 Supporting Documents for the Process and the Pilot Project 11

PAGE 4

Page 4 of 11 Executive Summary The University of Florida George A. Smathers Libraries created the Facilitated Peer Review Committee in 2012 to establish a facilitated pee r review process to support peer review of alternative scholarly works for the entire campus and to do so specifically in relation to the tenure and promotion process. Such a process emerged as a critical need across many fields to support scholarly efforts in multimedia publication, data curation, publicly engaged scholarship, collaborative works, and other forms of alternative scholarly products that do not fit within established disciplinary traditions. In order to support the se needs, the c ommittee activities and accomplishments for 2012 2013 in clude creating the facilitated peer review process with supporting documentation, and test ing the process with sup p orting a pilot project. As currently designed, t he Facilitated Peer Review Committee is akin to members of a journal staff editors in providing the framework for peer review, or the program officers for a granting a gency that supports and facilitates the expe rts who are conducting peer review. T he c ommitte e establishes and refines the necessary policies, procedures, and documentation; locates the appropriate experts to conduct peer review; and supports the process of peer review as performed by experts. Peer r eview is conducted by experts in the field. Committee members had rich backgrounds and experiences that informed and provided particular insights that enabled creating this process. These backgrounds included familiarity with administrative procedure, aw areness of and involvement with alternative scholarly works, expertise from service as peer reviewers (for journals, conferences, grant proposals, and other scholarly forms), and expertise for the tenure and promotion process at UF with service on tenure a nd promotion committees During th e past year, the c ommittee developed the process and documentation with all materials available on the c ommittee website. After creating the process, the c ommittee supported the facilitated peer review of an alternative scholarly work for the pilot project. The c ommittee found the pilot to be successful in terms of supporting the needs of individual scholars, institutional review and process needs, and the needs associated with peer review for improvement and validation f or scholarly rigor. The scholars and peer reviews for the pilot project also provided positive feedback on the process. Based on the activities and accomplishments to date, the c ommittee members recommend continuing the Facilitated Peer Review Committee S everal current co mmittee members will rotate off and new members will need to be appointed. After the new members are appointed and oriented, the c ommittee will need to refine the process with terms of service for members and will revise the current charge appropriately As with the initial c ommittee, a minimum of five and preferably six members are recommended and a similar mix of diverse backgrounds and expertise areas should be represented on the committee for expertise and experience in

PAGE 5

Page 5 of 11 alternative scholarly works, administrative processes, and peer review for different venues and types of scholarly work I t is recommended that all members commit to approximately five hours per month for the committee with the c ommitte e to develop new guidelines and recommendations if the need results in a demand that exceeds this time commitment With the new members and these agreed upo n parameters, the committee would then be available to support UF scholars by providing facilitated peer review of alternative scholarly works as needed and serving as experts to refer and recommend other resources and venues when appropriate For the next stage, the current committee specifically recommends working through another project submission from a non library app licant(s) as well as work i n: 1) focusing on a particular type of alternative work ; and/ or, 2) creating informational resources about and tracking types of alternative work for use in inspiring and informing others

PAGE 6

Page 6 of 11 Overview and Committee Activities Overview The Fac ilitated Peer Review Committee w as established in 2012 with the approval of the Dean of the Libraries to support the peer review process for innovative and/or nontraditional forms of scholarship and schol arly work, which is particularly needed for new works of digital scholarship. This process is critically needed across many fields to support scholarly efforts in multimedia publication, data curation, publicly engaged scholarship, collaborative work, and other forms of alternative scholarly products that do not fit within established disciplinary traditions. The need to support such a process at UF arose with a digital scholarship project created by two faculty members in the L ibraries, and that project serv e d as the pilot project for the c ommittee. Potential committee members were approached based on their rich backgrounds and experiences that could inform and provide insights to best develop this process. The final committee members who agree d to serve collectively offered a wide varied of expert knowledge and diverse backgrounds which included familiarity with administrative procedure, awareness of and involvement with alternative scholarly works, expertise from service as peer reviewers (for journals, conferences, grant proposals, and other scholarly forms), and expertise for the tenure and promotion process at UF with service on tenure and promotion committees From 2012 2013, the committee establish ed the requirements for the facilitated pe er review process with the committee acting akin to an editorial board that would facilitate, but not conduct, peer review specifically for innovative forms of scholarship. The committee establish ed the norms, standards, and documentation for the f aci litated peer review process, and refined and applied the process for the pilot project. The committee activities required approximately 5 hours per m onth for each committee member, with additi onal time required of the chair. The committee used this time for developing, reviewing, and refining documentation and for facilitating peer review for new projects For the pilot project, the scholars contributed time to complete the application for peer review and to revise the application based on feedback from t he committee. The peer reviewers contributed their time by serving as reviewers in reviewing the alternative scholarly work, completing the peer review evaluation form, and providing feedback to the committee on the process. Committee Activities In all activities the committee reaffirmed the importance of supporting alternative scholarly works in a manner that upholds peer review and is in alignment with individual and institutional needs The committee engaged in creating the facilitated peer review pr ocess to support and

PAGE 7

Page 7 of 11 uphold the sacredness of peer review as a process for fair evaluation that respects diversities 1 with the needs of individual authors and institutional units in mind in terms of measurement, communicability, and reporting. The committe e designed the process to mirror traditional peer review (following models of peer review for publications and tenure and promotion and grant awards) as closely as possible with all materials peer reviewed in accordance with already established standards o f excellence. For the operational aspects, t he majority of the committee's activities were conducted over email T he primary c ommittee activities were developing and refining the process documentation throughout the year The committee create d documentation ( available on the committ ee website and with this report) as the necessary documentation to support the process and this includes documentation to provide context for the process for scholars, peer reviewers, and committee members. The commit tee created : Peer review guide for peer reviewers and submitting authors Application for authors to request peer review of alternative scholarly works Letter template to peer reviewers explaining the facilitated peer review process and requesting peer re view Peer review form for use by peer reviewers Th e documents were initially developed from February November 2012. In November 2012, the scholars for the pilot project used the documentation and submitted their application to request peer review. The com mittee reviewed the application and discussed questions with the scholars from November 2012 through January 2013. In January 2013, the scholars submitted a revised application, which the committee approved to move to the peer reviewer stage. By March, the peer reviewers completed the reviews. The committee reviewed the responses and developed the final response to scholars, sent in May 2013. The final committee activity in drafting this report T he committee members, scholars, and peer reviewers provided p roductiv e feedback on the facilitated peer review process All provided substantive engagement and feedback that clarified and improve d the process. In addition to the committee focus on th e immediate, local needs at UF, the committee members were also contacted and discussed the process with colleagues at other institutions who expressed great interest in the process. These conversations provided additional p roductive fe edback and informed the process 1 See Michle Lamont, How Professors Think (Harvard University Press: 2009), for a discussion of peer review as a standardized and ritualized process wherein peer review becomes and is sacred.

PAGE 8

Page 8 of 11 Findings and Recommendations The committee was successfully created with members with rich backgrounds and experiences that made it possible to create this process. The committee successfully accomplished its goals in developing and testing the facilitated peer review process for the pilot project. The research done as part of creating the process confirmed the lack of alternatives and the clear need for this process for specific, immediate needs and for the larger activities needed in creating support for alternative scholarly works. The high level of engagement with the scholars for the pilot project greatly enriched the resulting project documentation which embodies the overall findings. One example of the pilot project informing the overall process occurred for the use of equivalencies. From committee discussions, early findings and recommendations emer ged, including that the use of equivalencies Equivalences have been found problematic for innovative research ; yet equivalences were d eemed useful for the facilitated peer review process to ensure support for individual scholars and institutional needs. The scholars and peer reviewers found equivalencies useful, although noting that equivalencies do require a som ewhat awkward process to establish the equivalency where the alternative scholarly work must be mapped to an established mode of scholarly production. The pilot project was particularly informative for this process, with the scholars recommending that the work be equivalent to two journal articles. This mapping for type and impact as quantity was informative and led to a productive discussion and process refinement to best utilize equivalencies Based on the activities and accomplishments to date for the Facilitated Peer Review Committee for 2012 2013 the committee members specifically recommend: C ontinuing the Facilitated Peer Review Committee with changes in membership. Changing the membership so that s everal current committee members will rotate off and new members will be appointed. Framing n ew committee activities to include: o o Refining the process to add terms of service for members o Refining the process to add guidelines for membership O perations of the initial committee provide one model with: a minimum of five and preferably six members ; and, recommended member commit ment to approximately f ive hours per month for the c ommittee with the committee to develop new guidelines and recommendations if the need results in a demand that exceeds this time commitment.

PAGE 9

Page 9 of 11 Activities of the initial committee showed the critical importance of backgrounds and expertise in relevant areas for members, 2 and representation should continue for critical areas while also seeking members who can augment and provide additio nal diversity in terms of backgrounds and experience to further inform the process and future concerns. o Supporting appropriat e, concrete activities for the immediate next phase: Identifying and working through another project submission with project from a non library app licant(s) Performing work that does either or both of the following: focuses activities and develops resources on a particular type of alternative work, and/or creates informational resources about and track types of alternative work for use in inspiring and informing others o Supporting appropriate activities for the immediate next phase and beyond: Refini ng all processes and documentation with new member input and committee discussion S upport ing the UF campus with the facilitated peer review process for alternative scholarly works as needed Supporting the UF campus with expert refer ral and recommend ation to appropriate resources and venues when available, for pee r review in addition to or instead of the facilitated peer review process D isseminating information about the process to c ampus Reviewing all processes and documentation with supporting the facilitated peer review process for additional projects changing and refining the processes and documentation to best support all Reporting regular ly with reports publicly accessible to support review, improvement, and new opportunities Additional recommendations from the pilot project may arise over the course of the next year. One of the scholars for the pilot project is going through the promotion process in 2013 2014. It is expected that discussions with current committee members, the scholar, and the review committee will occur and will cover the pilot project review and the promotion review process Discussing the processes together should provide information that can inform changes, new discussions and new directions for the facilitated peer review process. In addition to these specific recommendations, the committee recognizes that feedback and discussion on all aspects of process for the pilot project are needed, and that feedback from many 2 Including administrative p rocedure, awareness of and involvement with alternative scholarly works, expertise from service as peer reviewers (for journals, conferences, grant proposals, and other scholarly forms), and expertise for the tenure and promotion process at UF with service on tenure and promotion committees.

PAGE 10

Page 10 of 11 perspectives will best suppo recommends seeking out additional feedback and engagement from interested parties as time allows.

PAGE 11

Page 11 of 11 Supporting Documents Th e facilitated peer review process is evolving and this report serves to document the process to date. C urrent versions of all supporting documents are available onlin e through the committee website. 3 C opies of the supporting documents are also included with this report. Committee and process documents : Charge for the F acilitated Peer Review Committee Peer Review Form for Alternative Scholarly Works Application for Facilitated Peer Review Support for Alternative Scholarly Works Letter template with specifics for pilot project Peer Review Guide for Peer Reviewers and Sub mitting Authors P ilot project documents : Application to request peer review, completed by the scholars Peer Review Forms, one from each of the two peer reviewers Letter by the committee synthesizing the evaluations from the peer reviewers 3 http://www.uflib.ufl.edu/committees/fprc



PAGE 1

Page 1 of 11

PAGE 2

Page 2 of 11 With Gratitude: The F acilitated P eer R eview C ommittee members thank the pilot project scholars Brian Keith and Bonnie Smith, and expert peer reviewers Emily Backe and Kim Burhop, for their review s upporting both peer review of the pilot project and improving the whole of the facilitated peer review process. Thanks go to the c ommittee members for the time, insight, thoughtful reflection, and engaged discussion in making possible th e pilot and supporting future processes Facilitated Peer Review C ommittee Members : Sophia Acord, Associate Director for the Center for Humanities and the Public Sphere, College of Liberal Arts & S ciences Alan Dorsey, Professor of Physics and Associate Dean, College of Liberal Arts & Sciences (1/2012 4/2012) Rebecca Jefferson, Librarian for the Isser and Rae Price Library of Judaica, George A. Smathers Libraries Betsy Simpson, Chair of the Catalog ing and Metadata Department, George A. Smathers Libraries Laurie Taylor, Digital Humanities Librarian, George A. Smathers Libraries (Chair) Ricky Telg, Professor of Agricultural Education and Communication, IFAS

PAGE 3

Page 3 of 11 Table of Contents Page Executive Summary 4 Overview and Committee Activities 6 Findings and Recommendations 8 Supporting Documents for the Process and the Pilot Project 11

PAGE 4

Page 4 of 11 Executive Summary The University of Florida George A. Smathers Libraries created the Facilitated Peer Review Committee in 2012 to establish a facilitated pee r review process to support peer review of alternative scholarly works for the entire campus and to do so specifically in relation to the tenure and promotion process. Such a process emerged as a critical need across many fields to support scholarly efforts in multimedia publication, data curation, publicly engaged scholarship, collaborative works, and other forms of alternative scholarly products that do not fit within established disciplinary traditions. In order to support the se needs, the c ommittee activities and accomplishments for 2012 2013 in clude creating the facilitated peer review process with supporting documentation, and test ing the process with sup p orting a pilot project. As currently designed, t he Facilitated Peer Review Committee is akin to members of a journal staff editors in providing the framework for peer review, or the program officers for a granting a gency that supports and facilitates the expe rts who are conducting peer review. T he c ommitte e establishes and refines the necessary policies, procedures, and documentation; locates the appropriate experts to conduct peer review; and supports the process of peer review as performed by experts. Peer r eview is conducted by experts in the field. Committee members had rich backgrounds and experiences that informed and provided particular insights that enabled creating this process. These backgrounds included familiarity with administrative procedure, aw areness of and involvement with alternative scholarly works, expertise from service as peer reviewers (for journals, conferences, grant proposals, and other scholarly forms), and expertise for the tenure and promotion process at UF with service on tenure a nd promotion committees During th e past year, the c ommittee developed the process and documentation with all materials available on the c ommittee website. After creating the process, the c ommittee supported the facilitated peer review of an alternative scholarly work for the pilot project. The c ommittee found the pilot to be successful in terms of supporting the needs of individual scholars, institutional review and process needs, and the needs associated with peer review for improvement and validation f or scholarly rigor. The scholars and peer reviews for the pilot project also provided positive feedback on the process. Based on the activities and accomplishments to date, the c ommittee members recommend continuing the Facilitated Peer Review Committee S everal current co mmittee members will rotate off and new members will need to be appointed. After the new members are appointed and oriented, the c ommittee will need to refine the process with terms of service for members and will revise the current charge appropriately As with the initial c ommittee, a minimum of five and preferably six members are recommended and a similar mix of diverse backgrounds and expertise areas should be represented on the committee for expertise and experience in

PAGE 5

Page 5 of 11 alternative scholarly works, administrative processes, and peer review for different venues and types of scholarly work I t is recommended that all members commit to approximately five hours per month for the committee with the c ommitte e to develop new guidelines and recommendations if the need results in a demand that exceeds this time commitment With the new members and these agreed upo n parameters, the committee would then be available to support UF scholars by providing facilitated peer review of alternative scholarly works as needed and serving as experts to refer and recommend other resources and venues when appropriate For the next stage, the current committee specifically recommends working through another project submission from a non library app licant(s) as well as work i n: 1) focusing on a particular type of alternative work ; and/ or, 2) creating informational resources about and tracking types of alternative work for use in inspiring and informing others

PAGE 6

Page 6 of 11 Overview and Committee Activities Overview The Fac ilitated Peer Review Committee w as established in 2012 with the approval of the Dean of the Libraries to support the peer review process for innovative and/or nontraditional forms of scholarship and schol arly work, which is particularly needed for new works of digital scholarship. This process is critically needed across many fields to support scholarly efforts in multimedia publication, data curation, publicly engaged scholarship, collaborative work, and other forms of alternative scholarly products that do not fit within established disciplinary traditions. The need to support such a process at UF arose with a digital scholarship project created by two faculty members in the L ibraries, and that project serv e d as the pilot project for the c ommittee. Potential committee members were approached based on their rich backgrounds and experiences that could inform and provide insights to best develop this process. The final committee members who agree d to serve collectively offered a wide varied of expert knowledge and diverse backgrounds which included familiarity with administrative procedure, awareness of and involvement with alternative scholarly works, expertise from service as peer reviewers (for journals, conferences, grant proposals, and other scholarly forms), and expertise for the tenure and promotion process at UF with service on tenure and promotion committees From 2012 2013, the committee establish ed the requirements for the facilitated pe er review process with the committee acting akin to an editorial board that would facilitate, but not conduct, peer review specifically for innovative forms of scholarship. The committee establish ed the norms, standards, and documentation for the f aci litated peer review process, and refined and applied the process for the pilot project. The committee activities required approximately 5 hours per m onth for each committee member, with additi onal time required of the chair. The committee used this time for developing, reviewing, and refining documentation and for facilitating peer review for new projects For the pilot project, the scholars contributed time to complete the application for peer review and to revise the application based on feedback from t he committee. The peer reviewers contributed their time by serving as reviewers in reviewing the alternative scholarly work, completing the peer review evaluation form, and providing feedback to the committee on the process. Committee Activities In all activities the committee reaffirmed the importance of supporting alternative scholarly works in a manner that upholds peer review and is in alignment with individual and institutional needs The committee engaged in creating the facilitated peer review pr ocess to support and

PAGE 7

Page 7 of 11 uphold the sacredness of peer review as a process for fair evaluation that respects diversities 1 with the needs of individual authors and institutional units in mind in terms of measurement, communicability, and reporting. The committe e designed the process to mirror traditional peer review (following models of peer review for publications and tenure and promotion and grant awards) as closely as possible with all materials peer reviewed in accordance with already established standards o f excellence. For the operational aspects, t he majority of the committee's activities were conducted over email T he primary c ommittee activities were developing and refining the process documentation throughout the year The committee create d documentation ( available on the committ ee website and with this report) as the necessary documentation to support the process and this includes documentation to provide context for the process for scholars, peer reviewers, and committee members. The commit tee created : Peer review guide for peer reviewers and submitting authors Application for authors to request peer review of alternative scholarly works Letter template to peer reviewers explaining the facilitated peer review process and requesting peer re view Peer review form for use by peer reviewers Th e documents were initially developed from February November 2012. In November 2012, the scholars for the pilot project used the documentation and submitted their application to request peer review. The com mittee reviewed the application and discussed questions with the scholars from November 2012 through January 2013. In January 2013, the scholars submitted a revised application, which the committee approved to move to the peer reviewer stage. By March, the peer reviewers completed the reviews. The committee reviewed the responses and developed the final response to scholars, sent in May 2013. The final committee activity in drafting this report T he committee members, scholars, and peer reviewers provided p roductiv e feedback on the facilitated peer review process All provided substantive engagement and feedback that clarified and improve d the process. In addition to the committee focus on th e immediate, local needs at UF, the committee members were also contacted and discussed the process with colleagues at other institutions who expressed great interest in the process. These conversations provided additional p roductive fe edback and informed the process 1 See Michle Lamont, How Professors Think (Harvard University Press: 2009), for a discussion of peer review as a standardized and ritualized process wherein peer review becomes and is sacred.

PAGE 8

Page 8 of 11 Findings and Recommendations The committee was successfully created with members with rich backgrounds and experiences that made it possible to create this process. The committee successfully accomplished its goals in developing and testing the facilitated peer review process for the pilot project. The research done as part of creating the process confirmed the lack of alternatives and the clear need for this process for specific, immediate needs and for the larger activities needed in creating support for alternative scholarly works. The high level of engagement with the scholars for the pilot project greatly enriched the resulting project documentation which embodies the overall findings. One example of the pilot project informing the overall process occurred for the use of equivalencies. From committee discussions, early findings and recommendations emer ged, including that the use of equivalencies Equivalences have been found problematic for innovative research ; yet equivalences were d eemed useful for the facilitated peer review process to ensure support for individual scholars and institutional needs. The scholars and peer reviewers found equivalencies useful, although noting that equivalencies do require a som ewhat awkward process to establish the equivalency where the alternative scholarly work must be mapped to an established mode of scholarly production. The pilot project was particularly informative for this process, with the scholars recommending that the work be equivalent to two journal articles. This mapping for type and impact as quantity was informative and led to a productive discussion and process refinement to best utilize equivalencies Based on the activities and accomplishments to date for the Facilitated Peer Review Committee for 2012 2013 the committee members specifically recommend: C ontinuing the Facilitated Peer Review Committee with changes in membership. Changing the membership so that s everal current committee members will rotate off and new members will be appointed. Framing n ew committee activities to include: o o Refining the process to add terms of service for members o Refining the process to add guidelines for membership O perations of the initial committee provide one model with: a minimum of five and preferably six members ; and, recommended member commit ment to approximately f ive hours per month for the c ommittee with the committee to develop new guidelines and recommendations if the need results in a demand that exceeds this time commitment.

PAGE 9

Page 9 of 11 Activities of the initial committee showed the critical importance of backgrounds and expertise in relevant areas for members, 2 and representation should continue for critical areas while also seeking members who can augment and provide additio nal diversity in terms of backgrounds and experience to further inform the process and future concerns. o Supporting appropriat e, concrete activities for the immediate next phase: Identifying and working through another project submission with project from a non library app licant(s) Performing work that does either or both of the following: focuses activities and develops resources on a particular type of alternative work, and/or creates informational resources about and track types of alternative work for use in inspiring and informing others o Supporting appropriate activities for the immediate next phase and beyond: Refini ng all processes and documentation with new member input and committee discussion S upport ing the UF campus with the facilitated peer review process for alternative scholarly works as needed Supporting the UF campus with expert refer ral and recommend ation to appropriate resources and venues when available, for pee r review in addition to or instead of the facilitated peer review process D isseminating information about the process to c ampus Reviewing all processes and documentation with supporting the facilitated peer review process for additional projects changing and refining the processes and documentation to best support all Reporting regular ly with reports publicly accessible to support review, improvement, and new opportunities Additional recommendations from the pilot project may arise over the course of the next year. One of the scholars for the pilot project is going through the promotion process in 2013 2014. It is expected that discussions with current committee members, the scholar, and the review committee will occur and will cover the pilot project review and the promotion review process Discussing the processes together should provide information that can inform changes, new discussions and new directions for the facilitated peer review process. In addition to these specific recommendations, the committee recognizes that feedback and discussion on all aspects of process for the pilot project are needed, and that feedback from many 2 Including administrative p rocedure, awareness of and involvement with alternative scholarly works, expertise from service as peer reviewers (for journals, conferences, grant proposals, and other scholarly forms), and expertise for the tenure and promotion process at UF with service on tenure and promotion committees.

PAGE 10

Page 10 of 11 perspectives will best suppo recommends seeking out additional feedback and engagement from interested parties as time allows.

PAGE 11

Page 11 of 11 Supporting Documents Th e facilitated peer review process is evolving and this report serves to document the process to date. C urrent versions of all supporting documents are available onlin e through the committee website. 3 C opies of the supporting documents are also included with this report. Committee and process documents : Charge for the F acilitated Peer Review Committee Peer Review Form for Alternative Scholarly Works Application for Facilitated Peer Review Support for Alternative Scholarly Works Letter template with specifics for pilot project Peer Review Guide for Peer Reviewers and Sub mitting Authors P ilot project documents : Application to request peer review, completed by the scholars Peer Review Forms, one from each of the two peer reviewers Letter by the committee synthesizing the evaluations from the peer reviewers 3 http://www.uflib.ufl.edu/committees/fprc

PAGE 12

Charge for the Facilitated Peer Review Committee (Based on the standard format: http://www.uflib.ufl.edu/pers/develop/Committee.html ) Name: Facilitated Peer Review Committee Members: Sophia Acord, Associate Director, Center for Humanities and the Public Sphere Alan Dorsey, Professor of Physics and Associate Dean, College of Liberal Arts & Sciences Rebecca Jefferson, Judaica Librarian, UF Libraries Betsy Simpson, Ch air of the Cataloging Department, UF Libraries Laurie Taylor, Digital Humanities Librarian, UF Libraries (Chair) Ricky Telg, Professor of Agricultural Education and Communication, IFAS Description of Responsibilities: (describe responsibilities an d expecta tions) The Facilitated Peer Review Committee has been established with the approval of the Dean of the Libraries to support the peer review process for innovative and/or nontraditional forms of scholarship and scholarly work which is particularly needed f or new works of digital scholarship The need for this process arose with a digital scholarship project created by two faculty members in the Libraries, and this project will serve as the pilot project for the Committee. The Committee will establish the requirements for the facilitated peer review pro cess with the committee acting akin to an editorial board that would facilitate, but not conduct peer review specifically for innovative f orms of scholarship. This process is critically needed across many fields to support scholarly efforts in multimedia publication, data curation, publicly engaged scholarship, collaborative work, and other forms of alternative scholarly products that do not fit within established disciplinary tradition s The Committee will establish the norms, standards, and documentation for the facilitated peer review process. The Committee will refine these materials and apply them to the approved digital scholarship project which will serve as a pilot proj ect The Committee will document the applicatio n of the process to the pilot project for peer review of that project and for that project to serve as a model for facilitated peer review for other scholarly works as well as being a model for possible applic ation within other Colleges after the process is established for the pilot within the Libraries At this time, establishing the process with the facilitating committee, working through the full pilot project and documenting the entire process is expected to require up to 5 hours per month for each committee member primarily for reviewing and refining draft documentation on this process. Deadline (Anticipated) : The committee is expected to complete its work (developing materials that document the facilit ated

PAGE 13

peer review process as a model and processing the pilot project under the facilitated peer review process ) no later than August 2013 process and the outcome of the pilot project to the Dean of the Libraries. Committee Role: The Facilitated Peer Review Committee has been created specifically to address and support new modes of scholarship with a standard, rigorous, and scholarly peer review process. Because this is a new process being evaluated for its ability to support peer review and subsequent needs in regards to promotion and tenure, the committee role is advisory. Recommended Decision Authority: T he committee is advisory and not decision making T he process and pilot project will be evaluated by t he Dean of the Libraries and library faculty promotion and tenure standards for its faculty. The documentation and materials on this model will be available for use for other colleges and departments for their consideration as well Anticipated Date of Decision Concerning the Committee Recommendation: Th is i s a crucially needed process to support promotion and tenure of faculty in the Libraries and other areas of UF. Because of the crucial need and immediate application for the pilot project it is expected that the work will be applied to the pilot project within one month of th e committee completing its work and upon modificatio ns and final approval by the Dean of the Libraries. Minutes: Minutes will be kept and promptly distributed following each meeting to all committee members, and will serve to document the full process for facilitated peer review for the pilot project in particular and as for the model overall Minutes will be a brief summary of action taken at a meeting rather than a detailed accounting of what was said. Important motions, decisions, assignments, etc. will be included. Meeting Agenda: A preliminary agenda for the next meeting will be circulated once the date for the first meeting has been scheduled Subsequent meeting agendas and supporting documentation will be sent no les s than 48 hours prior to each scheduled meeting. Web Page: http://www.uflib.ufl.edu/committees/fprc

PAGE 14

Page 1 of 2 Peer Review Form for Alternative Scholarly Works Please complete this form and return it by email to Dr. Laurie Taylor UF Digital Humanities Librarian : Laurien@ufl.edu Note: All information from this form will be shared with the author ( s ) whose work is being reviewed. Please ensure all comments are frank constructive and relevant for the evaluation and validation processes. 1. Please choose one of the following in relation to the equivalency that represents an existing peer reviewed category in the traditional T&P process at the University of Florida (e.g.; journal article, book chapter, edited collection, book, etc.) as indicated in the attached Application for Facilitated Peer Review Support for Alternative Scholarly Works. __ Equivalency is appropriate as recommended by the author (s) as INSERTED BY COMMITTEE FROM AUTHOR APPLICATION Comments: __ Equivalency is not app ropriate as recommended by the author (s). However, the Work is a significant scholarly contribution, and the reviewer recommends the equivalency as______________________ Comments: __ Equivalency is not appropriate as recommended by the author (s). The Work is not at a level that would merit an equivalency. Comments: 2. Please indicate your judgment as to the quality of this work : Rankings have been attached to the following criteria as an aid to you. You may use this system or modify it as you see fit to assist in the final evaluation. Please rate the Work in the following areas: Excellent Good Adequate Inadequate Unacceptable Level of Context Sufficiency of background information provided Appropriateness of citations to prior work Methods/Process (if applicable) Clear description of process or methods to create the Work Quality of the data collection Impact Placement of the Work within the field, multiple fields, and/or within public scholarship Level and importance of contribution and impact for the field 1 Demonstration of new insights about the issue or problem Significant contribution to the understanding of the topic 1 Because the facilitated peer review process cannot have a journal impact factor or other measures, the peer reviewer assessment of the level and importance of the Work is critical.

PAGE 15

Page 2 of 2 Presentation of the Work Clarity and professionalism of written material Clarity and professionalism of visual and other material Appropriateness of language, terminology, and tone for the intended audience Usability Archival integrity (e.g., the work is located in an appropriate venue, is intuitive to locate, and has a stable presence) Overall Evaluation 3. Please provide comments for the author (s) and Facilitated Peer Review Committee Members: 2 Please provide a thoughtful rev iew and commentary, including any suggestions for improving the work : Overall assessment including whether the project accomplished the stated goals or intent its contribution to one or more disciplinary audiences, and c omments in general to its scholarly/intellectual value Comments on the appearance usability, and location of the Work (if applicable) Suggestions for future phases, possible further implications, and/or opportunities to leverage or maximize impact from the work 4 If you have any relation to the work being reviewed please note that relation 2 Reviewers should recognize that constraints may prevent the author (s) from implementing reviewer suggestions.

PAGE 16

Page 1 of 6 Application for Facilitated Peer Review Support for Alternative Scholarly Work s Important Notes : The facilitated peer review process utilizes traditional peer review with expert reviewers. Because this process needs to support individual authors as well as the needs of departments, colleges, and institutions, reviews from the facilitated pee r review process will be shared with the author (s). With author and reviewer participants in the process identified, peer reviewers are actively contributing to scholarly communication and discourse through their participation in this process. We intend for this process to be for substantively complete works. Alternative scholarly works can be unusually difficult to alter. The common option s of may not be appropriate Peer reviewers for the facilitated peer review process are asked to evaluate the Work in terms of the equivalency provided on this application. The equivalencies are those listed as an existing peer reviewed category in the traditional T&P process at the University of Florida (e.g.; journal article, book chapter, edited collection, book, etc.) 1 Peer reviewers review the Work and select one of the following options: Eq uivalency is appropriate a s recommended by the author (s) Equivalency is not appropriate as recommended by the author (s). However, the Work is a significant scholarly contribution, and the reviewer recommends the equivalency as another category listed in the T&P process. Equivalenc y is not appropriate as recommended by the author (s). The Work is not at a level that would merit an equivalency. The Peer Review Form for Alternative Scholarly Works provides additional information as an aid to reviewers. This information may also be inf ormative for completing the following application. 1 The University of Florida Promotion and Tenure (P&T or T&P) guidelines and information are online: http://www.aa.ufl.edu/tenure

PAGE 17

Page 2 of 6 1. Core Information Title/Name of W ork: Name of author (s) requesting facilitated peer review, and contribution to the W ork: Contact email address(es) for autho r (s), or alternate contact information: Please provide inform ation on how to access the Work. URL for the Work: If not online, mode/means of access: Name of additional contributor(s) who should be recognized for contributing to the Work, and role for the Work: 2 2 Alternative scholarly works are often produced by authors working with others. For example, information technology profession als and graduate students are frequent contributors to digital humanities scholarly works. In some cases, they may be collaborators who should be recognized as one of the authors In other cases, they may have contributed to some degree but should not be r ecognized as one of the authors for the work. Including contributors and their roles is needed to ensure full and appropriate attribution for work; to support further refinement of the facilitated peer review process; and to provide feedback to the releva nt institutional entities regarding the importance of the contributors for producing the work.

PAGE 18

Page 3 of 6 2. Information on the Work Please provide a short summary (approx. 200 500 words) of the Work in terms of goals, current status, and how it constitutes an impact or contribution to the field. Please explain if the Work is a contribution to multiple fields and/or public scholarship. How is the Work preserved or assured to be maintained and accessible over the long term ? For instance, a library may commit to supporting and preserving the W ork. How is the Work accessible? Please describe other review processes already in place for the Work For instance, please include information regarding competitive funding awarded for the Work; conference presentations on the Work; reports and papers about the Work (published formally, integrated support documents, etc.); reviews of the Work that have already been cond ucted; assessments of the Work; expert consu ltations and opinion s ; impact and integration measures (e.g.; usage statistics for an online site; evidence of an active core user community with partnerships and other similar arrangements); and the like.

PAGE 19

Page 4 of 6 For Works that are technical or technologically dependent please provide a short description of the Work Please note: this may be a description of the abstracted, logical technical elements and not necessarily the technologies in particular. If appropriate, please include contact information for a technical contact or representative for the Work. Please provide a brief explanation on the status of the Work in terms of the life cycle of the Work. Does the Work represent the completion of a project? Does it represent a milestone in an ongoing cycle of development for a long term program? What is the significant date range for the Work and how is it relevant? For instance, reference books are often created to cover a certain topic or period. Depending on their focus, a standard reference book may be reviewed and updated every five years. Similar activities may occur for data collection conducted for longitudinal studies. Please note any comparative genres or forms that could be informative for understanding this information for the Work.

PAGE 20

Page 5 of 6 3. Peer Review and Peer Reviewers Please provide a reco mmendation for the level or equivalency at which the Work Refereed Work that represents an equivalent impact in the field with the norm for a journal article or similar form The equi valency could take any form sole author book edited book, contributor of a chapter to a book refereed proceedings, refereed journal articl es etc. recognized as currency in the field. While equivalencies are imperfect, equivalencies may aid authors and institutions for the evaluation and proper crediting for alternative scholar ly works As such, a way to represent the Work within existing categories commonly included in tenure and promotion documents and CVs is needed. Authors in different fi elds may contribute in the same manner to a Work where the equivalency for each author because of the field or discipline, is different. Please explain how the equivalency relates to the discipline for the author (s). Please note: equivalencies should be based on impact Other measures, like hours of labor and costs should not be considered unless they are factors for impact Please provide names of at least two potential reviewers in the field; institutional affiliations; if applicable, affiliation or relationship of these reviewers to the Work; and brief bios/summaries of their expertise for reviewing the W ork.

PAGE 21

Page 6 of 6 4. Agreement and Submission All authors for the Work, as listed in Section 1, are required to sign the completed form before the review can begin. Submission of the completed application is the first formal step in the facilitated peer review p rocess. ________________________________ Signature of Author Requesting Review ________________________________ Printed or Typed Name of Author _______________ Date of Signature ________________________________ Signature of Author Requesting Review ________________________________ Printed or Typed Name of Author _______________ Date of Signature ___________________________ _____ Signature of Author Requesting Review ________________________________ Printed or Typed Name of Author _______________ Date of Signature ________________________________ Signature of Author Requesting Review ________________________________ Printed or Typed Name of Author _______________ Date of Signature ________________________________ Signature of Author Requesting Review __________ ______________________ Printed or Typed Name of Author _______________ Date of Signature ________________________________ Signature of Author Requesting Review ________________________________ Printed or Typed Name of Author _______________ Date of Signature Please designate a single corresponding author, if appropriate. If appropriate, p lease note contact information for any additional individual(s) to be included on correspondence with the author (s). This may include: department chair(s), mentor(s), and the like.

PAGE 22

The Foundation for The Gator Nation An Equal Opportunity Institution George A. Smathers Libraries 533 Smathers Library Scholarly Resources & Research Services PO Box 11700 0 Gainesville, FL 32611 700 0 352 273 2902 January 9 201 3 Dear POTENTIAL REVIEWER: The University of Florida George A. Smathers Libraries are currently working to establish a facilitated peer review process to support the peer review of alternative scholarly works. The facilitated peer review process is in development with the first pha se to establish template materials and a standard process and to apply the process to a pilot project. Such a process is critically needed across many fields to support scholarly efforts in multimedia publication, data curation, publicly engaged scholarsh ip, collaborative work, and other forms of alternative scholarly products that do not fit within established disciplinary traditions. This process is being established to support the peer review of alternative scholarly works specifically in relation to th e tenure and promotion process. 1 The Facilitated Peer Review Committee is comprised of UF faculty facilitate the new process of peer review. The Committee is akin to members of a journal editorial staff editors in providing the framework for peer review. The Committee establishes the necessary policies, procedures, and documentation; locates the appropriate experts to conduct peer review; support s the process of peer review performed by those expert s wherein the experts conduct peer review and provide quantitative and qualitative feedback using the provided peer review form; and then synthesizes feedback from all reviewer s into a letter and review packet to the author(s) The Committee is working to ensure the process is aligned with and supports individual autho rs as well as departmental, college, and institutional needs in terms of measurement and reporting for promotion and tenure within the University of Florida. Bri an W. Keith and Bonnie J. Smith, both from the UF Libraries, volunteered to have their Libraries Recruitment Efficacy and Outcomes Study ( http://www.uflib.ufl.edu/recruitmentstudy ) serve as the pilo t project for the process. They proposed you as a potential reviewer for the project. As Chair of the Facilitated Peer Review Committee, I am contacting you on behalf of the Committee and the UF Libraries to inquire regarding your availability to serve as a peer reviewer for the pilot project Included with this letter are several template components developed for use with all scholarly 1 The University of Florida Promotion and Tenure (P&T or T&P) guidelines and information are online: http://www.aa.ufl.edu/tenure

PAGE 23

The Foundation for The Gator Nation An Equal Opportunity Institution works that undergo the facilitated peer review process. The documents are: 1. Application for Facilitated Peer Review Su pport for Alternative Scholarly Works: provides background on the project and additional documentation about the impact, use, and reception of the work 2. Peer Review Guide for Peer Reviewers and Submitting Autho rs : provides an overview of th e facilitated peer review process for peer reviewers and authors 3. Peer Review Form for Alternative Scholarly Works : provides structure for responses D this process are online on the Committee website: http://www.uflib.ufl.edu/committees/fprc Please respond to this letter to confirm your willingness to serve as a peer reviewer or to discuss this process. I would greatly appreciate the opportunity to speak with you about this process whether or not you are able to serve as a peer reviewer. Please contact me with any questions, concerns, or to disc uss this process. I am available via email (Laurien@ufl.edu) and phone (352.273.2902) and I am happy to contact you at your convenience. Best regards, Laurie N. Taylor, Ph.D. Digital Humanities Librarian George A. Smathers Libraries University of Flori da P.O. Box 117000 Gainesville, FL 32611 Laurien@ufl.edu 533 Library West 352.273.2902 http://ufdc.ufl.edu

PAGE 24

Page 1 of 4 Peer Review Guide for Peer Reviewers and Submitting Authors The University of Florida George A. Smathers Libraries are establishing a facilitated peer review process to support the peer review of alternative scholarly works specifically in relation to the tenure and promotion process. 1 Such a process is critically needed across many fields to support scholarly efforts in multimedia publication, data curation, publicly engaged scholarship, collaborative work s and other forms of alternative scholarly pro ducts that do not fit within established disciplinary traditions. The process will support and uphold the sacredness of peer review as a process for fair evaluation that respects diversities 2 and has been established with the needs of individual author s and institutional units in mind in terms of measurement, communicability, and reporting. This process mirror s traditional peer review (following models of peer review for publications and tenure and promotion and grant awards ) as closely as possible with all materials peer reviewed in accordance with already established standards of excellence. Peer Review Process The steps/stages for the facilitated peer review process are : Author ( s ) submit s completed Application to C ommittee Committee reviews Application for completeness; communicating with author (s) as needed to obtain complete information Committee informs author (s) after the Application has been accepted as complete Committee confirms selection of at least two p eer r eviewers, with no less than 50% from the list provided by the author (s) Committee c hair creates projected timeline for review process Committee c hair contacts potential p eer r eviewers, supporting any discussion/questions Committee c hair contacts author (s) with the status of the review process and to share the projected timeline, and to support any discussion/questions Committee c hair maintains schedule for the process Peer r eviewers conduct peer review Peer r eviewers s ubmit completed reviews to the C ommit tee c hair consulting the Committee as needed Possible discussion and clarification at this stage 1 The University of Florida Promotion and Tenure (P&T or T&P) guidelines and information are online: http://www.aa.ufl.edu/tenure 2 See Michle Lamont, How Professors Think ( Harvard University Press: 2009), for a discussion of peer review as a standardized and ritualized process wherein peer review becomes and is sacred.

PAGE 25

Page 2 of 4 Committee chair reviews p eer r eviewer submissions for completeness consulting the Committee as needed Possible discussion and clarification at this stage Committee may select a third peer reviewer if needed, handled on a case by case basis Committee c hair in consultation with the Committee drafts a summary letter synthesizing the narrative reviews from the p eer r eviewers Committee c hair prepare s compilation of the draft summary letter, reviews, application, and any other documentation and provid es this compiled packet to the C ommittee Committee reviews, makes any changes, conducts final work if necessary, and grants final approval of the summary review letter with full review packet Committee c hair submits letter and review packet to author (s) Committee chair sends letters acknowledging the p eer r eviewers for their service Committee contacts author (s) and p eer r eviewers to request feedback to assess and improve the facilitated peer review process Roles and Responsibilities for the Peer Review Process Author (s) The facilitated peer review process begins with the author (s) completing the Application for Facilitated Peer Review Support for Alternative Scholarly Works. Author s provide contextual and background information as well as documentation to support the review of their alternative scholarly works. By completing the Application, author s provide succ inct information and context for alternative scholarly works. Facilitated Peer Review Committee The Facilitated Peer Review Committee is comprised of UF faculty and staff T role is to facilitate the process of peer review. The Committee is akin to members of a journal editorial board staff editors in providing the framework for peer review or the program officers for a granting a gency that supports and facilitates the experts who are conducting peer review The Committe e establishes the necessary policies, procedures, and documentation; locates the appropriate experts to conduct peer review; and supports the process of peer review as performed by experts. Peer review is conducted by experts in the field Additional role s expected of the Committee include: Full Committee o R eviews templates and processes no less than annually; makes appropriate updates and changes

PAGE 26

Page 3 of 4 Committee Chair o Schedules meetings, takes minutes, and distributes and maintains documentation regarding the pr ocess Peer Reviewers Peer reviewers are crucial contributors to the facilitated peer review process. By reviewing materials and providing thorough, informed and expert written reviews, peer reviewers ensure that scholarly works endorsed through the peer review process represent substantive i mpact or contribution to the field and that those works can be recognized and credited properly. This provides the necessary supports for individual author s, institutions and institutional un its, and more broadly, scholarly communications in terms of inquiry, excellence, and innovation. Peer reviewers also provide feedback to aid author s in improving and extendin g their work for maximum impact for the field. Identification of and Attribution for Author s, Committee, and Peer Reviewers Given that in many forms of alternative scholarship, as with online scholarly works, author s are identified, this process differs from the traditional peer review process in this one area where ins titutional facilitators, author (s), and reviewers are not anonymous. The facilit ated peer review process will follow the process used for tenure and promotion letters wherein the identities of institutional facilitators, author and reviewers are disclosed and author s may retain rights to review all feedback. Because this process needs to support individual author s as well as the needs of departments, colleges, a nd institutions reviews from the facilitated peer review process will be shared with the author (s) With both author (s) and reviewer s identified, peer reviewers are able to actively contribute to scholarly communication and discourse thr ough their participation in the process. Peer Review Form for Alternative Scholarly Work s The Peer Review Form for Alternative Scholarly Works contains contextual notes to support the feedback requested from peer reviews. not be appropriate. Peer reviewers for the facilitated peer review process are asked to evaluate the Work in terms of the equivalency provided on the application. The equivalencies are those listed as an existing peer reviewed category in the

PAGE 27

Page 4 of 4 traditional T &P process at the University of Florida (e.g.; journal article, book chapter, edited collection, book, etc.). 3 Peer reviewers review the Work and select one of the following options: Equivalency is appropriate as recommended by the author (s) Equivalency is not appropriate as recommended by the author (s). However, the Work is a significant scholarly contribution, and the reviewer recommends the equivalency as another category listed in the T&P process. Equivalency is not appropriate as recommended by the a uthor (s). The Work is not at a level that would merit an equivalency. The Peer Review Form for Alternative Scholarly Works provides additional information to aid reviewers which may also be informative for author s submitting W ork for review Additional Information Additional information on the peer review process, roles, and responsibilities is given in the Application for Facilitated Peer Review Support for Alternative Scholarly Works and the Peer Review Form for Alternative Scholarly Works. All materia ls are located on the Committee Website: http://www.uflib.ufl.edu/committees/fprc/ 3 The University of Florida Promotion and Tenure (P&T or T&P) guidelines and information are online: http://www.aa.ufl.edu/tenure