<%BANNER%>

Introduction to Systematic Reviews

University of Florida Institutional Repository
Permanent Link: http://ufdc.ufl.edu/IR00003445/00001

Material Information

Title: Introduction to Systematic Reviews
Series Title: Library Workshop Presentations
Physical Description: library workshop powerpoint presentation
Creator: Lyon, Jennifer
Copyright Date: 2011
Edition: 2012

Notes

Acquisition: Collected for University of Florida's Institutional Repository by the UFIR Self-Submittal tool. Submitted by Jennifer Lyon.
General Note: powerpoint presentation used for library workshops and to train UF faculty, students and staff

Record Information

Source Institution: University of Florida Institutional Repository
Holding Location: University of Florida
Rights Management:
This item is licensed with the Creative Commons Attribution Non-Commercial No Derivative License. This license allows others to download this work and share them with others as long as they mention the author and link back to the author, but they can’t change them in any way or use them commercially.
System ID: IR00003445:00001

Permanent Link: http://ufdc.ufl.edu/IR00003445/00001

Material Information

Title: Introduction to Systematic Reviews
Series Title: Library Workshop Presentations
Physical Description: library workshop powerpoint presentation
Creator: Lyon, Jennifer
Copyright Date: 2011
Edition: 2012

Notes

Acquisition: Collected for University of Florida's Institutional Repository by the UFIR Self-Submittal tool. Submitted by Jennifer Lyon.
General Note: powerpoint presentation used for library workshops and to train UF faculty, students and staff

Record Information

Source Institution: University of Florida Institutional Repository
Holding Location: University of Florida
Rights Management:
This item is licensed with the Creative Commons Attribution Non-Commercial No Derivative License. This license allows others to download this work and share them with others as long as they mention the author and link back to the author, but they can’t change them in any way or use them commercially.
System ID: IR00003445:00001

Full Text

PAGE 2

OBJECTIVES What is a Systematic Review? Different Types of Reviews Conducting a Review Develop a research question and protocol Searching process Choosing Databases Publication Bias Term Harvesting Grey Literature Specialty Techniques Evidence Rankings Presenting the Data: systematic reviews and meta analysis Writing the Methods Section

PAGE 3

WHAT IS A SYSTEMATIC REVIEW? empirical evidence that fits pre specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods that are selected to minimize bias, thus providing reliable findings from which conclusions can be drawn and decisions Liberati et al., 2009. The PRISMA statement for reporting systematic reviews and meta analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 2009; 62: e1 e34.

PAGE 4

WHY DO A SYSTEMATIC REVIEW? Save clinicians and researchers valuable time Take a large mass of literature and make concise sense of it Objectively evaluate the quality of evidence on a topic Resolve or highlight contradictions in the literature Guide clinical decisions F orm the basis for practice guidelines and health care policy Identify the need for additional research P revent unnecessary studies from being carried out

PAGE 5

KEY CHARACTERISTICS A systematic review: IS A RESEARCH STUDY Is based on a well defined research question Is comprehensive Is objective Evaluates the quality of included evidence

PAGE 6

WARNING is a systematic review or that it is a good quality systematic review

PAGE 7

TYPES OF REVIEWS Narrative reviews Broad perspective on topic (like a textbook chapter), no specified search strategy, significant bias issues, may not evaluate quality of evidence Structured reviews Includes a structured, but limited search, less bias, but not comprehensive, usually evaluates quality of evidence A partial systematic review Systematic reviews Comprehensive and minimized bias, qualitative, evaluates quality of evidence Based on randomized controlled clinical trials (RCTs) Best evidence Based on other types of clinical studies or literature Best available evidence Meta analysis A quantitative systematic review that applies statistical analysis

PAGE 8

SYSTEMATIC REVIEWS & META ANALYSIS Systematic Reviews are qualitative research studies in which primary studies summarized but not statistically combined Meta analyses are quantitative systematic reviews and involve combining the results of multiple primary studies using statistical methods M eta analysis is just a special (quantitative) type of systematic review.

PAGE 9

STEPS TO CONDUCT A SYSTEMATIC REVIEW 1. Develop a research question 2. Define inclusion and exclusion criteria 3. Locate studies 4. Select studies 5. Assess study quality 6. Extract data 7. Analyze and present results 8. Interpret results 9. Update the review as needed

PAGE 10

RESEARCH QUESTION: INTRODUCING PICO Method of structuring the research question: PICO Patient population Children aged 3 8 yrs diagnosed with croup Intervention Use of dexamethasone to treat croup Comparison Use of betamethasone to treat croup (optional, depending on the study question ) Outcomes Comparative relief of symptoms A, B, C; results of observations D, E, F; or test http://healthlinks.washington.edu/ebp/pico.html

PAGE 11

DEFINE INCLUSION/EXCLUSION CRITERIA Determined by research question Define prior to starting the search Broader inclusion criteria allows more generalization of the review May include choice of languages, publication dates, publication types, etc. Be aware limits may introduce bias as they may cause loss of valuable data.

PAGE 12

PRE ESTABLISH SEARCH PROTOCOL Identify databases and other resources Identify search terms, develop search strategies, conduct searches Need to have at least 2 independent reviewers to select studies based on pre established inclusion/exclusion criteria and another Review title/abstract Review full text Keep a log of excluded studies with reasons for exclusion Keep track of numbers of results from searches, number of duplicates found, number of studies excluded at each step

PAGE 13

CHOOSING DATABASES To reduce bias multiple databases must be searched Each database should be searched with a database specific search strategies Common minimum is usually Medline(PubMed), Embase Cochrane Central Registry of Controlled Trials Choice of others depends on nature of question: Bioscience L iterature Databases, Nursing Literature Databases, Pharmacology Literature Databases, Education Literature Databases, etc. Issues Balancing sensitivity and specificity When to stop? Effect of limits (inclusion/exclusion criteria) Access to resources Bias

PAGE 14

PUBLICATION BIAS that appears in the published literature is systematically (Rothstein, 2005) Systematic reviews aim to include ALL high quality studies about the research question. This is difficult to accomplish, but a missed study or studies may affect results and conclusions. Rothstein H, Sutton AJ, & Borenstein M. (2005). Publication bias in meta analysis prevention, assessment and adjustments. Chichester England; Hoboken, NJ: Wiley

PAGE 15

FROM THE COCHRANE COLLABORATION bias and other related biases can be summarised as statistically significant, 'positive' results being : m ore likely to be published (publication bias) m ore likely to be published rapidly (time lag bias) m ore likely to be published in English (language bias) m ore likely to be published more than once (multiple publication bias) m ore likely to be cited by others (citation bias ) All of these reporting biases make positive studies easier to find than those with non significant results, something that we can try to minimise by extensive searching http://www.cochrane net.org/openlearning/html/mod15 2.htm

PAGE 16

MINIMIZE BIAS Negative studies, equivocal results studies, non English studies are less likely to get published, less likely to get into the top journals, and less likely to get cited. Not everything gets published in peer reviewed journals Be aware you are introducing bias if you not looking beyond the obvious sources. dissertations, patents, etc. (Grey Literature) Be cautious of using language limits.

PAGE 17

TERM HARVESTING Methods for gathering and utilizing search vocabulary (search terms) for the literature review. Issues Clearly defined search question, inclusion/exclusion criteria, and outcomes required identified can be useful resources Knowledge of databases including scope, use of controlled vocabularies, search interfaces is valuable Knowledge of search strategy logic and construction is valuable Use available expertise consider collaborating with your medical librarian!

PAGE 18

SAMPLE TERM HARVESTING FORM PUBMED Concept: steroids croup young children MeSH Terms Pharmacological Action Terms Textwords Question: Are steroids effective in treating croup in young children?

PAGE 19

MORE ON TERM HARVESTING To find more terms: Read about topic (textbooks, older reviews, other articles, other sources) Talk to experts (other members of team, clinicians, colleagues, etc.) Look at indexing of pre Brainstorm, brainstorm, brainstorm Know how to use controlled vocabularies effectively and when to combine with textwords Know the idiosyncrasies of each database. Searching only one database is NEVER enough. Not all have controlled vocabularies Each controlled vocabulary tends to be unique How do they handle phrase searching? How do they handle limits? How do they handle field searching? What are the options for saving searches and exporting search results?

PAGE 20

DEVELOPING SEARCH STRATEGIES Develop search strategies for each database Dependent on specific search interface for each database Usually will have multiple search strategies for each database Usually will need to combine controlled vocabulary searching with textword searching Author searching can be helpful If you use automated limits, track how that affects the yield of your results. What are you losing? Record and keep ALL search strategies Record the yield of articles from each strategy Record the number of abstracts selected from each strategy

PAGE 21

GREY LITERATURE & OTHER SOURCES Definition 1997 The Luxembourg Convention on Grey Literature that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial Farace 1998) Synonyms: Fugitive literature, gray literature Forms of Grey Lit include: Reports, Conference Abstracts, Dissertations & Theses, Registered Clinical Trials, Interviews, Patents, Newsletters, White Papers, Book Chapters Farace DJ. GL '97 Proceedings. In: D. F, J. F, editors. Third International Conference on Grey Literature: Perspectives on the design and transfer of scientific and technical information. Luxembourg, 13 14 November 1997. Amsterdam: GreyNet / TransAtlantic ; 1998. p. iii.

PAGE 22

USING THE GREY LITERATURE Advantages Minimizes bias Finds valuable data that would otherwise be missed Increases currency and accuracy, Broadens applicability of review Disadvantages Takes time and effort Difficult to find and access; may have associated costs Indexing and search interfaces may be poor quality reviewed

PAGE 23

GREY LITERATURE SOURCES Clinical Trial Registers ISRCTN Register http://www.isrctn.org/ Clinicaltrials.gov http :// clinicaltrials.gov/ WHO ICTRP http ://apps.who.int/trialsearch / Clinical Study Results Databases PhRMA Clinical Study Results Database http://www.clinicalstudyresults.org/home/ IFPMA Clinical Trial Results Portal http :// clinicaltrials.ifpma.org/en/myportal/index.htm

PAGE 24

COLLECTING ARTICLES Two independent reviewers should run the searches and select abstracts Track the number of abstracts selected Count the number of duplicates eliminated Selection of abstracts is compared and disagreements resolved* An additional round (or two) of abstract weeding may be needed Agree on selected abstracts for which full text will be examined* Use of a citation management program is recommended to save time/effort Two independent reviewers should read and rate full text articles* Agree on selected articles for inclusion in the review Use selected articles as source for snowballing and handsearching *Have a third independent reviewer to help resolve differences

PAGE 25

HANDSEARCHING Handsearching is the process of Selecting journals of topical interest to the review question identify possibly missed studies Preferably with print, but can be done electronically (see http:// www.cochrane.org/training/handsearchers tscs for additional information) Why do it? Some journals are not indexed in major literature databases Journal supplements, news, editorials, letters, etc. may not be well indexed Many journals contain conference proceedings that may not be well indexed

PAGE 26

SNOWBALLING Snowballing is the process of citation tracking from one article to another using the reference lists of selected articles to locate other articles of importance (backtracking) s eeing who cited a selected article to locate other articles of importance (forward tracking) Why do it? Locate old articles of importance that might predate online literature databases or not be indexed in them Locate newer literature Gain a sense of the history and relationships of the literature on the topic I dentify the authors who publish the most on the topic (the experts)

PAGE 27

EVALUATING STUDIES BASIC STUDY DESIGNS EXPERIMENTAL Investigator manipulates a variable and examines effect on an outcome Randomized Controlled Trials Controlled Trials OBSERVATIONAL Investigator observes the outcome of naturally occurring differences in a variable Cohort Studies Case Control Studies Case Series

PAGE 28

EVALUATING STUDY QUALITY There are a number of scales used to evaluate the level of evidence quality in clinical studies. Some journals and professional associations have their own rankings. Choose an evidence ranking scale that is appropriate for the specialty area and targeted journal for publication of the review. Be consistent in your use of the one you create or select.

PAGE 29

EBM QUALITY RATING SCALES For RCTs, one of the most important quality reporting checklists is CONSORT (Consolidated Standards of Reporting Trials ) More general scales (beyond just RCTs) are provided from various sources including: The Centre for Evidence Based Medicine: http :// www.cebm.net/index.aspx?o=1025 American Association of Family Physicians: Levels of Evidence in AFP GRADE: http:// www.gradeworkinggroup.org/intro.htm MOOSE:( Meta analysis of Observational Studies in Epidemiology) http :// jama.jamanetwork.com/article.aspx?volume=283&issue=15&page=2008

PAGE 30

RESULTS STATS OR NOT The design and quality of selected studies will indicate whether the data is amenable to meta analysis (statistical analysis) or not. Meta analysis is most powerful when used to combine the results of highly similar randomized controlled trials. Similar populations Same intervention/control Same measurements Same outcomes recorded Full data available Note: If you plan to do meta analysis, get a statistician onboard early! Like your medical librarian, use available expertise to make things easier!

PAGE 31

PRESENTING THE DATA SYSTEMATIC REVIEW Schnuriger Beat; Inaba Kenji; Konstantinidis Agathoklis ; Lustenberger Thomas; Chan, Linda; Demetriades Demetrios Outcomes of Proximal Versus Distal Splenic Artery Embolization After Trauma: A Systematic Review and Meta Analysis Journal of Trauma Injury Infection & Critical Care. 70(1):252 260, January 2011. DOI: 10.1097/TA.0b013e3181f2a 92e

PAGE 32

PRESENTING DATA META ANALYSIS One of the most common ways to present meta analysis data is with a forest plot (AKA blobbogram ) The forest plot illustrates the relative strength of treatment effects (via Odds Ratios) from a set of quantitative clinical studies that address the same question. EXAMPLE FROM: Sukeik M, Alshryda S, Haddad FS, Mason JM. Systematic review and meta analysis of the use of tranexamic acid in total hip replacement. J Bone Joint Surg Br. 2011 Jan;93(1):39 46. Review. PubMed PMID: 21196541.

PAGE 33

WRITING UP THE METHODS SECTION Check author guidelines for targeted journal; look at other systematic reviews in that journal Follow one of the high quality guides such as PRISMA, Cochrane, or MOOSE More detail is better than less TRACK YOUR NUMBERS!

PAGE 34

PRISMA The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta Analyses) statement is the result of an international collaboration and agreement. This replaces the older QUORUM, These standards were developed to promote quality and transparency in reporting systematic review research. analyses. It was published simultaneously in the BMJ, Annals of Internal Medicine, Journal of Clinical Epidemiology, and PLoS Medicine http:// prisma statement.org/statement.htm Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). P referred R eporting I tems for S ystematic Reviews and M eta A nalyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097 Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gtzsche PC, et al. (2009) The PRISMA Statement for Reporting Systematic Reviews and Meta Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med 6(7): e1000100. doi:10.1371/journal.pmed.1000100

PAGE 35

Display your search process and track the number of citations through that process in a flow diagram. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). P referred R eporting I tems

PAGE 36

EXAMPLE: SEARCH FLOW CHART Blackwood B, Alderdice F, Burns K, Cardwell C, Lavery G, O'Halloran P. Use of weaning protocols for reducing duration of mechanical ventilation in critically ill adult patients: Cochrane systematic review and meta analysis. BMJ. 2011 Jan 13;342:c7237 PMID : 21233157; PMCID : PMC3020589 http ://www.ncbi.nlm.nih.gov/pmc/articles/PMC3020589/pdf/bmj.c7237.pdf

PAGE 37

THE COCHRANE COLLABORATION An organization that is internationally recognized for being an important leader in creating high quality systematic reviews and advancing Evidence Based Medicine is the Cochrane Collaboration Circle formed by the 'C' of Cochrane and the mirror image 'C' of Collaboration reflects the international collaboration that makes our work relevant globally The inner part of the logo illustrates a systematic review of data from seven randomized controlled trials (RCTs), comparing one health care treatment with a placebo Each horizontal line represents the results of one trial (the shorter the line, the more certain the result); and the diamond represents their combined results http://www.cochrane.org/

PAGE 38

A USEFUL GUIDE The Dartmouth Guide to Searching for Systematic Reviews: http:// researchguides.dartmouth.edu/sys reviews Provides useful information on term harvesting, search strategy development, and project management including downloadable forms that can be adapted to your needs.

PAGE 39

THINGS TO REMEMBER PLAN in advance! Remember that this takes time: a systematic review is a RESEARCH project. Use available expertise: involve a medical librarian and a biostatistician early. Keep careful and complete records. Follow the PRISMA Guidelines.

PAGE 40

Questions? Need Help? Contact your librarian! Gainesville: http:// www.library.health.ufl.edu/services/liaisons.html Borland: http:// www.library.health.ufl.edu/borland/index.html Or Contact Me! jalyon@ufl.edu 352 273 8441