|UFDC Home||| Help|
OBJECTIVES What is a Systematic Review? Different Types of Reviews Conducting a Review Develop a research question and protocol Searching process Choosing Databases Publication Bias Term Harvesting Grey Literature Specialty Techniques Evidence Rankings Presenting the Data: systematic reviews and meta analysis Writing the Methods Section
WHAT IS A SYSTEMATIC REVIEW? empirical evidence that fits pre specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods that are selected to minimize bias, thus providing reliable findings from which conclusions can be drawn and decisions Liberati et al., 2009. The PRISMA statement for reporting systematic reviews and meta analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 2009; 62: e1 e34.
WHY DO A SYSTEMATIC REVIEW? Save clinicians and researchers valuable time Take a large mass of literature and make concise sense of it Objectively evaluate the quality of evidence on a topic Resolve or highlight contradictions in the literature Guide clinical decisions F orm the basis for practice guidelines and health care policy Identify the need for additional research P revent unnecessary studies from being carried out
KEY CHARACTERISTICS A systematic review: IS A RESEARCH STUDY Is based on a well defined research question Is comprehensive Is objective Evaluates the quality of included evidence
WARNING is a systematic review or that it is a good quality systematic review
TYPES OF REVIEWS Narrative reviews Broad perspective on topic (like a textbook chapter), no specified search strategy, significant bias issues, may not evaluate quality of evidence Structured reviews Includes a structured, but limited search, less bias, but not comprehensive, usually evaluates quality of evidence A partial systematic review Systematic reviews Comprehensive and minimized bias, qualitative, evaluates quality of evidence Based on randomized controlled clinical trials (RCTs) Best evidence Based on other types of clinical studies or literature Best available evidence Meta analysis A quantitative systematic review that applies statistical analysis
SYSTEMATIC REVIEWS & META ANALYSIS Systematic Reviews are qualitative research studies in which primary studies summarized but not statistically combined Meta analyses are quantitative systematic reviews and involve combining the results of multiple primary studies using statistical methods M eta analysis is just a special (quantitative) type of systematic review.
STEPS TO CONDUCT A SYSTEMATIC REVIEW 1. Develop a research question 2. Define inclusion and exclusion criteria 3. Locate studies 4. Select studies 5. Assess study quality 6. Extract data 7. Analyze and present results 8. Interpret results 9. Update the review as needed
RESEARCH QUESTION: INTRODUCING PICO Method of structuring the research question: PICO Patient population Children aged 3 8 yrs diagnosed with croup Intervention Use of dexamethasone to treat croup Comparison Use of betamethasone to treat croup (optional, depending on the study question ) Outcomes Comparative relief of symptoms A, B, C; results of observations D, E, F; or test http://healthlinks.washington.edu/ebp/pico.html
DEFINE INCLUSION/EXCLUSION CRITERIA Determined by research question Define prior to starting the search Broader inclusion criteria allows more generalization of the review May include choice of languages, publication dates, publication types, etc. Be aware limits may introduce bias as they may cause loss of valuable data.
PRE ESTABLISH SEARCH PROTOCOL Identify databases and other resources Identify search terms, develop search strategies, conduct searches Need to have at least 2 independent reviewers to select studies based on pre established inclusion/exclusion criteria and another Review title/abstract Review full text Keep a log of excluded studies with reasons for exclusion Keep track of numbers of results from searches, number of duplicates found, number of studies excluded at each step
CHOOSING DATABASES To reduce bias multiple databases must be searched Each database should be searched with a database specific search strategies Common minimum is usually Medline(PubMed), Embase Cochrane Central Registry of Controlled Trials Choice of others depends on nature of question: Bioscience L iterature Databases, Nursing Literature Databases, Pharmacology Literature Databases, Education Literature Databases, etc. Issues Balancing sensitivity and specificity When to stop? Effect of limits (inclusion/exclusion criteria) Access to resources Bias
PUBLICATION BIAS that appears in the published literature is systematically (Rothstein, 2005) Systematic reviews aim to include ALL high quality studies about the research question. This is difficult to accomplish, but a missed study or studies may affect results and conclusions. Rothstein H, Sutton AJ, & Borenstein M. (2005). Publication bias in meta analysis prevention, assessment and adjustments. Chichester England; Hoboken, NJ: Wiley
FROM THE COCHRANE COLLABORATION bias and other related biases can be summarised as statistically significant, 'positive' results being : m ore likely to be published (publication bias) m ore likely to be published rapidly (time lag bias) m ore likely to be published in English (language bias) m ore likely to be published more than once (multiple publication bias) m ore likely to be cited by others (citation bias ) All of these reporting biases make positive studies easier to find than those with non significant results, something that we can try to minimise by extensive searching http://www.cochrane net.org/openlearning/html/mod15 2.htm
MINIMIZE BIAS Negative studies, equivocal results studies, non English studies are less likely to get published, less likely to get into the top journals, and less likely to get cited. Not everything gets published in peer reviewed journals Be aware you are introducing bias if you not looking beyond the obvious sources. dissertations, patents, etc. (Grey Literature) Be cautious of using language limits.
TERM HARVESTING Methods for gathering and utilizing search vocabulary (search terms) for the literature review. Issues Clearly defined search question, inclusion/exclusion criteria, and outcomes required identified can be useful resources Knowledge of databases including scope, use of controlled vocabularies, search interfaces is valuable Knowledge of search strategy logic and construction is valuable Use available expertise consider collaborating with your medical librarian!
SAMPLE TERM HARVESTING FORM PUBMED Concept: steroids croup young children MeSH Terms Pharmacological Action Terms Textwords Question: Are steroids effective in treating croup in young children?
MORE ON TERM HARVESTING To find more terms: Read about topic (textbooks, older reviews, other articles, other sources) Talk to experts (other members of team, clinicians, colleagues, etc.) Look at indexing of pre Brainstorm, brainstorm, brainstorm Know how to use controlled vocabularies effectively and when to combine with textwords Know the idiosyncrasies of each database. Searching only one database is NEVER enough. Not all have controlled vocabularies Each controlled vocabulary tends to be unique How do they handle phrase searching? How do they handle limits? How do they handle field searching? What are the options for saving searches and exporting search results?
DEVELOPING SEARCH STRATEGIES Develop search strategies for each database Dependent on specific search interface for each database Usually will have multiple search strategies for each database Usually will need to combine controlled vocabulary searching with textword searching Author searching can be helpful If you use automated limits, track how that affects the yield of your results. What are you losing? Record and keep ALL search strategies Record the yield of articles from each strategy Record the number of abstracts selected from each strategy
GREY LITERATURE & OTHER SOURCES Definition 1997 The Luxembourg Convention on Grey Literature that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial Farace 1998) Synonyms: Fugitive literature, gray literature Forms of Grey Lit include: Reports, Conference Abstracts, Dissertations & Theses, Registered Clinical Trials, Interviews, Patents, Newsletters, White Papers, Book Chapters Farace DJ. GL '97 Proceedings. In: D. F, J. F, editors. Third International Conference on Grey Literature: Perspectives on the design and transfer of scientific and technical information. Luxembourg, 13 14 November 1997. Amsterdam: GreyNet / TransAtlantic ; 1998. p. iii.
USING THE GREY LITERATURE Advantages Minimizes bias Finds valuable data that would otherwise be missed Increases currency and accuracy, Broadens applicability of review Disadvantages Takes time and effort Difficult to find and access; may have associated costs Indexing and search interfaces may be poor quality reviewed
GREY LITERATURE SOURCES Clinical Trial Registers ISRCTN Register http://www.isrctn.org/ Clinicaltrials.gov http :// clinicaltrials.gov/ WHO ICTRP http ://apps.who.int/trialsearch / Clinical Study Results Databases PhRMA Clinical Study Results Database http://www.clinicalstudyresults.org/home/ IFPMA Clinical Trial Results Portal http :// clinicaltrials.ifpma.org/en/myportal/index.htm
COLLECTING ARTICLES Two independent reviewers should run the searches and select abstracts Track the number of abstracts selected Count the number of duplicates eliminated Selection of abstracts is compared and disagreements resolved* An additional round (or two) of abstract weeding may be needed Agree on selected abstracts for which full text will be examined* Use of a citation management program is recommended to save time/effort Two independent reviewers should read and rate full text articles* Agree on selected articles for inclusion in the review Use selected articles as source for snowballing and handsearching *Have a third independent reviewer to help resolve differences
HANDSEARCHING Handsearching is the process of Selecting journals of topical interest to the review question identify possibly missed studies Preferably with print, but can be done electronically (see http:// www.cochrane.org/training/handsearchers tscs for additional information) Why do it? Some journals are not indexed in major literature databases Journal supplements, news, editorials, letters, etc. may not be well indexed Many journals contain conference proceedings that may not be well indexed
SNOWBALLING Snowballing is the process of citation tracking from one article to another using the reference lists of selected articles to locate other articles of importance (backtracking) s eeing who cited a selected article to locate other articles of importance (forward tracking) Why do it? Locate old articles of importance that might predate online literature databases or not be indexed in them Locate newer literature Gain a sense of the history and relationships of the literature on the topic I dentify the authors who publish the most on the topic (the experts)
EVALUATING STUDIES BASIC STUDY DESIGNS EXPERIMENTAL Investigator manipulates a variable and examines effect on an outcome Randomized Controlled Trials Controlled Trials OBSERVATIONAL Investigator observes the outcome of naturally occurring differences in a variable Cohort Studies Case Control Studies Case Series
EVALUATING STUDY QUALITY There are a number of scales used to evaluate the level of evidence quality in clinical studies. Some journals and professional associations have their own rankings. Choose an evidence ranking scale that is appropriate for the specialty area and targeted journal for publication of the review. Be consistent in your use of the one you create or select.
EBM QUALITY RATING SCALES For RCTs, one of the most important quality reporting checklists is CONSORT (Consolidated Standards of Reporting Trials ) More general scales (beyond just RCTs) are provided from various sources including: The Centre for Evidence Based Medicine: http :// www.cebm.net/index.aspx?o=1025 American Association of Family Physicians: Levels of Evidence in AFP GRADE: http:// www.gradeworkinggroup.org/intro.htm MOOSE:( Meta analysis of Observational Studies in Epidemiology) http :// jama.jamanetwork.com/article.aspx?volume=283&issue=15&page=2008
RESULTS STATS OR NOT The design and quality of selected studies will indicate whether the data is amenable to meta analysis (statistical analysis) or not. Meta analysis is most powerful when used to combine the results of highly similar randomized controlled trials. Similar populations Same intervention/control Same measurements Same outcomes recorded Full data available Note: If you plan to do meta analysis, get a statistician onboard early! Like your medical librarian, use available expertise to make things easier!
PRESENTING THE DATA SYSTEMATIC REVIEW Schnuriger Beat; Inaba Kenji; Konstantinidis Agathoklis ; Lustenberger Thomas; Chan, Linda; Demetriades Demetrios Outcomes of Proximal Versus Distal Splenic Artery Embolization After Trauma: A Systematic Review and Meta Analysis Journal of Trauma Injury Infection & Critical Care. 70(1):252 260, January 2011. DOI: 10.1097/TA.0b013e3181f2a 92e
PRESENTING DATA META ANALYSIS One of the most common ways to present meta analysis data is with a forest plot (AKA blobbogram ) The forest plot illustrates the relative strength of treatment effects (via Odds Ratios) from a set of quantitative clinical studies that address the same question. EXAMPLE FROM: Sukeik M, Alshryda S, Haddad FS, Mason JM. Systematic review and meta analysis of the use of tranexamic acid in total hip replacement. J Bone Joint Surg Br. 2011 Jan;93(1):39 46. Review. PubMed PMID: 21196541.
WRITING UP THE METHODS SECTION Check author guidelines for targeted journal; look at other systematic reviews in that journal Follow one of the high quality guides such as PRISMA, Cochrane, or MOOSE More detail is better than less TRACK YOUR NUMBERS!
PRISMA The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta Analyses) statement is the result of an international collaboration and agreement. This replaces the older QUORUM, These standards were developed to promote quality and transparency in reporting systematic review research. analyses. It was published simultaneously in the BMJ, Annals of Internal Medicine, Journal of Clinical Epidemiology, and PLoS Medicine http:// prisma statement.org/statement.htm Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). P referred R eporting I tems for S ystematic Reviews and M eta A nalyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097 Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gtzsche PC, et al. (2009) The PRISMA Statement for Reporting Systematic Reviews and Meta Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med 6(7): e1000100. doi:10.1371/journal.pmed.1000100
Display your search process and track the number of citations through that process in a flow diagram. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). P referred R eporting I tems
EXAMPLE: SEARCH FLOW CHART Blackwood B, Alderdice F, Burns K, Cardwell C, Lavery G, O'Halloran P. Use of weaning protocols for reducing duration of mechanical ventilation in critically ill adult patients: Cochrane systematic review and meta analysis. BMJ. 2011 Jan 13;342:c7237 PMID : 21233157; PMCID : PMC3020589 http ://www.ncbi.nlm.nih.gov/pmc/articles/PMC3020589/pdf/bmj.c7237.pdf
THE COCHRANE COLLABORATION An organization that is internationally recognized for being an important leader in creating high quality systematic reviews and advancing Evidence Based Medicine is the Cochrane Collaboration Circle formed by the 'C' of Cochrane and the mirror image 'C' of Collaboration reflects the international collaboration that makes our work relevant globally The inner part of the logo illustrates a systematic review of data from seven randomized controlled trials (RCTs), comparing one health care treatment with a placebo Each horizontal line represents the results of one trial (the shorter the line, the more certain the result); and the diamond represents their combined results http://www.cochrane.org/
A USEFUL GUIDE The Dartmouth Guide to Searching for Systematic Reviews: http:// researchguides.dartmouth.edu/sys reviews Provides useful information on term harvesting, search strategy development, and project management including downloadable forms that can be adapted to your needs.
THINGS TO REMEMBER PLAN in advance! Remember that this takes time: a systematic review is a RESEARCH project. Use available expertise: involve a medical librarian and a biostatistician early. Keep careful and complete records. Follow the PRISMA Guidelines.
Questions? Need Help? Contact your librarian! Gainesville: http:// www.library.health.ufl.edu/services/liaisons.html Borland: http:// www.library.health.ufl.edu/borland/index.html Or Contact Me! firstname.lastname@example.org 352 273 8441