Systematic Reviews Practicalities and Realities - PowerPoint PPT Presentation

About This Presentation

Systematic Reviews Practicalities and Realities


Systematic Reviews Practicalities and Realities Alison Brettle, Research Fellow (Information) Salford Centre for Nursing, Midwifery and Collaborative Research – PowerPoint PPT presentation

Number of Views:183
Avg rating:3.0/5.0
Slides: 38
Provided by: dkoufogi


Transcript and Presenter's Notes

Title: Systematic Reviews Practicalities and Realities

Systematic Reviews Practicalities and Realities
  • Alison Brettle,
  • Research Fellow (Information)
  • Salford Centre for Nursing, Midwifery and
    Collaborative Research
  • University of Salford

Session Overview
  • Practical overview of systematic reviews and what
    you need to conduct one
  • Discussion

What is a systematic review?
  • An overview of primary research studies conducted
    according to explicit and reproducible
  • A rigorous method of summarising research
  • Shows what we know and dont know about a topic
  • Provides evidence of effectiveness (or not) by
    summarising and appraising relevant evidence

Systematic reviews aim
  • To find all relevant research studies (published
    and unpublished)
  • To assess each study on basis of defined criteria
  • Synthesise the findings in an unbiased way
  • Present a balanced and impartial summary of the
    findings taking any flaws into consideration

Systematic review models
  • Medical/Health care
  • Cochrane Collaboration, NHS Centre for Reviews
    and Dissemination
  • Usually includes high quality research evidence
  • Often includes meta-analysis (mathematical
    synthesis of results of 2 studies that addressed
    same hypothesis in same way)
  • Social care/Social Sciences
  • SCIE, EPPI Centre, Campbell Collaboration
  • Often include wider range of studies including
  • Often narrative synthesis of evidence

Systematic review process
  • Define/focus the question
  • Develop a protocol
  • Search the literature (possibly 2 stages scoping
    and actual searches)
  • Refine the inclusion/exclusion criteria
  • Assess the studies (data extraction tools, 2
    independent reviewers)
  • Combine the results of the studies to produce
    conclusion can be a qualitative or quantitative
  • Place findings in context quality and
    heterogeniety of studies, applicability of

  • You are part of a systematic review team and this
    is your first meeting. Your task is to draft a
    protocol by the end of the meeting then present
    to the other groups

Components of a protocol
  • Background/context leading to question
  • Review question
  • Inclusion/Exclusion criteria
  • Methods for
  • Identifying evidence
  • Selecting studies
  • Extracting studies
  • Assessing quality of studies
  • Synthesising the results
  • Disseminating the findings

Lessons learned in systematic reviews

Workplace physical activity interventions a
systematic review of the research evidence
  • Lindsey Dugdill, Alison Brettle, Claire Hulme,
    Serena McCluskey, Andrew Long

  • Policy
  • Increased PA important for prevention of chronic
  • Trend on a worldwide and UK scale that PA levels
    are declining

  • To identify which types of workplace PA
    intervention were effective in changing PA
    behaviour for different sectors of the workforce
    and different types of workplace
  • To ascertain which aspects of intervention design
    and delivery contributed to effectiveness and
    contributory motivators, barriers and facilitators

Lesson 1. Focus the question
  • More focussed the question, easier to undertake
    literature search and decide on relevant studies
  • Need to maintain a balance between a clearly
    defined topic and a meaningful and relevant
  • Can help scope the literature to help develop the
  • Refine question as part of review process

Focussing the question
  • Options
  • empirically led -gt literature scoping -gt
    narrowing down topic on the basis of what is
    there in the evidence base
  • theoretical and conceptual -gt concept mapping,
    explore the meaning given to the concepts,
    embracing user and carer notions of desired
    outcomes (based on previous research) themselves
    central to a social care perspective
  • Funder/policy driven

Lesson 2. Scope the literature
  • Provides an indication of literature available
    and what needs to be examined
  • Helps focus the question
  • Helps clarify initial inclusion and exclusion

Lesson 3. Write a protocol
  • Plan what you are going to do in the review
  • Set out the background and objectives
  • Outline the resources you will use
  • Establish inclusion/exclusion criteria
  • How will data be extracted (what will be
  • How will you synthesise literature

Literature search
  • 12 electronic databases 1996-2007
  • Selected websites of known organisations
  • Citation tracking
  • Could also use
  • hand-searching of journals
  • grey literature

Lesson 4. Use multiple sources of information
  • Databases only part of the picture
  • Handsearching identifies further references
  • Citation tracking adds more references
  • SMI review 96 studies (73 databases, 23
    handsearching/citation tracking)
  • Counselling 47 studies (aprox 10 handsearching,
    4 citation tracking)

Lesson 5. Refine your search plan as you carry
out the literature searches
  • Theory
  • Develop your search plan
  • Everything that follows flows smoothly and easily
  • Practice
  • Modify your approach as the search progresses
  • Use spiral approach to refine
  • Ongoing discussions within team

Search spiral
  • Scoping (initial database searches)
  • Refining (indepth searches of range of sources)
  • Confirming (hand searches, citation tracking)

  • Intervention aimed to increase PA
  • Aimed at employed adults
  • Initiated or endorsed by employer
  • Outcome measures included a measure of PA
  • Evidence of behaviour change
  • English
  • Published post 1996
  • Took place in UK, Australia, NZ, Europe

  • Self employed or unemployed adults
  • Adults needing specialist advice regarding PA
  • No evidence of behaviour change or PA measure
  • Insufficient detail reported in study
  • Cost effectiveness
  • Located in US or Asia

Lesson 6. How will you refine the
inclusion/exclusion criteria?
  • Tighter the criteria
  • less papers to review
  • BUT
  • will your review draw any meaningful conclusions?
  • Focussing the question v quality of studies

Overview of search and appraisal process
  • 38 papers representing 33 studies
  • Each paper appraised by 1 reviewer, using a
    standardised protocol, 20 sample reviewed by 2
    reviewers and each checked as put on database
  • Quality of papers high, good, - poor
    evidence (using protocols/guides provided by
  • Categorised by intervention and quality rating
  • Summary table
  • Conclusions on the whole derived from those
    classified as high or good

Lesson 7 Know what you want to find out
  • Select your critical appraisal/data extraction
    tool carefully
  • Make sure it captures the elements that you wish
    to write about in your final report
  • Think what data need to extract
  • Narrative review v meta analysis
  • Quality of studies what evidence are you going
    to include? How are you going to assess the
  • Do you need to refine the inclusion/exclusion
    criteria as you go along?

Findings published systematic reviews
  • Inconclusive review level evidence that workplace
    PA interventions were effective in increasing PA

Findings Stairwalking
  • 7 studies (1 high, 2 good, 4 poor)
  • 4 demonstrated that posters and signs can
    increase stair use in the short term
  • Little evidence of effectiveness

Findings Walking interventions
  • 4 studies (2 good, 2 poor)
  • 3 used pedometers (objective measure)
  • All 4 also used self reported counts
  • Evidence that workplace walking interventions can
    increase daily step counts

Findings Active travel
  • 3 studies (1 good, 2 poor)
  • Evidence from 1 study that a walking and cycling
    to work campaign using written materials can
    increase walking to work in economically
    advantaged women

Findings multi component interventions
  • 16 studies (2 high, 1 good)
  • Wide range of interventions
  • Difficult to attribute which bit of the
    intervention was causing the effect
  • Limited evidence that counselling has effect on
    workplace PA

  • Quality of evidence
  • Exclusion of studies from US and Asia
  • Most of studies took place in large public sector
  • Reliance on self report measures

Lesson 9. Set up a system to keep track of the
review process
  • Use a package such as reference manager or
    endnote to keep track of references
  • Large reviews need method of keeping track of
    data extraction, exel, access or specialised
  • Update at every stage of the process
  • Compiles references at the end
  • Need a hard copy system too for the actual papers
    and a system for keeping track of results
  • If you do not know where you are up to it is
    unlikely that your review will be systematic

System to manage process the physical
  • Putting material into endnote
  • Direct export from many sources
  • From a file
  • Direct input
  • How are you going to manage the hard copies
  • What about your notes?
  • What systems do you use?

Reducing and organising
  • Using groups and adding notes to Endnote
  • Using a screening tool (once youve ditched the
    really irrelevant ones)

Data extraction/Critical Appraisal
  • Tools
  • Many widely available
  • Can adapt to suit your own study
  • Could design own
  • Assessing quality systems available be
    explicit if you are going to do this
  • Storing the data
  • Template in word/Table in word
  • Excel spreadsheet
  • Other?

Useful resources systematic reviews
  • Cochrane Collaboration
  • http//
  • http//
  • Centre for Reviews and Dissemination
  • http//
  • handbook for conducting systematic reviews,
  • http//
  • Searching for systematic reviews
  • EPPI-Centre Stages of a review
  • http//
  • SCIE - The conduct of systematic research reviews
    for SCIE knowledge reviews
  • http//
Write a Comment
User Comments (0)