Systematic Reviews Practicalities and Realities

1
Systematic Reviews Practicalities and Realities

• Alison Brettle,
• Research Fellow (Information)
• Salford Centre for Nursing, Midwifery and
  Collaborative Research
• University of Salford

2
Session Overview

• Practical overview of systematic reviews and what
  you need to conduct one
• Discussion

3
What is a systematic review?

• An overview of primary research studies conducted
  according to explicit and reproducible
  methodology
• A rigorous method of summarising research
  evidence
• Shows what we know and dont know about a topic
  area
• Provides evidence of effectiveness (or not) by
  summarising and appraising relevant evidence

4
Systematic reviews aim

• To find all relevant research studies (published
  and unpublished)
• To assess each study on basis of defined criteria
• Synthesise the findings in an unbiased way
• Present a balanced and impartial summary of the
  findings taking any flaws into consideration

5
Systematic review models

• Medical/Health care
• Cochrane Collaboration, NHS Centre for Reviews
  and Dissemination
• Usually includes high quality research evidence
  RCTs
• Often includes meta-analysis (mathematical
  synthesis of results of 2 studies that addressed
  same hypothesis in same way)
• Social care/Social Sciences
• SCIE, EPPI Centre, Campbell Collaboration
• Often include wider range of studies including
  qualitative
• Often narrative synthesis of evidence

6
Systematic review process

• Define/focus the question
• Develop a protocol
• Search the literature (possibly 2 stages scoping
  and actual searches)
• Refine the inclusion/exclusion criteria
• Assess the studies (data extraction tools, 2
  independent reviewers)
• Combine the results of the studies to produce
  conclusion can be a qualitative or quantitative
  (meta-analysis)
• Place findings in context quality and
  heterogeniety of studies, applicability of
  findings

7
Task

• You are part of a systematic review team and this
  is your first meeting. Your task is to draft a
  protocol by the end of the meeting then present
  to the other groups

8
Components of a protocol

• Background/context leading to question
• Review question
• Inclusion/Exclusion criteria
• Methods for
• Identifying evidence
• Selecting studies
• Extracting studies
• Assessing quality of studies
• Synthesising the results
• Disseminating the findings

9
Lessons learned in systematic reviews
10

Workplace physical activity interventions a
systematic review of the research evidence

• Lindsey Dugdill, Alison Brettle, Claire Hulme,
  Serena McCluskey, Andrew Long

11
Background

• Policy
• Increased PA important for prevention of chronic
  illness
• Trend on a worldwide and UK scale that PA levels
  are declining

12
Objectives

• To identify which types of workplace PA
  intervention were effective in changing PA
  behaviour for different sectors of the workforce
  and different types of workplace
• To ascertain which aspects of intervention design
  and delivery contributed to effectiveness and
  contributory motivators, barriers and facilitators

13
Lesson 1. Focus the question

• More focussed the question, easier to undertake
  literature search and decide on relevant studies
• Need to maintain a balance between a clearly
  defined topic and a meaningful and relevant
  question
• Can help scope the literature to help develop the
  question
• Refine question as part of review process

14
Focussing the question

• Options
• empirically led -gt literature scoping -gt
  narrowing down topic on the basis of what is
  there in the evidence base
• theoretical and conceptual -gt concept mapping,
  explore the meaning given to the concepts,
  embracing user and carer notions of desired
  outcomes (based on previous research) themselves
  central to a social care perspective
• Funder/policy driven

15
Lesson 2. Scope the literature

• Provides an indication of literature available
  and what needs to be examined
• Helps focus the question
• Helps clarify initial inclusion and exclusion
  criteria

16
Lesson 3. Write a protocol

• Plan what you are going to do in the review
• Set out the background and objectives
• Outline the resources you will use
• Establish inclusion/exclusion criteria
• How will data be extracted (what will be
  extracted)
• How will you synthesise literature

17
Literature search

• 12 electronic databases 1996-2007
• Selected websites of known organisations
• Citation tracking
• Could also use
• hand-searching of journals
• grey literature

18
Lesson 4. Use multiple sources of information

• Databases only part of the picture
• Handsearching identifies further references
• Citation tracking adds more references
• SMI review 96 studies (73 databases, 23
  handsearching/citation tracking)
• Counselling 47 studies (aprox 10 handsearching,
  4 citation tracking)

19
Lesson 5. Refine your search plan as you carry
out the literature searches

• Theory
• Develop your search plan
• Everything that follows flows smoothly and easily
• Practice
• Modify your approach as the search progresses
• Use spiral approach to refine
• Ongoing discussions within team

20
Search spiral

• Scoping (initial database searches)
• Refining (indepth searches of range of sources)
• Confirming (hand searches, citation tracking)

21
Inclusion

• Intervention aimed to increase PA
• Aimed at employed adults
• Initiated or endorsed by employer
• Outcome measures included a measure of PA
• Evidence of behaviour change
• English
• Published post 1996
• Took place in UK, Australia, NZ, Europe

22
Exclusion

• Self employed or unemployed adults
• Adults needing specialist advice regarding PA
• No evidence of behaviour change or PA measure
• Insufficient detail reported in study
• Cost effectiveness
• Located in US or Asia

23
Lesson 6. How will you refine the
inclusion/exclusion criteria?

• Tighter the criteria
• less papers to review
• BUT
• will your review draw any meaningful conclusions?
• Focussing the question v quality of studies

24
Overview of search and appraisal process
25
Appraisal

• 38 papers representing 33 studies
• Each paper appraised by 1 reviewer, using a
  standardised protocol, 20 sample reviewed by 2
  reviewers and each checked as put on database
• Quality of papers high, good, - poor
  evidence (using protocols/guides provided by
  NICE)
• Categorised by intervention and quality rating
• Summary table
• Conclusions on the whole derived from those
  classified as high or good

26
Lesson 7 Know what you want to find out

• Select your critical appraisal/data extraction
  tool carefully
• Make sure it captures the elements that you wish
  to write about in your final report
• Think what data need to extract
• Narrative review v meta analysis
• Quality of studies what evidence are you going
  to include? How are you going to assess the
  quality
• Do you need to refine the inclusion/exclusion
  criteria as you go along?

27
Findings published systematic reviews

• Inconclusive review level evidence that workplace
  PA interventions were effective in increasing PA

28
Findings Stairwalking

• 7 studies (1 high, 2 good, 4 poor)
• 4 demonstrated that posters and signs can
  increase stair use in the short term
• Little evidence of effectiveness

29
Findings Walking interventions

• 4 studies (2 good, 2 poor)
• 3 used pedometers (objective measure)
• All 4 also used self reported counts
• Evidence that workplace walking interventions can
  increase daily step counts

30
Findings Active travel

• 3 studies (1 good, 2 poor)
• Evidence from 1 study that a walking and cycling
  to work campaign using written materials can
  increase walking to work in economically
  advantaged women

31
Findings multi component interventions

• 16 studies (2 high, 1 good)
• Wide range of interventions
• Difficult to attribute which bit of the
  intervention was causing the effect
• Limited evidence that counselling has effect on
  workplace PA

32
Limitations

• Quality of evidence
• Exclusion of studies from US and Asia
• Most of studies took place in large public sector
  workplaces
• Reliance on self report measures

33
Lesson 9. Set up a system to keep track of the
review process

• Use a package such as reference manager or
  endnote to keep track of references
• Large reviews need method of keeping track of
  data extraction, exel, access or specialised
  software
• Update at every stage of the process
• Compiles references at the end
• Need a hard copy system too for the actual papers
  and a system for keeping track of results
• If you do not know where you are up to it is
  unlikely that your review will be systematic

34
System to manage process the physical

• Putting material into endnote
• Direct export from many sources
• From a file
• Direct input
• How are you going to manage the hard copies
• What about your notes?
• What systems do you use?

35
Reducing and organising

• Using groups and adding notes to Endnote
• Using a screening tool (once youve ditched the
  really irrelevant ones)

36
Data extraction/Critical Appraisal

• Tools
• Many widely available
• Can adapt to suit your own study
• Could design own
• Assessing quality systems available be
  explicit if you are going to do this
• Storing the data
• Template in word/Table in word
• Excel spreadsheet
• Other?

37
Useful resources systematic reviews

• Cochrane Collaboration
• http//www.cochrane.org/
• http//www.cochrane.org/docs/irmg.htm
• Centre for Reviews and Dissemination
• http//www.york.ac.uk/inst/crd/
• handbook for conducting systematic reviews,
• http//www.york.ac.uk/inst/crd/methods.htm
• Searching for systematic reviews
  http//www.york.ac.uk/inst/crd/revs.htm
• EPPI-Centre Stages of a review
• http//eppi.ioe.ac.uk/cms/Default.aspx?tabid89
• SCIE - The conduct of systematic research reviews
  for SCIE knowledge reviews
• http//www.scie.org.uk/publications/details.asp?pu
  bID111