Linking economic decisions and reviewers in the Campbell and Cochrane Collaborations

1
Linking economic decisions and reviewers in the
Campbell and Cochrane Collaborations

• Miranda Mugford
• Health economics group
• MED, UEA

2
Outline

• Evidence for economic decisions
• Economics and systematic review
• What are the Campbell and Cochrane
  collaborations...?
• The CC Economics Methods Group

3

• decisions about allocation of funds and
  facilities are nearly always based on the opinion
  of consultants but, more and more, requests for
  additional facilities will have to be based on
  detailed arguments with 'hard evidence' as to the
  gain to be expected from the patient's angle and
  the cost. Few could possibly object to this.

Cochrane A. Effectiveness and Efficiency.
Nuffield 1972
4
What needs to be known before adopting a
technology or new practice?

• Does it work?
• It it worthwhile?

5
Decision model health decision
6
Costs and decisions
7
Decision model economic decision
8
Economic decision model

• Effectiveness Outcome 1 Outcome 2
• Incremental cost Cost 1- Cost 2
• Cost effectiveness (incremental cost) /
  (incremental effectiveness)

9
Systematic review of effects of health care

• Finds and sums up current evidence
• Sets criteria for quality of evidence
• Increases chance of detecting a true effect of
  care
• Minimises chance of wrongly finding a form of
  care effective

10
aims to help people make well informed decisions
about health care by preparing, maintaining and
ensuring the accessibility of systematic reviews
of the effects of health care interventions.
11
The Campbell Collaboration
aims to help people make well-informed
decisions by preparing, maintaining, and
promoting access to systematic reviews of studies
on the effects of social and educational policies
and practices.
12
Key features of the Collaborations

• Involving professionals in reviewing own
  practices
• Comprehensive critical review and pooled evidence
  subject to clear criteria
• Regular updated publication of reviews
• Collaborative networks
• review groups
• methods groups
• centres

13

• Reviewers should always be cautious about
  reaching conclusions about implications for
  practice and they should avoid making
  recommendations.
• Alderson P, Green S, Higgins JPT, editors.
  Cochrane Reviewers Handbook 4.2.1 updated
  December 2003. In The Cochrane Library, Issue
  1, 2004. Chichester, UK John Wiley Sons, Ltd.

14
Economic evidence for decisions are there
accepted methods?
15
An exercise on the feasibility of carrying out
secondary economic analyses

• Economists have not yet developed a formal
  methodology for reviewing and summing up evidence
  from individual economic evaluations ... or
  indeed for assessing whether systematic reviews
  are possible in this context
• Jefferson T, Mugford M, Gray A, Demicheli V.
  Health Economics 19965155-165

16
Comparing guidelines on economic evaluation mid
1990s

• Disagreement
• Losses to production
• Discount rates
• Benefit measurement
• What baseline for comparison
• Evaluation type
• Research designs
• Valuing personal time
• many others
• ...

• Not mentioned
• Methods for estimation of costs
• Agreed
• opportunity costs should be measured
• assumptions and sources should be clear

17
Evidence from CEAs in court

• Journal of Health Policy, Politics and Law
• special issue Evidence its meanings in health
  care and in law Vol 26 no 2 April 2001
• article by Jacobson and Kanna
• commentaries by Rennie , Eddy

18
Evidence from economic evaluation

• Cost effectiveness is a widely used but
  imprecise term that means different things to
  different users
• Sooner or later, the dictates of cost containment
  will compel more widespread use of CEA. .. the
  courts will decide whether to defer to the
  medicalprofession or .. take cost effectiveness
  principles into account
• Jacobson and Kanna, 2001 J Health Politics Policy
  and Law

19
Problems with CEAs

• data timeliness
• bias in data sources
• disputes over measures of cost and effect
• subjectivity in specification and valuation
• dispute about benchmark CEA threshold
• no standard off the shelf methodology
• Jacobson and Kanna, 2001 J Health Politics Policy
  and Law

20

• CEAs tend to be riddled with bias and its not
  hard to see that money is the reason for this
• Given that there is no sensible alternative to
  CEA we need to improve the design, conduct,
  analysis and reporting of such analyses..
• The key to good reporting comes from recognising
  that CEAs are, in essence, reviews.
• Rennie 2001

21
Roles of methods groups in Cochrane and Campbell
Collaborations

• Handbook for reviewers
• Advice for training for reviewers
• Peer review of reviews
• Reviewing evidence for best methods
• Discussion forum on methods

22
Cochrane Methods Groups

• Applicability and Recommendations
• Economics ( Web Site ) jointly with Campbell
• Health-Related Quality Of Life
• Individual Patient Data Meta-Analyses
• Non-randomised Studies ( Web Site )
• Prospective Meta-Analysis ( Web site )
• Qualitative Methods ( Web site ) linked with
  Campbell
• Reporting Bias Methods Group
• Screening and Diagnostic Tests (See their Methods
  document and software.)
• Statistical Methods
• Information retrieval, linked with Campbell
• (Methodology Review Group)

23
Campbell Methods Groups

• Statistics Methods Group
• Implementation Process Methods Group
• Information Retrieval Methods Group

24
History of the Cochrane Economics Methods Group

• Oxford workshops 1992, 1993 (OHE)
• Appendix to Cochrane Reviewers Handbook
  (1994-1997)
• Cochrane economics discussion group 1994-98 led
  by Ron Akehurst, Martin Buxton
• Registration of CEMG 1998

25
Progress registering joint methods group

• 2001-2002 Networking to establish interest
• Nov 2002 Nuffield Trust, London. Draft
  registration document discussed
• Feb 2003 Campbell Collaboration Colloquium.
  Stockholm. Penultimate version agreed
• Feb 2004 registration document approved by both
  Steering groups
• Apr 2004 first meeting of convenors

26
Aims of the joint CC EMG group

• Promote and support consideration of economics in
  the Collaborations
• Develop economic methods
• Undertake empirical research
• Link reviewers with economics specialists
• Review validity of economics methods
• Disseminate valid methods and good practice
• Relate to other methods groups

27
Leaders and structure

• Group of convenors
• Chair of convenors
• Administrative coordinator
• Group members
• economists (with field or general interests)
  reviewers interested in economics
• mailing list only

28
Activities of the Cochrane Economics Methods Group

• Links with NHS Economic Evaluation Database
• Advising Cochrane reviewers on economic issues
  (training, peer review, membership of review
  groups)
• Empirical reviews of methods for addressing
  economic questions in RCTs and reviews of
  evidence
• Book on evidence based economic evaluation
• Developing country applications

29
NHS Economic evaluation database

• Pre-dated CEMG
• Autonomous entity based in CRD
• Included in Cochrane Library since 2000
• Research agenda
• links with Cochrane Reviews,
• uses of NEED abstracts..
• other topics ..

30
Including economics in Cochrane Reviews

• Getting reviewers interested
• Critical appraisal tools for economics studies
  understanding economics, looking at NHS EED
• Searching for economics studies new strategies
• What to summarise and synthesise using revman
• Is it a review of economics findings alongside an
  effectiveness review, or an economic evaluation?
• How to address international external validity
  and shelf life

31
Examples of economics methods of concern to
reviewers

• Posing economics questions
• Understanding and critiquing economic data
  collection and analyses
• Building economic models from systematic review
  of effectiveness
• External validity/transferability of economic
  (and other) data

32
Examples of economics approaches and methods of
concern to users of reviews of effectiveness

• The role of economic evaluation to inform policy
  development
• Resolving differences in approaches to economic
  evaluation and their application
• Measurement and presentation of costs and
  benefits
• Measurement or modelling of impact
• Implementation
• Implications for 'roll-out' / mainstreaming

33
The quality of systematic reviews of economic
evaluations in healthcare and what they are
telling us it is time for action

• Vittorio Demicheli (1,2 )
• Tom Jefferson (1,2)
• Luke Vale (2)
• 1. Health Reviews Ltd, UK
• 2. Campbell and Cochrane Economics Methods Group
• JAMA 2001

34
Evidence-based health economicsfrom
effectiveness to efficiency in health care

• Editors
• Cam Donaldson University of Calgary
• Miranda Mugford University of East Anglia
• Luke Vale, University of Aberdeen
• Contributors
• economists and reviewers involved in application
  of economic evaluation in providing and using
  evidence for health care decisions
• published by BMJ books, spring 2002

35
The balancing act

• Whether to address economics questions in
  Cochrane reviews at all?
• Most effective or most cost effective?
• Looking in and looking out of the Cochrane
  Collaboration
• Standardising CEA methods versus better economics
  
• Demand for economics advice outweighs capacity to
  develop methods or give practical input
• Whether economic priorities should determine
  which reviews are done?

36
The vanishing act?

• Funding and pro bono work
• Cochrane ethos
• Priority for review groups for DH cochrane
  funds
• Research council funding for theory driven
  research, not for messy networking?
• Maintaining and developing skills and
  perspectives
• many young economists are nervous of losing their
  spurs so do not become expert in health topics
• health service researchers can lose sight of
  purposes of economic input

37
The future ..

• A more secure base for the group?
• More active engagement of economists working
  with reviewers
• Strengthen links with economic evaluation
  methodologists
• More research on methods for
• synthesis of economic data
• reporting and generalisability
• transparency and detecting bias

38
(No Transcript)
39
Next steps

• Mail survey of members
• Appoint research coordinator at UEA
• Continue to provide workshops for reviewers
• Prioritise research agenda
• Set targets and seek funds