The need for better evidenced public health policy

1
The need for better evidenced public health policy

• Sally Macintyre

2
DECIPHer

• Why do we need centres of excellence in public
  health research?

3
10 years ago (Acheson Report) Evaluation Group

• lots of data documenting health inequalities
• lots of research attempting to explain health
  inequalities
• little information about effectiveness of
  interventions
• even less information about potential harms,
  costs or priorities
• evidence clearer for downstream than upstream
  interventions
• (Macintyre, Chalmers, Horton, Smith 1998)

4
Lack of evidence

• of published or funded public health research in
  UK, 4 deal with interventions rather than
  descriptions of the problem
• only 10 of them (0.4) deal with outcomes of
  interventions
• in specific topic areas evidence about
  inequalities, and tools for capturing social
  differences, not very robust
• very few systematic reviews have focused on
  effect of interventions on inequalities in health

(Millward L, Kelly MP Nutbeam D, 2001 Public
Health Intervention Research The Evidence,
London, HDA)
5
(Wanless report Securing good health for the
whole population 2004)

• Although there is often evidence on the
  scientific justification for action and for some
  specific interventions, there is generally little
  evidence about the cost-effectiveness of public
  health and preventative policies or their
  practical implementation
• little evidence about what works among
  disadvantaged groups to tackle some of the key
  determinants of health inequalities

6
Reducing Inequalities in HealthA European
Perspective

• Work policies - poor design, or health outcomes
  unevaluated
• food policies - little information on the long
  term effects .
• smoking - little direct evidence that permits
  any definitive judgements
• children - many interventions most are not well
  known and very few have been systematically
  evaluated
• access to healthcare - paucity of studies about
  the best ways to reach poorer people with
  appropriate and effective services
• (Mackenbach Bakker, 2002)

7
Scottish National Health Demonstration
Projects

• Poor evidence base in first place
• Evaluations set up too late
• Programmes non evaluable
• Decisions on phase 2 taken before evaluations
  complete

(Evaluation Task Force Review, Scottish
Executive, 2004)
8
Why Lack of evidence?

• Many evaluations focus on inputs, throughputs and
  customer or professional satisfaction rather than
  on outcomes
• When evaluations do look at outcomes, health is
  often not studied
• Few interventions are rolled out in ways which
  permit conclusive evaluation
• Most evaluations focus on, and have sufficient
  sample size for, assessment of the overall effect
  but not on differential effects by SES
• Policies may take some time to have the desired
  effects
• Lack of UK studies

9
Public Health/Health promotion antipathy to
biomedical model

• Techniques such as RCTs and systematic reviews
  seen as exclusively biomedical
• many public health researchers/activists already
  committed to particular research approaches
• many public health researchers/activists already
  committed to particular policies

10
House of Commons Health Committee

• the Government's approach to designing and
  introducing new policies which make meaningful
  evaluation impossible. . Even where
  evaluation is carried out, it is usually "soft",
  amounting to little more than examining processes
  and asking those involved what they thought about
  them.
• All too often Governments rush in with
  insufficient thought, do not collect adequate
  data at the beginning about the health of the
  population which will be affected by the
  policies, do not have clear objectives, make
  numerous changes to the policies and its
  objectives and do not maintain the policy long
  enough to know whether it has worked.

11
House of Commons Health Committee

• There are many area-based initiatives that
  are introduced one after the other. They are only
  given a few years to prove themselves and then,
  when they cannot prove themselves within that
  short period, they are stopped and something new
  comes along. There is a continual procession of
  area-based initiatives and that in itself is
  quite disruptive. Nothing is given time to really
  bed in and function.

12
Systematic review of area based regeneration
initiatives in the UK

• Little evidence of the impact of national urban
  regeneration on socio-economic or health
  outcomes.
• Changes often no different from national trends.
• However, some harms
• Single Regeneration Budget 1996 - 1999
  deterioration in self reported health
• Urban programme and City Challenge worsening of
  unemployment
• Estate Action increased housing costs
• Housing improvement in 1930s Scotland rents
  doubled, mortality rates increased

(Thomson et al, 2006)
13
House of Commons Health Committee

• There is an ethical imperative to develop and use
  evidence-based policy. All the reforms we have
  discussed are experiments on the public and can
  be as damaging (in terms of unintended effects
  and opportunity cost) as unevaluated new drugs or
  surgical procedures. Such wanton large-scale
  experimentation is unethical, and needs to be
  superseded by a more rigorous culture of
  piloting, evaluating and using the results to
  inform policy.
• Simple changes to the design of policies and how
  they are introduced could make all the
  difference. We recommend that all future
  initiatives to tackle health inequalities
  initiatives must, prior to their introduction,
  demonstrate adherence to the basic set of
  research guidelines we have detailed in this
  chapter, which include

14
Basic principles of evaluation

• importance of a counterfactual
• design should fit the specific features of the
  intervention
• randomisation prevents unknown biases
• prospective design, with baseline measures
• decide primary outcomes a priori
• measure direct and indirect impacts
• appropriate lengths of follow-up
• build in methods of measuring long-term outcomes
• objective assessment of both positive and
  negative outcomes
• non-suppression of negative findings
• explicit ideas about how the intervention is
  expected to workcollect information by gender,
  age, ethnicity and SES
• examine how the intervention is actually
  implemented
• include an economic evaluation.

15
DECIPHer

• Thats why we need centres of excellence like
  DECIPHer!
• Good luck.