Title: National Obesity Observatory The Standard Evaluation Framework Kath Roberts kath'robertsnoo'org'uk k
1National Obesity Observatory The Standard
Evaluation Framework Kath Roberts
kath.roberts_at_noo.org.uk/katharine.roberts_at_empho.
nhs.ukNick Cavillnick.cavill_at_cavill.net
2Bring me the Evidence!
3One question..
If evaluation was an animal what would it be?
4(No Transcript)
5(No Transcript)
6Finally........
7Has it achieved its objectives?
Has it worked?
Does it work?
What happened?
Was it effective?
Project
Do I need a double blind randomised control
trial?
Could it work?
What questionnaire should I use?
8Standard Evaluation Framework (SEF) for weight
management interventionswww.noo.org.uk/SEF
9Aim of the SEF
- To support high quality, consistent evaluation of
weight management interventions in order to
increase the evidence base
10Objectives of the SEF
- Increase the proportion of weight management
interventions that are evaluated - Increase the quality of evaluations
- Develop a core dataset to increase the
consistency and comparability of evaluations - Provide wider evaluation support
11Target audience for the SEF
- PCT commissioners (to support service
specifications) - PCT DH obesity leads
- Organisations running interventions
- External evaluators
- In some regions (NE), the SEF is a required
element of all new weight management programmes?
12Why use the SEF?
Time
13What interventions should it apply to?
- Any intervention that explicitly sets out to
manage or reduce body weight as an intermediate
or long-term health outcome (including the
primary prevention of weight gain) - Projects focusing on diet, physical activity, or
both in combination - Intended to be applicable to a range of
approaches including interventions conducted with
individuals on a one-to-one basis, or with people
in groups and in clinical or community settings.
14The SEF is not intended for
- Medical treatment e.g. bariatric surgery
prescribing - Wider environmental interventions e.g. changes to
the built environment (although the SEF could
support the evaluation of such interventions
alongside wider evaluation criteria)
15Consultation Processes
- Developed in consultation with academic experts,
representatives from Public Health Observatories,
Primary Care Trusts, Government Offices, Local
Authorities and other relevant organisations and
bodies - Initial list of criteria consulted on using an
online questionnaire - followed up by individual
interviews - Two consultation meetings to finalise criteria
and guidance. - NB the SEF itself will also be evaluated.
16What is in the SEF document?
- An introduction to the concepts of evaluation
- A list of data collection criteria divided into
essential and desirable - A guide to using the criteria and collecting the
data
17Introduction to Evaluation
- What is evaluation and why do it?
- What are the different types of evaluation?
- What are the differences between evaluation,
audit and monitoring? - What sorts of information need to be collected to
carry out evaluations? - What sorts of issues need to be considered as
part of an evaluation?
18SEF Criteria
- Programme details e.g. name of intervention,
aims/objectives - Demographics of individual participants e.g. age,
sex, ethnicity - Baseline data e.g. height/weight, dietary and
physical activity behaviours - Follow-up data (impact and process evaluation)
- Analysis and interpretation
19Supporting SEF guidance
- Details about why criteria have been categorised
as essential or desirable - What the data can and cannot tell you
- Standardised methods for collecting some of the
data - Suggested standardised definitions for data
- Suggested data collection tools
- Signposting to further guidance and useful
information
20Recommended Physical Activity Measurement Tools
- 4 instruments met the following criteria
- Relatively easy and practical to administer
- Mean difference between energy estimated by
self-report and doubly labelled water lt40 - Instrument should have been used in a trial of
exercise promotion and detected a significant
difference between intervention and control
participants - Population in the validity study generalisable to
wider population - Appendix to SEF by Dr Melvyn Hillsdon
21Recommended Physical Activity Measurement Tools
- International Physical Activity Questionnaire
(IPAQ) long version - 7day recall
- Valid in 15-69 year olds only
- Stanford 7-day recall
- Self report for sleep, moderate, hard and very
hard activities - Valid in adults only
22Recommended Physical Activity Measurement Tools
- 7-day physical activity diary
- Self-completion diary ticking 15min blocks of
activity - Valid in adults only
- New Zealand Physical Activity Questionnaire
- 7day Self-completion recall, using prompt cards
- Valid in middle aged adults only
23Limitations
- No existing physical activity measure perfectly
meets all criteria - Lack of suitable measures for children
- No gold standard for measuring sedentary
behaviours - Most measures have quite complex scoring systems
24Evaluating the SEF
- How we are evaluating the SEF
- SEF audit tool (currently piloted)
- Qualitative interviews with regional leads
- Qualitative interview with programme leads
- How you can help?
- Use the audit tool
- Take part in interviews
- Feedback your comments suggestions
- Measurement tools used why
- What measurement tools youve used for children
- Any problems using the SEF for physical activity
interventions - Anything you would like to see in the revised SEF
25Future SEF products
- Stand alone evaluation guide
- Stand alone criteria tables
- Example case studies
- Excel template for audit data
- Excel template for individual study data
- Online audit (currently taking place in the NE),
national audit under development.
26Exercise
- Each group to report back
- How is the SEF going to be used in your locality?
- What possible barriers are there to using the
SEF? - How could these barriers be overcome?
- What could NOO provide to support use of the SEF
further?
27Case Study Derbyshire FLIP
- Locally developed programmes to come together
under umbrella of Derbyshire FLIP - BMI
- Matching objectives with outcomes
- Clear referral routes
- Validated dietary and PA questionnaires
- Bringing 3 programmes together with common
weight management objectives - Data collection and analyses
28Questions?