Title: Obesity Evaluation Toolkit: Resources for Evaluating Community-Level Obesity Prevention Efforts
1Institute of Medicine
Obesity Evaluation Toolkit Resources for
Evaluating Community-Level Obesity Prevention
Efforts Webinar August 25, 2015
21. Introduction Leslie Sim, M.P.H. Institute
of Medicine of The National Academies of Science,
Engineering, and Medicine 2. Overview of
report, Evaluating Obesity Prevention Efforts A
Plan for Measuring Progress Lawrence W. Green,
Dr. P.H., M.P.H. University of California, San
Francisco
3- 3. Steps in evaluating community-level obesity
prevention efforts - Stephen Fawcett, Ph.D.
- Work Group for Community Health and Development,
University of Kansas - 4. Web-based resources to support your efforts
- Christina Holt, M.A.
- Community Tool Box, Work Group for Community
Health and Development, University of Kansas - 5. Questions
4- Submit questions - toolbox_at_ku.edu
- Issues connecting? - 1-866-770-8162
5Evaluating Obesity Prevention EffortsA Plan for
Measuring Progress
Lawrence W. Green, Chair and 13 members of The
Committee on Evaluating Progress of Obesity
Prevention Efforts
6Committee on Evaluating Progress of Obesity
Prevention Efforts
- Lawrence W. Green, Dr.P.H. (Chair)
- University of California, San Francisco
- Christina Bethell, Ph.D., M.P.H., M.B.A.
- Johns Hopkins University
- Ronette R. Briefel, Dr.P.H., R.D.
- Mathematica Policy Research
- Ross C. Brownson, Ph.D.
- Washington University in St. Louis
- Jamie F. Chriqui, Ph.D., M.H.S.
- University of Illinois at Chicago
- Stephen Fawcett, Ph.D.
- University of Kansas
- Brian R. Flay, D.Phil.
7- AN URGENT
- NEED FOR
- EVALUATION
8What we need to know from evaluation
- Where are we in making progress and with whom?
(current status) - How are we doing in making progress? (trends over
time in assessing needs and implementation of
policies and strategies) - What works in which populations?
- What are the unintended consequences?
9Committee on Evaluating Progress of Obesity
Prevention Efforts
- Study Charge
- to develop a concise and actionable plan for
measuring progress in obesity prevention efforts
for the nation and adaptable guidelines for
community assessments and evaluation.
10(No Transcript)
11The Concise and Actionable Plan
- Enhanced Leadership
- Enhanced Roadmap
- Enhanced Capacity and Infrastructure
- Included resources
- Indicators for the evaluation plans
- Tools and methods for assessing progress in
populations at greater risk for obesity - Community health assessment, surveillance, and
monitoring of interventions
12Interdependence of National and Community Obesity
Evaluation Plans
Core indicators, Data sources, Resources,
Methodologies
Community Obesity Evaluation Plan
National Obesity Evaluation Plan
Contextual data, Feasibility, Local innovation
13Recommendations
- Improve Leadership and Coordination
- Improve Data Collection
- Provide Common Guidance
- Improve Access to and Dissemination of
Information - Improve Workforce Capacity
- Address Disparities and Health Equity
- Support a Systems Approach
14To access the full report and related
dissemination materials(url http//iom.nationala
cademies.org/Reports/2013/Evaluating-Obesity-Preve
ntion-Efforts-A-Plan-for-Measuring-Progress.aspx)
Interactive indicator widget
Pull-out summary of indicators
4-page report brief
15Indicators of Progress (Excerpt from Table 4-1)
Indicator Topic Objective
OVERARCHING/SYSTEM-LEVEL INDICATORS OVERARCHING/SYSTEM-LEVEL INDICATORS
Obesity-adult Reduce the proportion of adults who are obese (body mass index (BMI) 30)
APOP GOAL AREA 1 PHYSICAL ACTIVITY ENVIRONMENT APOP GOAL AREA 1 PHYSICAL ACTIVITY ENVIRONMENT
Adult physical activity Increase the proportion of adults who meet current federal physical activity guidelines for aerobic physical activity and for muscle-strengthening activity
16National and State Level Indicators (excerpt,
Table 6-4)
Indicator Topica Data Source or Documentationb National Planc State Plansc WHO Proposed Indicatorsd
Overarching/System Level Overarching/System Level Overarching/System Level Overarching/System Level Overarching/System Level
Obesity-adult BRFSS NHANES Core
2. Obesity-adolescent NHANES YRBSS Core
3. Obesity-child NHANES Core
Goal Area 2 Food and Beverage Environment Goal Area 2 Food and Beverage Environment Goal Area 2 Food and Beverage Environment Goal Area 2 Food and Beverage Environment Goal Area 2 Food and Beverage Environment
28. Adult energy intake NHANES Expanded
29. Child and adolescent energy intake NHANES Expanded
30. Sugar-sweetened beverage policies in schools BTG CLASS SHPPS SNDA Expanded
17Community Level Indicators (excerpt, Table 7-2)
Indicator Topica Data Source Current Availability by Community Sizeb Current Availability by Community Sizeb
Larger Smaller
Overarching/System-Level
Obesity-adult BRFSS
Overweight-adult BRFSS
Obesity-adolescent YRBSS, School reports
Goal Area 2 Food and Beverage Environment
Sugar-sweetened beverage policies in schools (school district) SHPPS
Sugar-sweetened beverage consumption YRBSS (adolescent)
School policies to facilitate access to clean drinking water SHPPS
Note Larger population gt50,000 Smaller
population lt 50,000
18Some Steps in Evaluating Community-Level Obesity
Prevention EffortsStephen Fawcett, Work Group
for Community Health and Development, University
of Kansas
19Vision for IOM Report
- Assure collection and analysis oftimely and
meaningful datato inform and improve
obesityprevention efforts at national,
state,and community levels
20CONTEXT for Evaluation
- Multi-component, multi-sector, and multi-level
interventions - Indicators/measures of varying quality utility
- Varying capacity, capabilities, leadership, and
resources
21ACTIVITIES Develop Resources for Training,
Technical Assistance, and Dissemination
- Products/protocols/templates
- Web-based support training
- Curricula
- Clearinghouses for measures
- Communications
22Steps/Components of a Community Evaluation Plan
(Box 8-1)
- Design stakeholder involvement.
- Identify resources for the monitoring and
summative evaluation. - Describe the interventions framework, logic
model, or theory of change. - Focus the monitoring and summative evaluation
plan. - Plan for credible methods.
- Synthesize and generalize.
231. Design stakeholder involvement
- Identify stakeholders
- Consider the extent of stakeholder involvement
- Assess desired outcomes of monitoring and
summative evaluation - Define stakeholder roles in the monitoring and
summative evaluation
24Participatory Evaluation (CBPR)
252. Identify resources for monitoring and
summative evaluation
- Person-power resources
- Data collection resources
26Some Web-Based Data Sources(Table 7-5)
- Community Commons CHNA.org
- County Health Rankings
- USDA Food Atlas
- CDC Diabetes Interactive Atlas
- Census ACS and County/Zip Business Patterns
- HHS Community Health Status Indicators
27What indicators available at local level?
- Available for all/ many communities
- Adult obesity/overweight
- Activity Active transport by walking, bicycling,
density of recreational facilities, leisure-time
PA - Nutrition Adult F/V, food outlet density,
farmers market density, food deserts, SNAP/WIC
store density - Some larger communities also have
- Youth obesity/overweight
- Activity Youth PA, screen time, school PA
participation - Nutrition SSB consumption, youth F/V
28Data Availability (Table 7-2 Example)
293. Describe the interventions framework or logic
model, or theory of change.
- Purpose or mission
- Context or conditions
- Inputs resources and barriers
- Activities or interventions
- Outputs of activities
- Intended effects or outcomes
30Generic logic model for community obesity
prevention (Figure 8-1, adapted).
314. Focus the monitoring and summative evaluation
plan.
- Purpose or uses What does the monitoring and
summative evaluation aim to accomplish? - Set priorities by end-user questions, resources,
context - What questions will the monitoring and summative
evaluation answer? - Ethical implications (benefit outweighs risk)
32End-User Focused Evaluation QuestionsSome
examples
- How fully was intervention implemented?
- Did the intervention have desired effects?
- What was the impact on participants/ population?
- With whom?
- Under what conditions?
335. Plan for credible methods.
- Stakeholder agreement on methods
- Indicators of success
- Credibility of evidence
34Some Emerging Methods forData Collection
- Environmental change data
- Documentation of initiatives
- Unobtrusive observations
- Secondary data (e.g., GIS)
- Policy change data
- Documentation of initiatives
- Surveillance
- Systems change data
- Mapping changing relationships
35 Indicators of Success
- Translate expected effects (logic model) into
specific measurable units - Examples include
- Program Outputsunits of activities delivered
- Intermediate Outcomeschanges in communities and
systems (program, policy, environment) - Behavioral Outcomeschanges in diet and physical
activity - Population-level Outcomesreduced prevalence of
obesity
36Evaluation Designs Match to Goal/Context
Qualitative methods interviews, focus groups,
photo-voice, etc.
376. Synthesize and generalize.
- Disseminating and compiling studies
- Learning more from implementation
- Ways to assist generalization
- Shared sense-making and cultural competence
- Disentangling effects of interventions
38Logic Model Design and Shared Sensemaking
39Obesity Evaluation ToolkitWeb-based Resources
for Community Evaluation
- CONTEXT Distributed evaluation workforce
- People we will never see
- In places we will never be
- TOOLKIT Just-in-time resources for
- Training
- Technical Assistance
40Web-based resources to support your
efforts Christina Holt, M.A. Community Tool Box,
Work Group for Community Health and Development,
University of Kansas
41Navigating to the Toolkit
- The Obesity Evaluation Toolkit is available
online - http//iom.nationalacademies.org/activities/nutrit
ion/obesityprevprogress/resources-evaluating-commu
nity-level-obesity-prevention-efforts
42(No Transcript)
43What you will find
44(No Transcript)
45(No Transcript)
46(No Transcript)
47Evaluation Toolkit
48Community Tool Box Example Resource
49Troubleshooting Guide
50(No Transcript)
51(No Transcript)
52(No Transcript)
53(No Transcript)
54(No Transcript)
55(No Transcript)
56(No Transcript)
57Questions/ Discussion