Title: What Works in MultiSite Evaluations of Nutrition Education Interventions
1What Works in Multi-Site Evaluations of Nutrition
Education Interventions?
- Andy Fourney, Andrew Bellow, Patrick Mitchell,
Sharon Sugerman, Angie Keihner - California Nutrition Network for Healthy Active
Families - Research and Evaluation Unit
2California Nutrition Network for Healthy, Active
Families
- Targets low-income families
- Conducts a social marketing campaign to increase
- fruit and vegetable consumption
- physical activity
- promote the Food Stamp Program
- Contracts over 170 agencies
- Receives almost 106 million from USDA food stamp
program
3Evaluation for contractors receiving 350k
Strategies
Factors that Influence FVC
Change in Behavior
Process
Short-term Impact
Long-term impact
4Network Developed Methodology
- SOW Impact evaluation objective
- Compendium of Surveys
- Impact Evaluation Handbook
- Trainings and workshops
- Data entry templates
- Evaluation report template
- Monitoring
5Scope of Work Impact Evaluation Objective
- Impact objective By Sep. 30 20xx, a sample of
4th and 5th grade students at intervention sites
will report an increase in fruit and vegetable
consumption and an increase in one or more
factors related to fruit and vegetable
consumption, such as knowledge, preferences,
outcome expectations and self-efficacy.
6Impact Evaluation Handbook
7Compendium of Surveys
8Trainings
Workshops
9One-on-one TA
10Data Entry Templates
- Created in MS Excel
- Provides same results as SPSS
- Calculate pretest mean, posttest mean, difference
and p-value for a paired t-test - Customized for each survey
11Data Entry Templates
12Data Entry Templates
13Evaluation Report Template
- 18 questions
- Design
- Methods
- Results
- Reflection
- Successes
14Report Template - Design
- Please type your impact objective.
- What were the factors you measured?
- What intervention and strategies were implemented
to change the factors? - What survey tool did you use?
- Were there questions on the survey tool that were
not addressed in the intervention? If so, why? - Describe challenges faced during the intervention
implementation. - What type of design was used, e.g., pretest and
posttest, pre-posttest with a control, posttest
only or other?
15Report Template - Evaluation Survey
Implementation
- What type of design was used, e.g., pretest and
posttest, pre-posttest with a control, posttest
only or other? - Who were the survey respondents? (please give age
range) - Where were they administered (name and type of
locations, e.g., health department, community
center, mobile home park, etc.) - When were the pretest and posttest surveys
administered? (Please give dates.) - How many were administered? (If a different
number of pretests and posttest please explain.) - How long did it take the respondents to answer
the survey? (Please give range in minutes.)
16Report Template - Evaluation Results
- What were the results? How would you interpret
these results? How might these results be used? - How would you interpret these results?
- How might these results be used?
17Report Template - Reflection
- What were the big challenges you encountered
during the evaluation process? - Now, reflect on the successes youve had over the
past year. Describe the best moment you had as a
nutrition educator this year. Think about a
moment when you knew your nutrition education had
made a difference in someones life or give an
example of a time when you were proud to be a
nutrition educator. - Share a story about a one of the best moments you
had while conducting the evaluation this year?
Think about one of the highlights? - If Aladdin appeared with his magic lamp and
offered you a wish to have more of those best
moments or highlights what would you ask for?
18Monitoring sheet (in MS Excel) includes columns
for
- Contractor Name
- Contact Name
- Region
- Channel
- Federal Share
- Program Manager
- Required/Volunteer
- Contract External Evaluator
- Number of years in impact evaluation
19Monitoring sheet (continued)
- Evaluation Objective
- Factors (to be)/(that were) measured
- Design (posttest only, posttest with control,
pre-posttest, pre-post with control) - Participants age (grade, adults/youth, in years)
- Survey Used
- Have a copy of survey
- Pretest date(s)
- Posttest date(s)
- Number of Surveys expected (collected)
20Monitoring sheet (continued)
- Need Data Entry Template (date)
- Sent data entry template (date)
- Received Report (in email from date)
- Report Late/Received Draft
- Extension Granted (notes)
- Data Received
- Need Training
21What were the indicators of success?
22Who has participated?
23Funding ( of total budget)
(19)
(34)
(47)
24Resources
- www.ca5aday.com
- Research and Evaluation
- See impact evaluation section at the bottom
- www.harvestofthemonth.com
25Hoorah! More Matters