Creative or alternative methods for collecting data

1
Creative or alternative methods for collecting
data
2

• Surveys, end-of-session questionnaires,
  interviews and focus groups have become popular
  ways to collect data.
• What other ways can you collect data that may be
  more novel, more appropriate and yield better
  information?
• Remember, always check whether the proposed
  method will be culturally appropriate.

Let your mind roam.
3
Using existing records and forms

• Record books
• Entries in record books can provide insights
  about the environment in a club or project area
  participants perceptions of activities
  statements of learning and actions. For example,
  members may comment on growth in leadership
  skills, I have learned how to be more organized
  or I feel more confident in my leadership role
  or I led the group to
• You might take note of sections that are filled
  out and those that arent

4
Using existing records and forms

• Exhibits
• can be counted and compared to previous years
  entries a team of people can view exhibits to
  assess outcomes or list new ideas that may be
  used in program development judges comments can
  be gathered and summarized winning entries may
  be noted

5
Using existing records and forms

• Application, registration and scholarship forms
• Forms that are completed by members for awards
  and trips and other things such as camp counselor
  applications may contain useful data for
  example, answers to questions such as How has
  4-H enhanced your problem solving skills? or,
  How would you describe 4-H to a group of peers?
  Or, include questions to solicit data you need

6
Using existing records and forms

• Committee and participation lists
• You keep a variety of lists that can be used to
  track numbers and key demographics of who is (and
  who is not) signing up for what. You can monitor
  trends over time and get an idea of how youth are
  perceiving the program.

7
What other existing records and forms might you
use?

• Take a few minutes and think about the various
  records and forms you have that either
• already have data that you can use for your
  evaluation or
• you can modify to collect the data you need
  (perhaps, by adding a question or two).

8
Using games to collect data

• Value line or line rating
• In a value line, people position themselves along
  a line to denote their position about a topic.
• Signs are often posted on the wall (or along a
  line on the floor) that provide the rating scale
  (for example, strongly agree to strongly
  disagree very helpful to not helpful)
• Develop clear strong statements that relate
  clearly to the topic and what you want to measure
• You can use the value line at the end of an
  event or use it at the beginning and again at
  the end to collect pre-post data or with a sample
  of clubs

9
Example value line

• Program Developing club leadership. (Example
  provided by Mary Klemme)
• Part of your educational plan is to conduct some
  workshops about creating a positive environment
  for youth leadership development. You want to
  know how clubs are functioning in this area
  currently and how effective your workshops are.
  You decide to use a value line exercise with a
  sampling of clubs both before and after your
  workshops.

10

Statement Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree
Youth treasurers should sign all checks written by the club.
The clubs general leader should write the monthly agenda.
Parents should be able to vote on club issues at monthly meetings.
Older teens in the club do not need to attend club meetings

Count the number of people standing in the
different areas and record the numbers. You can
also use this process to facilitate discussion
about why people took various positions.
11
Games Card Sort

• Print brief explanations of program outcomes (or
  whatever you are seeking information about and
  wish people to rate or rank) on 3 x 5 cards. Make
  one set of cards for each participant. Or, let
  participants generate the items (e.g., benefits
  they gained from the program) and create one set
  of cards for each person.
• Write the rating scale (excellent-poor) or ranks
  (1st, 2nd, 3rd) on baskets placed on a table.
• Ask participants to sort and put their cards into
  the appropriate basket.
• You may wish to prioritize the cards in each
  basket.
• This can be done individually or as small groups
  working together.
• Simple key words or graphic images can be used to
  ease literacy requirements.

12
Games

• How might you use such games as
• Matching
• Jeopardy
• Scattagories
• Etc.
• to collect evaluation information?

13
Using photography to collect data

• Before and after photos of community service
  activities project activity skill development
  program, camp, etc.
• Photos taken over the course of a project to
  show development and changes
• Engage youth in taking and interpreting their own
  photos

14
Photography

• Example. You want to know what the camp
  experience means to youth. You randomly select a
  group of campers who are asked to take photos
  during camp. The photographers then gather in
  small groups to discuss their photos, why they
  took the photo, what it means to them, etc.
  Comments and remarks are recorded.

15
Photolanguage

• A qualitative data collection technique that
  uses photographs to help individuals express
  themselves more fully when asked questions. The
  photographs are typically black-and-white images
  in order to allow viewers to focus on the topic
  or essence of the photos.
• Select photos related to the evaluation topic
  that will stimulate conversation. For example,
  if you want to know what youth learned during the
  Winter Leadership Camp, you might say, Select a
  photo that helps you describe what you learned
  during the leadership camp.

16
Photolanguage continued

• Record what participants say analyze the data
• You can use photolanguage to assess needs,
  document outcomes and experiences.
• See an article in the Journal of Extension,
  Photos can inspire a thousand words by White,
  Sasser, Bogren and Morgan, June 2009
  http//www.joe.org/joe/2009june/iw1.php

17
Using creative expression to collect evaluation
data

• Various forms of creative expression can be used
  to collect evaluative data to document changes in
  knowledge, attitudes, skills, feelings and
  behaviors.
• Drawing
• Drama
• Role-playing
• Story telling storyboards
• Music

18

• Drawing is a natural mode of expression for boys
  and girls. It is a nonverbal
• Language and form of communication that can be
  analyzed for structure, quality and content.
• You might ask participants to draw a picture of
  themselves doing their favorite club activity,
  after-school activity, camp activity.

19
Storyboards as creative expression

• A series of pictures, illustrations or graphics
  that tell a story about a program, its impact and
  what youth experienced.
• Youth might create their storyboards individually
  or as a group.
• They might draw their storyboards on paper or
  using computer technologies.
• You might give each person/group a specific
  aspect to draw (relative to your evaluation
  questions) or let them identify topics they feel
  would provide useful evaluation information.
• Once the storyboards are complete, share and
  discuss. Record comments as additional
  evaluative data.

20
(No Transcript)
21
Journaling

• Journals are personal and allow youth to think
  about their experiences, what they are feeling
  and thinking and provides a useful ongoing
  account of an activity or experience.
• Helping young people journal effectively means
  more than giving them a notebook. When using
  journaling to collect evaluation data, be clear
  about what you want to know and give them a list
  of questions to address.
• Suggest a structure for the journal (or provide
  pre-printed pages). Space might be designated
  for days events feelings about the events
  reflection on what was learned what actions they
  will take.
• You might consider a group journal where all
  members write and share their experiences and
  thoughts in a combined effort.

22
Journaling

• Do a Google search. There are many resources on
  journaling on the web. One is from the
  University of Tennessee 4-HYD, titled Reflection
  journal at

http//www.utextension.utk.edu/4h/sos/whatisit/ref
lection/reflection_journal.htm
23
Action cards

• Action cards are index cards on which
  participants record steps, actions, and
  accomplishments during the course of a project or
  program. Individuals or groups write a short
  entry on each card - a few words or sentences.
  The cards help to monitor progress, critical
  incidents and accomplishments.
• Youth can fill in the cards at regularly
  scheduled meetings or specified times.
  Information on the cards become data for later
  analysis.

24
Group techniques

• Any of the common group techniques that we use in
  program planning are also useful for collecting
  evaluation information Brainstorming Nominal
  group technique Delphi technique Affinity
  diagramming

25
Group technique examples

• A talking circle youth form a talking circle
  and provide their comments about their
  experiences and benefits gained from the program
  comments are recorded
• A modified world café participants move from
  table to table building upon a series of
  evaluation questions the table host records
  key points
• Brainstorming groups small groups brainstorm
  answers to the evaluation questions with a
  summary list recorded on paper

• See also information in the presentation on
  Interviews Group interviews

26
Goal setting as data collection
Your Goal

• Participants set their own goal(s) at the
  beginning of the program (e.g., desired weight
  change change in exercise pattern amount of
  money to save increase in public speaking
  skills, etc). Depending upon the length of the
  program, you ask them to revisit their goal
  during the program (record progress) and/or at
  the end. Progress and achievement are recorded
  as evaluation data.

27
Fun with stickers!

• This idea may work well with young children.
  Count out a fixed number of stickers and place
  the same number in each of three cups (use any
  number of cups). Label each cup with learned a
  lot, learned a little, didnt learn anything,
  or whatever response options fit your question.
  Ask each youth to take a sticker from the cup
  that best answers the question you ask (your
  evaluation questions). Tally after each question.
  The youth get a fun sticker and you get
  evaluation data.

28
To evaluate a program at its conclusion
Draw a large head, heart and feet on flipchart
paper and post it. Distribute small slips of
paper and ask participants to write down the
major things they learned or got out of the
program. Have them post these in the appropriate
position on the diagram and discuss them. Record
comments. Or, invite participants to create
their own head, heart and feet and fill in the
form, either individually or with someone
else. You might want to write What you
think What you feel What you will do next
to the head, heart and feet.
Source Educating for a Change. Adapted from
Marsha Sfeir, a Toronto educator.
29
To evaluate a program at its conclusion
Another variation for quickly collecting data at
the end of a session or program is a card like
below
30
To evaluate a program at its conclusion
Here is another version of a card for quickly
collecting data at the end of a session or
program
31
See the PDF on the web site for additional ideas
using

• Creative expression
• Personal stories or testimonials
• Video taping or photography
• Expert review
• Diaries and journals
• Logs
• Case study

32
Using technology for collecting data

• Many of the ideas in this slide set might be
  adapted to the computer.
• Also, see another section on this web site for
  using technology in data collection.

33
Analyzing data when using creative methods

• While many of the methods covered in these slides
  seem fun and easy, think about how you will
  record and analyze the data you collect.
• Avoid collecting data unless you have a
  systematic process for analysis and plan for
  using the data.
• Many of these methods require the use of
  qualitative data analysis that involves
  categorizing the data, identifying themes and
  summarizing the key points. (see the section on
  qualitative data analysis)

34
Integrating evaluation into your programming

• As you can see from this presentation, there are
  many ways to integrate data collection into your
  programming so that it doesnt become an add
  on, boring or another questionnaire to complete.
  
• Use your own creativity to develop useful and
  culturally appropriate data collection methods.
  Make sure the method aligns with your evaluation
  purpose and what you want to know!

35
Reflection time

• What is one thing you learned (or had
  reinforced)from going through this presentation
  that you might try in your own evaluation?
• Good luck with your data collection efforts!