Online%20Survey%20and%20Assessment%20Techniques

1
Online Survey and Assessment Techniques

• Dr. Curtis J. Bonk
• Associate Professor, Indiana University
• President, CourseShare.com
• http//php.indiana.edu/cjbonk,
• cjbonk_at_indiana.edu

2
PauseWhat tools do you use or know about?
3

• Collecting Evaluation Data Online Evaluation
  Tools

4
Types of Data Collection

• Direct Observation in Work Setting
• By supervisor, co-workers, subordinates, clients
• Collect Data By Surveys, Interviews, Focus Groups
• Supervisors, Co-workers, Subordinates, Clients
• Self-Report by learners or teams
• Email and Chat
• Online Tests
• E-Portfolios or Learning Logs

5
I. Online Testing Tools(see http//www.indiana.e
du/best/)
6
(No Transcript)
7
Choice Select companies that specialize in
online assessment.
8
Or Use what the courseware package gives ya
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Some things in life are free!
15
(No Transcript)
16
Test Selection Criteria (Hezel, 1999)

• Easy to Configure Items and Test
• Handle Symbols
• Scheduling of Feedback (immediate?)
• Provides Clear Input of Exam Dates
• Easy to Pick Items for Randomizing
• Randomize Answers Within a Question
• Weighting of Answer Options

17
More Test Selection Criteria

• Recording of Multiple Submissions
• Timed Tests
• Comprehensive Statistics
• Summarize in Portfolio and/or Gradebook
• Confirmation of Test Submission

18
More Test Selection Criteria(Perry Colon, 2001)

• Supports multiple items typesmultiple choice,
  true-false, essay, keyword
• Can easily modify or delete items
• Incorporate graphic or audio elements?
• Control over number of times students can submit
  an activity or test
• Provides feedback for each response

19
More Test Selection Criteria(Perry Colon, 2001)

• Flexible scoringscore first, last, or average
  submission
• Flexible reportingby individual or by item and
  cross tabulations.
• Outputs data for further analysis
• Provides item analysis statistics (e.g., Test
  Item Frequency Distributions).

Web Resource http//www.indiana.edu/best/
20
II. Electronic Gradebooks (Vockell Fiore, 1993)

• Calculate scores, store info
• Weight scores
• Flag students with certain characteristics
• Print reports by individual or group
• Provide prompt feedback
• But inflexible, impersonal, can be incorrect

21
(No Transcript)
22
III. Online Survey Tools for Assessment
23
Perhaps you have heard of my survey techniques?
24
(No Transcript)
25
Survey Tool Features

• Maintain email lists and email invitations
• Conduct polls
• Adaptive branching and cross tabulations
• Modifiable templates
• Maintain library of past surveys
• Publish reports
• Technical support, chat advice
• Different types of accountshosted, corporate,
  professional, etc.

26
Still More Online Survey Features

• Online help, FAQs, demos, tutorials, white
  papers, glossaries, site maps, tips, overviews
• Website evaluation tools or options
• Supports different types of items (Likert,
  multiple choice, forced ranking, paired
  comparisons, etc.)
• Customizable audience lists
• Launch and close surveys

27
More Online Survey Features

• Online viewing of assessment results and question
  analysis (including various graphical displays)
• Easy to author surveys
• Instant feedback to participants
• Custom reporting, editable images and background,
  personalized greetings
• Storage of results in Access or SQL

28
Still More Features

• Ability to start and stop a survey
• Reports on subgroups
• Randomize qs to reduce bias
• Easy email reminders
• Personalized banner or logo
• Expanded question options
• Ability to copy surveys

29
(No Transcript)
30
Web-Based Survey Advantages

• Faster collection of data
• Standardized collection format
• Computer controlled branching and skip sections
• Graphics can reduce fatigue
• Easy to answer clicking
• Wider distribution of respondents

31
Web-Based Survey Problems Why Lower Response
Rates?

• Low response rate
• Lack of time
• Unclear instructions
• Too lengthy
• Too many steps
• Cant find URL
• Perceived as aggressive

32
Web-Based Survey Solutions Some Tips

• Send second request
• Make URL link prominent
• Offer incentives near top of request
• Shorten survey, make attractive, easy to read
• Credible sponsorshipe.g., university
• Disclose purpose, use, and privacy
• E-mail cover letters
• Prenotify of intent to survey

33
Tips on Authentification

• Check e-mail access against list
• Use password access
• Provide keycode, PIN, or ID
• (Futuristic Other Palm Print, fingerprint, voice
  recognition, iris scanning, facial scanning,
  handwriting recognition, picture ID)

34
Sample Survey Tools

• Zoomerang (http//www.zoomerang.com)
• IOTA Solutions (http//www.iotasolutions.com)
• QuestionMark (http//www.questionmark.com/home.htm
  l)
• Survey Solutions from Perseus (http//www.perseusd
  evelopment.com/fromsurv.htm)
• Infopoll (http//www.infopoll.com)

35
Sample Survey Tools

• Active Feedback
• (http//www.activefeedback.com/af)
• Majon International
• (http//www.coolsurveys.com)
• EZSurvey from Raosoft
• (http//www.raosoft.com/)
• SurveyShare (http//SurveyShare.com from
  Courseshare.com)

36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
Zoomerang Steps

1. Register Sign Up for Basic Membership (SKIP FOR
   NOW!)
2. Click on new survey.
3. Click on survey template (e.g., Education and
   Professor)
4. Delete, modify, or add 1-2 items.
5. Submit survey.

60
Zoomerang Steps

1. Modify Web greeting and submit.
2. Pick option 1 for submission.
3. Type Survey invitation email message
4. Enter email addresses for potential survey
   respondents
5. Decide on how share results

61
Zoomerang Steps

1. Launch survey
2. Click on My Surveys and browse
3. Click on Email List and browse
4. Click on Account Info
5. Other explorations

62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
SurveyShare Steps

1. Register Sign Up (perhaps take Quick Tour, Flash
   version)
2. Login
3. Click on Create Survey
4. Check out existing templates
5. Select one of the templates
6. Add one item, delete one item, modify one item

80
SurveyShare Steps

1. Edit Survey Introduction and Title
2. Edit Closing and Submit
3. Select Survey Invitation Via Email
4. Type emails to complete the survey
5. Click add (to move emails to right)
6. Give email subject and message

81
SurveyShare Steps

1. Enter email subject line and text
2. Preview survey
3. Activate survey
4. Go to MySurveys
5. Click on Survey Info
6. Go to mailing lists

82
SurveyShare Steps

1. Add a person
2. Add list
3. Check out resources (books)
4. Check out help, quick tour, and sitemap
5. Perhaps create a survey from scratch
6. Perhaps create a survey from My Previous
   Surveys

83

• Any Reactions to Online Surveys???

84

• Fitting Surveys into Evaluations Plans

85
Bonks Evaluation Plan
86
What to Evaluate?

1. Learnerattitudes, learning, use, performance.
2. Instructorpopularity, course enrollments.
3. Traininginternal and external components.
4. Task--relevance, interactivity, collaborative.
5. Tool--usable, learner-centered, friendly,
   supportive.
6. Courseinteractivity, participation, completion.
7. Programgrowth, long-range plans.
8. Organizationcost-benefit, policies, vision.

87
The 4 Levels

• Reaction
• Learning
• Behavior
• Results

88
(No Transcript)
89
Ziegler, April 2002, e-Learning

• the key is not to measure every possible angle,
  but rather to focus on metrics that are pragmatic
  and relevant to both human and business
  performance at the same time.

90
Some Final Advice
Or Maybe Some Questions???