2004 Public Health Training and Information Network (PHTIN) Series

1
2004 Public Health Training and Information
Network (PHTIN) Series
2
Site Sign-in Sheet

• Please mail or fax your sites sign-in sheet to
• Linda White
• NC Office of Public Health Preparedness
• and Response
• Cooper Building
• 1902 Mail Service Center
• Raleigh, NC 27699
• FAX (919) 715 - 2246

3
Outbreak Investigation Methods

• From Mystery to Mastery

4

5
2004 PHTIN Training Development Team

• Pia MacDonald, PhD, MPH - Director, NCCPHP
• Jennifer Horney, MPH - Director, Training and
  Education, NCCPHP
• Anjum Hajat, MPH Epidemiologist, NCCPHP
• Penny Padgett, PhD, MPH Epidemiologist
• Amy Nelson, PhD - Consultant
• Sarah Pfau, MPH - Consultant
• Amy Sayle, PhD, MPH - Consultant
• Michelle Torok, MPH - Doctoral Candidate
• Drew Voetsch, MPH - Doctoral Candidate
• Aaron Wendelboe, MSPH - Doctoral student

6
Future PHTIN Sessions

• October 12th. . . . . . . . . Analyzing Data
• December 14th. . . . . . . Risk Communication
• Each session will be on a Tuesday from 1000
  a.m. - 1200 p.m. (with time for discussion)

7
Session I VI Slides

• After the airing of each session, NCCPHP will
  post PHTIN Outbreak Investigation Methods series
  slides on the following two web sites
• NCCPHP Training web site
• http//www.sph.unc.edu/nccphp/phtin/
• North Carolina Division of Public Health, Office
  of Public Health Preparedness and Response
• http//www.epi.state.nc.us/epi/phpr/

8
Session IV

• Designing Questionnaires

9
Todays Presenters

• Sarah Pfau, MPH
• Consultant
• NC Center for Public Health Preparedness
• Kay Sanford, MSPH
• Head, Injury and Violence Prevention Branch,
  Division of Public Health,NC Department of Health
  and Human Services,

10
Designing Questionnaires Learning Objectives

• Upon completion of this session, you will
• Understand the role of questionnaire design in an
  outbreak investigation
• Know how to develop a hypothesis generating
  questionnaire
• Recognize key planning strategies for successful
  questionnaire design

11
Designing Questionnaires Learning Objectives

• Recognize key characteristics of well-designed
  questions
• Recognize three broad question types and when to
  use them
• Understand what different question types measure,
  and the type of data (quantitative versus
  qualitative) they yield

12
Designing Questionnaires Learning Objectives

• Know how to format questionnaires for interviewer
  administered or self-administered settings
• Recognize the impact of questionnaire design on
  View design and data analysis in Epi Info
  software

13
Designing Questionnaires
14
Basic Steps of an Outbreak Investigation

• Verify the diagnosis and confirm the outbreak
• Define a case and conduct case finding
• Tabulate and orient data time, place, person
• Take immediate control measures
• Formulate and test hypothesis
• Plan and execute additional studies
• Implement and evaluate control measures
• Communicate findings

15
Designing Questionnaires
16
Session Overview

• The importance of questionnaire design
• Hypothesis generating versus hypothesis testing
  questionnaires
• Preparing for questionnaire design
• Question design
• Question type
• Questionnaire format

17
Why is Questionnaire Design Important?

• The quality of the data will be no better than
  the most error-prone feature of the survey
  design.
• - Fowler, F.J. (1993). Survey Research Methods
  Second Edition. Sage Publications Newbury
  Park.

18
Why is Questionnaire Design Important?

• With an understanding of good questionnaire
  design principles, you will ask only about what
  you need to meet your research objectives.

19
Ask Only About What You Need. . .

• Example
• You ask respondents to list all medications that
  they are taking (difficult in terms of respondent
  recall / accuracy), when you really only need to
  know if they are taking antibiotics.

20
Why is Questionnaire Design Important?

• Question type and response option formatting
  impact how you can design an on-screen data entry
  form and / or analyze variables in your software
  program

21
Why is Questionnaire Design Important?

• Survey answers are not of interest
  intrinsically rather, the answers are important
  because of their relationship to what they are
  supposed to help you measure.

22
Hypothesis Generating Questionnaires
23
Which Questionnaire Type Should You Use?

• Hypothesis Generating?
• Hypothesis Testing?

24
(No Transcript)
25
Hypothesis Generating Questionnaire

• Include questions about
• Demographics
• Clinical details of the illness
• Health care provider visits
• Water exposure
• Exposure to other ill persons
• Exposure to children in day care
• Exposure to a farm or farm animals
• Travel outside of the immediate area

26
Hypothesis Generating Questionnaire

• If the pathogen can be spread through food or
  beverages, include questions about
• Food eaten in the home
• Food eaten in the homes of friends, family
• Food eaten at any restaurant
• Dates and times of food consumption and any
  suspicious observations

27
Hypothesis Generating Questionnaire
28
Hypothesis Generating Questionnaire
29
Hypothesis Testing Questionnaires
30
Hypothesis Testing Questionnaire

• Include detailed questions about the suspected
  source of infection.
• Example The local bakery is suspected as the
  source of a Hepatitis outbreak in multiple
  counties. The hypothesis testing questionnaire
  is used with both cases and non-cases, and
  includes only a food history for all possible
  items on the bakery menu to pinpoint the exact
  food item that is contaminated.

31
Hypothesis Testing Questionnaire
32
Hypothesis Generating versus Testing Questions
E. coli 0157H7

• Hypothesis Generating
• Account for consumption of ground beef, lettuce,
  alfalfa sprouts, un-pasteurized milk or juice,
  and swimming in or drinking sewage-contaminated
  water

• Hypothesis Testing
• beef suspected
• Beef brand, date of purchase, and grocery store
  or restaurant where beef was purchased or eaten

33
Preparing for Questionnaire Design
34
Preparing for Questionnaire Design

• There is more to questionnaire design than
  writing questions. Ideally, you should first
• Have a clear purpose and research objectives
• List variables to be measured
• Have an analysis plan
• Consider cost and other logistical aspects

35
Questionnaire Design

• Have a clear purpose and research objectives.
• Is the purpose of your outbreak investigation
  survey to generate a hypothesis or to test a
  refined hypothesis?

36
Questionnaire Design

• List variables to be measured
• Whether you are generating or testing a
  hypothesis, determine your variables of interest
  before you develop questions. You will avoid
  asking unnecessary questions or asking for
  unnecessary details.

37
Questionnaire Design

• Go to http//www.cdc.gov
• Under Health Safety Topics in the left
  margin, click on, Diseases Conditions
• Select a link to either an alphabetized list of
  all diseases or diseases by topic e.g., if you
  already know that a pathogen is water-borne
  versus food-borne

38
Questionnaire Design
39
Questionnaire Design

• Have an analysis plan.
• Guides the question types and response option
  categories used on the questionnaire
• Helps assure that the data collection leading up
  to analysis yields variable coding that your
  analysis software program can use efficiently.

40
Questionnaire Design

• Consider cost and other logistical aspects
• What is the survey sample size?
• What is the geographic distribution of the survey
  sample?
• Will questionnaires be interviewer administered
  or self-administered?
• What is your staff capacity to work within the
  parameters of a c above?

41
5 minute break
42
Question Design
43
Question Design

• Elements of good question design
• Reliability
• Validity
• Specificity versus ambiguity
• Simplicity
• Only one question asked
• Mutually exclusive answer choices
• Refers respondents to specific dates / times for
  recall
• When feasible, make sure data can be compared to
  existing sources of information

44
Question Design

• 1. A question that is designed to be reliable
  will assure that the words are interpreted the
  same way in any setting, and that respondents
  answer the same way in any setting.

45
Reliable Question Design

• Question
• Are you experiencing diarrhea?
• Interviewer then adds
• For the purposes of this survey, we consider
  diarrhea to be 3 or more loose bowel movements in
  a 24 hour period.

46
Question Design

• 2. A question that is designed to be valid will
  always yield information that can be used as a
  true measure of what you, the researcher, are
  looking for.

47
Valid Question Design

• Less Useful
• Which is your source of drinking water at home?
• Tap water
• Bottled water
• Better
• Which is your source of drinking water at home?
• Municipal tap water
• Municipal tap water with additional filtration
• Well water
• Commercially bottled water

48
Question Design

• 3. Avoid ambiguity in question wording.
• Less useful
• When did you have Disease X?
• Better
• How old were you when you had Disease X?

49
Question Design

• 3. Avoid ambiguity in question wording.
• Less useful
• Have you been examined by a physician in the
  past seven days?
• Better
• Have you been examined by a physician for these
  symptoms in the past seven days?

50
Question Design

• 4. Use simple language and keep questions
  short.
• Less useful
• Were you exposed to the fomite at the dinner
  party?
• Better
• Did you use a shared hand towel at the dinner
  party?

51
Question Design

• 5. Ask only one question.
• Two questions in one
• Did you eat mashed potatoes and giblet gravy?
• One question at a time
• Did you eat mashed potatoes? Yes No
• If Yes, did you eat them
• a. Plain or with butter
• b. With giblet gravy

52
Question Design

• 6. For closed-ended questions, make sure that
  response options are mutually exclusive.
• Not useful
• What is your age?
• 18 years old or younger
• 18 years old or older
• Useful
• What is your age?
• 17 years old or younger
• 18 years old or older

53
Question Design

• 7. Use specific date / time references to
  improve respondent recall.
• Less useful
• Have you been swimming in a public pool
  recently?
• Better
• Did you swim in a public pool between Monday,
  June 2nd and Monday, June 9th, 2004?

54
Question Design

• When feasible, make sure data can be compared to
  existing sources of information.
• Example
• Case-patient age
• Less than 2 years
• 2 5 years
• The U.S. Census reports population data for age
  groups less than 1, 1 2, 3, 4, 5, etc.

55
Question DesignGeneral Guidelines

• Design clear, specific questions.
• Use simple language and keep questions short.
• Ask only one question!

56
Question DesignGeneral Guidelines

• For closed-ended questions, make sure that
  response options are mutually exclusive.
• Use specific date / time references to improve
  respondent recall.
• Make sure data can be compared to existing
  sources of information if you need to compare
  local, state, and national statistics.

57
Question Type
58
Question Type

• Questions fall into three broad categories
• Closed-ended
• Open-ended
• Fill-in-the-blank
• Data collected and analysis options may differ
  by question type.

59
Closed-ended Questions
60
Question Type

• Closed-Ended Questions
• Provide answer choices in pre-coded categories
  that represent counts, ranges, or demographic
  information.
• Yield quantitative data.
• Are preferable for self-administered and
  hypothesis testing questionnaires.

61
Question Type

• There are two ways in which categorical
  measurement is carried out in closed-ended
  questions
• Nominal
• Ordinal

62
Question Type

• Nominal response options result in the
  classification of a respondent into a category
  for race, gender, marital status, etc.
• - Response options can be mutually exclusive
  (Select only one response)

63
Question Type

• Example Nominal response options in a
  closed-ended question
• Please select the one response that describes
  your marital status.
• Single
• Married
• Divorced
• Widowed

64
Question Type

• Ordinal response options result in
  classification into an ordered category that may
  or may not rank values that are equidistant.
• - Ordinal responses have a numeric value.

65
Question Type

• Example Ordinal response options in a
  closed-ended question
• When you turn off the frozen dairy dessert
  machine at the restaurants closing time, how
  frequently do you sanitize the dispenser?
• 1 Every time 4 Once a month
• 2 Every other time 5 Less than once a
  month
• 3 Once a week

66
Question Type

• Likert scales contain pre-coded ordinal
  responses with assigned values. You can then
  calculate averages to determine the most
  prevalent response.
• Example Please rate the severity of your
  abdominal cramp pain (circle one)
• 1 minimal 3 moderate 5 very painful
• 2 mild 4 painful

67
Question TypeLikert Scales
68
Question TypeLikert Scales

• Example Please rate the severity of your
  abdominal cramp pain
• 1 minimal 3 moderate 5 very painful
• 2 mild 4 painful
• Calculate the average of responses, where N
  35
• 1 10 people 2 5 people 3 20 people
• 4 0 5 0
• 10 x 1 10 2 x 5 10 3 x 20 60
• 10 10 60 80
• 80 / 35 2.29 (mild pain)

69
Question Type

• Using Dont Know or Not Applicable response
  options
• When you have categorical, dichotomous response
  options such as, Yes and No, you may
  sometimes choose to add a Dont Know or Not
  Applicable option.

70
Open-ended Questions
71
Question Type

• Open-Ended Questions
• Allow respondents to provide answers in their own
  words.
• Yield qualitative data.
• May yield unanticipated answers that contribute
  to the study.
• Are most appropriate for hypothesis generating
  versus testing questionnaires.

72
Question Type

• Example Open-ended questions
• What restaurants did you patronize in the past
  seven days?
• Please list the two main symptoms you are having
  with this illness

73
Fill-in-the-blank Questions
74
Question Type

• Fill-in-the-blank Questions
• Allow respondents to provide short answers in
  their own words.
• Yield qualitative data.
• Are most appropriate when possible response
  categories are too numerous to list.
• Are most appropriate when the question is
  measuring respondent characteristics versus
  attitudes, beliefs, or behaviors.

75
Question Type

• Example Fill-in-the-blank Questions
• 1. County of residence _____
• 2. Age in years ____
• 3. Number of children under age 18 in your
  household ____

76
Question and AnswerOpportunity
77
5 minute break
78
Activity

• Completion Time 5 minutes

79
Activity

• Instructions
• Three sample questions appear on the following
  slides. Your task is to critique those questions
  and write down one element that could be improved
  or modified to make the question better.

80
Activity

• Question 1
• Did you take any prescription medications for
  this illness? Yes No
• If yes, what medications? ___________

81
Activity

• Question 2
• Did you travel anywhere during the seven days
  before your illness? Yes No
• If airline travel, what airline? __________
• Outgoing flight number ____
• Returning flight number ____

82
Activity

• Question 3
• During the seven days before your illness, did
  you have any pets at home, have contact with
  household pets elsewhere, or visit a household
  with pets (including reptiles)? Yes No

83
Epi Info

• Impact of Question Design on View Design and Data
  Analysis

84
Epi Info Implications

• Overview
• Field type and properties
• Check code in MakeView
• Categorical data
• Dichotomous response options

85
Tables
The core of your database the largest unit of
information where data are stored by rows and
columns
86
Records

• Each record contains a complete set of
  information on one specific subject or
  case-patient
• E.g., record 1 John Does name, address, date
  of onset of illness, lab results, etc.
• E.g., record 2 Jane Does name address, date
  of onset of illness, lab results, etc.

87
Fields

• The smallest unit of information in your
  database. Each field contains one piece of
  information about a subject / case-patient.
• E.g., field 1 first name
• E.g., field 2 street address

88
Sample Database Tables
Records 01 and 02 represent individual
respondents
Fields Client ID, First Name, Last Name, Phone
Number, County
89
Sample Epi Info View
Each food or beverage is represented by an
individual field for data entry and analysis.
90
Field Type and Properties
91
Field Type and Properties

• Number Type
• Age in years _____
• Number of children living in this household _____
• Text Type
• County of residence _____
• First name _____

92
Field Type and Properties
Food item field format in Epi Info can be
either a. Checkbox b. Yes / No pictured
above
93
Line Listing
94
Frequency Table
95.7 of the ill people in the sample did NOT
consume milk.
95
Dichotomous Response Options

• Which of the following food items did you
  consume in March or April of this year?
• Commercial milk purchased locally Yes No
• Fresh clams purchased locally Yes No
• Fresh oysters purchased locally Yes No
• None of the above Yes No

Represents one field of data in database
Field Type
96
Contingency Table
93.1 of the people who are not ill did not
consume milk.
95.7 of the people who are ill did not consume
milk.
Dichotomous exposure variable Milk consumption
(Yes / No) Dichotomous outcome variable Ill?
(Yes / No)
97
Check Code in MakeView

• Eight Check Code commands in the Field
  Definition window
• Repeat Last Legal Values
• Required Codes
• Read Only Comment Legal
• Soundex Range
• Use only with number formatted fields

98
Check Code in MakeView
99
Check Code in MakeView
100
Field Type and Properties

• Please select the one response that describes
  your marital status.
• Single
• Married
• Divorced
• Widowed

101
Check Code in MakeView
102
Field Type and Properties
a.
b.
Field format in Epi Info can be a. Text with
Legal Values b. Option
103
Check Code in MakeView

• Comment Legal

104
Question and AnswerOpportunity
105
Questionnaire Format
106
Questionnaire Format

• We will now consider layout guidelines
• In general
• For interviewer administered questionnaires
• For self-administered questionnaires

107
General Formatting Guidelines
108
General Guidelines

• A well-designed questionnaire
• Minimizes respondent burden
• Minimal time to think about each response
• Minimal time to complete entire survey

109
General GuidelinesRespondent Perspective

• Format page layout with a vertical flow from
  question to question and from response option to
  response option.
• Maintain white space on the page.
• Number every question.

110
General GuidelinesRespondent Perspective

• Use consistent codes for response options.
• Arrange questions in a logical order even within
  sub-sections if possible.
• Use differentiating font for questions and
  response options.

111
General GuidelinesRespondent Perspective

• Provide clear but brief instructions for survey
  completion.
• Indicate units for fill-in-the-blank questions.
• Use clear cues (text or arrows) to guide
  respondents through skip patterns.
• Use transitional cues for each subsection of
  questions (The next series of questions will
  ask about . . . ).

112
General GuidelinesResearcher Perspective

• Include a unique identification number so you can
  maintain confidentiality and link data when
  applicable (e.g., in Epi Info).
• When applicable, use numbers versus check boxes
  for pre-coded response options so they can be
  used for data entry and analysis (e.g., in Epi
  Info).

113
General GuidelinesResearcher Perspective

• To reduce the difficulty of distinguishing
  between missing data and simply all no
  responses to a categorical list of response
  options, consider alternative formatting.
• Example standard question format
• Where did you see published information about
  this free HIV screening?
• Billboard on subway
• Doctors office
• Local newspaper

114
General GuidelinesResearcher Perspective

• Example alternative question format
• Where did you see published information about
  this free HIV screening?
• Billboard on subway Yes No
• Doctors office Yes No
• Local newspaper Yes No

115
Format Guidelines Interviewer Administered
Questionnaire
116
Questionnaire Format

• Interviewer administered questionnaire
• Differentiate between text instructions for the
  interviewer and questions to be asked (e.g.,
  italicize or bold instructions).
• Establish a clear system for skip patterns (e.g.,
  lines and arrows or clear instructions).

117
Questionnaire Format

• Interviewer administered questionnaire
• Put optional wording in parentheses.
• Example (his / her)
• Put all words for the interviewer to say in
  writing to assure standardized interviewing
  (e.g., introductions, transitions, explanations,
  or definitions).

118
Interviewer Administered Questionnaires

• Suggested Supporting Documentation for
  Interviewers

119
Supporting DocumentationInterviewers Manual

• Background
• Fieldwork
• Interviewing Techniques
• Telephone Interviewer Instructions (if
  applicable)
• Terms and Definitions
• Details of the Questionnaire
• Description of the Survey Area

120
Supporting Documentation

• Standard format for interviewer introductions
  for telephone or face-to-face interviews
• Interviewers name
• Health department (or organization) represented
• One sentence description of the surveys purpose
  (e.g., There has been an outbreak of disease X
  in your county and we are calling you to . . .)
• A conservative estimate of how long the interview
  will take

121
Supporting DocumentationInterviewer Information

• Who is the interviewer?
• Who is sponsoring / implementing the survey?
• Who is the primary contact?
• What is the purpose of the survey / study?
• How will results be used, and how will they help
  the respondent?

122
Supporting DocumentationInterviewer Information

• How many people will be participating?
• How did the interviewer obtain the respondents
  name?
• Will responses be confidential?
• Will survey results be available to respondents?

123
Supporting DocumentationTelephone Interviewer
Instructions

• What information to have available during an
  interview
• How to use the telephone system (and the CATI
  software, if applicable)
• How to log calls

Computer-assisted Telephone Interview
124
Supporting DocumentationTelephone Interviewer
Instructions

• How to handle angry respondents
• When to seek supervisor assistance
• What to do once a telephone interview has been
  completed

125
Format GuidelinesSelf-Administered Questionnaire
126
Questionnaire Format

• Self administered questionnaire
• Completion of the questionnaire should be
  self-explanatory do not use detailed
  instructions, because not all respondents will
  read them.
• Restrict the design to closed-ended questions
  (e.g., checking a box or circling a response
  option).
• Order questions from most topic-related and easy
  to answer at the beginning, to most likely to be
  objectionable or of a sensitive nature at the
  end.

127
Questionnaire Format

• Self administered questionnaire
• Strive for consistency in question type and
  response option formatting whenever possible.
• Use clean formatting with plenty of white space
  so you do not overwhelm the respondent with
  clutter on the page.
• Use skip patterns only where necessary use
  visual cues versus written instructions where
  needed.

128
Format Guidelines

• Interviewer versus
• Self-Administered Questionnaires

129
Format Guidelines
130
You are not finished yet. . .

Pilot Test Questionnaires!
131
Pilot Test Questionnaires

• Are questions yielding the information that they
  are supposed to yield?
• Do respondents understand all wording?
• Do respondents interpret the questions the same
  way?
• Do closed-ended questions have a response option
  that applies to each respondent?
• Are skip patterns followed correctly?

132
Question and AnswerOpportunity
133
Guest Expert Lecturer

• Kay Sanford, MSPH
• Head, Injury Epidemiology Unit
• Injury and Violence Prevention Branch
• NC DHHS Division of Public Health
• kay.sanford_at_ncmail.net
• 919.715.6444

134
NC Rapid Needs Assessment Hurricane Isabel,
September 2003
Injury and Violence Prevention Branch, NC-DHHS DPH
135
Rapid Needs Assessment

• Background
• 3rd week September 2003, Hurricane Isabel en
  route to NC coast (Beaufort County)
• RNA process had been used in other disasters,
  e.g., Ankara earthquake 1999 Maine Ice Storm,
  1998 Hurricane Opal, 1995,
• Decision to perform RNA made as Isabel approached
  the NC coast

Injury and Violence Prevention Branch, NC-DHHS DPH
136
Development of Questionnaires
Injury and Violence Prevention Branch, NC-DHHS
DPH
137
Who needed the RNA data?

• In 2003, no mandate for RAPID hurricane data
  within the state
• PHPR - DPH initiative - useful to
• Division Emergency Management
• Local emergency operations programs
• Local Health Departments
• RNA now written into State Hurricane Plan

Injury and Violence Prevention Branch, NC-DHHS DPH
138
Who was involved in developing the RNA
Questionnaire?

• Epidemiology Section
• Public Health Preparedness Response
• Injury and Violence Prevention Branch
• Division of Environmental Health
• Division of Mental Health
• CDC specialists in mass trauma surveillance
• PHRST members

Injury and Violence Prevention Branch, NC-DHHS DPH
139
How we developed the RNA Questionnaire by
committee (1)

• Selected a TEAM leader
• Identified examples to serve as prototypes
• Agreed upon goals of data collection
• Promoted our individual areas

Injury and Violence Prevention Branch, NC-DHHS DPH
140
Strategies for Designing the RNA Survey Instrument

• Limit forms to one-page
• Number all questions
• Preference for closed-ended questions
• Omitted extremely sensitive questions
• Allowed for Dont Know as a legitimate answer
• Header completed by interviewer
• Questions read verbatim by interviewer

Injury and Violence Prevention Branch, NC-DHHS DPH
141
How we developed the RNA Questionnaire by
committee (2)

• Wrote / revised individual questions
• Prepared QxQ instructions procedures
• COMROMISED on questions, definitions, and
  protocol.

Injury and Violence Prevention Branch, NC-DHHS DPH
142
How many questionnaires were needed?

• 1. Tracking Form
• 2. Survey Form
• 3. Referral Form

Injury and Prevention Branch, NC-DHHS DPH
143
Tracking Form
144
What data needed to be collected for Hurricane
Isabel?

• TRACKING FORM Sampling information
• county
• cluster number
• block number
• Interview team
• Sample sequence No., Address
• Date, Time

Injury and Violence Prevention Branch, NC-DHHS DPH
145
What data needed to be collected for Hurricane
Isabel?

• TRACKING FORM Response status
• Interview Conducted
• Interview started but not completed
• Interview refused
• Language barrier
• Residence not occupied
• Evacuated Vacant Seasonal occupant
• Dwelling destroyed Reason Unknown
• Other (specify)

146
Survey Instrument
147
What data needed to be collected for Hurricane
Isabel?

• The SURVEY FORM
• Damage to homes
• Availability of usable utilities
• Incidence of hurricane related illness / injury
• Access to medical care
• Access to food and water
• Self-identified immediate needs

Injury and Violence Prevention Branch, NC-DHHS DPH
148
Respondent Referral Form
149
Strategies for collecting the data

• Paper form - delayed data entry in the field
  more delayed data transmission
• Access database Epi Info for data analysis
• Electronic form on laptop/iPAQ - direct data
  entry delayed data transmission
• Hurricane Isabel Paper form w/ Delayed
  DE-Delayed DT
• Hurricane Charley Direct DE w/ Delayed DT
• Next time Direct DE/Direct DT

Injury and Violence Prevention Branch, NC-DHHS DPH
150
What else was needed?

• Introductory script
• English and Spanish
• Question by Question instructions
• Standard in formal epidemiologic research
• Often only form of procedure manual when
  preparation time limited RNA situations
• Often deciding factor in complicated analysis

Injury and Violence Prevention Branch, NC-DHHS DPH
151
Scripting English Introductory Script
152
Introductory Script

• Identifies and legitimizes the interviewer
• States reasons for conducting the survey
• Safe living conditions
• Need within household for medical care
• Assures respondent of the confidentiality of
  responses
• Selects the household respondent

Injury and Violence Prevention Branch, NC-DHHS DPH
153
Question by Question Instructions

• Refresher guide to interviewers after training
• Standardized definitions
• Instructions for probing
• Instructions for recording or reading response
  categories
• Instructions for when to refer

Injury and Violence Prevention Branch, NC-DHHS DPH
154
Survey Instrument

• Header
• Date
• Interviewer
• Sampling information
• Respondent information

Injury and Violence Prevention Branch, NC-DHHS DPH
155
Survey Instrument

• Household information
• type
• usual number of residents
• residents during the storm
• persons at increased risk

Injury and Violence Prevention Branch, NC-DHHS DPH
156
Survey Instrument

• Damage to house from the hurricane
• habitability of structure
• tarp needed
• amount of flood water in the house
• missing amount of mud or debris

Injury and Violence Prevention Branch, NC-DHHS DPH
157
Survey Instrument

• Current status of household utilities
• running water
• power
• toilet
• telephone
• radio
• use of generator
• use of charcoal

Injury and Violence Prevention Branch, NC-DHHS DPH
158
Interviewer Training
Injury and Violence Prevention Branch, NC-DHHS DPH
159
Interviewer Training Recap from August Lecture

• Training is NOT (never) optional!
• Trainings must be interactive
• Interviewers must practice reading questions out
  loud
• Provide support documentation (manual)

Injury and Violence Prevention Branch, NC-DHHS DPH
160
Interviewer Training

• Planning and implementation logistics
• Explanatory notes for interviewers
• Scripting
• Terms defined
• Instructions for probing
• Rapport and feedback
• Instructions for disaster relief referrals
• Other. . .

Injury and Violence Prevention Branch, NC-DHHS DPH
161
Interviewer Training

• Maintaining rapport is more complex during a
  public health crisis than during a routine
  survey.
• An interviewer should be
• Nonjudgmental
• Noncommittal
• Objective
• And yet, interviewer must provide reassurance.

Injury and Violence Prevention Branch, NC-DHHS DPH
162
Interviewer Instructions
163
Interviewer TrainingTerms Defined in QxQs

• 10. INTERVIEWER What type of dwelling is this?
  ___
• 1 single family
• 2 mobile home
• 3 2-5 family
• 4 6 or more family
• 5 other
• QxQ Selection based on type of housing, not the
  number of families living in the premises

Injury and Violence Prevention Branch, NC-DHHS DPH
164
Interviewer TrainingTerms Defined in QxQs

• 11. How many people lived in this residence
  before the hurricane? ____
• QxQ Enter the number of people who routinely
  slept and ate at least one of their meals at this
  residence prior to 9/18/03, e.g., exclude
  students or military personnel living away from
  home, overnight guests, etc. Enter 99 for Dont
  Know.
• At issue simple question complex definition
  probing likely provision for Dont Know.

Injury and Violence Prevention Branch, NC-DHHS DH
165
Interviewer TrainingTerms Defined in QxQs

• 13. How much external damage was there from the
  hurricane to the residence? ___
• 1 none or minimal
• 2 damaged, but habitable
• 3 damaged, but uninhabitable
• At issue categories are read aloud habitable
  means safe to live in based on respondents
  answer skip pattern.

Injury and Violence Prevention Branch, NC-DHHS DPH
166
Interviewer TrainingTerms Defined in QxQs

• 16.e Does the indoor toilet work? ___
• 1 yes 2 no
• 3 never had 9 Dont know
• Enter YES (1) even if waste material can be
  flushed by pouring water into commode. Enter
  Never Had (3) when the residences does not have
  any kind of indoor standard toilet.
• At issue complex definitions skip pattern
  potential for referral.

Injury and Violence Prevention Branch, NC-DHHS DPH
167
Lessons Learned
Injury and Violence Prevention Branch, NC-DHHS DPH
168
Lessons Learned (1)

• Include representative from Latino (minority)
  community from the beginning
• Cultural sensitivity / correctness
• Translation of forms, scripts
• Include question on the presence of mud or debris
  in survey questionnaire
• Appropriate for mountains and flatlands

Injury and Violence Prevention Branch, NC-DHHS DPH
169
Lessons Learned (2)

• Discuss at outset need for including questions on
  current conditions versus risk factors for
  potential problems
• Landslides often occur days after hurricane
• Floods often occur days or weeks after event

Injury and Violence Prevention Branch, NC-DHHS DPH
170
Lessons Learned (3)

• Emotional / mental stress is frequent after mass
  trauma how to craft questions that will
  identify stress that is not easily abated
• Cost-benefit of asking about highly emotionally
  charged events, e.g., death of family member(s)

171
Appendix
172
Spanish Introductory Script
173
Spanish RNA Survey Instrument
174
Session Summary
175
Session Summary

• With an understanding of good questionnaire
  design principles, you will ask only about what
  you need to meet your research objectives.
• In preparation for questionnaire design, you
  should have clear research objectives list
  variables to be measured identify an analysis
  plan and consider cost and logistical aspects.

176
Session Summary

• Use a hypothesis generating questionnaire to
  explore all potential sources of infection, but
  with a small number of cases and no comparison
  group.
• A hypothesis testing questionnaire literally
  tests a research hypothesis based on data
  collected via a hypothesis generating
  questionnaire. Include both cases and controls in
  the survey sample population.

177
Session Summary

• The three broad question types are
  closed-ended open-ended and fill-in-the-blank.
  You may use only one or a combination of all,
  depending on the purpose of your survey.
• Key elements of question design are
  reliability validity specificity simplicity
  asking only one question mutually exclusive
  answer choices providing date / time references
  and using response options that parallel existing
  data sources.

178
Session Summary

• A well-formatted questionnaire minimizes
  respondent burden by using question order,
  spacing, distinctive font, symbols, and simple,
  short instructions to facilitate navigation and
  using customized components for interviewer
  administered versus self-administered settings.
• Question type and response option formatting
  impact how you can design an on-screen data entry
  form and / or analyze variables in your software
  program.

179
References and Resources

• American Statistical Association (1999). What Is
  a Survey? Designing a Questionnaire. Alexandria,
  VA Section on Survey Research Methods.
• American Statistical Association (1997). What Is
  a Survey? More About Mail Surveys. Alexandria,
  VA Section on Survey Research Methods, American
  Statistical Association.
• Centers for Disease Control and Prevention, Food
  borne and Diarrheal Diseases Branch, Food borne
  Outbreak Response and Surveillance Unit (2003).
  Outbreak Investigation Toolkit Standard
  Questionnaire. Online resource
  http//www.cdc.gov/foodborneoutbreaks/standard_que
  stionnaire.htm
• Data Skills Online web site, Office of
  Continuing Education, Institute for Public
  Health, UNC Chapel Hill.
• An Overview of Primary Data Collection
  Instruments and Designing Questionnaires
  online self-instructional tools.
  http//www.sph.unc.edu/toolbox/.

180
References and Resources

• Centers for Disease Control and Prevention.
  Standard Food Borne Disease Outbreak
  Questionnaire. (PDF file) at http//www.cdc.gov/f
  oodborneoutbreaks/question/standard_questionnaire.
  pdf
• Fowler, F.J. (1993). Survey Research Methods
  Second Edition. Sage Publications Newbury
  Park.
• Reingold, A. (1998). Outbreak Investigations A
  Perspective. Emerging Infectious Diseases. Vol.
  4 No. 1 January March 1998.

181
References and Resources

• Salant, P. and Dillman, D. (1994). How to
  Conduct Your Own Survey. John Wiley Sons,
  Inc. New York.
• Stehr-Green, J.K. (2002). A Multi-state Outbreak
  of E. coli 0157H7 Infection Case Study
  Instructors Guide. Atlanta, GA U.S. Department
  of Health and Human Services, Public Health
  Service, Centers for Disease Control and
  Prevention. http//www.phppo.cdc.gov/phtn/casestud
  ies/classroom/ecoli.htm

182
References and Resources

• Stehr-Green, J. and Stehr-Green, P. (2004).
  Hypothesis Generating Interviews. Module 3 of a
  Field Epidemiology Methods course being developed
  in the NC Center for Public Health Preparedness,
  UNC Chapel Hill.
• Torok, M. (2004). FOCUS on Field Epidemiology.
  Case Finding and Line Listing A Guide for
  Investigators. Volume 1, Issue 4. NC Center for
  Public Health Preparedness.
• Wiggins, B. and Deeb-Sossa, N. (2000).
  Conducting Telephone Surveys. Chapel Hill, NC
  Odum Institute for Research in Social Science.

183
Slides from Todays Session

• Following this program, please visit one of the
  web sites below to access and download a copy of
  todays slides
• NCCPHP Training web site
• http//www.sph.unc.edu/nccphp/training/index.html
• North Carolina Division of Public Health, Office
  of Public Health Preparedness and Response
• http//www.epi.state.nc.us/epi/phpr/

184
Next Session October 12th1000 a.m. - Noon

• Topic Analyzing Data

185
Site Sign-in Sheet

• Please mail or fax your sites sign-in sheet to
• Linda White
• NC Office of Public Health Preparedness
• and Response
• Cooper Building
• 1902 Mail Service Center
• Raleigh, NC 27699
• FAX (919) 715 - 2246