How to design the test and related issues. Enumerator ... F - PowerPoint PPT Presentation

1 / 29
About This Presentation
Title:

How to design the test and related issues. Enumerator ... F

Description:

How to design the test and related issues. Enumerator ... Full field test ... One sub sample of respondents have cognitive test (core extended are embedded) ... – PowerPoint PPT presentation

Number of Views:62
Avg rating:3.0/5.0
Slides: 30
Provided by: lindar8
Category:

less

Transcript and Presenter's Notes

Title: How to design the test and related issues. Enumerator ... F


1
Field test options instrument, instructions,
non-response/ refusals, and Interviewer
debriefing Washington Group Regional Training
Workshop Rio de Janeiro 19 20 September
2005 Margie Schneider HSRC, South Africa
2
Recap
  • What we have covered so far
  • Purpose of the questions and where they came from
  • Core set of questions and extended set and Q X Q
    specifications
  • Objectives of the test
  • Translation protocol
  • What we still need to go through
  • How to design the test and related issues
  • Enumerator training
  • Plan for analysis and report writing

3
Different components of the testing process
  • Pre-testing and expert review
  • Testing internal validity on people with known
    disability status (quota sampling and linked to
    cognitive testing)
  • Pilot testing as for full Census or survey
  • Full field test
  • Different sampling approaches will be discussed
    in relation to different testing protocols

4
Field Test options
  • 2 questions to be answered
  • What set of questions should be used in field
    test?
  • What field test approach should be used?
  • Consider your own context, need, funding, skills,
    etc. in deciding

5
Question sets
  • Core set 4 2 (6 domains of functioning)
    Compulsory
  • Core extended set on core domains highly
    recommended
  • Core extended set further additional
    questions nice to have for analysis
  • 2 additional domains (learning and interpersonal
    interactions)
  • Psychological distress
  • Country specific question set nice to have for
    comparison with prior data

6
What approach to testing?
  • What factors to consider in deciding
  • Country needs
  • Capacity to undertake test
  • Funds available
  • Time available
  • Planned activities, e.g. other surveys
  • ?

7
Census or national survey
  • Use of question set in planned Census or national
    survey
  • Advantages
  • Large sample (5000 10 000 or whole pop)
  • Minimal additional cost for disability questions
  • Disadvantages
  • Not sufficient space for extended set
  • Need to administer extended set and cognitive
    test to sub-sample (costs of time and additional
    training)

8
Special study small sample
  • Select sample on basis of known disability
    status
  • Allows for construction of 2X2 table for
    sensitivity and specificity calculations and
    identifying true and false positives and
    negatives use of gold standard
  • Suggest around 200 true positives and 200 true
    negatives
  • Interviewer does not know status of respondent
    (unless visible and obvious)

9
Special study (contd.)
  • Advantages
  • Larger set of questions
  • Close observation of interview
  • Cost is not exorbitant
  • Disadvantages
  • Might not get full population representation
  • Cost of establishing true status is high
  • What counts as the true positive? (beyond the
    easily observable)
  • What is the gold standard?

10
Special study large sample
  • Uses population based sample
  • Sample size based on expected prevalence of the
    different types of activity limitations
  • Prevalence rates are usually low and so need
    large sample to yield enough disabled
    respondents

11
Special study large sample (contd)
  • Advantages
  • Large data set
  • Detailed set of questions - more than extended
    set
  • Provides an indication of prevalence using core
    and extended sets
  • Disadvantages
  • Costly and time consuming
  • Requires capacity to run and analyse

12
Field test and cognitive test
  • All respondents have core set
  • One sub sample of respondents have cognitive test
    (core extended are embedded)
  • A second sub-sample of respondents do
  • Core extended
  • Core extended further Qs
  • Core extended country specific questions
  • Some do all ?

13
Non-responses and refusals
  • How do you define these?
  • How do you deal with these?
  • Aim to
  • Reduce non-response and refusals
  • Manage them when they do arise

14
Reducing NR and refusals
  • Use up to date sampling frame
  • Clearly defined reasons for data collection
  • Preparing the way prior contact, letters, etc.
  • Adequate interviewer training
  • Allow budget for call-backs
  • Plan for follow up of NR and refusals
  • Separate refusals, part refusals, non-contacts
    and sample loss (e.g. vacant dwellings)

15
Effect of each on analysis
  • Sample loss does not create bias but reduces
    sample size need large enough initial sample to
    take these into account
  • Refusals and non-contact bias where these
    respondents might be different to those
    respondents reached
  • Need to understand reasons for refusals and
    non-contacts during pre-testing and pilot stage

16
Interviewer debriefing
  • Interviewers are well placed to evaluate process
    and content at an early stage
  • Interviewers should note comments in margins of
    questionnaires
  • Different approaches
  • Group discussion (focus group technique)
  • Interviewer rating forms
  • Standardised interviewer questionnaires
  • Combination of all three

17
Enumerator training
  • Washington Group Regional Training Workshop
  • Rio de Janeiro
  • 19 20 September 2005
  • Margie Schneider
  • HSRC, South Africa

18
General points
  • Select some disabled interviewers
  • Importance of all interviewers having a good
    understanding of what disability is and is not
    and how it relates to the questions asked
  • Avoid using term disabled or with
    disabilities
  • All interviews are face to face
  • Translation training

19
General points (contd.)
  • Confidentiality and understanding what this means
  • Getting informed consent (ethics clearance?)
  • Interviewers must be able to explain purpose of
    survey
  • Read questions in set order and with set wording
  • Editing in field of completed questionnaires
  • Submitting of completed interviews to head office

20
Types of interviews
  • Direct respondent answers for him or herself
  • Interpreted an interpreter translates and
    respondent answers directly
  • Facilitated a third party assists in explaining
    (e.g. intellectually disabled person)
  • Proxy a person responds for another (e.g. child)

21
Interviewing disabled people
  • Show respect and treat the person like anyone
    else
  • Dont use first names unless permitted
  • Address the person directly (not their attendant)
  • Ask how you can adapt your presentation to make
    it easier (no need to ask what is wrong with
    person)

22
Hearing difficulties
  • Lip reading
  • Lighting
  • Face the person
  • Get persons attention before speaking
  • Reduce background noise
  • Set the context especially when changing topics
  • Use written communication (literate)

23
Physical difficulties
  • Accessibility of building where conducting
    interviews
  • Presence of attendant and confidentiality issues
  • Get to same level (e.g. sitting for person using
    wheelchair)
  • Person to be seated comfortably
  • Address person directly
  • Pointing may be difficult
  • Person may need breaks to move around

24
Visual difficulties
  • Large print and small print for cue cards
  • Braille versions of cue cards
  • Good contrast printing for pictures and print
    (black on white or yellow)
  • Identify yourself and others in the room verbally

25
Communication difficulties
  • Clarify preferred mode of communication
  • Repeat what you think was said to clarify unclear
    speech
  • Limit to yes / no questions

26
Specific learning difficulties
  • Manage problems in spatial orientation, hand-eye
    coordination
  • Limit auditory, visual and tactile distractions
  • Avoid written text
  • Explain carefully (if verbal language skills are
    affected)

27
Intellectual difficulties
  • Be careful with informed consent
  • Explain terms simply
  • Listen carefully
  • Have familiar person (friend or relative) close
    by
  • Use pictures or role play with little human or
    animal figures

28
Emotional or mental health difficulties
  • Side effects of medication
  • Break up interview if too fatigued
  • Give encouragement and support
  • Manage expressions of frustration
  • Manage stress

29
Hidden difficulties
  • Might not come forward with information because
    of fear of stigma
  • Effect of medication
  • May need to break up interview
Write a Comment
User Comments (0)
About PowerShow.com