Towards Best Practices in Surveying People with Disabilities: Lessons Learned From the Field

1
Towards Best Practices in Surveying People with
Disabilities Lessons Learned From the Field

• Thilo Kroll
• University of Dundee (Scotland, UK)
• Alliance for Self Care Research
• School of Nursing Midwifery (Social Dimensions
  of Health Institute)
• ISDS State-of-the Art Conference, Washington DC,
  July 12-13, 2006
• t.kroll_at_dundee.ac.uk

2
Overview

• Brief review of reasons for survey exclusion of
  people with disabilities
• Examples to remedy survey exclusion taken from a
  forthcoming book
• Suggestions for future research

3
Who are we talking about?

• According the American Community Survey (ACS)
  approximately 12 of the working age adult
  population reported a disability in 2004
• There is evidence that the number of people
  reporting disabling conditions is increasing
  (MMWR 2001)
• But
• Who defines disability? (e.g. policy, research,
  consumer)
• How is disability defined? (e.g. ICF vs.
  activity limitation)
• Who reports disability? (e.g. self-report,
  proxy report, researcher)
• What is the specific purpose of disability
  reporting? (e.g. return to work, health,
  insurance)
• How is disability measured? (e.g. dichotomous
  item, categorical, multidimensional)

4
What do we know?

• Interagency Committee on Disability Research
  (ICDR) hosted a 2-day conference in April 2004 on
• Best Practices for Surveying People with
  Disabilities
• Federal sector, private organizations and
  universities
• Topics
• Participatory action research and the survey
  process
• Methods for improving survey measurement and
  response
• Modes of data collection and survey participation
• The role of technology and the Internet
• Quality assurance in survey development and
  implementation

5
Who is excluded?

• People with
• Physical (Mobility Impairment) Example Cannot
  write answer (without accommodation)
• Sensory (Vision, Hearing, Speech) Example
  Cannot read question (without accommodation)
• Cognitive (Attention, Memory, Concentration,
  Learning) Example Cannot remember question
  Cannot understand question (without
  accommodation)
• Psychiatric (Depression, Psychosis) Example
  Does not trust interviewer (without
  accommodation)
• disabilities
• People from
• Socially marginalized groups (e.g. income,
  education, living conditions)
• Example Homeless adult with diabetes-related
  disability and limited formal education lives in
  shelter 2 out of 7 nights
• Ethnic and linguistic minorities (e.g. Spanish
  speaking adults with cognitive or speech
  impairments)
• Multiple social disadvantage

6
Research question and method
What do we want to know?
Why do we need to know? (relevance)
Who wants to Know?
Is it the correct question?
Yes
No
Which method can answer the question?
7
Research standardardization vs.
Inclusiveness/Flexibility
Survey/Correlational Designs
Qualitative/Ethnography
Experiment/RCT

• Standardization
• Sampling
• Data collection
• Data analysis

• Diversity of
• Impairments
• Environments
• Accommodation needs

Mixed Method Designs
8
(Some) reasons for exclusion

• General population surveys not flexible enough to
  be customizable to specific respondent needs
• People do not conform to disability
  conceptualization of researcher, policy
• People cannot be found
• Researchers dont look
• Researchers/interviewers not sufficiently trained
• Time constraints
• Lack of funds for alternative formats

9
Some further reasons for exclusion

• Conceptual (e.g. Activity limitation vs. ICF)
• Cultural (e.g. Deaf vs deaf)
• Language (e.g. ASL vs. Standard US English)
• Impairment (e.g. speech impairments)
• Reliance on proxy respondents (e.g. no efforts
  made to include individuals directly)
• Research question (e.g. consumer relevance
  policy relevance)
• Study instrument (e.g. highly structured,
  flexible, semi-structured)
• Study administration (e.g. phone, face-to-face)
• Format (e.g. paper-pencil, alternative formats)
• Item appropriateness (e.g. relevance, cognitive
  appropriateness)
• Researcher training (e.g. interviewer
  preparation)
• Reach (e.g. sampling through advocacy groups,
  random dialling population-based, multiple
  sampling approaches)
• Costs (alternative formats more costly)

10
Spectrum of inclusion/exclusion
Invisible
Multiple modifications to standard survey
practice needed
Simple modifications to standard surveys needed
No Modifications needed
11
Example Deaf Persons Language, Culture and
Research Practice Issues

• Barbara Allen, Nancy Meyers, John Sullivan
  Melissa Sullivan (2006) Using American Sign
  Language in Assessing the End-of-Life Care
  Educational Needs of Deaf Persons Lessons on
  Language, Culture, and Research Practices
• Approximately 23 million people in the US are
  categorized has having a hearing loss
• Deaf or hard of hearing persons are
  underrepresented in US government surveys of
  citizens well being (telephone surveys exclude
  the group)
• Following Spanish, American Sign Language (ASL)
  is the second most commonly used minority
  (non-English) language in the US
• Approximately 2 Million Americans are estimated
  to use ASL every day
• No database with demographic information (poses
  problem for sampling)
• Language differences may impede quality of
  healthcare
• Barriers to communication identified in hospice
  care settings
• Obstacles to healthcare and information (health
  literacy)
• Accuracy
• Completeness

12
Example Deaf Persons Language, Culture and
Research Practice Issues

• ASL is a visual language signs are not
  adequately represented in written form
• The use of written English language
  questionnaires between Deaf and hearing
  individuals may be potentially unreliable
• Fourth grade reading level questions may fail to
  reflect cognitive sophistication and depth of
  experience of Deaf respondents
• Question translation may vary across ASL
  interpreters
• Differences in language (e.g. metaphors,
  homonyms, idioms) may give ambiguous meaning to
  validated English-language measures
• English language constructs may not be
  interpretable in ASL or even known
• Gaps in background knowledge need to be addressed
  before the study can be conducted

13
Example Deaf Persons Language, Culture and
Research Practice Solutions

• Community-Based Participatory Research approach
  Deaf community initiated collaboration
• CBPR partnerships join participants with diverse
  skills, knowledge, and expertise to address
  complex problems
• To reflect the relational networks at the core of
  the Deaf culture
• Goal To identify and adopt best practices
  within studied communities
• Visual quality of ASL makes face-to-face
  communication essential
• Glossing GLOSS is a unique ASL linguistic method
  for translating and transcribing the cognitive
  equivalent of an English word or phrase
• Reading, understanding and signing GLOSS possible
  for most Deaf respondents
• Likert Scale representations ranging from left to
  right may not work with ASL users who more
  typically arrange degrees of measurements
  vertically
• Example of modification of scale end points
• not very important ----? IMPORTANT LITTLE

14
Example Deaf Persons Language, Culture and
Research Practice Further developments

• Full community participation is necessary to
  construct a meaningful research tool
• Deaf people are integral to administering the
  research survey and in interpreting the results

15
Example Modification to telephone interviewing
Issues

• Ciemnecki, A.B. CyBulski, K.A. (2006). Removing
  the Barriers Modifying Telephone Survey
  Methodology to Increase Self-Response Among
  People with Disabilities
• SSI survey experience (type of impairment unknown
  prior to sampling)
• Three principal challenges related to telephone
  interviewing
• Communication barriers
• Stamina barriers
• Cognitive barriers

16
Example Modification to telephone interviewing
Solutions

• Modification to Instrumentation
• Reduction of High-Frequency Sounds
• Questions cannot be understood if they are not
  heard
• Example (High frequency sounds) How satisfied
  are you with the overall quality of care you
  receive as a member of NAME OF MANAGED CARE PLAN?
  Are you satisfied, somewhat satisfied, neither
  satisfied nor dissatisfied, or very
  dissatisfied?
• Example (low frequency sounds) How would you
  rate the overall quality of the medical care you
  get as a member of NAME OF MANAGED CARE PLAN? Is
  it excellent, very good, good, fair or poor?

17
Example Modification to telephone interviewing
Solutions

• Interviewer Checkpoints
• Fatigue probes (e.g. Would you like to take a
  break?)
• Encouragement probes (e.g. Your answers are very
  helpful for this study)
• Structured probes (appeared on CATI screens)
• Modification to procedures
• Interviewer Training and Supervision
• General sensitivity training
• To accommodate hearing impairments
• Use normal tone of voice, no restriction to
  single syllable words
• Adjust headsets to amplify outgoing sounds
• Use text telephone (TTY/TTD) relay operator

18
Example Modification to telephone interviewing
Solutions

• To accommodate speech impairments
• Adjust headsets to amplify incoming sounds
• Ask respondents to repeat what she or he said
• Patience (speech easier to discern after a while)
• Repeat and read out loud what was understood
  (clarification)
• Not to pretend to have understood something the
  interviewer did not understand
• Not to make assumptions about people based on
  tone of voice
• To accommodate stamina challenges
• Scheduling appointments for times when
  respondents most alert
• Breaks
• Rescheduling

19
Example Modification to telephone interviewing
Solutions

• To accommodate cognitive challenges
• Repeat respondents name often
• Keeping respondents free of distraction
• Refrain from exaggerating the inflection or tone
  of voice (distracting)
• Production Standards
• Estimates of respondent burden (extra time needed
  per interview for interviewer)
• Contact Procedures
• Advance information by mail
• Interviewer continuity
• Interviewer Assessments
• Interviewers rated respondents ability to
  understand and answer questions

20
Example Modification to telephone interviewing
Further developments

• Telephone interviews may not work for everyone
• Self-response should take precedence over proxy
  response
• Facilitate assisted interviews
• Combine proxy and self-response by telephone
• Mixed mode designs (add in-person, paper, web
  modes)

21
Interviewer preparation and training Issues

• Glazier, R (2006) Training Temporarily
  Able-Bodied Survey Interviewers
• Diversity in disability
• Disability type and impairment spectrum
• Socioeconomic status
• Educational background
• Cultural, linguistic, ethnic background
• Self- vs. researcher defined disability

22
Interviewer preparation and training Solutions

• Interviewer Disability Training Manual
• Generic pointers
• Communication
• Language (choice of appropriate terminology)
• Stamina (fatigue)
• Expectations and Attitude
• Privacy
• Proxy respondents (keep in mind who is the actual
  respondent eye contact)
• Interview environment
• Explanations
• Patience (without a condescending tone)
• Disability-specific pointers
• Cognitive disabilities (e.g. false
  generalizations to a range of intellectual
  functions)
• Psychiatric disabilities (e.g. false assumptions
  about intellectual abilities)
• Sensory disabilities (e.g. guide dogs lighting
  response options background noise positioning)
• Mobility disabilities (e.g. dont assume
  connection between physical impairment and mental
  faculties dont lean on wheelchair or invade
  personal space offering assistance)

23
Thinking outside the box
24
Thinking outside the box

• Who to involve?
• What questions to ask?
• How to ask questions?
• What sampling strategies to use?
• What methods to apply?
• How to make sense of the data

http//www.invo.org.uk
25
Number of PAR papers published between 1996 and
2005 OVID MEDLINE, CINAHL, PSYCINFO (duplicates
removed)
26
Future developments

• Design
• Participatory strategies (think about who is
  representative reflect what participatory
  means not always CBPR)
• Mixed method strategies and designs
• Sampling
• Going outside known channels use wider community
  outlets
• Think social exclusion in broader terms
• Mixed mode interviewing
• Use more than one approach
• New technologies
• MM cellphones SMS/Text PDAs Multi-media story
  telling/focus groups etc. via instant messaging,
  Skype, ASL videostream and other Internet
  applications
• Evidence and Quality assurance procedures
• Good research

27
Summary and conclusion

• Need for standardization vs full inclusion
  represent two end-points of the research spectrum
• Meaningful (not tokenistic) participatory
  research approaches are required to be fully
  inclusive of people with disabilities
• Creative approaches to sampling and data
  collection are needed to enhance inclusiveness
• Review of the process of evidence production
• Surveying people with disabilities needs to be
  sensitive to the cultural and socioeconomic
  context in which people operate

28
Thank you! More information

• All examples were based on chapters of the
  forthcoming book by
• Kroll, T., Keer, D., Cyril, J., Placek, P.
  Hendershot, G. (eds) (2006). Towards Best
  Practices for Surveying People with Disabilities.
  Hauppage, NY NovaPublishers.
• The book can be ordered through
• https//www.novapublishers.com/
• Books distributed in UK through Gazelle Books