The Definition and Measurement of Disability: The Work of the Washington Group

1
The Definition and Measurement of Disability The
Work of the Washington Group

• Jennifer Madans
• National Center for Health Statistics, USA
• for the Washington Group

2
The Washington Group on Disability Statistics

• In June of 2001, the UN International Seminar on
  the Measurement of Disability recommended the
  development of principles and standard forms for
  global indicators of disability to be used in
  censuses
• There was a broad consensus on the need for
  population based measures of disability for
  country use and for international comparisons

3
The Washington Group was set up to

• Foster international cooperation in the area of
  health and disability statistics
• Develop disability measures suitable for censuses
  and surveys that will provide basic information
  on disability
• Untangle the web of confusing and conflicting
  disability estimates

4
Current Problem

• Currently national censuses in developing
  countries use one of three types of questions
  that provide widely differing estimates of
  national prevalence of disability
• The three types of questions include
• A. Generic question about the presence of a
  condition
• B. Generic questions about the presence in the
  household of a person with a disability followed
  by a list of impairments
• C. Checklist of impairments

5
Variety in National Estimates

• Developing Countries
• Cyprus 4.0
• Uganda 1.2
• Nigeria 0.5
• Developed Countries
• United Kingdom 12.5

6
Washington Group Objectives

• Develop a small set/s of general disability
  measures
• Recommend extended set/s of items to measure
  disability as components of population surveys /
  supplements
• Address methodological issues associated with
  disability measurement

7
Preliminary work to meet objectives

• Clarify the purpose of data collection in order
  to identify appropriate measures
• Understand choices being made when time, expenses
  and respondent burden limit number of questions

8
Moving from Concept to Definition to Measurement
The Conceptual Model

• ICF selected as the conceptual model
• Common point of reference
• Common vocabulary
• Does not provide an operational definition or a
  way to measure the concepts

9
Moving from Concept to Definition to Measurement
The Definitional Paradox

• There is no single operational definition of
  disability
• Different operational definitions lead to
  different estimates
• The question you are trying to answer (the
  purpose) will determine which definition to use
• Need to understand the choices that are being
  made when a purpose and a definition are chosen
• Need to understand the choices that are being
  made when time, expenses and respondent burden
  limit number of questions

10
Purpose of Data Collection

• 3 major classes of purposes at aggregate level
• Service Provision
• Monitoring functioning in the population
• Assess equalization of opportunities
• 2 criteria for selection of a purpose
• Relevanceparticularly for policy makers and
  program officials
• Feasibility

11
Purpose Service provision

• Seeks to identify those with specific needs,
  usually the most serious problems
• Requires detailed information about the person
  and the environment
• Influenced by the organization and structure of
  service organizations within a particular culture

12
Purpose Monitoring functioning in the population

• Seeks to identify all those with activity or
  participation limitation
• Response comparability problematic since
  participation is culturally and environmentally
  determined

Population reporting work limitation
13
Purpose Equalization of opportunities

• Seeks to identify all those at greater risk than
  the general population for limitations in
  activity or participation
• Disability as a demographic

Employed
14
Applying criteria to select purpose

• Service provision
• Level of detail necessary not feasible in a
  census format
• Nature of service provision varies across
  cultures
• Monitoring functioning
• Response comparability problematic since
  participation is culturally and environmentally
  determined
• Assessing equalization of opportunities
• If we conceive of disability toward the most
  basic elements of activity, without tying it to
  participation, we limit the number / types of
  questions thus enhancing feasibility

15
Locating Risk in the ICF Model
Health Condition
ACTIVITY
?
Body Functions Structure
Participation
Environmental Factors
Personal Factors
Source ICF, WHO, 2001
16
Moving from Concept to Definition to Measurement
Measurement of equalization of opportunities

• Locate the definition of disability at the most
  basic level of activity/participation in core
  domains
• This level is associated with the ability or
  inability to carry out basic bodily operations at
  the level of the whole person (i.e. walking,
  climbing stairs, lifting packages, seeing a
  friend across the room)

17
Benefits of this approach

• Development of a demographic means of
  understanding disability (can compare persons
  with and without disability)
• Connection between disability and participation
  can be made during data analysis
• Effectiveness of programs / policies to promote
  full participation can be monitored

18
Possible types of questions

• Questions that measure functioning in various
  domains such as mobility, cognition, sensory,
  etc.
• A qualifier would need to ascertain that the
  action was accomplished without human or
  mechanical assistance

19
Possible question options

• Mobility
• Walking
• Climbing stairs
• Bending or stooping
• Reaching or lifting
• Using hands
• Sensory
• Seeing
• Hearing
• Communicating
• Understanding
• Speaking

• Cognitive functions
• Learning
• Remembering
• Making decisions
• Concentrating
• Emotional functioning
• Interpersonal interactions
• Psychological well-being

20
Criteria for inclusion of domains

• Cross cultural comparability
• Suitability for self-report
• Parsimony
• Validity across various methodological modes

21
WG draft questions for censuses (short
disability measure)

• Core Questions
• Do you have difficulty seeing even if wearing
  glasses?
• Do you have difficulty hearing even if using a
  hearing aid?
• Do you have difficulty walking or climbing
  stairs?
• Do you have difficulty remembering or
  concentrating?
• Additional Questions
• Do you have difficulty with (self-care such as)
  washing all over or dressing?
• Because of a physical, mental, or emotional
  health condition, do you have difficulty
  communicating (for example understanding others
  or others understanding you)?
• Response categories No - no difficulty Yes
  some difficulty Yes a lot of difficulty
  Cannot do at all

22
Objectives

• Identify persons with similar types and degree of
  limitations in basic activities regardless of
  nationality or culture
• Represent the majority (but not all) persons with
  limitations in basic activities in any one nation
• Represent the most commonly occurring limitations
  in basic activities within any country
• Capture persons with similar problems across
  countries

23
Intended use of data

• Development of a demographic means of
  understanding disability Can compare persons
  with and without disability on levels of
  participation in employment, education, or family
  life to see if persons with disability have
  achieved social inclusion
• Monitor effectiveness of programs / policies to
  promote full participation
• Monitor prevalence trends for persons with
  limitations in specific basic activity domains

24
Limitations

• One set of measures will not satisfy multiple
  needs for disability data
• Excluded populations
• Very young children
• Institutionalized population
• Homeless
• Floating populations
• It is not our purpose to
• identify every person with a disability within
  every community
• replicate a population evaluated across a wider
  range of domains that would be possible with
  other forms of data collection

25
Constraints of the census format

• Limited space
• Indicated domains reflect types of questions that
  would identify largest population for this
  purpose and those most culturally comparable
• Allows some flexibility for additional culturally
  relevant questions for specific countries

26
Pre-test implementation

• Implementation protocols developed
• Objectives and evaluation plan for field /
  cognitive testing
• Cognitive test plan
• Translation
• Enumerator training
• Sample design issues
• Plan for tabulation, analysis and report writing
• Two regional workshops successfully implemented
• June 20-22, 2005 / Nairobi, Kenya
• September 19-20, 2005 / Rio de Janeiro, Brazil

27
Standardized testing

• 15 countries participating
• 13 funded via World Bank grant, 2 self-funded
• Cognitive tests in 12 countries
• Congo, Egypt, Gambia, India, Kenya, Lesotho,
  Mauritius, Philippines, Uganda, Mexico, Tanzania,
  Vietnam
• Field tests in 2 countries
• Gambia, Vietnam
• Combined cognitive/field test in 3 countries
• Argentina, Brazil, Paraguay

28
Pre-test objectives

• To determine if
• the questions are being interpreted as intended
  by the developers in that they are capturing the
  important aspects of the functional domains
  selected and
• the questions are interpreted consistently across
  countries.

29
Pre-test evaluation

• Evaluation
• Validity
• Content validity How well WG question set
  compares with expanded disability measures
• Criterion related validity How well individual
  WG questions compare to relevant similar concept
  in a comparison measure
• Face validity Does the measure look to be
  valid?
• Reliability test/re-test

30
Cognitive test

• Cognitive test
• Objective to determine if questions are being
  interpreted as intended and if interpretation is
  consistent across countries
• WG cognitive test more structured than usual
• Ensures a greater level of standardization across
  test sites
• Understand how the response mechanisms operate in
  the different countries

31
Cognitive test

• Components of cognitive test
• Purposive samples
• Asked WG questions and series of detailed
  questions in the same domains
• Traditional cognitive probes
• Collected information on response process
• Analyzed problematic response patterns
• Interviewer report on problems respondent had
  with questions

32
Detailed questions on functioning (example,
cognition)

• WG question
• Do you have difficulty remembering or
  concentrating?
• Detailed questions
• Do you have difficulty remembering the names of
  people or places?
• Do you have difficulty remembering appointments?
  
• Do you have difficulty remembering how to get to
  familiar places?
• Do you have difficulty remembering important
  tasks, like taking medications or paying bills?
• Do you have difficulty concentrating on doing
  something for ten minutes?
• ?Do you have difficulty learning a new task, for
  example, learning how to get to a new place?
• Do you have difficulty finding solutions to
  problems in day to day life?

33
Information on response process

• Questions to interviewers
• Need to repeat any part of the question?
• Any difficulty in using response options?
• Did respondents ask for clarification or qualify
  their answers?
• Follow-up probes to understand response process

34
Correlates of cognitive difficulty
35
Correlates of cognitive difficulty
36
Socio- demographic correlates of cognitive
difficulty
37
Overall, questions worked well

• Well understood
• Responses consistent
• Problem areas
• Glasses clause WG is revisiting but solution
  has a lot to do with translation protocol
• Communication Question WG will revisit to see
  if question can be simplified
• Introduction to the question set

38
Field test

• Conditions closely approximate how final study
  will be done
• See how WG core questions function in different
  countries
• Useful to compare WG set to a larger set of more
  detailed questions to determine whether the same
  population is identified by each set

39
Field test

• Undertaken in Gambia, Vietnam, and South Africa
• Checking for internal consistency
• Results comparable to changes seen in other
  countries with move towards functional approach
  to measuring disability

40
Field test Analysis

• Constructed three thresholds for disability based
  on WG census question
• D11 if any difficulty
• D21 if a lot
• D31 if unable to do
• Constructed three thresholds for disability based
  on extended questions
• ED1
• ED2
• ED3
• What was the D measure picking up compared to the
  ED measures?

41
Example Mobility question

• WG question
• Do you have difficulty walking or climbing steps?
  
• D11 if some difficulty, a lot of difficulty,
  or cannot do at all
• D21 if a lot of difficulty or cannot do at
  all
• D31 if cannot do at all
• Extended Questions
• Do you have difficulty going outside of your
  home?(0)
• Do you have difficulty walking a long distance
  such as a kilometer (or a mile)?
• ED11 if some, a lot or cannot do for
  either question
• ED21 if a lot or cannot do at all for either
  question
• ED31 if cannot do at all for either question

42

43
Disability data collection recommendations

• Do not use the word disability
• Functional approach
• Separate functional domains
• Scaled responses
• Definition of disability can be made ex post, and
  hinge on domains and thresholds used

44
Meeting products and more information

• Executive summaries of meetings, presentations
  and papers posted on the Washington Group
  website
• http//www.cdc.gov/nchs/citygroup.htm
• Publication of key papers in a special issue of
  Research in Social Science and Disability