Sampling Issues for Telephone Surveys in Scotland

1
Sampling Issues for Telephone Surveys in Scotland

• Gerry Nicolaas
• Survey Methods Unit
• National Centre for Social Research
• 13 January 2004
• email G.Nicolaas_at_NatCen.ac.uk

2
This session will focus on

• Surveys of the general population
• Probability sampling

3
Potential sampling advantages of telephone
interviewing

• No need for geographical clustering.
• Inclusion of more remote areas.
• Cheaper recalls

4
Telephone methods are not widely used in the UK
for social surveys among the general population
5
Obstacles to selecting a representative and
unbiased probability sample of the general
population

• Households with no telephone
• Unlisted telephone numbers
• Mobile phones

6
(1) Households without telephone

• Proportion of households with no phone in
  Scotland declined from 5 in 1998 to less than
  0.5 in 2002.
• Households with no phone tend to belong to the
  most socially and economically deprived groups.
• The exclusion of households with no phone would
  not introduce notable bias in a general
  population survey.

Source Taylor, 2003
7
(2) Unlisted telephone numbers

• In 1998 26 of telephone owning households (fixed
  lines) in Scotland were ex-directory
• Households with unlisted phone numbers tend to
  be
• - Smaller than average
• - Headed by someone in social classes IV or V

Source Beerten and Martin, 1999
8
(3) Mobile phones

• Proportion of households in Scotland with at
  least one mobile phone increased from 13 in 1998
  to 80 in 2002
• 5 of households only have mobile phone
• Mobile phone owners without a fixed line tend to
  be younger, lower socio-economic group,
  lower income, unemployed.
• Mobiles tend to be not listed in phone
  directories.
• Mobile phone numbers cant be linked to geogr.
  area.
• Mobiles belong to individual rather than
  household.

Source Taylor, 2003 Ofcom 2003
9
Sampling from telephone directories

• Select samples systematically or use random
  numbers to select page numbers and phone numbers.
• Non-coverage of
• - households with no telephone,
• - households with only mobile phones,
  and
• - households with unlisted telephone
  numbers.
• Coverage about 65 of Scottish population.
• Under-representation of
• - most socially economically
  deprived groups,
• - young mobile people,
• - people living in small households.

10
Plus digit sampling

• Sample is selected from the telephone directory
  and a fixed number is added to the last digit.
• Similar procedure replaces the last one or more
  digits with a random number.
• Sample will include unlisted numbers, but
• - proportion of unlisted numbers is lower than in
  the population
• - profile of the achieved sample reflects that of
  listed telephones rather than that of all
  telephones
• - probabilities of selection are unknown vary
  because of unequal distribution of listed
  unlisted numbers.

11
Random Digit Dialling (RDD)

• RDD uses comprehensive list of of valid area
  codes and prefixes and adds randomly generated
  suffixes.
• OFCOM database of blocks of 10,000 numbers
• e.g. 0131 557 xxxx
• All telephone households have a known non-zero
  chance of selection (fixed telephones and
  mobiles).
• Exclusion of households without phone will not
  introduce notable bias
• But High proportion of non-working numbers and
  other ineligibles (e.g. business) about 80

Sources http//www.ofcom.org.uk/working_w_ofcom/
numbers/numbers_administered/?a87101geog1 Nicola
as and Lynn (2002)
12
RDD hit rates can be improved

• Advanced telephony systems can screen out most
  non-working numbers
• RDD samples can be matched against yellow pages
  to remove listed business numbers
• Adopt sample designs that improve hit rate,
• - Mitofsky-Waksberg two-stage method
• - List-assisted methods

13
Should mobile phone numbers be included in RDD
sample?

• Exclusion of mobile phone numbers will result in
  under-coverage of younger people, lower
  socio-economic group, lower income group, the
  unemployed.
• Inclusion of mobile phone numbers will result in
  further reduction in effective sample size due to
  post-weighting of households with more than one
  phone (fixed mobile)
• Inclusion of mobile phone numbers will increase
  ineligibles for surveys covering areas smaller
  than UK (e.g. Scotland).

14
Dual frame sampling

• E.g, a sample of listed telephone numbers
  supplemented with a RDD sample.
• Directory status of each RDD interview must be
  known
• Sampling administration is more complicated.
• Data to be combined using post-stratified
  dual-frame estimators.
• Unclear whether this approach is feasible and
  whether the gains are worthwhile.

15
Single frame, dual mode approach

• Use a frame with good coverage (e.g. PAF)
• Telephone interviews for those sampled cases with
  a matched telephone number.
• Postal questionnaires or face-to-face interviews
  for those without a matched telephone number.
• But
• - low matching rates
• - costly if unmatched cases interviewed
  face-to-face
• - relatively low response rates if postal
  questionnaires are sent to unmatched cases
• - possibility of mode effects.

16
Conclusion

• None of the sampling methods are perfect.
• Choice of sampling method depends on specific
  survey.
• On the whole, RDD appears to be superior method
• - Complete coverage of all households with
  phones.
• - Non-coverage of households without phone can be
  ignored
• - Sampling frame is accurate (Ofcom database).
• - Precision of estimates is high (unclustered
  sample).
• - Relatively cheap and easy to select RDD sample.
• - Fieldwork efficiency can be improved.
• But the proportion of mobile only households
  needs to be monitored.
• Response rates?

17
Another sampling issue Respondent selection

• Include all eligible household members or select
  one at random with post weighting for unequal
  selection probabilities.
• Random selection of one eligible household member
  tends to be most common approach for telephone
  surveys
• Gold standard is Kish Method but some claim
  this requires too much info up front and may
  reduce response.
• Most common method is Last/Next Birthday method.
• NatCen experiment showed no significant
  difference between the two methods in response
  rates nor sample compositions.

Source Tipping Nicolaas (2001)
18
Suggested reading

• Beerten Martin (1999) Household ownership of
  telephones and other communication links
  implications for telephone surveys. Surv.
  Methodol. Bull., 44 1-7.
• Collins (1999) Sampling for UK telephone surveys.
  JRSS(A) 162 1-4.
• Collins Sykes (1987) The problems of
  non-coverage and unlisted telephone numbers in
  telephone surveys in Britain. JRSS(A) 150(3)
  241-253
• Lepkowski (1988) Telephone sampling methods in
  the United States. In Groves et al (eds),
  Telephone Survey Methodology. John Wiley and
  Sons New York
• Nicolaas Lynn (2002) Random-digit dialling in
  the UK viability revisited. JRSS(A) 165
  297-316.
• Taylor, S. (2003) Telephone surveying for
  household social surveys the good, the bad and
  the ugly. Surv. Methodol. Bull., 52 10-21.
• Tipping Nicolaas (2001) Respondent selection
  procedures for telephone surveys. Survey Methods
  Newsletter, 21(1) 4-7, Nat. Centre for Soc. Res.