Survey design issues and child outcome measures in the Millennium Cohort Study

1
(No Transcript)
2
Survey design issues and child outcome measures
in the Millennium Cohort Study
Sub-brand to go here

• Lisa Calderwood
• Senior Survey Manager
• Centre for Longitudinal Studies

CLS is an ESRC Resource Centre based at the
Institute of Education
3
Outline of presentation

• Introduction to the Millennium Cohort Study
• Methodological and design issues
• Child outcome measures

4
An introduction to the Millennium Cohort Study
5
Millennium Cohort Study (MCS)

• Longitudinal birth cohort study following over
  19,000 children born in the UK in 2000/2001
• Four sweeps so far at 9 months, 3 years, 5 years
  and 7 years, next planned for 11 years
• Funded by UK Economic and Social Research Council
  (ESRC) and UK government departments
• Over sampled places in Scotland, Wales and
  Northern Ireland, areas with high child poverty
  and in England areas with higher minority ethnic
  populations
• One of four British Birth Cohort Studies

6
British Birth Cohort Studies

• MCS the first national birth cohort study for
  30 years
• Previous UK/GB National Studies
• 1946 MRC National Survey of Health
  Development
• 1958 National Child Development Study (NCDS)
• 1970 1970 British Birth Cohort Study
• 1958, 1970 MCS based at CLS

7
Objectives of MCS

• To chart the initial conditions of the social,
  economic and health advantages and disadvantages
  facing new children in the new century and their
  consequences
• To capture information for the future
• To compare patterns of development with other
  cohorts
• To collect information on previously neglected
  topics, such as fathers involvement and child
  care
• To investigate the wider social ecology of the
  family, including community and services,
  splicing in geo-coded data

8
MCS Study Design

• Cohort born over a 12 month period
• Includes children in England, Scotland, Wales and
  Northern Ireland
• Geographically clustered by electoral ward
• Over represents children from different ethnic
  groups and those in deprived areas

9
MCS Sample Design

• Stage 1 Selection of electoral wards (398)
• Ethnic
• Disadvantaged
• Advantaged
• Stage 2 Selection of families (27201)
• From child benefit records held by Department of
  Work and Pensions
• 18,552 families recruited at 9 months 692
  families at 3 years 19,244 families

10
MCS Number of interviewed families by sweep
COUNTRY Total Cohort MCS1 Achieved Sample (Total) MCS2 Achieved Sample (Total) MCS3 Achieved Sample (Total) MCS4 Achieved Sample (Total)
ENGLAND 12,225 11,532 10,050 9,717 8,839
WALES 2,760 2,761 2,261 2,181 2,018
SCOTLAND 2,336 2,336 1,814 1,814 1,628
N IRELAND 1,923 1,923 1,465 1,534 1,372
TOTAL UK 19,244 18,552 15,590 15,246 13,857
11
MCS Sponsors

• UK Economic Social Research Council
• ONS consortium of UK Government Departments
• Office for National Statistics
• Department for Children, Schools and Families
• Department for Work and Pensions
• Department of Health
• Welsh Assembly Government
• Scottish Government
• NI Executive
• SureStart - National Evaluation in England
• Childrens Fund - National Evaluation in England

12
Content of MCS Surveys
9m Age 3 Age 5 Age 7 Age 11
Interview (and self-completion) with both resident parents ? ? ? ? ?
Cognitive assessments ? ? ? ?
Physical measurements ? ? ? ?
Child self-completion ? ?
Older Siblings ? ?
Interviewer Observations ? ? ? ?
Teacher Survey ? ? ?
13
Content of MCS Parent Interviews

• Pregnancy and Child Birth
• Child Development and Behaviour
• Education and Childcare
• Child Health
• Family Demographics, Family Relationships and
  Parenting
• Parental Health
• Parental Employment, Education and Income
• Attitudes
• Social and Community Context

14
Administrative data linkage on MCS
Cohort Member Siblings Parents
Health ? ? ?
Education ? ?
Economic ?
15
MCS Add on studies

• Collection of bio-markers
• Oral fluid sample for exposure to infections at
  age 3
• Shed milk teeth for lead from age 6 upwards
• Saliva samples for DNA from mothers, fathers and
  children at age 11?
• Physical activity monitoring at age 7
• Sub-studies
• Mothers who had assisted fertility treatment
• Nursery observations at age 3
• Methodological studies
• Predicting and preventing non-response

16
Methodological and design issues
17
Theoretical framework

• Groves at al. (1989) Total Survey Error

Non-observational Observational
Coverage Measurement
Sampling
Non-response
18
Outline of section on methodology

1. Non-response
2. Measurement
3. Surveying children

19
Methodology 1 Non-response
20
Non-response in longitudinal surveys

• Different sources of attrition location, contact
  and co-operation e.g. Lepkowski and Couper
  (2002), Lynn et al. (2002)
• Many longitudinal surveys have developed highly
  successful strategies to minimise attrition
  through failure to locate and failure to
  co-operate

21
Response rates at MCS1(Baseline)

• Response rate at baseline sweep was 72
• Response rate differed by country
• lowest in Northern Ireland (66)
• Response rate differed by ward type
• lowest in ethnic areas (66)
• lower in disadvantaged than advantaged (72 vs.
  76 in England)

22
Response rates (RR) from MCS1-MCS4
MCS1 MCS2 MCS3 MCS4
Achieved sample 18,552 15,246 15,590 13,857
Study RR 72 58 58 52
Sweep RR 81 79 81 76
Contact rate 96 90 94 95
Co-operation rate 85 88 85 78
Field RR 81 79 86 86
23
Longitudinal patterns of participation on MCS
Productive these sweeps N
All four sweeps S1,2,3,4 11,721 60.9
Three out of four sweeps S1,2,3 1,513 7.9
S1,3,4 1,029 5.3
S1,2,4 445 2.3
Two out of four sweeps S1,2 1,219 6.3
S1,3 415 2.2
S1,4 168 0.9
First sweep only S1 2,042 10.6
All three sweeps S2,3,4 468 2.4
Two out of three sweeps S2,3 100 0.5
S2,4 26 0.1
Sweep two only S2 98 0.5
24
Response to individualelements
9m Age 3 Age 5 Age 7
Main respondent (mother) interview and self-completion 99 99 99 99
Partner respondent (father) interview and self-completion 85 83 88 84
Cognitive assessments - 78 98 99
Physical measurements - 90 99 99
Child self-completion - - - 94
Teacher Survey 68 70
25
Fieldwork
26
Division of responsibilities Between CLS and
Agency
Calderwood, 2009
27
Fieldwork procedures securing informed consent

• Advance letter and general information leaflet
• Additional leaflets with more detailed
  information e.g. child elements, teacher survey,
  data linkage
• Information leaflet for child introduced at age 7
  
• Written consent from parents for own and childs
  participation, teacher survey and data linkage
• Verbal assent from child at age 11

28
Fieldwork proceduresresponse maximisation

• Telephone contact (for some) at MCS3 and MCS4
• Minimum requirements for call attempts
• Refusal conversion
• Multiple visits and interviewer flexibility

MCS3 MCS4
Telephone contact attempted 68 66
Telephone contact made 48 52
Appointment made by telephone 44 47
29
Interviewer tracking procedures

• Multiple attempts to contact sample members,
  current occupiers, neighbours and stable contacts
  through personal visits, telephone, mail, email,
  text

MCS3 MCS4
Tracing required ( of issued) 13 14
Traced by interviewers ( of movers) 53 54
Traced by CLS ( of movers) 18 15
Untraced ( of movers) 29 31
30
Cohort maintenance
31
Cohort maintenance procedures used on MCS

• Send feedback on results every year
• Update contact information every year
• Encourage families to contact us with change of
  address i.e. Freephone number, email address and
  website
• Office-based tracking (and through admin data)
  for those known to have moved
• Additional/bespoke mailings for certain groups
  e.g. refusals, emigrants

32
Cohort maintenance procedures used on MCS

• Annual updating of contact information
• Reply-slip pre-printed with full contact details
  i.e. Home address and phone, names of parents and
  child, parents phone numbers (mobile and work),
  email address and stable contacts
• Proportion with different types of contact
  information

Parent 1 (P1) Parent 2 (P2)

Home Telephone Number 83 n/a
Work Telephone Number 16 15
Mobile Telephone Number 62 47
Email Address 53 36
Stable contact 72 26
33
Tracking procedures used on MCS

• Office-based tracking
• Attempt to contact sample members, current
  occupiers and stable contacts through telephone,
  mail, email and text
• Use electoral and phone records and postal
  directories which are available publicly,
  centrally and electronically
• Attempt tracking through administrative data
  sources

34
Residential mobility, trackingand non-response
on MCS

• Conditional on successful location, mobile
  families in MCS are no less likely to co-operate
  at next sweep than non-mobile families (Plewis et
  al., 2008, Calderwood, 2009)
• Tracking methods used on MCS are successful over
  90 of mobile families are successfully located
  at next sweep (Calderwood, 2009)
• Comparable to other major longitudinal studies
  (Couper and Ofstedal, 2009)

35
Methodology 2 Measurement
36
Measurement in longitudinal surveys

• Measurement of change and the contribution of
  dependent interviewing e.g. Moore et al., 2009,
  Jackle, 2009
• Panel conditioning e.g. Sturgis et al. 2009
• Mode changes between sweeps e.g. Dillman, 2009
• Uses of administrative data e.g. Calderwood et
  al, 2009
• Role of interviewers e.g. Lynn, 2008

37
Methodology 3 Researching children
38
Conducting longitudinal survey research with
children

• Data collection mode use of web surveys?
• Contact and communication email, text, web
  including social networking sites
• Ethical and legal issues relating to consent to
  participate (and link to administrative data)
• Giving children a voice advisory/participation
  panel?

39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
Child outcome measures
44
Outline of section on child outcome measures

1. Cognitive development
2. Behavioural development
3. Physical growth and development

45
Child outcomes 1 Cognitive development
46
Cognitive Assessment at 3

• Bracken School Readiness Assessment
• The sub-tests administered evaluates 88 concepts
  relating to
• Colours - represents both primary colours and
  basic colour terms.
• Letters- measures knowledge of both upper- and
  lower-case letters.
• Numbers/Counting - measures recognition of
  single- and double-digit numbers and samples the
  ability to assign a number value to a set of
  objects.
• Sizes - includes concepts that describe one, two,
  and three dimensions.
• Comparisons - measures ability to match and/or
  differentiate objects based on one or more of
  their salient characteristics.
• Shapes - includes one-, two-, and
  three-dimensional shapes. The one-dimensional
  category includes linear shapes two-dimensional
  shapes are represented by concepts such as the
  circle, square, and triangle and
  three-dimensional shapes include concepts such as
  the cube and pyramid.

47
Cognitive Assessment at 3 and 5

• BAS Naming Vocabulary
•  
• Selected from British Ability Scales (BAS) Used
  at age 3 at age 5
•  
• A verbal scale for children aged 2 years 6 months
  to 7 years 11 months. It assesses the spoken
  vocabulary of young children.
• The interviewer shows a booklet of coloured
  pictures of objects which the child is shown one
  at a time and asked to name.
• Measures expressive language ability, and
  successful performance depends on the childs
  previous development of a vocabulary of nouns.
  The items require the child to recall words from
  long-term memory rather than to recognise or
  understand the meaning of words or sentences.
•  

48
Practical problems in administering assessment

•  Anyone else present during Assessment?
• If so who mother, father, children, other?
• Effect on child's performance of this presence.
• What was childs energy level during Assessment?
• Language
• Vocab cannot be assessed if child speaks another
  language
• Child may be too shy, or too young at 3 to
  attempt the assessment
• Ca 2500 missing cases, ca 16 at age 3

49
Cognitive Scores findings

• Both BAS and Bracken showed marked patterning by
  socio economic variables, gender, and ( at age 3)
  ethnic group
• Although they cannot are not repeated, at all for
  Bracken, and beyond age 5 for BAS Naming
  Vocabulary, they are also highly predictive of
  later cognitive and school outcomes, known so far
  to age 7
• As age-appropriate instruments change it is a
  challenge to monitor a childs progress
  longitudinally
• Note also that more of the children who were
  too slow to do cognitive assessments at age 3
  start to be able to do so.
• Need to adjust for age at interview.

50
Child outcomes 2 Behavioural development
51
Behavioural Development

•  
• Since age 3, the surveys have included the
  Strengths and Difficulties Questionnaire
• In the main self-completion instrument, mother
  report
•  25 items form 5 subscales  
• 1. Emotion Symptoms Scale
•  2. Conduct problems 
• 3. Hyperactivity
• 4. Peer Problems
• 5. Pro-social Scale
• This standard instrument, author Goodman, who
  insists it is used in its entirety, is also put
  to teachers at age 7 and 11
•  
•  

52
Scoring the SDQ

• Total Difficulties sum of 4 problem
  sub-scales, excluding Pro-social
• 5 items per sub-scale
• Each item can score 0 Not true
• 1 Somewhat true
• 2Certainly true
• Maximum theoretical score 40
• gt 17 clinically
  relevant , severe problem
• 14-16 borderline
• Mean ca 9 sd ca 5

53
Behaviour findings

• More attention has been paid to Total
  Difficulties than Pro social or other individual
  subscale.
• Score is sometimes treated a continuous variable
  and sometimes as dichotomy
• It has strong patterns , not only with
  socio-economic indicators, but also parental
  relationships and parental health
• It is mildly predictive of the childs cognitive
  outcomes and is expected to be predictive of
  delinquency.

54
Child outcomes 3 Physical growth and
development
55
Anthropometry

• A direct measurement of body size is a record of
  growth, and also of great interest in the
  context of obesity
• Childs height and weight was successfully
  measured by generalist interviewers at age 3, 5,
  7 and is planned for age 11. Parent needs to be
  present . Yields BMI weight( kg)/height(m)2
• Measurements have also included waist
  circumference (age 5,7,11), fat mass (age 7 and
  11) and accelerometry (age 7) to record physical
  activity.

56
Anthropometry findings
57
Anthropometry findings

• Only a minority of children were found to be
  obese but, by age 7 the obesity rate of MCS
  children, 6, was double that recorded a decade
  earlier for 7 year olds in ALSPAC. Evidence of
  advancing obesity epidemic.
• IOTF ( international Obesity Task Force)
  definitions used for the obesity and overweight
  thresholds, which should facilitate international
  comparison.

58
Website
www.cls.ioe.ac.uk Follow CLS on Twitter
www.twitter.com/CLScohorts
59
References

• Calderwood, L. (2009). The relationship between
  survey quality and survey processes in
  longitudinal studies. CLS Working Paper
  (forthcoming)
• Calderwood., L. (2009). Keeping in touch with
  mobile families in the UK Millennium Cohort
  Study. Proceedings of the XXV International
  Methodology Symposium Statistics Canada
  (forthcoming)
• Calderwood, L, and Lessof, C. (2009) Enhancing
  Longitudinal Surveys by Linking to Administrative
  Data. In P.Lynn (ed.) Methodology of Longitudinal
  Surveys, London John Wiley Sons, Inc
• Couper, M.P. and Ofstedal, M.B. (2009). Keeping
  in Contact with Mobile Sample Members. In P.Lynn
  (ed.) Methodology of Longitudinal Surveys,
  London John Wiley Sons, Inc
• Dillman, D.A. (2009) Some Consequences of Survey
  Mode Changes in Longitudinal Surveys. In P.Lynn
  (ed.) Methodology of Longitudinal Surveys,
  London John Wiley Sons, Inc

60
References

• Groves, R. M. (1989) Survey Errors and Survey
  Costs. New York John Wiley
• Groves, R. M., and Heeringa, S., (2006)
  Responsive design for household surveys tools
  for actively controlling survey errors and costs.
  Journal of the Royal Statistical Society Series
  A Statistics in Society, 169, 439-457 Part 3.
• Jäckle, A. Dependent Interviewing A Framework
  and Application.
• to Current Research In P.Lynn (ed.) Methodology
  of Longitudinal Surveys, London John Wiley
  Sons, Inc
• Lepkowski, J.M. and Couper, M.P. (2002)
  Non-response in longitudinal household surveys.
  In R.M. Groves, D.Dillman, J.Eltinge, and
  R.J.A.Little (Eds.), Survey Nonresponse New York
  John Wiley Sons, Inc
• Lynn, P. (2001). A Quality Framework for
  Longitudinal Studies. (Draft) Last update
  26-9-2001.

61
References

• Lynn, P. and Clarke, P, (2002) Separating Refusal
  Bias from Non-contact Bias evidence from UK
  National Surveys Journal of the Royal Statistical
  Society Series D (The Statistician) edition 3,
  vol51, 319 - 333, 2002
• Lynn, P. (2008) The effect of interviewer
  continuity on measurement error in panel surveys
  Seminar on the Role of Interviewers in Ensuring
  Quality in Longitudinal Surveys, Royal Society
  Edinburgh Foundation
• Moore, J., Bates, N., Pascale, J. and Okon, A.
  (2009) Tackling Seam Bias Through Questionnaire
  Design. In P.Lynn (ed.) Methodology of
  Longitudinal Surveys, London John Wiley Sons,
  Inc
• Plewis, I, Ketende, S, Joshi, H., Hughes, G
  (2008) The Contribution of Residential Mobility
  to Sample Loss in a Birth Cohort Study Evidence
  from the First Two Waves of the UK Millennium
  Cohort Study Journal of Official Statistics 24.
  365-385

62
References

• Plewis, I. (2007) Non-response in a birth cohort
  study the case of the Millennium Cohort Study
  International Journal of Social Research
  Methodology 10. 325-334
• Statistics Canada (1998) Statistical Quality
  Guidelines (3rd Edition). Ottawa Statistics
  Canada
• Sturgis,P., Allum, N. and Brunton-Smith, I.
  (2009) Attitudes Over Time The Psychology of
  Panel Conditioning In P.Lynn (ed.) Methodology
  of Longitudinal Surveys, London John Wiley
  Sons, Inc