A view from overseas, National Longitudinal Survey of Children and Youth (Canada)

1
Key issues for longitudinal research
A view from overseas, National Longitudinal
Survey of Children and Youth (Canada)
Stephanie Lalonde, Statistics Canada
April 20, 2011
2
Todays presentation

• Survey Overview
• History of NLSCY
• Survey Design
• Survey Content
• Direct Assessments
• School Collection
• Research
• Lessons Learned

3
What is the NLSCY?

• A long-term study of Canadian children that
  follows their development and well-being from
  birth to early adulthood.
• Conducted by Statistics Canada and funded by
  Human Resources and Skills Development Canada
  (HRSDC).
• Development began in 1992 and data from the final
  collection were released in 2010.

4
Objectives of the NLSCY

• To determine the prevalence of risk and
  protective factors for children and youth.
• To understand how these factors, as well as life
  events, influence childrens development.
• To make this information available for developing
  policies and programs that help children and
  youth.
• To collect information about a wide variety of
  topics biological, social, economic.
• To collect information about the environment in
  which a child is growing up family, peers,
  school, community.

5
History of NLSCY

• Planning report May 1993
• First data collection 1994-95
• Data collection every two years
• Last data collection 2008-09

6
Origins of NLSCY

• May 1992, the Canadian government announced
  Brighter Futures initiative.
• What works for children Information
  Development Program is a component of initiative
• Mandate to develop the first multi-disciplinary
  national longitudinal and cross-sectional
  database on children.

7
Implications on survey design

• Need to provide longitudinal and cross-sectional
  information
• The ecological or holistic approach to the
  measurement of risk and outcomes
• The need to provide children and family
  information
• The need to gather and integrate community
  information
• The need to collect information on selected
  children from teachers

8
SURVEY DESIGN
9
NLSCY overview
0 original cohort 11
1 2 3 4 5 6 7 8
2 13
0 1
0 1
2 3
4 15
0 1
2 3
4 5
6 17
0 1
2 3
4 5
8 19
10 21
2 3
4 5
0 1
12 23
8 9
4 5
6 7
2 3
0 1
14 25
2 3
4 5
6 7
8 9
10 11
0 1
Cohort 3
Cohort 4
Cohort 5
Cohort 6
Cohort 1
Cohort 2
Cohort 7
Cohort 8
10
Longitudinal sample size and response rates
11
Cycle 8 sample and response rates
Number of sampled children and response rate, by age at Cycle 8        
Age as ofDecember 31, 2008 Sampled In-scope Respondents Cycle 8 responserate
Years Number Number Number
0-1 5,482 5,463 4,106 75.2
2-3 5,580 5,555 4,372 78.7
4-5 5,404 5,372 4,130 76.9
6-7 4,271 4,256 3,450 81.1
14-15 3,134 3,129 2,501 79.9
16-17 2,238 2,235 1,770 79.2
18-19 2,523 2,515 1,635 65.0
20-21 2,361 2,354 1,366 58.0
22-23 2,418 2,406 1,470 61.1
24-25 2,382 2,368 1,466 61.9
Total 35,793 35,653 26,266 73.7
12
Collection strategy

• Every 2 years - Mixed mode of collection
• Lasts 9 months (Sept June)
• Multi level data Household, Child, Parent (PMK),
  Schools
• Direct measures
• Community surveys

13
SURVEY CONTENT
14
Context
Public Programs
Family
Work
Resources
Community School
Social
Outcomes
Physical Health
Transitions Illness/Injury Accidents Divorce/sepa
ration Death of Family Member Spell of
Poverty Puberty School Entry Graduation First
Job Marriage First Child
Emotional
Social
Cognitive/ Learning
Language/ Communication
Space
Time
15
Person most knowledgeable (PMK)

• Person Most Knowledgeable (PMK)
• Person who answers question about the child
• Usually the biological mother, but not always

Age PMK biological mother PMK biological father
0-7 88.4 9.6
14-17 83.5 11
16
Questionnaires
PMK Child/Youth Teacher/Principal (cycles 1-5)
Household Adult Child Self-completes Youth Kindergarten Elementary
Direct Assessments
17
Survey Overview
18
Adult component

• List of subjects covered
• Education
• Labour force
• Income
• Health
• Family functioning
• Neighbourhood safety
• Social support
• Socio-demographic characteristics

19
Subjects Covered by Child Component

• Behaviour
• Sleep habits
• Motor/Social/Cognitive Development
• Relationships
• Parenting
• Custody
• Expectations (Aspirations)
• Socio-demographic characteristics

• Education
• Health
• Medical/Biological
• Mothers work after childs birth
• Temperament
• Literacy
• Communication
• Activities
• Developmental Milestones
• Childcare

20
Subjects Covered by Youth Component

• Criminal behaviour
• Relationships
• Sexual Health
• Activities
• Self-esteem
• Emotional Quotient
• Social Support
• Family formation and fertility

• Moving out of parental home
• Health
• Education
• Labour Force
• Career Aspiration
• Income
• Suicide
• Political engagement
• Self-assessment of abilities

21
Self-completed questionnaires

• Family and friends
• School
• Self-Esteem
• Emotional Intelligence
• Puberty
• Dating
• Smoking, Drinking and Drugs
• Activities
• Health
• Work
• Feelings and Behaviours (suicide)

22
DIRECT ASSESSMENTS
23
Direct Assessments
Age Assessment Measures
Grades 2 to 10 (usually 7 to 15 years) Math tests Achievement of basic academic skills
16-17 Problem solving exercise Reading comprehension, problem solving decision making
18-19 Literacy assessment Prose literacy Document literacy
20-21 Numeracy assessment Numeracy
24
Direct Assessments 4-5 year olds

• PPVT-R
• Measure of receptive vocabulary
• Who Am I?
• Measure of level of development
• Number Knowledge
• Measures understanding of numbers

25
Choosing direct assessments

• Increase focus on early childhood development
• Review of framework
• Selection of direct measures based on literature
  review and review of other surveys
• 16 measures selected for more detailed review
• 11 measures informally tested
• 5 measures field tested

26
Choosing direct assessment - criteria

• Available in English and French (or easily
  adaptable)
• Appropriate for administration in the childs
  home
• Easy to administer by lay interviewers
• Easy to score

27
Assessments tested

• Who Am I?
• Number Knowledge
• Weschler Preschool and Primary Scales of
  Intelligence (WPPSI-R)
• Early Screening Inventory
• Ravens Coloured Progressive Matrices

28
Field test of assessments Who Am I?

• Measures developmental level and acquired
  knowledge and skills
• Developed by Dr. Molly de Lemos at the Australian
  Council of Educational Research (ACER)
• Can be used for children aged 3 to 7
• General development assessed using a copying
  shapes task and drawing picture of onself
• Acquired knowledge and skills assessed through
  writing symbols such as numbers, letters, words
  and sentences
• Field test found that the assessment was
  relatively easy to administer and enjoyed by
  children
• The assessment is inexpensive
• Dropped the drawing task to save time
• Added to the NLSCY in Cycle 4

29
Field test of assessments Number Knowledge

• Assesses childrens understanding of quantity and
  the system of whole numbers
• Developed by Dr. Robbie Case and colleagues at
  the Institute of Child Study - University of
  Toronto
• Four developmental levels (pre-dimensional,
  uni-dimensional, bi-dimensional, and integrated
  bi-dimensional)
• Levels are attained at approximately 4, 6, 8 and
  10 years of age.
• Only first three levels used in test
• The assessment included counting to 10, concepts
  of quantity, number line, simple additions and
  subtractions and some problem solving.
• Added to Cycle 4 of the NLSCY with some
  modifications.

30
Field test of assessments Block Design

• Sub-test of the Wechsler Preschool and Primary
  Scales of Intelligence-Revised (WPPSI-R)
• WPPSI-R can be used as an intelligence test.
  Block design is one of the performance sub-tests
  and examines logical reasoning.
• The child must design with blocks a shape copying
  either one the interviewer constructs or from a
  booklet.
• The task proved too difficult for the
  interviewers to administer consistently so was
  not added to the NLSCY.

31
Field test of assessments Early Screening
Inventory

• A brief developmental screening instrument that
  is individually administered to children from 3
  to 6 years of age.
• Developed by Dr. Samuel Meisels and colleagues at
  the University of Michigan.
• Designed to identify children who may need
  special educational services to participate
  successfully in school.
• Provides an overview of the childs development
  in three major areas Visual-Motor/Adaptive (fine
  motor skills, eye-hand co-ordination and
  short-term memory skills), Language (not used in
  test) and Cognition and Gross Motor.
• Some difficulties in administration but
  recommended for inclusion in NLSCY
• Final decision was not to use the ESI

32
Field test of assessments - Ravens Coloured
Progressive Matrices

• Assesses childs capacity for analogical
  reasoning as one aspect of intelligence.
• Child must select the missing element to complete
  a pattern
• Test is non-verbal
• Well liked by parents, children and interviewers
• Not added to the NLSCY

33
SCHOOL COLLECTION
34
School collection

• Changes cycle to cycle
• Dropped from Cycle 6 (2004) on
• Requires signed parental permission
• Requires co-operation from all ten provinces
• Relatively low response rates
• Logistically complex

35
Survey content teachers

• Child/students education
• Child/students behaviour and attendance
• Involvement of parent and guardian
• Teaching practices
• Teachers perceptions of the school
• Personal information

36
Survey content - Principals

• Students in the school
• Involvement of parent(s) and guardian(s)
• Characteristics of school
• Principals perceptions of the school
• Personal information

37
COMMUNITY SURVEYS
38
Community surveys

• Part of Understanding the Early Years initiative
• Designed to give communities information to
  enhance community resources and services
• Worked with community groups
• Includes a mapping project to map community
  resources and services
• Goal is to allow each community to use
  information to improve early childhood development

39
Collection for Communities
Vocabulary Test PPVT
Community Resource Use
Who Am I
Child Questionnaire
Household Contact
Parent Questionnaire
Number Knowledge
Phone interview
EDI Teacher completed
Paper Questionnaire
Questionnaire CAI
Tests
Collected in the school
40
SELECTED FINDINGS FROM NLSCY
41
Growing Up in Canada

• Hyperactivity biggest risk factor in slowing math
  skills
• Aggressive behaviour tended to decrease as
  children grew up
• Positive parenting can make a difference in
  disadvantaged families

42
Vulnerable children

• Edited by J. Douglas Willms
• Development of Vulnerability Index
• Childhood vulnerability only weakly linked to
  income
• Effects of good parenting outweigh effects of
  income
• Vulnerability varies amongst communities

43
Parenting style and children's aggressive
behaviour

• Change in parenting environment predicts change
  in child's behaviour
• Aggressive behaviour linked to parenting style,
  regardless of sex or income

44
Other examples

• Relationship between youth depression and changes
  in relations with parents and peers.
• Description of Child care in Canada
• Description of Readiness to Learn of Five Year
  Olds
• Canadian Nine Year Olds at School
• Successful Transitions conference

45
LESSONS LEARNED
46
Challenges

• Breadth of content
• Large number of age groups
• Mixed requirements
• Longitudinal
• Cross-sectional
• Changes from cycle to cycle

47
Example of changes

• Age age at time of collection vs. reference age
• Most questions asked based on reference age but
  norms based on actual age
• Change of PMK over time

48
Lessons Learned

• Keep it simple wherever possible
• Have adequate resources from the start to design
  the best survey vehicle
• Think longitudinally
• Re-evaluate decisions made at previous cycles to
  determine if they are still appropriate
• Do research and analysis to develop new methods
• Ensure that there is good documentation
• That explain the why and allow others to
  replicate the work
• Have clear long-term objectives
• NLSCY is trying to please everyone focus on a
  few things and do them well

49
Contact Information

• Stephanie Lalonde
• stephanie.lalonde_at_statcan.gc.ca
• General NLSCY inquiries ssd_at_statcan.gc.ca