Developing a Severity-of-Disability Scale and Modeling Early Reading and Math Performance in a Longitudinal Study of Preschoolers with Disabilities - PowerPoint PPT Presentation

View by Category
About This Presentation
Title:

Developing a Severity-of-Disability Scale and Modeling Early Reading and Math Performance in a Longitudinal Study of Preschoolers with Disabilities

Description:

Developing a Severity-of-Disability Scale and Modeling Early Reading and Math Performance in a Longitudinal Study of Preschoolers with Disabilities – PowerPoint PPT presentation

Number of Views:259
Avg rating:3.0/5.0

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Developing a Severity-of-Disability Scale and Modeling Early Reading and Math Performance in a Longitudinal Study of Preschoolers with Disabilities


1
Developing a Severity-of-Disability Scale and
Modeling Early Reading and Math Performance in a
Longitudinal Study of Preschoolers with
Disabilities
  • Elaine Carlson
  • Tamara Daley
  • Frank Jenkins
  • Westat
  • The Pre-Elementary Education Longitudinal Study
    is funded by the U.S. Department of Education.

2
PEELS design
  • 6-year national longitudinal study of
    preschoolers with disabilities, currently in its
    fourth year.
  • Designed to describe
  • the children,
  • the services they receive,
  • their transitions from early intervention to
    preschool and preschool to elementary school, and
  • their early academic and functional skills.
  • Sample includes 3,104 children who were ages 3-5
    and were receiving special education services in
    2003-04.
  • Produces statistical estimates that generalize to
    the national population of children with
    disabilities ages 3-5.

3
PEELS design, contd
  • Data collection
  • a one-on-one assessment of early academic skills,
  • a parent telephone interview,
  • mail questionnaires to childrens teachers and
    school-, district-, and state-level
    administrators, and
  • indirect assessments of students social skills,
    problem behaviors, academic skills, and motor
    skills

4
Sample design
  • Nationally representative sample of 223 LEAs (208
    in Wave 1, plus 15 added in Wave 2)
  • LEAs stratified by
  • enrollment size
  • geographic region
  • wealth

5
LEA participation
  • Main sample
  • 709 released for recruiting
  • 245 agreed in 2001
  • 189 participated in 2003-04 (Wave 1)
  • Nonresponse sample
  • 32 of 464 non-recruited LEAs selected
  • 19 participated in 2003-04 (Wave 1)
  • Supplemental sample to add 1 previously
    nonparticipating SEA
  • 15 of 24 participated in 2004-05 (Wave 2)
  • Total Sample 223 LEAs

6
Family participation
  • Wave 1 main and nonresponse samples
  • (ages 3-4 in 2003-04)
  • 52,871 on sampling frame
  • 5,330 selected
  • 4,072 eligibility determined
  • 2,906 agreed (81 of those eligible)

7
Family participation
  • Wave 2 supplemental sample
  • (ages 4-6 in 2004-05)
  • 7,727 on sampling frame
  • 542 selected
  • 433 eligibility determined
  • 198 agreed (63 of those eligible)
  • Total sample 3,104 families

8
PEELS schedule
9
Unweighted response rates, by wave
Wave
1 2 3
SEA questionnaire 100 - -
LEA questionnaire 84 - -
Principal/program director questionnaire 73 77 56
Teacher questionnaire 79 84 84
Parent interview 96 93 88
Child assessment 96 94 93
English direct assessment 84 87 88
Alternate assessment 11 7 5
-Not applicable QED data were used to
supplement these data, bringing percentage of
children with some school data in waves 1, 2, 3
to 94, 95, 94, respectively
10
Demographic characteristics
Gender
Male 70.7
Female 29.3
Race/ethnicity
American Indian/Alaska Native 3.3
Asian 3.2
Black or African American 14.2
White 84.6
Pacific Islander 0.8
Hispanic, any race 21.0
NOTE Demographic and household characteristics
are for the unweighted sample. SOURCE U.S.
Department of Education, National Center for
Special Education Research, Pre-Elementary
Education Longitudinal Study (PEELS), Parent
Interview (Wave 1 for main and nonresponse
samples, Wave 2 for supplemental sample),
previously unpublished tabulation.
11
Household characteristics
Mothers education
ltH.S diploma 18.9
H.S. diploma or GED 31.0
Some college 29.4
4-year degree or higher 21.0
Household Income
20,000 or less 26.3
20,000-40,000 28.3
gt40,000 45.3
SOURCE U.S. Department of Education, National
Center for Special Education Research,
Pre-Elementary Education Longitudinal Study
(PEELS), Parent Interview (Wave 1 for main and
nonresponse samples, Wave 2 for supplemental
sample), previously unpublished tabulation.
12
Primary disabilities
Speech or language impairment 49.8
Developmental delay 26.4
Autism 6.7
Mental retardation 4.0
Learning disability 2.5
Other health impairment 2.4
Orthopedic impairment 1.6
Emotional disturbance 1.1
Low incidence disability 5.5
NOTE Low incidence includes visual impairment,
hearing impairment, deaf-blindness, traumatic
brain injury, multiple disabilities. SOURCE U.S.
Department of Education, National Center for
Special Education Research, Pre-Elementary
Education Longitudinal Study (PEELS), Early
Childhood and Kindergarten Teacher Questionnaires
(Wave 1 for main and nonresponse samples, Wave 2
for supplemental sample), previously unpublished
tabulation.
13
Developing a Severity-of-Disability Scale

14
Severity
  • Under IDEA, children classified according to one
    of 13 disability categories
  • Children in each category have a range of
    abilities
  • Heterogeneity within categories often overlooked
  • Categorical labels especially problematic for
    preschoolers

15
Approaches to measurement
  • Pediatric health conditions (PEDI, WeeFim)
  • Self-care, mobility, and social cognition
  • Require a certain threshold of impairment
  • Single disability conditions in children
  • Hearing, vision, language, depression
  • ICIDH (WHO, 1980)
  • Published as a research tool
  • Not widely adopted for use

16
Approaches to measurement
  • ABILITIES Index (Simeonsson Bailey, 1991)
  • Severity in 9 domains
  • Audition
  • Behavior and social skills
  • Intellectual function
  • Limbs
  • Intentional communication
  • Tonicity
  • Integrity of physical health
  • Eyes
  • Structural status

17
Approaches to measurement (contd)
  • Since its original publication, 6 new domains
    added (Simeonsson, 2006)
  • Regulation of attention,
  • Regulation of activity level, and
  • Regulation feeling/emotions,
  • Academic skills,
  • Motivation, and
  • Impulse control

18
ABILITIES index
  • In Special Education Expenditures Project (SEEP)
    (Chambers et al., 2004)
  • Federal disability category able to explain only
    10 of the variation in total expenditures
  • In contrast, the ABILITIES Index alone was able
    to account for 40 of the variation in total
    educational expenditures for special education
    students
  • Including the federal disability categories
    increased the variance accounted for by 2

19
Question of current study
  • To what extent do functional markers of severity
    of childhood disability predict measurement of
    child outcomes?

20
Overall method
  • Identify items from parent interview similar to
    ABILITIES domains
  • Create composite domains where necessary
  • Examine distribution of severity within the 15
    domains
  • Examine regressions of the 15 domains on multiple
    outcomes
  • Create short version of index using 6 domains
  • Compare 6 domain and 15 domain versions of index
    to outcomes to select better version
  • Compare 6 domain version of index with other
    indicators of severity

21
Outcome measures
Wave 1 data
  • Cognitive
  • PPVT-III
  • Woodcock-Johnson III Letter-Word Identification
  • Woodcock-Johnson III Applied Problems
  • Social/Behavioral
  • PKBS Social Competence
  • PKBS Problem Behaviors
  • Alternate Assessment
  • ABAS Conceptual Domain
  • ABAS Practical Domain
  • ABAS Social Domain

22
Creating the severity measure
  • Used parent report from Wave 1 CATI
  • Created 4-point scale for each domain
    (normal/typical, mild, moderate, severe)
  • 11 domains already with 4 point scales
  • 4 levels of severity
  • 4 domains with 3 point scales
  • 1 ? 1, 2 ? 2, 3 ? 4 to preserve range

23
Creating the severity measure
  • Some domains matched to a single question
  • Use of arms (within limbs domain) How well does
    child use his/her arms and hands for things
    like throwing, lifting, or carrying?
  • Motivation Some children try to finish things,
    even if it takes a long time. How much does this
    sound like child
  • For domains with multiple questions, worked with
    ABILITIES Index author to best reflect original
    intention of domain

24
Example Combining Items to Create Communication
with Others
When CHILD talks to people he/she doesnt
know well, is he/she 1 Very easy to
understand 2 Fairly easy to understand 3
Somewhat hard to understand 4 Very hard to
understand 5 DOES NOT OR WILL NOT TALK AT ALL
Compared with other children about the same age,
how well does CHILD make his/her needs known
to you and others? Communication can be any form,
for example crying, pointing or talking. Would
you say he/she 1 Communicates just as well as
other children 2 Has a little trouble
communicating 3 Has a lot of trouble
communicating 4 Does not communicate at all?
1 Communicates just as well as other children
and very easy to understand 2 Some difficulties
communicating or being understood 3 Moderate
difficulties communicating or being understood 4
Does not communicate at all or very hard to
understand
25
ResultsPopulation weighted percentages (n
2,986)
  • Audition, vision, use of arms, legs, regulation
    of emotions
  • 80-95 in normal/typical category
  • Inappropriate or unusual behavior, overall
    health, social skills, use of hands,
    understanding
  • 50-60 in normal/typical, 20-30 mild

26
ResultsPopulation weighted percentages (n
2,986)
  • Motivation, regulation of attention, regulation
    of activity level,
  • 20-30 in normal/typical, 33-42 mild, 16-30
    severe
  • Communicating with others
  • 29 in normal/typical, 9 mild, 45 moderate, 17
    severe
  • Intellectual function
  • 10 in normal/typical, 43 mild, 32 moderate,
    14 severe

27
Regressions on outcome variables
  • All predictors entered simultaneously examined
    beta weights and bivariate correlations between
    predictors and each outcome
  • Significant predictors of at least three
    outcomescognition, communicating with others,
    understanding, overall health, and regulation of
    activity level
  • Significant predictor of two outcomesregulation
    of attention
  • The remaining severity domains generated a
    mixture of significant and nonsignificant
    associations
  • No significant loading on any outcome variables
    use of hands and arms and use of legs

28
Creation of indices
  • Index A, 15 items Sum of all domains
  • Index B, 6 items cognition, communicating with
    others, understanding, overall health, regulation
    of activity, regulation of attention
  • No significant differences in correlations
    between Index A and outcomes and Index B and
    outcomes

29
Index A and B correlations with outcomes
Index A 15 var Index B 6 var
PPVT -.32 -.36
Letter-Word Identification -.22 -.26
Applied Problems -.40 -.45
PKBS Social Skills Composite -.47 -.43
PKBS Problem Behavior Composite .35 -.35
ABAS Conceptual Domain -.53 -.46
ABAS Practical Domain -.53 -.43
ABAS Social Domain -.40 -.35
The associated p-values are less than .0001 for
all coefficients.
30
Final severity measure items
  • Cognition
  • Communicating with others
  • Understanding
  • Overall health
  • Regulation of activity
  • Regulation of attention

31
Distribution of final severity measure
32
Validation correlations between Index B and
other indicators
  • Wave 1 Parent report Age at which children
    began receiving special education or therapy
    servicer (2,802) -.22, p lt .0001
  • Wave 1 Teacher report Amount of modification
    needed to curriculum materialsr (248) .42, p
    lt .0001
  • Wave 1 Teacher report Number of services the
    child receives in school r (2,014) .37, p lt
    .0001

33
Validation comparison of mean scores on severity
measure
  • From teacher/parent declassification measure,
    Wave 1 Children remaining in special educationM
    13.2Children no longer receiving special
    education M 10.7 , p lt .0001
  • From assessment measures, Wave 1Children taking
    the alternate assessment M 16.3Children
    completing the direct assessmentM 12.4, p lt
    .0001

34
Summary
  • To what extent do functional markers of severity
    of childhood disability predict measurement of
    child outcomes?
  • Of the 15 domains examined, most were significant
    predictors of at least two outcomes
  • An index of only six variables was as effective
    as the longer version
  • Severity was significantly correlated with
    intervention variables
  • Severity differentiated children in two groups

35
Modeling Early Reading and Math Performance

36
Hierarchical analysis
  • PEELS has children in naturally-occurring
    hierarchies
  • time points within children
  • children within districts
  • districts

37
  • What we want to know
  • What factors relate to childrens cognitive
    growth over 3 years?
  • What we have
  • observations of PEELS children over 3 years.
  • Yearly information about childs SES, health,
    severity, and services received
  • Yearly measures of 3 academic outcomes
  • Adapted Peabody Picture Vocabulary Test (PPVT)
  • Woodcock-Johnson III Letter-Word Identification
  • Woodcock-Johnson III Applied Problems

38
Problem How to make sense out of longitudinal
data
  • Hierarchical data is clustered i.e., repeated
    measures are not independent observations.
    Standard regression assumes independent
    observations.
  • Ordinary repeated measures analyses do not allow
    for missing time points or clustering.
  • Repeated cross-sectional analyses ignore the
    growth of individual children. Mean growth is not
    the same as growth of individuals.

39
Solution Hierarchical linear modeling
  • Data is modeled at 3 levels of hierarchy at the
    same time.
  • Most of the clustering in the sample is accounted
    for, leading to correct statistical tests.
  • Focus is on individual growth profiles. Modeling
    seeks to explain differences in growth between
    children.

40
Hierarchical structure of data
41
HLM model
  • Level 1 Repeated observations within child
  • is the outcome for individual i measured at
    wave t .
  • is the age of the child at time at time t.
  • is the growth intercept average achievement for
    the individual
  • is the growth curve slope How much the outcome
    changes over years.
  • is a random error term.

42
HLM modelLevel 2 Children nested within
districts.
is an intercept for child ji,
is the individual growth curve for child ji
are child factors that predict growth.
to
is the deviation from average achievement for
each child.
is the deviation from average growth for each
child.
43
HLM modelLevel 3 Districts
  • Where,
  • is the achievement status of district j ,
  • is the grand mean of achievement, and
  • is the district effect on achievement.

44
Hypothetical growth curve HLM
  • Individual slopes estimated for high low
    severity children


  • lt- Low
    severity

45
  • Individual slopes estimated for high low
    severity children


  • lt- Low
    severity


  • lt- High severity

46
Four Sets of Predictors
  • Predictors are added in four sets of similar
    variables. These sets include
  • SES e.g., mothers education, SES scale
  • Severity e.g., severity scale, age services
  • started
  • Health e.g., childs general health, health
    scale
  • Services e.g., time in a regular classroom,
    parent
  • involvement scale

47
Predictors of PPVT growth
Predictor Factor Effect Prob
Intercept 229.55 .00
Slope 54.24 .00
Years in special education Severity -1.37 .01
Severity of disability scale Severity -.78 .00
Ease of transition Service -1.85 .01
Time in a regular classroom Service .03 .03
Accounted for 22 of PPVT growth
Note Controlled for age and cohort
48
Predictors of Letter-Word growth
Predictor Factor Effect Prob
Intercept 318.18 .00
Slope 37.76 .00
Years in high poverty school SES 2.10 .00
Mother had some education after HS SES 2.92 .00
Years in special education Severity -1.62 .00
Age services started Severity .09 .02
Severity of disability scale Severity -.69 .00
Ease of transition Service -2.61 .00
Time in regular classroom Service .08 .00
Parent involvement scale Service 1.49 .03
Accounted for 57 of Letter-Word growth
Note Controlled for age and cohort
49
Predictors of Applied Problems growth
Predictor Factor Effect Prob
Intercept 393.26 .00
Slope 24.00 .00
Household Income SES -1.26 .03
Child is Hispanic SES 1.53 .03
Mother had some education after HS SES 1.42 .02
Parent SES scale SES 1.38 .01
Problems with health Health 2.50 .00
Health scale Health 1.18 .05
Childs general health Health 2.49 .00
50
Predictors of Applied Problems growth (continued)
Predictor Factor Effect Prob
Years in special education Severity -1.02 .00
Age services started Severity .09 .00
Severity of disability scale Severity -.24 .02
Time in regular classroom Service .03 .00
Parent involvement scale Service .75 .06
Accounted for 12 of Applied Problems growth
Note Controlled for age and cohort
51
Summary
  • Scores increased from 1½ to 2 standard deviations
    over 3 years
  • SES, severity, health, service predictors
    accounted for 12 to 57 of growth

Percent of growth accounted for Percent of growth accounted for

PPVT 22
Letter-Word 57
Applied Problems 12
52
Summary (cont.)
  • Service-related predictors of growth
  • Time in a regular classroom
  • Positively related to growth for all outcomes
  • Parent involvement
  • Positively related for 2 of 3 outcomes
  • (Letter-Word Applied Problems)
  • Ease of transition
  • Positively related for 2 of 3 outcomes
  • (PPVT Letter-Word)

53
Summary (cont.)
  • Other predictors of growth
  • Years child was in special education
  • Predicted lower growth for all outcomes
  • Parents rating of severity
  • Predicted lower growth for all outcomes
  • Mothers education
  • Positively related to growth for 2 of 3 outcomes
  • (Letter-Word Applied Problems)

54
WEBSITE WWW.PEELS.ORG
About PowerShow.com