Title: Two Year Outcomes of Autism Early Intervention in BC
1Two Year Outcomes of Autism Early Intervention in
BC
- January, 2005
- Pat Mirenda, Ph.D., Project Director
- Veronica Smith, Anat Zaidman-Zait, Paula
Kavanagh, and Karen Bopp, Research Assistants - Bruno Zumbo, Ph.D., Statistical Consultant
- The University of British Columbia
2What is Early Intensive Behavioral Intervention
in BC?
- Early intensive behavioral intervention (EIBI)
was implemented in May, 2001 - There are three EIBI sites
- Delta Association for Child Development
- Queen Alexandra Centre for Children (Victoria)
- Thompson Okanagan Autism Project (TOAP) (child
development centres in Penticton, Kelowna,
Vernon, Kamloops)
3What Services Do EIBI Programs Provide?
- EIBI consists of
- Year-round, at least 20 hours/week of 11
intervention - Highly structured teaching, based on applied
behavior analysis (ABA) principles - Integrated therapies (SLP, OT, etc.)
- Component of integration with typical peers
- Positive behavioural support for problem
behaviour - Family involvement in training and intervention
- Each EIBI sites is funded to provide services to
a minimum of 25 children and families at any time
4What is Interim Early Intensive Intervention in
BC?
- The Interim Early Intensive Intervention (IEII)
funding model (also known as Individualized
Funding) was implemented in June, 2002 for all BC
children with autism spectrum disorders, ages 0-6
- Families receive up to 20,000 per year to
purchase services for their children from a list
of Qualified Service Providers - Behaviour consultants and interventionists
- Speech-language pathologists
- Occupational and physiotherapists
5The Evaluation Project
- The evaluation project was initiated at the very
beginning of the EIBI and IEII programs. - So, the results only apply to the children and
families who were initially involved in these
programs, which have developed considerably since
the evaluation was completed - Results may be different if the evaluation was
conducted today
6Overview of the Evaluation
- The evaluation project was approved by UBCs
Behavioral Ethics Review Board - All EIBI and IEII families were invited but not
required to participate in the project - 50 of EIBI children/families who gave consent in
each site were selected at random for the
evaluation. The total number of EIBI families in
the project was 39. - All IEII families who consented to participate
and whose children were eligible to receive at
least one year of IEII funding were also
included. The total number of EIBI families in
the project was 31.
7Evaluators
- Evaluators assessed each family and child at the
EIBI site office or at home - Multidisciplinary team of evaluators
- Psychologist (same person each time)
- Speech-language pathologist (SLP) (same person
each time) - Family interviewer (different evaluators each
time) - Psychologist, SLP each spent several hours with
each child, often on separate days - Family interviewers spent several hours with
family member(s) - Children and families were assessed before
intervention started (T1), 6 months later (T2), 1
yr later (T3), and 2 yrs later (T4)
8Participants by Region
BC Region EIBI Participants IEII Participants
Kootenays 0 1
Okanagan 9 0
Thompson/Caribou 4 0
Upper Fraser 0 0
South Fraser 13 6
Simon Fraser 0 6
Coastal North Shore 0 0
Vancouver/Richmond 0 12
North 0 3
Upper Island 2 2
Victoria 11 1
TOTAL 39 31
9Who Were the Children?
Variable EIBI (N 39) IEII (N 31)
Age at T1 46 mo (range 21-68 mo) 55 mo (range 28-72 mo)
Gender 33 boys (85), 6 girls 25 boys (81), 6 girls
T1 diagnosis 29 autism (74), 10 PDDNOS 27 autism (87), 4 PDDNOS
Ethnicity 74 Euro-Canadian, 21 Asian-Can, 5 other 26 Euro-Canadian, 42 Asian-Can, 32 other
Primary language at home 97 English, 3 other 13 spoke more than 1 language at home 58 English, 16 Chinese, 26 other 61 spoke more than 1 language at home
10Children (Cont)
- Before intervention started
- 61 of EIBI children and 97 of IEII children
were in preschool or day care - Most children both groups had received small
amounts of intervention from infant development
consultants, speech-language pathologists,
occupational therapists, or other professionals - Four EIBI and three IEII children had received
ABA therapy three received more than 900 hours,
while the others received fewer hours
11Who Were the Families?
Variable EIBI IEII
Marital status 29 two-parent families (74), little change over 2 yrs 26 two-parent families (84), little change over 2 yrs
Average family education Professional diploma/some University Some University
Average family occupation Semi-skilled worker/skilled manual worker Semi-skilled worker/skilled manual worker
12Treatment Provided
? Pat Mirenda, Ph.D., 2005
13Treatment Services Received
- As part of their intervention, all EIBI and
almost all IEII children received - At least some structured ABA teaching or behavior
support services - Either direct or consultative speech-language and
occupational therapy - Some EIBI and IEII children received various
recreation therapies - All but two children attended day care,
preschool, or school as well
14Average Hours of Treatment Over 2 Years
15Was There a Relationship Between Child Progress
and Hours of Treatment?
- Using statistical tests, we found no significant
relationship between child progress on any
measure and either - Total hours of treatment without school or
- Total hours of treatment including school
- This means that factors other than hours of
treatment were better predictors of child outcomes
16Outcomes over 2 Years (T1-T4)
T1 average age 42 T4 average age 61
? Pat Mirenda, Ph.D., 2005
17Understanding the Results.
- Sometimes, test scores can change just because a
child has good day or a bad day when he/she
is tested -- but the difference is not really
meaningful or important -- it could have occurred
by chance - We used a number of statistical tests to
determine whether children made more progress
over 2 years they would have made just by chance - We use the term significant in this
presentation to indicate that there is a 5 or
less likelihood that a change occurred by chance,
and code this in green - We use the term not significant to indicate
that the result could have occurred by chance,
and code this in red
18Understanding the Results.
- First, growth curve analyses were done
- To determine whether the rate of change over 2
yrs of treatment was greater than expected by
chance (i.e., significantly different) and - To obtain estimates of the childrens rate of
progress on various measures prior to
intervention - Then, the data were analyzed to examine specific
predictors of progress over 2 yrs
19Were There Differences In Progress Between the
EIBI and IEII Groups?
- There no significant differences between the two
service delivery groups on any measure - On average, the EIBI and IEII children made
similar progress over 2 years - So, we combined the two groups into one group of
70 children to answer the rest of the questions
20Were There Differences in Progress Between
Children with Autism and PDD-NOS?
- There were no significant differences between the
two diagnostic groups on any measure - Children with autism and children with PDD-NOS
(pervasive developmental disorder-not otherwise
specified) made similar progress - So, we analyzed the results across both children
with autism and those with PDD-NOS to answer the
rest of the questions
21Was More Progress Made by Children Who Were
Younger at T1?
- There were no significant differences
between children who started treatment younger
and those who started when they were older on any
measure - Children who started when they were older made as
much progress as children who started when they
were younger - But remember At T1, EIBI children were 46 months
old, on average (range 21-68 mo) and IEII
children were 55 months old, on average (range
28-72 mo), at T1 -- that is, no children were
older than age 6 at T1
22Were There Differences for Children Who Were
Testable vs. Untestable?
- Untestability was determined on a test-by-test
basis, using different criteria for each measure - Children were considered to be untestable if they
met the criterion established for each test at
both T1 and T2 - Significant differences were found between
testable and untestable children on several
measures - In other words, children who could not achieve at
least a minimum test score on these tests both
before they began treatment as well as 6 months
later made less progress over 2 years, compared
to children who could be tested successfully - So, the results for testable and untestable
children will be presented separately in the
slides that follow
23Reading the Graphs
- We used growth curve analysis to find out whether
the childrens scores on various tests changed
significantly over the 2 yrs - We charted the results using graphs that look
like this
24What Do the Graphs Mean?
- Each grey line is one child
- These are the childs test score
- These are when the child was tested
- 0 before intervention (T1)
- 6 months later (T2)
- 12 months later (T3)
- 24 months later (T4)
25What Do the Graphs Mean?
- The red line is the average test score across all
70 children - So, in this graph, the average score increased
from 60 to just over 80 between T1 and 2 years
later
26Test Results
- The next slides present the test results over 2
years for all 70 children
27Autism Severity and Symptoms CARS and ABC
- CARS Childhood Autism Rating Scale
- Administered by psychologist
- Based on observation of child and family
interview - Provides descriptions of symptoms and severity
rating - ABC Autism Behavior Checklist
- Administered by family interviewer
- Based on parent report of behaviors
28CARS
- On the CARS, low scores less severe autism
- Average score at baseline 35.9
- Average score at 2 yrs 34.2
- This is not a meaningful decrease in autism
severity, as observed by the psychologist
29ABC
- On the ABC, low scores fewer autistic behaviors
- Average score at baseline 61
- Average score at 2 yrs 41
- This is a significant decrease in autistic
behaviors, as reported by parents
30Temperament and Atypical Behavior Scale (TABS)
- Administered by family interviewer, based on
parent report - Provides subscale scores related to four clusters
of atypical behaviors - Detached behaviors related to being in his/her
own world - Hypersensitive easily frustrated, tantrums,
aggressive, impulsive - Underreactive socially unresponsive
- Dysregulated sleep problems, difficult to
comfort
31TABS Standard Scores
- On the TABS, high scores fewer unusual
behaviors - On average, the childrens lowest scores were on
the detached subscale and remained atypical
after 2 years of intervention - Scores for the other subscales improved somewhat,
but the children were still in the TABS at risk
category
32IQ Testable Children
- These are standard scores, not raw scores
- Typical childrens standard scores tend to be
stable over time - Testable children
- Started with a mean score of 60
- Ended with a mean score of 83.7 at T4 (23.7 pts)
- This is significant
33IQ Untestable Children
- Untestable children
- Started with a mean score of 45.8
- Ended with a mean score of 49.7 at T4 (3.9 pts)
- This is significant, even though it is a small
increase
34One More Lesson!
- To read the next set of graphs, you need to know
what this dotted red line means - This is a statistical estimate of what the
average childs test score would have been
without intervention - In general, the bigger the gap between the solid
and the dotted red lines, the greater the impact
of the intervention
35Adaptive Behavior VABS
- VABS Vineland Adaptive Behavior Scales
- Administered by psychologist
- Based on parent interview
- Provides total score and subscale scores in four
areas - Communication
- Daily living skills
- Socialization
- Motor skills
36VABS Communication
- Typical children (ages 0-8) gain approx. 1.15 raw
score points per month (ppm) on this subscale - EIBI and IEII children gained, on average
- .7 ppm prior to intervention
- 1.07 ppm during intervention
- This is a significant gain of 7 months more than
would have occurred without intervention - 31 of the change was due to treatment 69 was
due to maturation
37VABS Socialization
- Typical children (ages 0-8) gain approximately
1.0 raw score ppm on this subscale - EIBI and IEII children gained, on average
- .72 ppm prior to intervention
- .74 ppm during intervention
- This is not a significant gain compared to what
would have occurred without intervention - 4 of the change was due to treatment 96 was
due to maturation
38VABS Motor Skills
- Typical children (ages 0-6) gain approximately
1.0 raw score ppm - EIBI and IEII children gained, on average
- .87 ppm prior to intervention
- .74 ppm during intervention
- This is not a significant gain compared to what
would have occurred without intervention in
fact, it is a slight decrease in gain
39VABS Daily Living Skills
- Typical children (ages 0-8) gain approximately
1.35 raw score ppm - EIBI and IEII children gained, on average
- .7 ppm prior to intervention
- 1.15 ppm during intervention
- This is a significant gain of 6 months more than
would have occurred without intervention - 46 of the change was due to treatment 54 was
due to maturation
40Preschool Lifestyle Inventory
- The PLI measures the number of different leisure
and personal management activities done by the
child across nine areas in the past 30 days, by
parent report - Also measures the amount of support required by
the child in activities (1 no support
independent, 4 substantial support), by parent
report
41Results Leisure Activities
- Children were engaged in significantly more
- Play activities (e.g., puzzles, drawing, lego,
play-dough, doll play, looking at books, board
games) - Exercise activities (e.g., riding a tricycle,
going on swings/slides, skating, throwing a ball)
- Media activities (e.g., using a computer,
watching TV/videos - Community activities (e.g., going to the park,
movies, swimming, church, parties) - Other leisure activities identified by the parent
42Results Daily Living Skills
- Children were also engaged in significantly more
- Food-related activities (e.g., using spoon/fork,
ordering food in restaurant, making a snack) - Space and belongings activities (e.g., putting
away toys, setting the table, pet care) - Personal hygiene and community activities (e.g.,
toileting, dressing, using a schedule)
43Results Support
- Over 2 years, children required significantly
less support for personal hygiene and community
activities (e.g., toileting, dressing, washing
hands, brushing teeth)
44Social Network Analysis Form (SNAF)
- Measures the number of socially important people
in the childs life within the past 30 days, by
parent report - Family members
- Preschool/daycare/school contacts
- Friends
- Neighbours
- Paid staff
- Other
45Social Network Results
- Significant increases were found in the number of
childrens - Preschool/school peers
- Friends
- Paid staff
- No significant differences were found in other
areas (e.g., number of family members,
neighbours, etc.)
46Receptive Language Tests
- PLS-AC Preschool Language Scale-3
- Administered by speech-language pathologist (SLP)
- Provides receptive language subscale score
(global language comprehension) - PPVT Peabody Picture Vocabulary Test (IIIA and
IIIB) - Administered by SLP
- Measures single word vocabulary comprehension
47PLS-AC Testable Children
- Typical children (ages 0-8) gain approximately
0.6 raw score ppm - Testable children gained, on average
- 0.4 ppm prior to intervention
- .75 ppm during intervention
- This is a significant gain of 12 months more
than would have occurred without intervention - 39 of the change was due to treatment 61 was
due to maturation
48What Does This Mean?
- For these children, skills gained as a result of
the change in rate of progress include the
ability to understand - Advanced spatial concepts (e.g., under, in back
of) - Advanced descriptors (e.g., long ,short)
- Time concepts (e.g., day versus night)
- Advanced quantities (e.g., Which one has
five?) - Complex directions (e.g., Give me the small red
ball in the box) - Passive voice (e.g.,The boy was chased by the
dog)
49PLS-AC Untestable Children
- Typical children (ages 0-8) gain approx. 0.6 raw
score ppm - Untestable children gained, on average
- 0.2 ppm prior to intervention
- 0.5 ppm during intervention
- This is a significant gain of 12 months more
than would have occurred without intervention - 67 of the change was due to treatment 33 was
due to maturation
50What Does This Mean?
- For these children, skills gained as a result of
the change in rate of progress include the
ability to identify - Pictures
- Body parts (e.g., hair, mouth, eye, nose, etc.)
- Action words (e.g., eat, sleep, drink, play,
wash) - Basic spatial concepts (e.g., in, off, out of)
- Pronouns (e.g., me, my, him)
- Early quantity concepts (e.g., some, the rest of)
- Functional object use (e.g., scissors are used
for cutting paper) - Basic descriptors (e.g., big, little, wet)
51PPVT Testable Children
- Typical children (ages 0-8) gain approx. 1.1 raw
score ppm - Testable children gained, on average
- 0.4 ppm prior to intervention
- 1.6 ppm during intervention
- This is a significant gain of 27 months more
than would have occurred without intervention - 77 of the change was due to treatment 23 was
due to maturation
52PPVT Untestable Children
- Typical children gain approximately 1.1 raw score
ppm - Untestable children gained, on average
- 0 ppm prior to intervention
- .4 ppm during intervention
- This is a significant gain of 20 months more
than would have occurred without intervention - Unable to estimate the amount of change due to
treatment because of very low T1 scores
53Expressive Language Tests
- PLS-EC Preschool Language Scale
- Administered by SLP
- Measures global expressive language
- EOWPVT Expressive One-Word Picture Vocabulary
Test - Administered by SLP
- Measures single word vocabulary output
- MacArthur Words Gestures, Words Sentences
- Administered by family interviewers
- Measures parent report of of words child
understand and says, from a list of 680 words
54PLS-EC
- Typical children gain approx. 0.6 raw score ppm
- Both testable and untestable children gained, on
average - 0.3 ppm prior to intervention
- 0.5 ppm during intervention
- This is a significant gain of 6 months more than
would have occurred without intervention - 39 of the change was due to treatment 61 was
due to maturation
55What Does This Mean?
- For the average child, skills gained as a
result of the change in rate of progress include
the ability to - Use possessives (e.g., The boys cat)
- Tell how an object is used in a short sentence
(e.g., I use a spoon to eat) - Answer questions logically (e.g., What do you do
when youre sleepy? I go to bed) - Use pronouns such as I, she, we, they
56EOWPVT Testable Children
- Typical children gain approximately 1.0 raw score
ppm - Testable children gained, on average
- 0.4 ppm prior to intervention
- 1.4 ppm during intervention
- This is a significant gain of 25 months more
than would have occurred without intervention - 71 of the change was due to treatment 29 was
due to maturation
57EOWPVT Untestable Children
- Typical children gain approx. 1.0 raw score ppm
- Untestable children gained, on average
- 0 ppm prior to intervention
- .2 ppm during intervention
- This is a significant gain of 15 months more
than would have occurred without intervention - Unable to estimate the amount of change due to
treatment because of very low T1 scores
58MCDI Words Said
- Approximately one-third (34.3) of the children
could say only 1-49 words on the MCDI at T1,
compared to only 12 at T4 - Only 4.3 of the children could say 650 words or
more on the MCDI at T1, compared to almost
one-third (32.7) at T4
59MCDI Words Said
- The childrens average number of words on the
MCDI increased from 164 at T1 to 424 at T4 -- a
260 increase
60What Predicted Change?
- We used statistical tests to determine whether
any T1 variables or measures were good predictors
of how much change would occur in individual
children over the 2 years - In other words, was it possible to predict before
intervention which children would benefit the
most and, if so -- how?
61Predictors
- None of the following were good predictors of
change over 2 years - Type of service (EIBI vs. IEII)
- Diagnosis (autism vs. PDD-NOS)
- Age at baseline
- Total hours of treatment, either with or without
school/ preschool/daycare - Combinations of the following were good
predictors of child change on some measures, over
2 years - Testability at T1 and T2 but not at T1 alone
- T1 CARS scores (less severe autism predicted more
progress) - T1 IQ scores (higher IQ scores predicted more
progress) - T1 number of words said on the MCDI (more words
predicted more progress) - There may be other (and better) predictors as
well that were not examined in this analysis
62Parenting Stress PSI-SF
- PSI Parenting Stress Inventory-Short Form
- Administered by family interviewer
- Based on parent report
- Provides total parenting stress score and
subscale scores in four areas
63Parenting Stress PSI-SF
- PSI Parenting Stress Inventory-Short Form
- Administered by family interviewer
- Based on parent report
- Measures total parenting stress -- that is,
stress related to parenting (in this case) the
child with autism, not stress caused by other
factors (e.g., marital problems, family illness,
financial difficulties, etc.) - Scores of 90 or above are considered problematic
(see blue line on next slide)
64PSI-SF
- Average score at baseline 96.6
- 90 or more 64 of families
- Average score at 2 yrs 82.6
- 90 or more 34 of families
- This is a significant decrease but is still
worrisome for the 34 of families with scores
above 90
65What Child Factors Predicted PSI Scores?
- We used statistical tests to determine if
specific child factors were associated with
(i.e., predicted) changes in PSI scores over the
2 years - We examined all of the test results (IQ,
expressive and receptive language, daily living
skills, problem behavior, etc.) - The only significant predictors were changes in
problem behavior scores on the TABS and changes
in autistic behavior scores on the ABC --
together, these accounted for 77 of the PSI
score change - This suggests that reduced problem behavior is
the factor most related to reduced parenting
stress and should be a high priority for direct
intervention from the outset of treatment
66Parent Perceptions and Satisfaction
? Pat Mirenda, Ph.D., 2005
67Parent Satisfaction PSQ
- PSQ Parent Satisfaction Questionnaire
- Adapted from an instrument used in Lovaas (ABA)
research sites - Administered by the family interviewer
- Measures parents perceptions of child change in
several areas and satisfaction with the EIBI or
IEII programs as whole - Completed at each evaluation point here, we
summarize only the results at 32 mo (T5) for EIBI
families and 24 mo (T4) for IEII families
68Parent Perceptions of Child Change
- Parents were asked if they noticed improvements
in their childs - Language and communication skills
- Social skills
- Play and leisure skills
- Aggression and tantrums
- Self-stimulatory and ritualistic behaviour
- Self-help skills
69Parent Perceptions of Child Change
- Parents rated the amount of change in each area
from 1-7 - 1 this is no longer a problem for my child
- 2 significantly improved
- 3 slightly improved
- 4 no change
- 5 slightly worse
- 6 significantly worse
- 7 much worse
70Parent Perceptions of Child Change
71Parent Satisfaction
- Parents were also asked to rate their
satisfaction in a number of areas - In general, the rating scale was
- 1 very negative
- 2 or 3 somewhat or slightly negative
- 4 neutral or just right
- 5 or 6 somewhat or mostly positive
- 7 very positive
72How Do You Feel About
the treatment methods used with your child?
73How Do You Feel About
the therapists working with your child?
74How Do You Feel About
the workload for you and your child?
75How Has Treatment Affected
your stress and that of your child?
76How Has Treatment Affected
your family, overall?
77Quality Compared to Other Services Received
78How Has Treatment Affected
your confidence and hopefulness for your childs
future?
79Recommend to Others?
80Overall Ratings
81Parent Benefits
82Parent Benefits
83Summary
- Over 2 years, children receiving EIBI and IEII
made more progress per month than they are likely
to have made without treatment, on almost every
measure - On average, both EIBI and IEII parents saw
significant improvements over 2-3 yrs in their
children, and were very satisfied with the
services they received
84Many Thanks to
- Psychologists
- Ron Buen
- Kathryn Cass
- Julie Conry
- Linda Eaves
- Peggy Koopman
- Karl Mueller
- Tara Tunstall
- Edith van de Watering
- Speech-language pathologists
- Barbara Cotter
- Marta Eveson
- Betsy Niely
- Mary McKenna
- Shannon Muir
- Liz Payne
- Lisa Prokopowitz
- Pat Savinkoff
- Veronica Smith
- Rhoda Zacker
85Many Thanks to
- Additional UBC research assistants
- Karen Ott VandeKamp
- Kim Hurd
- Robyn Teske
- Project assistant Jackie Brown
- And, most of all, all the families and children
who participated!
- Family interviewers
- Kirsten Turoldo
- Lynn Edwards
- Brenda Fossett
- Kim Hurd
- Margaret Gauthier
- Jane Kelty
- Liana Maione
- Karen Ott vandeKamp
- Laurie Reid
- Tawnya Schulz
- Robyn Teske
- Sandra Waddle
- Krista Zambolin
86The End
? Pat Mirenda, Ph.D., 2005