Building Quality Child Assessment Systems for IDEA Infant/Toddler and Preschool Programs

1
Building Quality Child Assessment Systems for
IDEA Infant/Toddler and Preschool Programs

• Where Weve Been Lately and Where We Might Be
  Going Next
• Mary McLean
• 8/27/2008

2
Where Weve Been Lately

• Working on Outcomes Assessment!

3
IDEA 2004 and Accountability

• Is money spent on programs for young children
  with disabilities producing good outcomes?

4
Decisions, decisions.

• What to measure ?
• Status vs progress ?
• One instrument or many ?
• Direct assessment or observation-based?
• Sample or report on all children?

5
Part C and Preschool Child Outcomes

• of children who demonstrate improved
• Positive social emotional skills (including
  positive social relationships)
• Acquisition and use of knowledge and skills
  (including early language/ communication and
  early literacy)
• Use of appropriate behaviors to meet their needs

6
Reporting Categories

1. of children who did not improve functioning
2. of children who improved functioning but not
   sufficient to move nearer to functioning
   comparable to same-aged peers
3. of children who improved functioning to a level
   nearer to same-aged peers but did not reach it
4. of children who improved functioning to reach a
   level comparable to same-aged peers
5. of children who maintained functioning at a
   level comparable to same-aged peers

7
Timelines

• __X__December 2005 States submit State
  Performance Plans
• __X__Feb 2007 APR Child status at entry data
• __X__Feb 2008 APR First progress data
  
  
• _____2009 - First report to the public
• _____2010 Targets to be set

8
Time to Celebrate!!
9
Where We Are Now

• Working on Quality Assessment Practices

10

• As states work through the many challenges
  involved in developing accountability systems, we
  may find that the push for accountability has the
  unintended positive consequence of building
  better assessment practices.
• Hebbeler, Barton Mallik, 2007

11
Building a better assessment system

• for children, families and programs

12
Recommended Practices for Assessment

• Neisworth, J. Bagnato, S. (2005). DEC
  recommended practices Assessment. In Sandall,
  Hemmeter, Smith McLean (Eds) DEC recommended
  practice A comprehensive guide for application.
  Longmont, CO Sopris West Publishing Co.
• NAEYC and NAECS/SDE (2003). Early childhood
  curriculum, assessment, and program evaluation
  Building an effective accountable system in
  programs for children birth through age 8.
  http//www.naeyc.org/about/positions/cape.asp
• DEC (2007). Promoting positive outcomes for
  children with disabilities Recommendations for
  curriculum, assessment and program evaluation.
  www.dec-sped.org
• Snow, C. VanHemel, C. (2008). Early childhood
  assessment Why, what and how? Washington, DC
  National Academies Press.

13
Characteristics of a Quality Assessment System

• Useful (utility)
• Authentic
• Collaborative
• Universal

14
Useful

• assessment utility

15
The Birth to 6 Child Outcome System
1. Assess at entry and exit.
4. The state collects this data and reports to
OSEP percentages of children meeting criteria
for each reporting category
2. Determine status ratings at entry and exit
3. Provide this information to the state.
5. The state determines goals and improvement
activities.
COSF
Social Emotional
Acquiring knowledge and skills
Actions to meet needs
16
Formative and Summative Assessment

• Formative Assessment
• Assessment for learning
• Ongoing during intervention
• Informs instruction
• Benefits the children who are being assessed

• Summative Assessment
• Assessment of learning
• Completed as a child is exiting a program
• Informs accountability or program evaluation
• Benefits programs

17
Wisconsin Model Early Learning StandardsTeaching
Cycle
Ongoing Assessment Gathering information to
determine what the child can do and what the
child is ready to learn
Planning Deciding what should be done to promote
development and what we want children to learn.
Implementation Providing meaningful, experiential
activities that support individual and group
goals guided by supportive interaction and
relationships
18
Ongoing Assessment

• Helps decide what to teach
• Helps decide how to teach
• Helps to know when to make changes in teaching
• Is assessment in the service of instruction
• (McAfee Leong, 2002)

19
An Assessment System with High Utility

• Informs program planning and progress monitoring
  and accountability
• Informs general early childhood
  planning/monitoring and IFSP/IEP
  planning/monitoring

20
The Birth to 6 Child Outcome System
1. Utilize authentic, on-going assessment
practices.
4. The state collects this data and reports to
OSEP percentages of children meeting criteria
for each reporting category
2. Determine status ratings at entry and exit
3. Provide this information to the state.
5. The state determines goals and improvement
activities.
COSF
Social Emotional
Acquiring knowledge and skills
Actions to meet needs
21
Authentic
22
Authentic assessment

• Observation of child behavior over time in
  typical routines and activities

23
Standardized_________________________________Authe
nticDirect Assessment__________________Obse
rvation-based assessment
24
the science of the strange behavior of
children in strange situations with strange
adults for the briefest possible period of time
Bronfenbrenner (1977)

• (standardized assessment)

25
The best way to understand the development
of children is to observe their behavior in
natural settings while they are interacting with
familiar adults over prolonged periods of time.
Bronfenbrenner, 1977

• (authentic assessment)

26
Observation_______Direct Assessment

• Procedures for consistency in administration and
  scoring are built in
• Behavior sampled may not be representative of
  childs typical behavior

• Consistency depends on teacher training and
  monitoring of implementation
• Behavior measured is childs typical behavior
• Increased utility for instruction
• (Mathematica, 2007)

27
Strategies for authentic assessment

• Observation and documentation
• anecdotal notes
• event sampling
• activity protocols
• portfolio assessment
• 
  

28

• Embed assessment into ongoing routines and
  activities
• (Raver, S.,2003 Sandall Schwartz, 2008)

29
Activity Matrix(Sandall Schwartz, 2008)
Schedule Shelby Matthew Davion
Arrival Respond to peers Remove and hang up coat Follow 2-step directions
Free Play Engage in activity Join ongoing play Request preferred item
Circle Imitate adult words Imitate adult actions Tell full name
Outside Throw ball with 2 hands Join ongoing play Request preferred item
Snack Use pincer grasp Pour juice
Free Play 2-word utterance Imitate adult actions Sort shapes
Circle/Departure 2-word utterance Identify positional concepts
Transitions Follow 2-step direction
30
Anecdotal notes
31
Event Sampling
Schedule Shelby Matthew Davion
Arrival Respond to peers 2/2 Remove and hang up coat with verbal prompt Follow 2-step directions 2/2
Free Play Engage in activity 8 minutes Join ongoing play 2/2 Request preferred item 3/4
Circle Imitate adult words 5/7 Imitate adult actions 4 Tell full name with model
Outside Throw ball with two hands 4/4 Join ongoing play 1/1 Request preferred item 1/2
Snack Use pincer grasp 4/9 Pour juice with physical guidance
Free Play 2-word utterance 4 Imitate adult actions 3 Copy square 2/2
Circle/Departure 2-word utterance 3 Identify positional concepts 2/2 under, on top
Transitions Follow 2-step direction 2/2
32
Activity Protocols

• An activity protocol provides a list of skills
  from a curriculum-referenced assessment that are
  likely to be observed in a particular activity or
  routine
• Grisham-Brown, Hemmeter Pretti-Frontczak, 2005

33
Activity Protocol Playdough

• AREA ITEM CHILD BEHAVIORS
  
• Motor M-1 Holds object with one hand
  and
• manipulates
  with other
• Cognitive C-3 Demonstrates understanding
  of size concepts
• Adaptive A-4 Fastens button on art
  smock

34
Portfolio Assessment

• Paper
• Electronic

35
Use of Authentic Assessment to Inform Multiple
Measures

• To inform curriculum-referenced assessments which
  measure progress toward EC learning targets or
  curricular goals
• To measure progress toward IEP goals or IFSP
  outcomes
• To inform measures of accountability

36
Collaborative

• with families and teams

37
Gathering information from families

• Family involvement expands the validity of
  assessment information to home and community
  environments

38
Strategies for Gathering Information

• Utilize home visits as a strategy to connect with
  families
• Make periodic requests for information from
  families (describe specific skills to watch for
  at home.)
• Use assessment tools that have family report
  forms AEPS, HELP.
• Use existing informal communication mechanisms
  (traveling notebook, daily conversations, e-mail,
  telephone.)

39
Gathering information from other service providers
40
How to Include Information from Other Providers
Systematically

• Plan collaborative activities for observations
• Schedule periodic team meetings or staffings
• Share information and request information

41
Universal

• and individually appropriate

42
Universality

• Design and/or accommodations which enable all
  children to demonstrate their underlying
  functional capabilities
• Bagnato, Neisworth, Pretti-Frontczak, (in
  preparation).
• Assessment must be valid for all children
  including children who are English Language
  Learners and children with disabilities

43
Assessment and Learning Targets
Individualized
Universal
44
Universal Design

• The design of products to be useable by all
  people to the greatest extent possible
  (Thompson, Johnstone Thurlow, 2002)
• Center for Applied Special Technology (CAST)
• http//www.cast.org/research/udl/index.html
• Desired Results access Project
• California Department of Education
• www.draccess.org

45
Accommodations/Adaptations

• Adaptations are changes in environmental
  arrangements or differences in observed behavior
  that allow children with disabilities to be
  accurately assessed in the natural environment
• Augmentative or alternative communication system
• Alternative mode for written language
• Visual support
• Assistive equipment or device
• Functional positioning
• Sensory support
• Alternative response mode
• www.draccess.org

46
(No Transcript)
47
Where Are We Going Next?

• Innovative Practices in Measurement

48
Innovative Models and Frameworks

• Assessment Framework - RTI
• Measurement Model - IRT / Rasch

49
Assessment Framework RTI

• Center for Response to Intervention in Early
  Childhood
• http//www.crtiec.org/
• Recognition and Response An Early Intervening
  System for Young Children At Risk for Learning
  Disabilities
• http//www.fpg.unc.edu/randr/

50
Where did RTI come from?

• Initiatives in the field of learning disabilities
• Written into IDEA 2004 early intervening
• Part B money can be used to fund programs for
  children who have not yet been found eligible for
  special education

51
RTI Tiered Model

• Tier 1 Universal periodic screening and
• resulting interventions
• Tier 2 Research-based small group
• interventions
• Tier 3 Individualized interventions and
• possible referral for special education

52
What Might RTI Mean for Early Childhood?

• Universal and Periodic Screening (Prevention)
• Progress Monitoring
• Outcomes Measurement

53
Screening and Progress MonitoringIndividual
Growth and Development Indicators(IGDIs)

• General Outcomes Measurement (GOM)

54
IGDIs Communication Trajectory
Childs Observed Trajectory
Normative Trajectory
Below Average (-1.5 SD) Trajectory
www.igdi.ku.edu
55
IGDIs Available

• Infant and Toddler
• Early Communication
• Early Problem Solving
• Early Movement
• Early Social
• www.igdi.ku.edu

• Preschool
• Picture Naming
• Alliteration
• Rhyming
• www.ggg.umn.edu

56
Accountability Crosswalk to OSEP Outcomes
57
Measurement Model

• Rasch Scaling
• Item Response Theory

58
Rasch/IRT Measurement Model

• Allows the creation of interval-scaled
  calibrations when raw scores are ordinal
• Translation a criterion or
  curriculum-referenced instrument can be
  calibrated so that it yields scaled scores (equal
  interval scores)
• The childs ability is described relative to
  position on a specific developmental path not
  relative to scores from the norming population.

59
Classic Psychometric Theory Normal Curve
2
14
34
34
14
2
Mean Median Mode Standard Deviation
60
Rasch Scaling Demonstrating higher level skills
results in a higher score
More of the construct or skill (later developing
skill)----
Less of the construct or skill (earlier
developing skill)----
61
So..

• Allows measurement of progress as acquisition of
  skills not as position relative to scores from
  the norming population
• Allows creation of an item bank which prevents
  narrowing of the curriculum or teaching to the
  test
• (Meisels, 2002)

62
Rasch/IRT Instruments

• Desired Results Developmental Profile access
• (DRDP access)
• www.draccess.org
• Ages and Stages Questionnaire (ASQ)
• www.brookespublishing.com/store/books/bricke
  r-asq/
• Assessment, Evaluation and Programming System for
  Infants and Children (AEPS)
• www.brookespublishing.com/store/books/bricke
  r-aeps/

63
What Do We Need?
64
What Do We Need?

• Research on
• direct assessment and observation-based
  assessment with young children
• new assessment instruments
• assessment systems that weave general early
  childhood and EI/ECSE
• procedures for assessing children who are English
  Language Learners
• accommodations for children with disabilities

65
What Do We Need?

• Personnel Development
• pre-service preparation that provides a solid
  foundation in assessment skills
• system for continuous in-service training to
  address local needs
• emphasis on measurement skills in doctoral
  programs

66
What Do We Need?

• Policy
• alignment of early learning guidelines,
  assessment practices, curricular practices and
  accountability requirements across agencies and
  programs serving young children

67
Thank You!
68

• Mary McLean, Ph.D.
• Kellner Professor of Early Childhood Education
• Department of Exceptional Education
• School of Education
• University of Wisconsin-Milwaukee
• 414-229-2213
• mmclean_at_uwm.edu