New Tools in the Tool Box: What We Need in the Next Generation of Early Childhood Assessments

1
New Tools in the Tool Box What We Need in the
Next Generation of Early ChildhoodAssessments

• Kathy Hebbeler
• ECO at SRI International
• Presented at the OSEP EC Conference,
• December 2008

2
Todays Topics

• The new National Academy of Sciences Report
  Early Childhood Assessment Why, What, How
• What should the next generation of assessment
  tools look like?

3
(No Transcript)
4
Reason for the Committee

• Controversy over Head Start National Reporting
  System (NRS)
• Congressional mandate
• review and provide guidance on appropriate
  outcomes and assessments for young children.
• 2 key topics
• Identification of key outcomes for children 0-5
• Quality and purpose of different state-of-the art
  techniques

5
Organization of the Report

• Overview of purposes
• What to assess Outcomes
• And Quality of Environment
• How to assess
• Technical discussion of validity
• Issues related to minority, ELL, children with
  disabilities
• Thinking systematically

6
2 Key Principles

• The purpose of the assessment should guide
  assessment decisions
• Assessment should be conducted within a coherent
  system of medical, educational, and family
  support services, that promote optimal
  development for all children

7
Purposes

• Determining an individual childs level of
  functioning
• Individual-focused screening
• Community-focused screening
• Diagnostic testing
• Establishing readiness

8
Purposes (Contd)

• Guiding intervention and instruction
• Evaluating the performance of a program or
  society
• Program effectiveness
• Program impact
• Social benchmarking
• Advancing knowledge of child development

9
Guidelines from Multiple Groups

• Assessment should benefit children
• Assessments should meet professional, legal,
  ethnical, standards
• Assessments should be age-appropriate or
  developmentally/individually appropriate
• Parents/family should be involved in the
  assessment whenever possible
• Assessments should be linguistically and
  culturally appropriate/responsive

10
Guidelines from Multiple Groups (Contd)

• Assessments should assess developmentally/educatio
  nally significant content
• Assessment information should be gathered from
  familiar contexts (NEGP), realistic settings and
  situations (NAEYC), or be authentic (DEC).
• Information should be gathered from multiple
  sources.
• Screening should be linked to follow-up
  assessment.

11
Recommendations Related to Purpose

• Make assessment purpose explicit and public
• Assessment strategy must match purpose

12
Which Outcomes

• Why this domain?
• Evidence of consensus around value
• Evidence of continuity within domain over
  development or links to other domains
• Evidence that it is a target for intervention and
  affected by the environment

13
Outcome Domains (NEGP)

• Physical well-being and motor development
• Socioemotional development
• Approaches to learning
• Language (and emerging literacy)
• Cognitive skills (including mathematics)

14
Classification Challenges

• Boundaries are artificial
• Researchers (and test developers) use different
  classification schemes

15
Recommendations related to Domains and Outcomes

• Go beyond the standard domains of language and
  mathematics
• Need support for better measures of
  socioemotional development
• D-4 For children with disabilities and specials
  needs, domains-based assessments may need to be
  replaced or supplemented with more functional
  approaches

16
Validity and Reliability

• Weight of evidence
• Not whether an assessment is valid and reliable
• Use accumulation of evidence to judge whether the
  assessment is suitable for the task for which it
  is intended.
• Evidence pertains to specific type of uses.

17
Assessing All Children

• Much more research needs to be conducted on the
  validity of tools with minority children, ELL,
  and children with disabilities.
• Children with disabilities
• Construct irrelevant skills as a threat to
  validity
• Interrelatedness of domains
• Functional outcomes
• Universal design
• Many tools used with children with disabilities
  have not been validated for the purposes for
  which they are being used.

18
Instrument Selection Recommendations

• Select assessment with acceptable levels of
  validity
• Examine data produced with the assessment for
  validity for purposes for which the assessment is
  being used.
• Test developers need to collect and make
  available evidence related to validity for
  minority, ELL, children with disabilities.
• Be very cautious about reaching conclusions about
  a group not well represented in the validation
  sample.

19
Situating Assessment in a System

• EC System has
• A goal for children
• Strategies to achieve this goal
• Infrastructure supporting the goal and the
  strategies
• Selects assessment to be compatible with other
  elements of the system.

20
Systems Recommendations

• EC assessment system must be part of a larger
  system with strong infrastructure to support
  early care and education.
• Standards
• Assessment
• Reporting
• Professional development
• Opportunity to learn
• Inclusion
• Resources
• Monitoring and evaluation

21
Systems Recommendations

• Coherence
• Horizontally coherent curriculum, instruction,
  assessment aligned with early learning standards
• Vertically coherent shared understanding of
  goals at all levels consensus about purposes of
  assessment
• Developmentally coherent taking into account
  what is known about young childrens skills and
  understanding develop

22
Research Needed

• New conceptual frameworks
• Application of technical advances, e.g., IRT
• Span birth to 6 or 7
• Social emotional measures
• Capture childrens growth toward being able to
  meaningfully participate in variety of everyday
  setting
• Use of technology

23
Challenges with Current Assessments for All
Children/Children with Disabilities

• Domains-based
• When should assessment look at only one domain?
  When should it look at the whole child?
• How functional are current tools?
• Varies
• Need for tools to assess the three outcomes
• How authentic? How real?
• How much is lost with structured requests? And
  for which children?

24
Challenges with Current Assessments for All
Children/Children with Disabilities

• Tool structure
• E.g., Work Sampling
• Need for specificity sometimes..
• E.g., AEPS High Scope COR
• Do we want predictive validity?
• Universal design principles
• DRDP as the exception
• Adaptations and accommodations

25
Challenges with Current Assessments for All
Children/Children with Disabilities

• Diagnosis vs. intervention planning purpose
• Must these be separate tools?
• Are the things we want to know about children to
  prove eligibility so different from what we want
  to know to develop an intervention plan?
• AEPS in JEI, December 2008
• Recognition and Response/RTIs tools for young
  children
• IGDIs
• How do these fit with curriculum-based assessment?