Measuring Progress: Improved Outcomes for Children and Families - PowerPoint PPT Presentation

Loading...

PPT – Measuring Progress: Improved Outcomes for Children and Families PowerPoint presentation | free to download - id: 20a654-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Measuring Progress: Improved Outcomes for Children and Families

Description:

... EI ... Critical Events in EI Accountability. 1992 Osborne and Graebler, ... OSEP begins to ask states for EI child outcome data (and funds the ECO ... – PowerPoint PPT presentation

Number of Views:61
Avg rating:3.0/5.0
Slides: 72
Provided by: FPG261
Learn more at: http://www.infantva.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Measuring Progress: Improved Outcomes for Children and Families


1
Measuring Progress Improved Outcomes for
Children and Families
  • Kathy Hebbeler
  • The Early Childhood Outcomes Center
  • SRI International

Virginia EI Conference Creating Connections
Strengthening Partnerships between Families and
Providers Roanoke, VA, March 2007
2
Objectives
  • Review history and rationale related to the
    selection of child indicators
  • Describe what we have learned about implementing
    measurement systems
  • Describe why child indicators are important for
    improved services

3
Goal of the ECO Center
  • Promote the development and implementation of
    child and family outcome measures for infants,
    toddlers and preschoolers with disabilities that
    can be used in local, state, and national
    accountability systems

4
Being clear
  • For VA, the word indicator refers to the 3 OSEP
    outcomes
  • to distinguish them from IFSP outcomes
  • ECO Centers materials use the word outcomes to
    refer to both
  • I will try to use indicator for the OSEP
    outcomes unless I mean all kinds of outcomes

5
  • What are Outcomes?

6
The Concept of Outcomes
  • An outcome is the result of some action or set of
    actions
  • Often expressed in the form of a statement
  • Examples
  • All passengers will wait 10 minutes or less in
    the airport security line.
  • All children will be fully immunized by 2.

7
The Concept of Outcomes
  • Two parts
  • The expected result
  • The action(s) that produces the result
  • May not be stated but always present
  • Outcomes are the effect in cause and effect

8
Outcomes Closer to Home
  • EI programs provide high quality services and
    supports
  • EI services are family centered
  • Families help their child develop and learn
  • Children take appropriate action to meet their
    needs

9
Different Kinds/Levels of Outcomes
  • Program or systems outcomes
  • EI programs provide high quality services and
    supports
  • EI services are family centered
  • EI services are coordinated
  • What are the actions/activities that produce
    these results?

10
Different Kinds/Levels of Outcomes
  • Child or family outcomes
  • Families know their rights and advocate
    effectively for their children
  • Children have positive social relationships
  • What are the actions/activities that produce
    these results?

11
Logic Model for Child and Family Outcomes
Adequate funding
Good outcomes for children and families
High quality services and supports for children
0-5 and their families
Good State policies and programs
Good Local policies and programs
Good Federal policies and programs
Strong Leadership
  • Supports
  • Preservice training
  • Inservice training

12
Two Critical Outcome/Indicator Questions for
Building High Quality Programs
  • What is the intended result?
  • What is the action/set of action that will
    produce the intended result?

13
  • How Looking at Child Indicators
  • for EI Became Important

14
Focus on Accountability
  • Followed an era of focusing on and measuring
    actions/activities
  • Call for interest in the ultimate result child
    and family outcomes/indicators
  • Cut across both the public and private sectors
  • Funders demanded data to determine whether a
    program was doing what it was supposed to do

15
Critical Events in EI Accountability
  • 1992 Osborne and Graebler, Reinventing
    Government
  • 1993 GPRA (Government Performance and Results
    Act) passed
  • Intervening years.
  • 2002 PART finds there are no data on outcomes
    for Part C
  • 2003 OSEP begins to ask states for EI child
    outcome data (and funds the ECO Center!)
  • 2005, 2006 OSEP releases revisions to the
    reporting requirements

16
PART Review for Part C and Part B Preschool
  • Results Not Demonstrated
  • Part C
  • While the program has met its goal relating to
    the number of children served, it has not
    collected information on how well the program is
    doing to improve the educational and
    developmental outcomes of infants and toddlers
    served.
  • Part B Preschool
  • The Department has no performance information
    on preschool children with disabilities served by
    this program.
  • Read more at Expectmore.gov

17
Intervening Years
  • Special Education
  • National study found poor outcomes for special
    education adolescents Results
  • Push to include students with disabilities in
    statewide assessment systems
  • Early Childhood
  • Debate about whether child outcomes should be
    measured at all
  • Discussion of all the problems in trying to
    measure outcomes for young children with
    disabilities

18
Measuring Child Indicators for EI
  • The PART findings put an end to the debate about
    whether or not to do it
  • Unfortunately, almost no progress had been made
    in the intervening years as to HOW to do it

19
Meanwhile Re-thinking Assessment in Early
Childhood
  • Major changes in last 15 years in how assessment
    of young children is viewed
  • Old position Do not test little kids
  • New position Ongoing assessment is part of a
    high quality early childhood program

20
What changed
  • New and different tools became available
  • Curriculum-based assessments were developed
  • General EC Tools for 3-5 came first 0-3 tools
    are coming now
  • Interesting sidebar Curriculum-based
    assessments for programs serving children 0-5
    with disabilities have been around for years

21
What changed
  • The purpose of assessment was redefined
  • Not about sorting, labeling, using to deny
    access (the unready)
  • Now about Getting a rich picture of what
    children can do and cant do and using that
    information to help them acquire new skills
  • progress monitoring

22
What changed
  • Assessment had always been seen as a process with
    multiple purposes
  • Distinctions were made between good and bad uses
    of assessment with young children
  • Good uses are now promoted
  • For more information NAEYC web site (Position
    statement on Curriculum, Assessment and
    Evaluation)

23
Interesting Irony
  • Even though the disability community had
    developed many curriculum-based assessment
    tools, currently many programs do not
    practice ongoing assessment
  • Assess for eligibility only
  • The push for ongoing assessment to monitor how a
    child is doing and plan for instruction/interventi
    on is coming from the general education community

24
Where We Are Now
  • Push for measuring outcomes for accountability
    (top down but not just federal)
  • Push for ongoing assessment of progress
    (monitoring outcomes) as best practice in EC
    programs
  • Done well this could be a powerful blending of
    forces to improve outcomes for children

25
  • Why These 3 Indicators?

26
OSEP Reporting Requirements Child Indicators
  • Positive social emotional skills (including
    positive social relationships)
  • Acquisition and use of knowledge and skills
    (including early language/ communication and
    early literacy)
  • Use of appropriate behaviors to meet their needs

27
Origin of the Child Indicators
  • ECO Center stakeholder meetings followed by
    public comment period (2004)
  • First, collected themes and ideas
  • Then, drafted and re-drafted indicator wording
  • Made recommendation to OSEP (2005)

28
Themes from the Stakeholders Child Indicators
  • Consistent with IDEA and legislative intent
  • Reflect what EI and ECSE are trying to do
  • One set for birth to 5
  • Reflect what is known about development and
    learning

29
Themes from the Stakeholders
  • Be compatible with best practice (esp.
    transdisciplinary service models, functional
    behaviors)
  • Do not base them on domains
  • Have potential to influence practice in a
    positive way
  • Incorporate universal design
  • Be readily understood

30
Make Outcomes/Indicators Functional
  • Functional refers to things that are meaningful
    to the child in the context of everyday living
  • Refers to an integrated series of behaviors or
    skills that allow the child to achieve the
    outcomes
  • They are not
  • a single behavior, nor are they
  • the sum of a series of discrete behaviors

31
Functional
  • What does a child typically do?
  • Actual performance across settings and situations
  • How child uses his/her skills to accomplish tasks
  • Not the childs capacity to function under
    unusual or ideal circumstances

32
Recommendations
  • Decision One set of indicators birth to 5
  • Decision Functional outcomes
  • Decision Global, not specific

33
The need for an introductory statement
  • Family and child outcomes are linked
  • There are overarching goals for children and
    family that cut across the outcomes
  • Helpful to frame the outcomes with these
    overarching goals

34
Concepts in the introduction Goal for children
  • The ultimate goal is for young children to be
    active and successful participants now and in the
    future in a variety of settings in their homes,
    in their child care, preschool or school
    programs, and in the community

http//www.fpg.unc.edu/eco/pdfs/eco_outcomes_4-13
-05.pdf
35
Concepts in the introduction Goal for children
  • Active and successful participants
  • Now and in the future
  • In variety of settings

36
Concepts in the introduction Goal for families
  • Enable families to provide appropriate care for
    their child
  • Have resources they need to participate in
    community activities

37
Concepts in the introduction Effective programs
  • Support families in their quest for a
    satisfactory quality of life
  • Provide needed supports and services in a timely
    fashion

Remember Outcomes are the result of actions
38
Concepts in the preface Outcomes and
accountability
  • Outcomes determined by variety of factors
  • Not all families and children will achieve all
    outcomes
  • BUT, system should still strive to achieve them

39
ECO Family Outcomes
  • Understand their childs strengths, abilities,
    and special needs
  • Know their rights and advocate effectively for
    their children
  • Help their children develop and learn
  • Have support systems
  • Access desired services, programs, activities in
    their community

40
  • The OSEP Reporting Categories

41
OSEP Reporting Categories
  • Percentage of children who
  • a. Did not improve functioning
  • b. Improved functioning, but not sufficient to
    move nearer to functioning comparable to
    same-aged peers
  • c. Improved functioning to a level nearer to
    same-aged peers but did not reach it
  • d. Improved functioning to reach a level
    comparable to same-aged peers
  • e. Maintained functioning at a level comparable
    to same-aged peers

3 outcomes x 5 measures 15 numbers
42
Functioning
13
43
Entry
14
44
Entry
Exit
15
45
Entry
Exit
16
46
CISF Numbers to OSEP Reporting Categories
  • Each possible combination of numbers (and answer
    to the b question at entry and exit generates a
    line, i.e., a developmental trajectory)
  • Each trajectory corresponds to an OSEP category
  • Taking measurements more frequently than entry
    and exit generates more powerful information

47
Functioning
13
48
OSEP Reporting Categories Child Outcomes
  • Percentage of children who
  • Did not improve functioning
  • Improved functioning, but not sufficient to move
    nearer to functioning comparable to same-aged
    peers
  • Improved functioning to a level nearer to
    same-aged peers but did not reach it
  • Improved functioning to reach a level comparable
    to same-aged peers
  • Maintained functioning at a level comparable to
    same-aged peers

3 outcomes x 5 measures 15 numbers
49
Comparing to Same Aged Peers
  • Deficit model? Not individualized?
  • Goal of EI Active and successful participation
    now and in the future
  • Kindergarten, school readiness, having friends,
    community participation
  • Setting high expectations
  • Moving from 1 to 2 moving from
  • 4 to 5
  • How much progress is enough?

50
  • COSF/CISF Implementation Issues

51
Overall
  • Child indicators have been very well received
  • Many states have embraced outcomes measurement at
    some level

52
(No Transcript)
53
Assessing functioning
  • Process helps teams think about the child
    functionally across a variety of settings and
    situations
  • Requires good assessment practices (and suggests
    need for improving current practice)
  • Assessment only at entry. What is the role of
    assessment in EI?
  • Moving from domains to functioning

54
Assessing functioning
  • Providers need a way to have a conversation with
    families about child functioning across settings
    and situations
  • Providers need training in explaining assessment
    results to parents
  • soft pedal

55
Age Expectations
  • Requires providers understand typical child
    development and how children of different ages
    are expected to function in all 3 indicators
  • Reveals need for training in this area

56
Working with Families
  • Changing the conversation between providers and
    families?
  • Changing the nature of IFSPs?
  • Do we write IFSPs to these indicators?
  • Would that be such a bad thing?

57
Opportunity?
  • Measurement of indicator data can encourage more
    frequent collection of assessment data
  • Progress monitoring
  • Improve practice
  • And provide outcome indicators

58
Validity and Reliability
  • Validity is not a characteristic of a tool
  • Validity is a characteristic of a set of scores
    produced by a tool
  • CISF is judgment-based
  • Substantial research to support that with defined
    criteria, providers and parents can reach
    reliable judgments

59
Validity and Reliability
  • ECO working on procedures to help states assess
    validity of their data
  • Good training and ongoing supervision will be
    critical
  • Quality assurance will be critical

60
Some Concerns
  • Being implemented very quickly
  • Not all implementers are well trained
  • Few states have procedures in place for assuring
    the process is high quality and reliable
  • States doing the minimum will not have very
    useful data

61
How Will These Data Be Used?
  • Hopefully not at all -- UNTIL checks are in place
    to insure they are of high quality
  • Meaning needs to be attached to the findings by
    those closest to the program
  • Craft your own message

62
Next steps for ECO
  • More support and training materials
  • More guidance on supervision and quality
    assurance
  • More guidance on using data for program
    improvement

63
  • A Vision
  • (What Does This Mean for You)

64
Logic Model for Child and Family Outcomes
Adequate funding
Good outcomes for children and families
High quality services and supports for children
0-5 and their families
Good State policies and programs
Good Local policies and programs
Good Federal policies and programs
Strong Leadership
  • Supports
  • Preservice training
  • Inservice training

65
Building a high quality system
  • What if every piece of the system functioned as
    it was supposed to?
  • What information is needed to know if this is
    happening?
  • What information is needed to know how to
    identify and address weaknesses in the system?

66
Your contribution
  • Where do you fit?
  • What is your contribution to good outcomes? What
    actions do you take that produce good child and
    family outcomes?
  • What information would you need to know if your
    actions were producing good outcomes?

67
  • The ideal outcome system gives persons at each
    level of the system the information to know if
    they are producing good outcomes -and if not, to
    change their practices/policies/procedures so
    eventually they will.

68
Remember the goal for children
  • Active and successful participation now and in
    the future

69
And an overarching goal for families
  • To enable families to provide care for their
    child and have the resources they need to
    participate in their own desired family and
    community activities

70
The Take Home Message
  • Measuring outcomes is not primarily about data
  • It is about doing good things for children and
    families
  • And using data as a tool to help programs and
    providers to know whether what they are doing is
    making a difference

71
Finding out more
  • Additional information about OSEP requirement and
    state activities
  • New resources will be coming including training
    materials, resources on assessment and typical
    child development, and materials for parents
  • www.the-eco-center.org
About PowerShow.com