Wyoming DDD Monitoring Manual

1

ECO Suggestions on Indicators C3 and B7 Kathy
Hebbeler, ECO Lynne Kahn, ECO/NECTAC August 13,
2009
2
Take-home points for today

• The change in the measurement table involves 2
  new calculations from data already being reported
• The change reduces the possible number of targets
  for these Indicators.
• The calculation described in the Indicator
  Measurement Table can be done by an ECO
  calculator.
• The summary statements provide data describing
  program effectiveness.

3
Timeline

• February 2007 states report data on indicators
  C3 and B7 for the first time
• February 2010 states are to set targets for
  these indicators

4
C3 and B7 Three Child Outcomes

• Children have positive social-emotional skills
  (including social relationships)
• Children acquire and use knowledge and skills
  (including early language/communication and
  early literacy)
• Children use appropriate behaviors to meet their
  needs

5
Reporting Categories (Measures) for C3 and B7

• Percentage of children who
• a. Did not improve functioning
• b. Improved functioning, but not sufficient to
  move nearer to functioning comparable to
  same-aged peers
• c. Improved functioning to a level nearer to
  same-aged peers but did not reach it
• d. Improved functioning to reach a level
  comparable to same-aged peers
• e. Maintained functioning at a level comparable
  to same-aged peers

3 outcomes x 5 measures 15 numbers
6
The Problem

• Data reported includes
• 5 progress categories
• For each of 3 outcomes
• Total of 15 numbers reported each year
• OSEP heard Too many targets for one Indicator
• OSEP asked ECO for a recommendation

7
The Solution

• Summary Statements
• (a way to reduce the data so states did not have
  to set 15 targets for the indicator)

8
Broad input to develop the Summary Statements

• ECO presented options to states and ECO work
  groups via conference calls
• Posted on the ECO web site for comments
• OSEP put the summary statements out for public
  comment
• Comments came in that were thoughtful, but not
  necessarily consistent with one another

9

• Paper documenting the process and alternatives
  considered on the ECO website
• Setting Targets for Child Outcomes

10
The concepts are easier than the words or the
formulas

• Summary Statement 1 How many children changed
  growth trajectories during their time in the
  program?
• Summary Statement 2 How many children were
  functioning like same aged peers when they left
  the program?

11
(No Transcript)
12
The change to the Measurement TableThe
Summary Statements

• Of those children who entered the program
  below age expectations in each Outcome, the
  percent who substantially increased their rate of
  growth by the time they turned 6 years of age or
  exited the program.
• The percent of children who were functioning
  within age expectations in each Outcome by the
  time they turned 6 years of age or exited the
  program.

13

• Measurement for Summary Statement 1
• Percent of infants and toddlers reported
  in progress category (c) plus of infants and
  toddlers reported in category (d) divided by
  of infants and toddlers reported in progress
  category (a) plus of infants and toddlers
  reported in progress category (b) plus of
  infants and toddlers reported in progress
  category (c) plus of infants and toddlers
  reported in progress category (d) times 100.

14
Summary Statement 1

• c d
• _________________________
• a b c d

X 100
15

• Measurement for Summary Statement 2 Percent
  of infants and toddlers reported in progress
  category (d) plus of infants and toddlers
  reported in progress category (e) divided by the
  total of infants and toddlers reported in
  progress categories (a) (b) (c) (d) (e)
  times 100.

16
Summary Statement 2

• d e
• _____________________________
• a b c d e

X 100
17
A shortcut to the calculations
Summary Statements Calculator -April 14, 2009
18
Example of State Indicator Data for 2008-2009
19
SS1-What are we calculating??
760 (a, b, c, and d) or 76 of the children
entered or exited the program functioning below
age expectations
240 (e) or 24 of the children entered and exited
functioning at age expectations
20
SS1 (continued)
570 (c and d) of the 760 (a, b, c, and d) changed
their growth trajectories (made greater than
expected progress)
270 300 570 760
75
21
SS2-What are we calculating??
30 of the children reached age expectations by
exit and 24 of the children entered and exited
at age expectations
300240 540 1000
54
22
Summary Statement 1 is one type of evidence of
program effectiveness

• Required Summary Statement 1
• Of those children who entered the program
  below age expectations in Outcome A, the percent
  who substantially increased their rate of growth
  by the time they exited the program 75
•  75 of the children who were below age
  expectations made greater than expected gains in
  their social relationships, made substantial
  increases in their rates of growth. i.e. changed
  their growth trajectories

23
Summary Statement 2 is another type of evidence
of program effectiveness

• Required Summary Statement 2
• The percent of children who were functioning
  within age expectations in Outcome A when they
  exited the program 54
• 54 of the children were functioning like same
  age peers in their social relationships when they
  exited the program.

24
Questions and comments?
25
More Information re Target Setting

• Conference calls,
• Individualized TA
• Resources at
• The-ECO-Center.org

26
What else can we say?

• 96 of children participating in Part C made
  progress in their social relationships while they
  were enrolled.
• The 4 of children who did not make progress
  included children with the most severe
  disabilities and/or degenerative conditions. Can
  you describe them?

27

• 24 of the children participating in Part C were
  functioning at age expectations at entry and at
  exit in their social relationships.
• Can you describe them?