Child Outcome Data

1
Broad Data Analysis

• Child Outcome Data

2
Broad Analysis Child Outcomes

• Does our states data look different than the
  national data?
• Are our state child outcomes trends stable over
  time?
• Is the data trending upwards?
• Is the data trending downwards?
• Is our state performing more poorly in some
  outcomes than others?
• Are the child outcomes similar across programs?
• What about data quality? Can we be confident in
  our data?

3
Child Outcomes State vs. National
4
Child Outcomes State vs. National
5
National Vs. State Meaningful Differences
6
Child Outcomes National vs. State FFY11 and
State FFY12
7
Data Quality Trends
8
Virginia Trends
9
Virginia Trends
Outcome 3 Use of Appropriate Behaviors to Meet
Needs
Outcome 2 Acquisition and Use of Knowledge and
Skills
10
Virginia Trends
11
(No Transcript)
12
(No Transcript)
13
Child Outcome Results FFY11/FFY12
Meaningful Differences Calculation
14
Child Outcomes National vs. State FFY11 and
State FFY12
15
Child Outcomes Local vs. State
16
Child Outcomes Local vs. State
17
Child Outcomes Local vs. State
18
Child Outcomes Local vs. State
19
Child Outcomes Local vs. State
20
Child Outcomes Local vs. State
21
Data Quality Elements

• Completeness of data
• number of children reported for the
  outcome/number who exited
• Virginias results average 65 range for
  Local Systems 17 - 100
• Expected Patterns for Progress Categories
• Virginias state date is within these parameters
  for all three outcomes
• Child Outcomes State Trends Over Time
• As noted on previous slides, Virginias results
  do not show wide variations which would trigger
  concerns about data quality

22
Broad Data Analysis

• Family Outcome Data

23
Broad Analysis Family Outcomes

• Does our states data look different than the
  national data?
• Are our state family outcomes trends stable over
  time?
• Is the data trending upwards?
• Is the data trending downwards?
• Is our state performing more poorly in some
  outcomes than others?
• Are the family outcomes similar across programs?
• What about data quality? Can we be confident in
  our data?

24
(No Transcript)
25
Family Outcomes State Trends over Time
26
(No Transcript)
27
Family Outcomes Local vs. State 2012-20134A EI
has helped the family know their rights
28
Family Outcomes Local vs. State 2012-20134B EI
has helped the family communicate their
childrens needs
29
Family Outcomes Local vs. State 2012-20134C EI
has helped the family help their child develop
and learn
30
Data Quality

• The data analysis for Virginia survey is quite
  extensive, using rigorous data analysis
  standards.
• Virginias response rate, like other states that
  use mailing as the means to disseminate the
  family survey, is lower than states who use other
  mechanisms.

31
Broad Infrastructure Analysis
32
Analysis Mechanisms

• Use of federal monitoring tools and procedures
• Communication with Local Systems through
• Ongoing TA
• Regional Meetings
• Trainings
• System Manager Meetings
• Monitoring Results
• Record Reviews
• QMRs
• Individualized TA
• Local System Contract Deliverables

33
Infrastructure AnalysisInformation Sources

• Local self-reporting
• Stakeholder input
• LSM surveys
• Training/meeting evaluations
• Monitoring/QMR
• Dispute Resolutions
• Observations through TA and other interactions

34
Broad Infrastructure Analysis

• Governance - Mixed
• Strengths
• Code of Virginia establishes infrastructure
• VICC
• State Interagency Agreement interagency
  partnerships
• DBHDS local contract with Local Lead Agencies
  specifies LS infrastructure requirements
• Weaknesses
• Variable job roles and skill sets of LSMs
• Variable strength and specificity of LLA/provider
  contracts
• Variable support from LLA leadership

35
Broad Infrastructure Analysis

• Fiscal - Mixed
• Strengths
• Additional state funding
• Medicaid EI Services program
• Trainings on fiscal issues
• Strong working relationship with fiscal office at
  the state lead agency (DBHDS)
• Weaknesses
• Variable fiscal skill set of LSMs
• Code of Virginia does not require fiscal
  commitment at local level
• Variety of Local Lead Agencies
• Inconsistent reporting of fiscal data at local
  level
• Lack of fiscal data in state data system
• Perception/reality that there is not enough money

36
Broad Infrastructure Analysis

• Quality Standards Not sure
• Practice Manual articulates expected practices,
  but these are not labeled quality practices
• Monitoring and Accountability Strength
• Local self-monitoring and supervision for
  continuous improvement variable
• Professional Development Strength
• Technical Assistance Strength
• Data Weakness

37
Closing Thoughts

• Based on this broad review, did anything strike
  you as an area of focus for our Systemic
  Improvement Plan?