Assuring%20the%20Quality%20of%20Data%20from%20the%20Child%20Outcomes%20Summary%20Form

1
Assuring the Quality of Data from the Child
Outcomes Summary Form
2
Four-state Panel

• Wendy Whipple (NV) Part C
• Carolee Eslinger and Carrie Mori (ID) Part C
• Ted Maloney and Erica Swanson (MT) Part C
• Margy Hornback, Dale Walker, and Charles
  Greenwood (KS) Birth-5

3
Each state will describe

• Approach to COSF quality assurance (QA)
• COSF QA process
• What happens as a result of QA activities
• Issues, challenges, strategies, and lessons
  learned

4
Nevadas approach to COSF quality assurance

• Have a task force that meets regularly with
  representation from all regional programs
• Using a consensus model for decision making
• Have created a matrix for all decisions that have
  been made to date and a plan for how to implement

5
Nevadas COSF QA process

• Comparing COSF data with TRAC 618 data to ensure
  all entries and exits are completed
• Creating reports for regional programs to utilize
  to manage COSF requirements
• Have instituted statewide training
• Just beginning to look at individual COSF forms
  to identify concerns
• Revised COSF form to provide assistance to staff
  in rating

6
What happens in Nevada as a result of your QA
activities?

• Creating reports for regional programs
• Analyzing data for children who do not remain in
  early intervention for six months to see if there
  are trends
• Just beginning to look at exit COSF forms in
  comparison with entry
• Determining if the information on the entry form
  was sufficient to assist with completing an
  accurate exit form
• Next step to create a quality assurance technical
  assistance process to address concerns with
  completion of COSF forms based on review

7
Idahos approach to COSF quality assurance

• Developed training and protocol with key
  stakeholders
• Provided statewide training around the state
• Worked closely with Part B to facilitate COSF
  data sharing

8
Idahos COSF QA process

• Generate reports form DataTot to assure entry and
  exit scores are recorded
• Share reports with Regional Programs
• Planning to conduct a review of data across
  regions to check for significant
  discrepancies/scoring errors

9
Idahos COSF QA process

• Collaborated with Boise State University Early
  Childhood Studies Program to assess the
  reliability of the process
• Scenario created with a brief history, test
  scores, comments from parents, and other
  supporting materials
• Regional teams reviewed the scenario, scored the
  COSF, and provided feedback on the process
• Results reported to Central Office for follow-up

10
What happens in Idaho as a result of your QA
activities?

• Small workgroup convened to recommend process and
  policy revisions
• Proposed revisions shared with SDE (Part B) to
  assure continued compatibility between systems
• Revisions adopted and implemented
• Data will continue to be monitored for
  completeness and quality

11
Montanas approach to COSF quality assurance

• Review each COSF, team COSF procedures, and
  agency COSFs for
• Item completion
• Composition of COSF teams
• Parent/family COSF outcome input
• Number of sources for making each COSF outcome
  rating
• Assessments used for each COSF outcome rating
• Observations/reports used for each COSF rating

12
Montanas COSF QA process

• COSFs compiled by Part C agency and sent to lead
  agency on a quarterly basis
• Review every COSF for an agency for each item on
  the COSF and compile data
• Complete a quarterly report for each agency
• Provide feedback and technical assistance for
  each agency for each quarter

13
What happens in Montana as a result of your QA
activities?

• Provide feedback to each Part C agency,
  especially concerning systemic issues and
  positive support
• Provide general feedback to stakeholders
• Review for state systems issues
• Look for possible future COSF issues (e.g., exit
  ratings)

14
Kansas approach to COSF quality assurance

• Developed web-based application for data entry
• Provided training and technical assistance,
  including documentation of supporting evidence
  guided by decision tree
• Partnered with researchers for data analysis

15
Kansas COSF QA process

• Content analysis of individual child COSF data
• Analysis of data collected in web-based
  application
• Analysis of video tapes
• Survey
• Validity studies

16
What happens in Kansas as a result of your QA
activities?

• Used to improve the quality of the process
  (technically correct and level of quality)
• Refine the web-based application fields
• Improve training and technical assistance
• Refine research questions
• Begin to plan for intervention strategies to
  improve outcomes