Measuring Benefits Planning Services and Outcomes

1
Measuring Benefits Planning Services and Outcomes

• September 20, 2007

2
Session Goals

• MIG-RATS Benefits Counseling Workgroups Overview
• Modeling Benefits Counseling Services and
  Outcomes
• Hands On Developing Measurement Strategies
• Next Steps

3
What is Fidelity Measurement?

• Fidelity adherence to the service protocol as
  originally developed
• Measurement determines the extent to which the
  service
• Includes all key elements
• Excludes extraneous elements
• Delivers elements consistently across programs
  and/or consumers

4
Challenges to Measuring Benefits Counseling
Fidelity

• Wide range of services offered
• No accepted ideal service model
• Very limited access to data on service delivery
• Limited ability to collect new data on benefits
  counseling services

5
Workgroup Fidelity Measurement Strategy

• Use existing understanding and expertise to
  identify key domains/features of benefits
  counseling
• Develop measurement tool (survey) for Benefits
  Counselors/Specialists to identify features of
  services they deliver
• Tool will show services and consistency of
  delivery, but wont define a correct model

6
What is Outcomes Measurement?

• Attempts to determine the impact of benefits
  counseling services on consumers
• Knowledge
• Attitudes
• Behaviors
• Ideally, would eliminate other sources of
  influence

7
Challenges to Measuring Benefits Counseling
Outcomes

• Limited prior demonstrated outcomes
• No research model to follow
• Identifying and accessing data
• Eliminating effects of other potential influences

8
Workgroup Outcomes Measurement Strategy

• Identify potential outcomes based on workgroup
  input
• Identify possible data sources for measuring
  proposed outcomes
• Develop analysis plan for examining relationship
  of outcomes to services

9
Bringing it all together

• A model of benefits counseling service delivery
  and outcomes
• Based on a logic model of benefits counseling
• Shows progress the workgroups have made in
  identifying domains of service delivery and
  potential outcomes of services

10
(No Transcript)
11
The work in working session

• Break into small groups
• Begin (or continue) to operationalize the service
  domains or outcomes
• Define items to measure domains (fidelity)
• Discuss data sources or collection strategies
  (outcomes)

12
Questions for fidelity groups

• What is the full range of topics to cover within
  a domain?
• What questions (and responses) will address the
  topics within a domain?
• Do the proposed domains capture all of the key
  elements of service delivery?
• Note that measures should be developed for
  Benefits Counselors

13
Questions for outcomes groups

• How might the identified outcomes be measured?
• What are potential data sources for these
  outcomes?
• What are the characteristics of the suggested
  data?
• Are there other outcomes that could be considered?

14
Next Steps

• Develop draft of survey instrument for fidelity
  measurement
• Populate spreadsheet with outcomes data source
  suggestions and develop analysis plan
• Joint Workgroup Follow-up call October 2, 230-4
  ET