Getting Your HMIS Data to the Next Level: Preparing for Program Reporting, Evaluation and Monitoring - PowerPoint PPT Presentation

Loading...

PPT – Getting Your HMIS Data to the Next Level: Preparing for Program Reporting, Evaluation and Monitoring PowerPoint presentation | free to view - id: 4516f-ZDc1Z



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Getting Your HMIS Data to the Next Level: Preparing for Program Reporting, Evaluation and Monitoring

Description:

Utilization Rates the canary in the coal mine. Data Quality Checks Big brother is watching. Preparing Data for Program Analysis do you have a dog in this fight? ... – PowerPoint PPT presentation

Number of Views:60
Avg rating:3.0/5.0
Slides: 34
Provided by: paulasch4
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Getting Your HMIS Data to the Next Level: Preparing for Program Reporting, Evaluation and Monitoring


1
Getting Your HMIS Data to the Next Level
Preparing for Program Reporting, Evaluation and
Monitoring
  • Matt White,
  • Abt Associates Inc.

2
Learning Objectives
  • To give HMIS project managers and program staff
    practical tips on improving data quality
  • To learn how to monitor data quality through
    utilization rates, custom reports and regular
    user meetings.
  • To learn of management techniques to prepare HMIS
    data for evaluation and monitoring questions.

3
Overview
  • Utilization Rates the canary in the coal mine
  • Data Quality Checks Big brother is watching
  • Preparing Data for Program Analysis do you have
    a dog in this fight?

4
What are Bed Utilization Rates?
  • The bed utilization rate is the percentage of
    beds that are used on a particular day or the
    percentage of beds that are used on an average
    day during a particular time period.
  • Utilization is calculated as follows
  • of clients served on an average day
  • of beds available
  • Bed Utilization Rate

5
Bed Utilization Rates An Example
  • For example, if a community serves 180
    individuals in emergency shelters on an average
    day during a particular month and has 200 beds
    available for emergency shelters serving
    individuals, the bed utilization rate is
  • 180 200 90
  • Bed utilization rates can be calculated for a
    single provider, an entire community, or any
    other level of aggregation.
  • For non-residential providers, a similar
    utilization rate can be calculated in terms of
    service slots.

6
Inaccurate Bed Utilization Rates Means
Inaccurate Information for Making Policy Decisions
  • Inaccurate bed utilization rates means your
    program does not know
  • The number of homeless people using shelters
  • The number of days people are using shelters
  • The characteristics of sheltered homeless people
    served on a particular day or period of time
  • System wide what types of providers need more
    capacity and what types have extra capacity.

7
Reasons for Excessively High Utilization Rates
(over 100)
  • Missing exit dates
  • Missing exit dates for people who leave the
    shelter leads to an over-count of homeless people
    served on a particular day or time period.
  • Inaccurate, low bed counts
  • Inadequate deduplication (affects over time
    counts and average daily utilization based on
    over time counts)
  • Accounting for people who use overflow beds, but
    not the beds

8
Reasons for Excessively Low Utilization Rate
(below 50)
  • Not entering information on everyone served in
    HMIS leading to undercount of the number of
    people served.
  • Inaccurate, high bed counts
  • For family programs, not all beds in occupied
    unit may be used

9
Suggestions for Improving Accuracy of Counts of
Number of People Served
  • Regularly review utilization rates by program and
    ask for confirmation of number of people served
    for unusual rates (see sample worksheet)
  • Compare utilization rates across similar
    providers to identify possibly erroneous rates.
  • Use the counts and utilization rates for funding,
    policy decisions, and evaluations. When people
    see the numbers are important to decisions, they
    have and incentive to ensure they are accurate.
  • Compare the number of people served from HMIS to
    point-in-time count reported for CoC Application
    (if from a different source)

10
More Suggestions for Improving Accuracy of Counts
of Number of People Served
  • Match people listed in HMIS to bed logs if
    separate logs are kept.
  • Automatically enter exit dates for overnight
    shelters.
  • Identify specific records for confirmation if
    length of stay seems excessive.
  • If long-term problem, exit everyone and have
    provider re-enter only those persons still being
    served.
  • Design system so persons cannot be listed as
    staying at two residential providers at the same
    time.
  • Increase training of front-line staff who enter
    the information.

11
Suggestions for Improving Accuracy of Number of
Beds Available
  • Regularly review utilization rates by program and
    ask for confirmation of bed inventory for unusual
    rates (see sample worksheet)
  • Calculate bed utilization rate with overflow beds
    if appropriate
  • Make sure CoC Housing Inventory information is
    internally consistent.
  • Some apps have family units listed, but not beds
  • Others have of beds participating in HMIS, but
    nothing in column for total of beds available
  • Some list providers that do not serve homeless
    people
  • Some have wrong geocode for location of program
  • Aim for 100 accuracy of information needed for
    deduplication

12
Worksheet to Monitor Bed Utilization Rates Monthly
  • Include seasonal beds in the denominator if
    they are open during the month.

13
Utilization Wrap Up
  • Inaccurate bed utilization rates point to
    problems in the count of the number of sheltered
    homeless people or to inaccurate counts of beds
    available to serve homeless people
  • Accurate utilization rates are important they
    can help you determine whether you have too few
    or too many resources for serving certain types
    of residential providers
  • They also point to problems in all the
    information you have on sheltered homeless people
  • You can address the issue the main emphasis has
    to be on entering information on everyone who is
    served and exiting people no longer being served

14
Data Quality Checks
Timeliness Completeness Accuracy Consistency
15
Ensuring Quality HMIS Data
  • Monitor program activity
  • Clients served, services provided, needs, etc.
  • Monitor client-level completion rates
  • Clients without service transactions, DOBs,
    disability status,program entry dates or other
    Universal Data Elements
  • Compare data from different reports
  • APR vs. Clients Served Report

16
Identifying Data Inconsistencies
  • Run reports that are clear and concise
  • Be familiar with data being entered and reported
  • Entries should be done in real time
  • Use data quality reports to track user entries
  • Disability incongruities (yes, but no type)
  • Chronic homeless (flag doesnt match LOS)
  • Zip codes (no quality indicator)

17
Potential User Entry Problems
  • Overload on users from data entry backlogs
  • Users who are not computer literate
  • Users who do not understand HMIS at the
    beginning/training level
  • Users who rush through data entry
  • Lack of communication between users, System
    Administrators and HMIS providers
  • Users who dont have a dog in the fight

18
Solving User Entry Challenges
  • Be selective in staff who will be entering data
    at the beginning
  • Integrate program functions with HMIS data entry
    functions
  • Hire additional data entry support (or use
    existing personnel)
  • Use specific staff for data entry to reduce
    errors
  • Use/hire additional staff during peak times

19
Solving User Entry Challenges
  • Constant communication done at training level
    will ensure a comfort level with HMIS
  • Re-train individuals until comfortable with HMIS
  • Run reports on individual users to ensure quality
    data entry
  • Run random reports that match up with paper
    trails
  • Follow up with client entries with users that are
    new to system

20
Preparing Data for Program Analysis
  • Fitness for Use Tests

21
Fitness for Use
  • Completeness
  • Information is entered on all consumers
  • Information on the consumer is complete
  • Accuracy
  • Data reflects reality
  • Data is entered correctly
  • Data has face validity reflects what we know
  • Consistency
  • Performance information is consistent over time

22
Fitness for Use
  • Program staff reviews reports monthly for
    completeness, accuracy and consistency.
  • Clear protocols for correcting data.
  • Agency signs off on reports monthly
  • Errors systematically result in corrective action
  • Procedures for correcting are defined
  • Software has error checking functions (out of
    range, missing values, incongruous data).
  • Staff look at data reliability and validity
    issues prior to data analysis and publishing
    reports.

23
The Next Step
  • Preparing for Reporting, Evaluation, and
    Monitoring

24
Process
  • Define research/monitoring question
  • Identify necessary data variables required for
    analysis
  • Determine exceptions, limits, exclusions
  • Define query and run report
  • Review/test results

25
Reporting, Evaluation, Monitoring
  • Prepare data for the following analysis
  • Is the male Veteran LOS longer than the average
    LOS for my program?
  • What of families have a positive exit outcome
    within 90 days?

26
Is the male Veteran LOS longer than the average
LOS for my program?
  • Universal Data Elements
  • Program entry/exit
  • Gender
  • Race/Ethnicity
  • Date of birth to calculate age
  • Disability status
  • Program Level Data Elements
  • Employment/Income/Non-Cash Benefits
  • Education
  • Health/Physical and/or Developmental
    Disability/HIV/AIDS
  • Veteran status
  • Residence prior to
  • program entry
  • ZIP of last permanent
  • address
  • Mental Health/
  • Substance Abuse/
  • Domestic Violence
  • Services Received
  • Reason for Leaving and
  • Destination

27
Is the male Veteran LOS longer than the average
LOS for my program?
  • Universal Data Elements
  • Program entry/exit
  • Gender
  • Race/Ethnicity
  • Date of birth to calculate age
  • Disability status
  • Program Level Data Elements
  • Employment/Income/Non-Cash Benefits
  • Education
  • Health/Physical and/or Developmental
    Disability/HIV/AIDS
  • Veteran status
  • Residence prior to
  • program entry
  • ZIP of last permanent
  • address
  • Mental Health/
  • Substance Abuse/
  • Domestic Violence
  • Services Received
  • Reason for Leaving and
  • Destination

28
Is the male Veteran LOS longer than the average
LOS for my program?
29
Is the male Veteran LOS longer than the average
LOS for my program?
30
Is the male Veteran LOS longer than the average
LOS for my program?
  • Male Veteran LOS 16.5 (round up to 17)
  • Ave LOS 25
  • Answer No, the male Veteran LOS is 8 days less
    than the average of all other program
    participants.

31
What of families have a positive exit outcome
within 90 days?
32
What of families have a positive exit outcome
within 90 days?
33
What of families have a positive exit outcome
within 90 days?
  • Answer 22 (2 of 9) families had a positive
    outcome within 90 days.
About PowerShow.com