PERFORMANCE METRICS IN INTEGRATED JUSTICE: Measuring Success and the Need for Improvement

1
PERFORMANCE METRICS IN INTEGRATED
JUSTICEMeasuring Success and theNeed for
Improvement
presented by Bob Roper, CIO Colorado Judicial
Branch Teri B. Sullivan, SEARCH Group
2
DO NOT PANICTHIS IS NOT A STATISTICS COURSE
3
PRESENTATION OBJECTIVES

• DEVELOP A RATIONALE FOR MEASURING CJIS
  OBJECTIVES Why Measure Anything?
• IDENTIFY METHODS FOR GATHERING DATA
• OUTLINE SOME CJIS GOALS AND OBJECTIVES
• SPECIFY SOME CJIS MEASUREMENTS LINKED TO GOALS
  AND OBJECTIVES
• CONVERT DATA INTO INFORMATION DISPLAYING THE
  METRICS
• PROVIDE SOME CAUTIONS IN MEASURING CJIS
  OBJECTIVES
• IDENTIFY SOME ADDITIONAL RESOURCES

4
WHY MEASURE CJIS OBJECTIVES?
5
WHY EVALUATE A CJIS PROJECT?Are We Beyond the
trust me Stage?

• INFORMATION IS CONTROL
• PROVIDES FEEDBACK TO IMPROVE PRGM.
• PROVIDES INFO FOR RESOURCE ALLOCATION
• ENABLES EFFECTIVE PLANNING
• TESTS GENERALIZATIONS vs INDIVIDUAL ASSUMPTIONS
• MARKET TO, AND DEVELOP SUPPORT AMONG FUNDING
  BODIES, CONSTITUENTS, AND STAFF

6
LEGISLATION THAT REQUIRE QUANTIFIABLE OBJECTIVES

• Chief Financial Officers Act (CFO) of 1990
• Government Performance and Results Act (Results
  Act) of 1993
• Federal Acquisition Streamlining Act (FASA) of
  1994
• Paperwork Reduction Act of 1995
• Clinger-Cohen Act of 1996
• Performance Based Budgeting

7
METHODS FORGATHERING DATA
8
MEASURING THE SUCCESS OF THEPROGRAM
Compared to What?

• OTHER PROGRAMS
• THE CURRENT SYSTEM - BASELINING
• OTHER JURISDICTIONS - BENCHMARKING
• OVER TIME

9
USE MORE THAN ONE MEASUREThe Balanced Scoreboard

• CUSTOMER PERSPECTIVE
• INTERNAL BUSINESS PERSPECTIVE
• INNOVATION AND LEARNING PERSPECTIVE
• FINANCIAL PERSPECTIVE

10
INTEGRATED CRIMINAL JUSTICE OBJECTIVES THAT CAN
BE MEASURED
11
DEFINING THE EVALUATION PROCESSHow Do I Know It
Worked?

• GOALS
• Broad Statements of Interest
• PROJECT OBJECTIVES
• Done on Time, Within Budget and Within Specs
• FUNCTIONAL OBJECTIVES
• Did It Perform As Designed?
• MEASURABLE BUSINESS OBJECTIVES
• Why Did We Pursue the Program in the First
  Place?

12
SAMPLE GOALS OF INTEGRATED JUSTICE

• ENHANCE PUBLIC SAFETY
• IMPROVE ACCOUNTABILITY OF THE JUSTICE SYSTEM
• IMPROVE PUBLIC TRUST AND CONFIDENCE IN THE
  JUSTICE SYSTEM
• IMPROVE CASEFLOW MANAGEMENT
• IMPROVE STAFF EFFICIENCIES
• ENHANCE THE QUALITY OF DECISION MAKING

13
CHARACTERISTICS OF MEASURABLE BUSINESS OBJECTIVES

• BE INCREASINGLY SPECIFIC ABOUT THE
  MEASURETHE MORE SPECIFIC, THE MORE RELIABLE THE
  MEASURE
• WHAT IS THE MEASURE?
• WHATS THE OBJECT OF THE MEASURE?
• WHATS THE VALUE OF THE MEASURE FOR THAT OBJECT?
• WHERE IS IT GOING TO BE MEASURED?
• WHEN IS IT GOING TO REACH THAT VALUE?
• BE SURE YOUR MEASURABLE BUSINESS OBJECTIVE IS
  RELATED TO A GOAL.

14
EVOLUTION OF A MEASURABLE BUSINESS
OBJECTIVEGOAL IMPROVE CASEFLOW MGMT

• Reduce the time to disposition
• Reduce the time to disposition of misdemeanor
  cases
• Reduce the time to disposition of misdemeanor
  cases to an average of 6 months
• Reduce the time to disposition of misdemeanor
  cases to an average of 6 months Statewide
• Reduce the time to disposition of misdemeanor
  cases to an average of 6 months Statewide by
  6/30/03

15
EVOLUTION OF A MEASURABLE BUSINESS
OBJECTIVEGOAL IMPROVE QUALITY OF DECISION-MKG

• Reduce the number of false arrests
• Reduce the number of false arrests of felony
  cases
• Reduce the number of false arrests of felony
  cases to less than 4 a month
• Reduce the number of false arrests of felony
  cases to an average of 6 months in Denver
• Reduce the number of false arrests of felony
  cases to an average of 6 months Statewide by
  9/30/02

16
EVOLUTION OF A MEASURABLE BUSINESS
OBJECTIVEGOAL IMPROVE PUBLIC SAFETY

• Increase the percent of court dispositions that
  match to an arrest
• Increase the percent of court dispositions that
  match to an arrest in felony cases
• Increase the percent of court dispositions that
  match to an arrest in felony cases to 80
• Increase the percent of court dispositions that
  match to an arrest in felony cases to 80
  Statewide
• Increase the percent of court dispositions that
  match to an arrest in felony cases to 80
  Statewide by 6/30/02

17
SOME MEASURABLE OBJECTIVES FOR INTEGRATED
JUSTICE GOALS
18
THESE ARE ONLY THE BASIC MEASURES THAT NEED TO BE
EXPANDED AS DESCRIBED IN THE PRECEDING 3 EXAMPLES
IN ORDER TO BECOME FULL MEASURABLE OBJECTIVES.
19
GOAL ENHANCE PUBLIC SAFETYSome Measurable
Objectives Are

• Increase the of court dispositions that match
  to an arrest incident
• Decrease the of times court cases are not
  reported in criminal histories
• Decrease the average response time it takes to
  receive a positive identification
• Decrease the of incidents where the wrong
  person is released from custody
• Decrease the of incidents where criminal
  records are associated with the wrong person
• Decrease the recidivism rate
• Reduce the of escapes during prisoner transport
• Decrease the discrepancy between active
  warrants on the court system and those on the
  State Registry
• Decrease the amount of time it takes to issue
  victim notices

20
GOAL IMPROVE ACCOUNTABILITYSome Measurable
Objectives Are

• Increase the of Mgmt Reports available to the
  public on the WWW
• Increase the of hits on the CJIS Web page
• Increase the hrs the general public views data
  electronically

21
GOAL IMPROVE PUBLIC TRUST/CONFIDENCESome
Measurable Objectives Are

• Reduce the hrs it takes to respond to a request
  from the general public
• Reduce the minutes it takes to complete a
  criminal history background check
• Increase the of the general public that trusts
  the courts
• Increase the of the general public that is
  satisfied with local law enforcement
• Reduce the of civil complaints against local
  law enforcement

22
GOALIMPROVE CASEFLOW MGMTSome Measurable
Objectives Are

• Reduce the of continuances per case originating
  from scheduling conflicts
• Reduce the of cases without a next scheduled
  event
• Increase the of hearings held as scheduled
• Reduce the average days from arrest to
  arraignment
• Reduce the average days held waiting for bond
  decisions
• Reduce the days to process cases from arrest to
  disposition
• Reduce the processing time required to process
  inmates at the Department of Corrections

23
GOAL ENHANCE STAFF EFFICIENCIESSome Measurable
Objectives Are

• Reduce the hrs staff spends entering data
  electronically
• Reduce number of hrs to collect data manually
• Reduce of phone-calls among governmental
  entities
• Reduce the costs of copying paper for other
  governmental entities
• Reduce the costs of storing paper
• Reduce the costs of buying paper supplies
• Reduce the number of hrs spent filing paper
  manually
• Reduce the mailing costs among governmental
  entities
• Reduce the of hrs spent searching other
  automated systems
• Reduce the of hrs spent by staff looking for
  hard copy files
• Increase the of law enforcement personnel that
  can be reassigned from a desk to the street
• Increase the of data transfers among the
  agencies
• Reduce the costs associated with producing forms

24
GOAL IMPROVE DECISION MAKING QUALITYSome
Measurable Objectives Are

• Reduce the of false arrests
• Reduce the of hrs it takes to enter a court
  disposition into the State Criminal History
  Repository
• Reduce the amount of missing data among the
  governmental entities
• Reduce the of data errors reported for
  correction
• Decrease the amount of time it takes to enter a
  warrant into the State Warrant Registry
• Decrease the amount of time it takes to enter a
  protective order into the Statewide Protective
  Order Registry
• Increase the of query hits among the agencies
• Reduce the of inaccurate background checks

25
DISPLAYING AND PUBLISHING THE METRICS
26
In order to reach the goals and objectives, one
needs to measure and publish because, People
behave based on how they are measured.
27
DISPLAYING AND PUBLISHING THE RESULTSCONVERTING
DATA INTO INFORMATION

• CONVERT DATA INTO PICTURES AND GRAPHICS
• COLOR HELPS TO HIGHLIGHT YOUR POINTS
• K.I.S.S. YOUR DISPLAY
• PUBLISH REGULARLY
• DONT OVERWHELM WITH DATA

28
GRAPHIC DISPLAY OF DATADISPOSITION MATCHING IN
COLORADO
29
GRAPHIC DISPLAY OF DATAWARRANT TRANSFER IN
COLORADOA PICTURE IS WORTH A THOUSAND WORDS
30
GRAPHIC DISPLAY OF DATARANKING LEADS TO
IMPROVEMENT
31
GRAPHIC DISPLAY OF DATABENCHMARKING AGE OF
CRIMINAL CASES
32
CAUTIONS IN MEASURING CJIS OBJECTIVES
33
THINGS TO AVOID

• TOO COMPLEX
• GENERALIZING FROM LIMITED DATA
• CANNOT COLLECT ACCURATE OR COMPLETE DATA
• TOO MANY MEASURES
• SPURIOUS RELATIONSHIPS
• ASSUMPTIONS
• BE CAREFUL THE STATS YOU PUBLISH WILL BE PUBLIC

34
THINGS TO DO

• BE SURE TO ID THINGS/CONCEPTS THAT ARE IMPORTANT
  AND CAN BE MEASURED
• PILOT THE MEASURES
• RECEIVE FEEDBACK
• MODIFY MEASURES
• REASSESS SUCCESS
• BROADCAST SUCCESS GET AN AGENT
• DONT FORGET TO TALK ABOUT SOME THINGS THAT CANT
  BE MEASURED
• MEASURE THE INTEGRITY/QUALITY OF DATA

35
COPING WITH DATA QUALITY

• DATA RELIABILITY
• COMPLETE
• ACCURATE
• TIMELY
• DATA VALIDITY Are we measuring what we intend
  to measure?
• DECIDE ON DAY FORWARD OR CONVERSIONSTHEN
  ANNOUNCE
• INSERT DATA EDITS
• PUBLISH THE RESULTS
• GET UPPER LEVEL MANAGEMENT SUPPORT
• PROVIDE MECHANISMS FOR CORRECTING DATA

36
MEASURES SHOULD BE

• SPECIFIC
• MEASURABLE
• ACTIONABLE
• RELEVANT
• TIMELY

37
RESOURCES

• Executive Guide Measuring Performance and
  Demonstrating Results of Information Technology
  Investments GAO/AIMD-98-89
• Performance Measurement for Government,
  Introduction to Performance Measurement,
  http//accounting.rutgers.edu/raw/seagov/pmg/perfm
  easures
• Manny DeVera, Putting Performance Measures and
  Capital Planning Into Practice, pg. 1 25,
  March 2001 http//wwwoirm.nih.gov/itmra/perform.ht
  ml
• Robert S. Kalpan and David P. Norton, The
  Balanced Scorecard Measures that Drive
  Performance, Harvard Business Review, Jan-Feb
  1993
• Dave Trimble, How to Measure Success Uncovering
  the Secrets of Effective Metrics, pg. 1- 7,
  http//www.prosci.com/metrics.htm
• Dave Trimble, Benchmarking Uncovering Best
  Practices and Learning from Others, pg. 1 8,
  http//www.prosci.com/benchmarking.htm
• Mark Glover, A Practical Guide for Measuring
  Program Efficiency and Effectiveness in Local
  Government, Tampa, FL The Innovation Groups,
  1994
• Office of Management and Budget, OMB Circular
  No. A-11 (2000)
• Performance Measures for the Criminal Justice
  System, Discussion Papers from the BJS-Princeton
  Project, October 1993, NCJ-143505