From Outputs to Outcomes: Measuring Results of Enforcement and Compliance Activities National Environmental Partnership Summit May 10, 2006

1
From Outputs to OutcomesMeasuring Results of
Enforcement and Compliance ActivitiesNational
Environmental Partnership SummitMay 10, 2006

• Lisa Lund
• Deputy Director
• Office of Compliance
• U.S. EPA

2
Evolution of Measurement in OECA

• In the past, emphasis on activity counts and
  direct results of activity (outputs) per unit of
  time
• Number of inspections
• Number of administrative orders, referrals, etc.
• Dollar value of penalties assessed, years of
  incarceration, etc.

3
Evolution of Measurement in OECA

• Today, emphasis on data for planning, managing
  and evaluating
• Need a combination of outputs and outcomes
• Change in behavior of regulated entity as result
  of compliance assistance
• Number of pounds of pollutant reduced as result
  of enforcement actions
• Dollar value of health benefits achieved by
  pollutant reductions

4
Outcome Measures - Definitions

• Outcomes reflect the results of outputs
• Intermediate outcome measures progress toward a
  final outcome
• Intermediate outcome of enforcement action
  change in facility management practices
• Final outcome measures ultimate result the
  program is designed to achieve
• Final outcome of enforcement action reduction
  in number of people living in area where
  pollutant standards were exceeded

5
Hierarchy of Indicators
6
Steps to Improve Outcome Measurement

• OECA measurement program review
• Exploring use of hazard and exposure data
• Seeking research on measuring deterrent effect
• Engaging with international enforcement and
  compliance programs
• Moderated two international discussion forums
  seeking input on best practices around the globe

7
Steps to Improve Outcome Measurement

• Environmental Assistance Network - developing
  common measures to demonstrate results under Goal
  5 of EPA Strategic Plan
• State Review Framework ensuring consistency
  among states in implementing enforcement and
  compliance programs, improving data quality
  within and among states
• Element 13 developing method for getting
  outcome information on Elements 1-12

8
Environmental Assistance Network

• Help EPA better plan, coordinate, and measure its
  environmental assistance efforts
• Achieve the maximum efficiency and effectiveness
  in helping organizations improve their
  environmental performance
• Present a more organized, cohesive face to
  external stakeholders
• More effectively demonstrate results under Goal 5
  of the Agencys strategic plan

9
Common Measures for Goal 5

• Entities reached by environmental assistance (EA)
  activities
• Entities reporting increased understanding as a
  result of EA activities
• Entities reporting changes in behavior as a
  result of EA activities
• Entities reporting a reduction in pollution/
  environmental impact as a result of EA activities
• Entities reporting reduced operating costs as a
  result of EA activities

10
EA Network Sector Focus Area

• Construction sector selected as the first EA
  Network focus area. Sector activities may
  include
• identifying metrics of common interest or gaps in
  measurement
• proposing and tracking standard cross-program
  metrics
• identifying where standard metrics may be applied
  (e.g., in OECAs commitment system supporting
  national priorities) and promoting their use

11
State Review Framework

• Tool to gauge state performance in compliance
  assurance
• Developed jointly by ECOS Compliance Committee,
  EPA OECA, regions and states
• Provides consistency in environmental protection,
  consistent level of activities in states and a
  consistent level of oversight by EPA regions
• Based on 12 elements and a 13th optional element
• Uses common set of output measures to gauge
  performance

12
Element 13

• Provides mechanism for state to share outcome
  information related to Elements 1-12
• Allows states to
• Seek recognition from EPA that innovative program
  is important part of its core program
• Seek reduction in core program activities in
  order to conduct alternative/innovative program
  that leads to improving compliance
• Data must demonstrate outcomes that comport with
  Agency goals

13
Challenges

• Inconsistent data quality
• Effective outcome measures depend on complete and
  accurate reporting by states and regions
• Requires common set of measures
• Resource intensive
• Complex and time-consuming design and
  development require careful consideration of
  numerous criteria
• Scope of regulated universe makes development of
  overall compliance rate infeasible
• 11 statutes x 28 programs x possibly hundreds of
  points of compliance x thousands of regulated
  entities
• Few environmental and human health models
  appropriate for our use exist

14
Conclusion

• Viability and future funding of our programs
  depend on robust performance measurement, using
  outcomes to the greatest extent possible
• Weve made great progress, more work is needed

15
Contact Information

• Lisa C. Lund
• Deputy Director
• Office of Compliance
• U.S. Environmental Protection Agency
• [phone]
• lund.lisa_at_epa.gov