Innovations in Accountability Federal Government Directions in Strengthening Accountability and the

1
Innovations in AccountabilityFederal Government
Directions in Strengthening Accountability and
the Central Role of EvaluationTerry
HuntA/Senior Director, Centre of Excellence for
Evaluations

• Canadian Evaluation Society --
• New Brunswick Chapter

2
What is Evaluation?

• In depth examination of program or policy
  success, relevance and cost-effectiveness.
• Independent voice to management regarding what
  the department is getting for its money.
• Only function in government that provides
  reliable, independent information on whether
  Government is achieving intended results and
  effectively responding to citizens needs.
• Enables an evidence-based discussion on
  allocation and re-allocation of expenditures.

3
Audit and Evaluation Are Different

• Skills set, methods, practices and scope of
  analysis
• Evaluation assesses program or policy
  effectiveness and delivery efficiency (i.e., a
  deeper look at value for money)
• An evidence-based trigger for more in-depth
  examination of key management and accountability
  issues that may lead to comprehensive audits
• A systematic method to engage Canadians in the
  identification of program need and delivery
• Audit examines management practices whereas
  evaluation looks at results
• Audit looks at how we are doing things
  evaluation examines whether we are doing the
  right things and if we are doing them in a
  cost-effective manner

4
Background Evaluation Practice in the Federal
Government

• Formal evaluation policy in Canada for over 30
  years
• Evaluation seen as key enabler of results-based
  management
• Established standards and role of Office of the
  Auditor General key to longevity

5
Context for Change

• Building Evaluation Capacity
• Policy and Standards took effect April 01, 2001
• Greater TBS support and leadership
• Management standards for evaluation
• Investment strategy for functional units in
  departments
• However, we still have a long way to go
• Budget 2004 Strengthening Public Sector
  Management
• a modern expenditure management and oversight
  system
• Commitment to re-establishment of Office of
  Comptroller General at TBS responsible for the
  oversight of all government expenditures
• increased evaluation activity and capacity

6
Results-based Practices in Support of Evaluation

• Management Accountability Framework (MAF)
• Requiring deputy heads to put in place risk-based
  evaluation plans and a strong evaluation system
• Evaluation provides information to address MAF
  expectations
• Departmental evaluation units to support
  department heads ability to monitor
  implementation and use of MAF
• Program Activity Architectures
• Department-level logic model approach, linking
  programs to strategic outcomes
• Assessing achievement of strategic outcomes,
  using established measures
• Expenditure Review Process
• ERCs assessment of individual programs will
  require the granularity provided by program
  evaluation

7
TBS Management Accountability Framework
8
Benefits Strengthened Transparency,
Accountability and Decision-making

• Clear public performance reporting
• Demonstrates stewardship

CANADIANS / PARLIAMENTARIANS

• Assessment of relative priority of programs and
  activities

EXECUTIVE / CABINET COMMITTEES
DEPARTMENTS / AGENCIESPrograms, Stakeholders
Beneficiaries

• Monitor, report and manage based on results

9
Role of Federal Evaluation Function

• Embeds principles of evaluation into management
  practice (life cycle of programs)
• Evaluation as a facilitator and advocate for
  results-based management
• Broadens scope beyond programsto policies
  initiatives cross-jurisdictions
• Commitment to transparency and management in
  full public view
• Investment strategy for functional units in
  departments
• Centre of Excellence for Evaluation (CEE) within
  TBS created

10
Results-Based Management A Life Cycle Approach
Managing Cultural Change
5
11
Evaluation Policy Objective

• Policy Objective ensuring government has
  timely, strategically focussed, objective and
  evidence-based information on the performance of
  its policies, programs and initiatives to produce
  better results for Canadians.
• Departments should use evaluation to inform/guide
  their decision-making process
• Evaluation can be useful in addressing
  Expenditure Review questions (relevance,
  effectiveness and efficiency)
• Bottom-line Evaluation is a key decision-making
  tool for that can help shape the way government
  designs, delivers, and reports on programs.

12
(No Transcript)
13
Key Activities Required

• Departmental Head of Evaluation must provide
  leadership and direction to the practice of
  evaluation
• strategic, risk-based evaluation plans
• work with managers to help improve program design
  and performance measurement
• inform senior management of issues of concern
• make reports publicly available
• apply evaluation standards
• Departmental managers must manage for results
• draw on the organisations evaluation capacity
• ensure that they have reliable, timely, objective
  and accessible information for decision-making
  and performance improvement

14
Federal Evaluation Process

• 1. Results-based Management and Accountability
  Frameworks
• Blueprint for results-based management
• Provides information on what a policy, program or
  initiative is expected to achieve
• Applies evaluation methodology to strengthen
  accountability and results-based planning
  practices in the design of programs
• Requires identifying performance reporting needs
  at the front end
• 2. Departmental Audit and Evaluation Committee
• 3. Risk-based Departmental Audit and Evaluation
  Plan

15
Effective Evaluation Practices

• Organizational structure
• Environment conducive to results-based management
• A direct line to departmental decision makers
• Level of resources (activity and capacity)
• Planning
• Incorporate evaluation planning in corporate
  planning process
• Evaluation incorporated in program design
• Quality control
• Formal tracking of implementation of evaluation
  recommendations
• Ensuring timeliness of evaluation products
• Use of briefing deck to report preliminary
  findings
• Use targeted studies to address specific
  management issues

16
Preconditions for Using Evaluations for
Decision-making

• Asking The Tough Questions
• Validity Reliability
• Independence and Objectivity
• Accessibility

17
The Role of TBS

• Monitor quality of evaluations
• Feed evaluations into Cabinet decision-making
  (ERIC)
• Annual Report to Treasury Board Government of
  Canadas Evaluation Plan and Evaluations Role
  in Strengthening Evaluation

18
Challenges

• Addressing multiple needs Need to ensure
  function supports deputy, Minister and Centre
• Focus on quality of evaluations
• Cost-effectiveness information
• Moving beyond formative assessments to address
  the central questions What impact did the
  program have? Did it make a difference?
• Building internal capacity
• Following up on use of evaluations

19
(No Transcript)