Performance Management and Measurement: Balanced scorecard and other tools

1
Performance Management and Measurement
Balanced scorecard and other tools
David Akopyan OPB/BOM October 2007
2
Way forward proposals under discussion in
the context of of Strategic Plan and Enhanced
Results Based Management project. 1.
Performance Management/ Measurement and RBB2.
PM in 2007 BSC new indicators,3. Recent BSC
usage survey4. Way forward for Performance
management options Strategic/ operational BSC
September 2007 workshop in NY.
3
Why performance management system?

• Less than 10 of strategies effectively
  formulated are effectively executed. For
  majority of failures it isnt bad strategy -its
  bad execution.
• Fortune magazine
• If you want your organization to focus on
  strategy, make sure that your management systems
  are designed to manage strategy.
• D Norton

4
PERFORMANCE MANAGEMENT SYSTEM

• Targets and indicators - linked to results and
  objectives, measurable and well defined clear
  for monitoring and reporting also hierarchy and
  links
• The business units/owners - to define, plan,
  target, report and held accountable
• The systems and tools for measure, monitor and
  report.
• Data integration and the linkages among
  corporate, regional, functional units and
  country offices.

5
Performance Management System for Strategic Plan
LMS
Results Reporting
Strategic Plan Drive for Results
Lyris
CARDS
Balanced Scorecard, CO scans, Executive snapshot
Communications Office DB
Snapshot
Surveys GSS Partners survey HQPSS
ATLAS
6
Performance Monitoring Architecture
Strategic Plan
Corporate Balanced Scorecard
Unit Level Balanced Scorecard
RBB / Unit Work Plan
RCA Plan

7
Cascading Down Performance Management Structure
Report to the ExB
Strategic Plan 2008-2011
Corporate Scorecard Corporate Core Indicators
Regional Indicators Regional Scorecard Corporate Core Indicators Regional Indicators
CO Indicators CO Indicators Office Scorecard Corporate Core Indicators CO Indicators CO Indicators
Individual Scorecard RCA Individual Scorecard RCA Individual Scorecard RCA Individual Scorecard RCA Individual Scorecard RCA
Results
Strategy / Goals
8
Performance Monitoring Assessment
Indicators, targets, and baseline data are
derived from the unit Work plan.
1
Result Indics Performance
Expenditure Target Baseline
Actual Planned Actual F1


RBB-MR1
Indic1 75 Indic2
100 RBB-MR2 Indic3
75 F3
RBB-MR5 Indic4 95
Indic5 50
Actual performance is measured using multiple
data sources depending on the specific indicator.
2
Planned expenditure retrieved from ATLAS budget
allocation.
3
Actual expenditure retrieved from ATLAS Finance.
4
2
3
4
1
Multiple Sources (Snapshot, ATLAS, LMS, Excel
etc)
RBB-MR1 F1 RBB-MR2
F1 RBB-MR5 F3
spent F1 spent F3
F1 budgeted F3 budgeted
Budget Allocation ATLAS PBM
Financial ExpenditureATLAS PM
UNIT Work Plan
9
ERBM System Design - Proposal

Operational Data
BSC Unit Work Plan Results
Development Data
Corporate
2008-11 Angola CO Balanced Scorecard
Strategic View / Choices(Strategic BSC)
Unit (CO/RC/HQ unit)
2008 RBB / Unit Work Plan Results
Function RBB Results 2008 Deliverables Budgeted Spent
Function 2 Result 2.1 OOOOO XXXXX XXXXX XXX
Function 4 Result 4.3 OOOOO YYYYY XXXXX XXXX
Function 10 Result 10.3 OOOOO ZZZZZ XXXXX XX
Function 13 Result 13.1 OOOO SSSSSSS XXXXX X
Function 17 Result 17.2 OOOOO QQQQQQ XXXXX XXXX
Individual (RCA)
Management View BSC for RBB
Customized Data
Remove Data
10
UNDP Balanced Scorecard
Balanced Scorecard? - Management System
translating vision and strategy into action
and a) to drive the change process and b) to
align organization behind the strategy.

• In Each Perspective
• Strategic Objectives
• Targets
• Results

11
BSC new indicators in 2007
Clients and partners satisfaction - Government satisfaction, UN, IFI, other donors satisfaction website updated and reflect UNDP priorities (new criteria) - Number of UN joint programmes Internal Efficiency - Implementation of audit recommendations, - NEX audit management, - Adm. to total ratio, Financial data quality (new criteria) Sound project management and monitoring in ATLAS MOSS compliance
Organization capacity - staff perception, - female/total ratio - Learning programme participation - Knowledge network participation - Knowledge sharing b/w COs Financial Stewardship Volumes of delivery Volumes of non-core RM Cost Recovered From Programme Country Cost Sharing Cost Recovered From Trust Funds and Third-party Cost Sharing policy compliance, b) actual collection?
12
Sound Project Management and Monitoring in Atlas
Perspective Internal efficiency Internal efficiency Internal efficiency Internal efficiency Internal efficiency Internal efficiency Internal efficiency
Strategic Objectives Strengthen capacity in project management and monitoring Strengthen capacity in project management and monitoring Strengthen capacity in project management and monitoring Strengthen capacity in project management and monitoring Strengthen capacity in project management and monitoring Strengthen capacity in project management and monitoring Strengthen capacity in project management and monitoring
Definition Index (0-12 scale, 1-9 mandatory and 3 additional bonus) resulting from assessment of the use of the Project Management Section in Atlas for a sample of development projects. Index (0-12 scale, 1-9 mandatory and 3 additional bonus) resulting from assessment of the use of the Project Management Section in Atlas for a sample of development projects. Index (0-12 scale, 1-9 mandatory and 3 additional bonus) resulting from assessment of the use of the Project Management Section in Atlas for a sample of development projects. Index (0-12 scale, 1-9 mandatory and 3 additional bonus) resulting from assessment of the use of the Project Management Section in Atlas for a sample of development projects. Index (0-12 scale, 1-9 mandatory and 3 additional bonus) resulting from assessment of the use of the Project Management Section in Atlas for a sample of development projects. Index (0-12 scale, 1-9 mandatory and 3 additional bonus) resulting from assessment of the use of the Project Management Section in Atlas for a sample of development projects. Index (0-12 scale, 1-9 mandatory and 3 additional bonus) resulting from assessment of the use of the Project Management Section in Atlas for a sample of development projects.
Target Corporate Corporate Corporate Reporting Reporting OSG/OPB/CBS/BDP OSG/OPB/CBS/BDP
Data Source Atlas Atlas Atlas Owner Owner OSG OSG
Rating gt8 8 5 5 gt
Note Evaluation criteria 1 - Risks and Issues entered and followed 2 - Output baseline, targets and results entered 3 - Activity purpose, description, and Quality criteria, method and assessment entered Note Evaluation criteria 1 - Risks and Issues entered and followed 2 - Output baseline, targets and results entered 3 - Activity purpose, description, and Quality criteria, method and assessment entered Note Evaluation criteria 1 - Risks and Issues entered and followed 2 - Output baseline, targets and results entered 3 - Activity purpose, description, and Quality criteria, method and assessment entered Note Evaluation criteria 1 - Risks and Issues entered and followed 2 - Output baseline, targets and results entered 3 - Activity purpose, description, and Quality criteria, method and assessment entered Note Evaluation criteria 1 - Risks and Issues entered and followed 2 - Output baseline, targets and results entered 3 - Activity purpose, description, and Quality criteria, method and assessment entered Note Evaluation criteria 1 - Risks and Issues entered and followed 2 - Output baseline, targets and results entered 3 - Activity purpose, description, and Quality criteria, method and assessment entered Note Evaluation criteria 1 - Risks and Issues entered and followed 2 - Output baseline, targets and results entered 3 - Activity purpose, description, and Quality criteria, method and assessment entered Note Evaluation criteria 1 - Risks and Issues entered and followed 2 - Output baseline, targets and results entered 3 - Activity purpose, description, and Quality criteria, method and assessment entered
13
NEX Audit management
Perspective Internal Efficiency Internal Efficiency Internal Efficiency Internal Efficiency Internal Efficiency Internal Efficiency Internal Efficiency
Strategic Objective Enhance oversight and accountability Enhance oversight and accountability Enhance oversight and accountability Enhance oversight and accountability Enhance oversight and accountability Enhance oversight and accountability Enhance oversight and accountability
Definition indicator composed from 4 below sub-indicators materiality of net financial audit qualifications severity of audit results adequacy of audit scope appropriate administration of audit exercise. indicator composed from 4 below sub-indicators materiality of net financial audit qualifications severity of audit results adequacy of audit scope appropriate administration of audit exercise. indicator composed from 4 below sub-indicators materiality of net financial audit qualifications severity of audit results adequacy of audit scope appropriate administration of audit exercise. indicator composed from 4 below sub-indicators materiality of net financial audit qualifications severity of audit results adequacy of audit scope appropriate administration of audit exercise. indicator composed from 4 below sub-indicators materiality of net financial audit qualifications severity of audit results adequacy of audit scope appropriate administration of audit exercise. indicator composed from 4 below sub-indicators materiality of net financial audit qualifications severity of audit results adequacy of audit scope appropriate administration of audit exercise. indicator composed from 4 below sub-indicators materiality of net financial audit qualifications severity of audit results adequacy of audit scope appropriate administration of audit exercise.
Target Corporate Corporate Corporate Reporting Reporting Automated Automated
Data Source OAPR analysis OAPR analysis OAPR analysis Owner Owner OAPR OAPR
Rating 2.34- 3.00 1.67- 2.33 1.00-1.66
Note OAPR will review the annual NEX audit both the reports and the process in each country office and assign a rating based on 4 dimensions measured 40 - materiality of net financial audit qualifications, 30 -severity of audit results, 20 - adequacy of audit scope and 10 - appropriate administration of audit exercise.  After the completion of the year additional 5-6 months needed to complete the process. Recruitment of audit company, audit report and submission to CO and after to OAPR, OAPR rating. The overall implementation rate is then published in the Balanced Scorecard in June of the next after the year reviewed. Note OAPR will review the annual NEX audit both the reports and the process in each country office and assign a rating based on 4 dimensions measured 40 - materiality of net financial audit qualifications, 30 -severity of audit results, 20 - adequacy of audit scope and 10 - appropriate administration of audit exercise.  After the completion of the year additional 5-6 months needed to complete the process. Recruitment of audit company, audit report and submission to CO and after to OAPR, OAPR rating. The overall implementation rate is then published in the Balanced Scorecard in June of the next after the year reviewed. Note OAPR will review the annual NEX audit both the reports and the process in each country office and assign a rating based on 4 dimensions measured 40 - materiality of net financial audit qualifications, 30 -severity of audit results, 20 - adequacy of audit scope and 10 - appropriate administration of audit exercise.  After the completion of the year additional 5-6 months needed to complete the process. Recruitment of audit company, audit report and submission to CO and after to OAPR, OAPR rating. The overall implementation rate is then published in the Balanced Scorecard in June of the next after the year reviewed. Note OAPR will review the annual NEX audit both the reports and the process in each country office and assign a rating based on 4 dimensions measured 40 - materiality of net financial audit qualifications, 30 -severity of audit results, 20 - adequacy of audit scope and 10 - appropriate administration of audit exercise.  After the completion of the year additional 5-6 months needed to complete the process. Recruitment of audit company, audit report and submission to CO and after to OAPR, OAPR rating. The overall implementation rate is then published in the Balanced Scorecard in June of the next after the year reviewed. Note OAPR will review the annual NEX audit both the reports and the process in each country office and assign a rating based on 4 dimensions measured 40 - materiality of net financial audit qualifications, 30 -severity of audit results, 20 - adequacy of audit scope and 10 - appropriate administration of audit exercise.  After the completion of the year additional 5-6 months needed to complete the process. Recruitment of audit company, audit report and submission to CO and after to OAPR, OAPR rating. The overall implementation rate is then published in the Balanced Scorecard in June of the next after the year reviewed. Note OAPR will review the annual NEX audit both the reports and the process in each country office and assign a rating based on 4 dimensions measured 40 - materiality of net financial audit qualifications, 30 -severity of audit results, 20 - adequacy of audit scope and 10 - appropriate administration of audit exercise.  After the completion of the year additional 5-6 months needed to complete the process. Recruitment of audit company, audit report and submission to CO and after to OAPR, OAPR rating. The overall implementation rate is then published in the Balanced Scorecard in June of the next after the year reviewed. Note OAPR will review the annual NEX audit both the reports and the process in each country office and assign a rating based on 4 dimensions measured 40 - materiality of net financial audit qualifications, 30 -severity of audit results, 20 - adequacy of audit scope and 10 - appropriate administration of audit exercise.  After the completion of the year additional 5-6 months needed to complete the process. Recruitment of audit company, audit report and submission to CO and after to OAPR, OAPR rating. The overall implementation rate is then published in the Balanced Scorecard in June of the next after the year reviewed. Note OAPR will review the annual NEX audit both the reports and the process in each country office and assign a rating based on 4 dimensions measured 40 - materiality of net financial audit qualifications, 30 -severity of audit results, 20 - adequacy of audit scope and 10 - appropriate administration of audit exercise.  After the completion of the year additional 5-6 months needed to complete the process. Recruitment of audit company, audit report and submission to CO and after to OAPR, OAPR rating. The overall implementation rate is then published in the Balanced Scorecard in June of the next after the year reviewed.
14
Security and MOSS compliance
Perspective Internal Efficiency Internal Efficiency Internal Efficiency Internal Efficiency Internal Efficiency Internal Efficiency Internal Efficiency
Strategic Objectives Staff security ensured Strategic Plan output 3 Enabling environment for safe programme delivery ensured Strategic Plan output 5 Staff security ensured Strategic Plan output 3 Enabling environment for safe programme delivery ensured Strategic Plan output 5 Staff security ensured Strategic Plan output 3 Enabling environment for safe programme delivery ensured Strategic Plan output 5 Staff security ensured Strategic Plan output 3 Enabling environment for safe programme delivery ensured Strategic Plan output 5 Staff security ensured Strategic Plan output 3 Enabling environment for safe programme delivery ensured Strategic Plan output 5 Staff security ensured Strategic Plan output 3 Enabling environment for safe programme delivery ensured Strategic Plan output 5 Staff security ensured Strategic Plan output 3 Enabling environment for safe programme delivery ensured Strategic Plan output 5
Definition 1. MOSS compliance for main offices for 2007, and from late 2007 introduce a quantified system to measure MOSS compliance 2. MOSS compliance for sub offices and project offices from 2008 3. Security risk management mainstreamed into UNDP programming from 2008 and beyond. 1. MOSS compliance for main offices for 2007, and from late 2007 introduce a quantified system to measure MOSS compliance 2. MOSS compliance for sub offices and project offices from 2008 3. Security risk management mainstreamed into UNDP programming from 2008 and beyond. 1. MOSS compliance for main offices for 2007, and from late 2007 introduce a quantified system to measure MOSS compliance 2. MOSS compliance for sub offices and project offices from 2008 3. Security risk management mainstreamed into UNDP programming from 2008 and beyond. 1. MOSS compliance for main offices for 2007, and from late 2007 introduce a quantified system to measure MOSS compliance 2. MOSS compliance for sub offices and project offices from 2008 3. Security risk management mainstreamed into UNDP programming from 2008 and beyond. 1. MOSS compliance for main offices for 2007, and from late 2007 introduce a quantified system to measure MOSS compliance 2. MOSS compliance for sub offices and project offices from 2008 3. Security risk management mainstreamed into UNDP programming from 2008 and beyond. 1. MOSS compliance for main offices for 2007, and from late 2007 introduce a quantified system to measure MOSS compliance 2. MOSS compliance for sub offices and project offices from 2008 3. Security risk management mainstreamed into UNDP programming from 2008 and beyond. 1. MOSS compliance for main offices for 2007, and from late 2007 introduce a quantified system to measure MOSS compliance 2. MOSS compliance for sub offices and project offices from 2008 3. Security risk management mainstreamed into UNDP programming from 2008 and beyond.
Target Corporate Corporate Corporate Reporting Reporting Security Team and DSS Security Team and DSS
Data Source Security Team analysis Security Team analysis Security Team analysis Owner Owner Security Team Security Team
Rating

15
Website Updated and Reflects Key UNDP Priorities
Perspective Clients Satisfaction Clients Satisfaction Clients Satisfaction Clients Satisfaction Clients Satisfaction Clients Satisfaction Clients Satisfaction
Strategic Objectives Strengthen communication function Strengthen communication function Strengthen communication function Strengthen communication function Strengthen communication function Strengthen communication function Strengthen communication function
Definition Index (0-10 scale) resulting from the evaluation of the office website by COA according to a set of criteria validating the alignment of the website content with key UNDP priorities. Index (0-10 scale) resulting from the evaluation of the office website by COA according to a set of criteria validating the alignment of the website content with key UNDP priorities. Index (0-10 scale) resulting from the evaluation of the office website by COA according to a set of criteria validating the alignment of the website content with key UNDP priorities. Index (0-10 scale) resulting from the evaluation of the office website by COA according to a set of criteria validating the alignment of the website content with key UNDP priorities. Index (0-10 scale) resulting from the evaluation of the office website by COA according to a set of criteria validating the alignment of the website content with key UNDP priorities. Index (0-10 scale) resulting from the evaluation of the office website by COA according to a set of criteria validating the alignment of the website content with key UNDP priorities. Index (0-10 scale) resulting from the evaluation of the office website by COA according to a set of criteria validating the alignment of the website content with key UNDP priorities.
Target Corporate Corporate Corporate Reporting Reporting Automated Automated
Data Source COA database COA database COA database Owner Owner COA COA
Rating 8 8 gt 6 6 gt
Note Evaluation criteria 1 - Corporate branding. 2 - Alignment with corporate practices/strategic plan Core concepts mainstreamed. 3 MDG content. 4 - Transparency. 5 Search engine optimization and analytics 6 UNDPs role in UN partnership/country team 7 - Innovation and excellence (bonus point) Note Evaluation criteria 1 - Corporate branding. 2 - Alignment with corporate practices/strategic plan Core concepts mainstreamed. 3 MDG content. 4 - Transparency. 5 Search engine optimization and analytics 6 UNDPs role in UN partnership/country team 7 - Innovation and excellence (bonus point) Note Evaluation criteria 1 - Corporate branding. 2 - Alignment with corporate practices/strategic plan Core concepts mainstreamed. 3 MDG content. 4 - Transparency. 5 Search engine optimization and analytics 6 UNDPs role in UN partnership/country team 7 - Innovation and excellence (bonus point) Note Evaluation criteria 1 - Corporate branding. 2 - Alignment with corporate practices/strategic plan Core concepts mainstreamed. 3 MDG content. 4 - Transparency. 5 Search engine optimization and analytics 6 UNDPs role in UN partnership/country team 7 - Innovation and excellence (bonus point) Note Evaluation criteria 1 - Corporate branding. 2 - Alignment with corporate practices/strategic plan Core concepts mainstreamed. 3 MDG content. 4 - Transparency. 5 Search engine optimization and analytics 6 UNDPs role in UN partnership/country team 7 - Innovation and excellence (bonus point) Note Evaluation criteria 1 - Corporate branding. 2 - Alignment with corporate practices/strategic plan Core concepts mainstreamed. 3 MDG content. 4 - Transparency. 5 Search engine optimization and analytics 6 UNDPs role in UN partnership/country team 7 - Innovation and excellence (bonus point) Note Evaluation criteria 1 - Corporate branding. 2 - Alignment with corporate practices/strategic plan Core concepts mainstreamed. 3 MDG content. 4 - Transparency. 5 Search engine optimization and analytics 6 UNDPs role in UN partnership/country team 7 - Innovation and excellence (bonus point) Note Evaluation criteria 1 - Corporate branding. 2 - Alignment with corporate practices/strategic plan Core concepts mainstreamed. 3 MDG content. 4 - Transparency. 5 Search engine optimization and analytics 6 UNDPs role in UN partnership/country team 7 - Innovation and excellence (bonus point)
16
Balance Scorecard Usage survey

• Survey covered all RRs and DRRs - 85 responses
• Opinions on BSC
• It is an excellent monitoring tool but is used as
  a "stick with no carrot"
• Despite of some feebleness, BSC is very
  interesting tool for management and strategic
  planning
• Regional Bureaux do not use the tool to work with
  COs but rather just to grade them
• Focus is correct, but not all indicators strategic

17
Choices - Strategic/operational,
Development/management

• Need more focus on strategic positioning and
  programme results.
• It must evolve with changing organizational
  priorities and align with the Strategic Plan
• Should include more elements on programmatic
  results and impact.
• Important to have some bird's eye view on several
  strategic issues.
• Is perceived as a mechanism to control the
  management and staff performance and not tied to
  the strategic goals and objectives.
• The BSC designed by accountants, and accountants
  are usually not put in charge of strategy? .
• Emphasis is needed on development results. But
  perhaps there needs to be two separate
  scorecards?
• Management side has made a lot of progress but
  not strategic.
• BSC transforms strategic planning from an
  academic exercise into the nerve center of an
  organization.