An Introduction to Practical Tools to Support Delivery

1
An Introduction to Practical Tools to Support
Delivery VII CONFERENCE OF THE MONITORING
EVALUTATION NETWORK IN LATIN AMERICA AND THE
CARIBBEAN November 2011
Ray Shostak, CBE
2
Objectives of the Workshop

• The workshop will introduce a range of practical
  tools to accelerate the delivery of a governments
  key priorities.
• By the end of the session we will have
• Considered the imperative of delivery
• Introduced three potential analytical tools
• Considered the challenges of using tools
• Explored the potential uses of the tools

3
Shape of the Workshop
Time Focus Process
2.00 Introduction and Framework Presentation
2.20 Preparing for Delivery Presentation
2.40 Are you ready for action? Group Workshop 1/2 and Plenary
3.15 Understanding Delivery Systems Presentation
3.30 What does it look like? Group Workshop 2/3 and Plenary
4.15 From a Citizens point of view Presentation
4.45 Concluding questions and Remarks Plenary
4
Context.
?

• Individually, think of a policy area
• that is a government priority for delivery
• Identify the outcomes desired
• Identify your role in delivery

5
Outcomes Driven Performance
Outcomes for and with citizens
6
Key Lessons on Delivery
Shared and clear strategic vision Sound
governance and accountability
arrangements Prioritising key programmes
Leadership, focus, capacity and clarity
Knowing what motivates and stimulates delivery in
each part of the delivery system Understanding
the strengths/weaknesses of general and specific
interventions Citizen insight
Understanding what works and engaging the
delivery system
Good metrics Understanding what drives
contributing partners Good feedback loops that
are used
Effective performance management and good data
7
Features of a Delivery Plan

• Clarity about vision and outcomes
• Sets out priority actions to deliver the outcomes
• Effective performance, programme and risk
  management
• Effective delivery partner and stakeholder
  engagement
• Effective governance arrangements
• Effective performance, programme and risk
  management

8
Features of a Delivery Plan/1
Key characteristics
WHAT NEEDS TO BE IN PLACE
PITFALLS TO AVOID
Clarity about vision and outcomes

• A clear description of the aim of the project and
  the outcomes being sought.
• A narrative of what needs to be delivered by the
  lead Ministry and their delivery partners to
  achieve the outcomes
• An explanation of how short term progress and
  long term impact will be demonstrated
• A clear explanation of the impact when the
  project has been delivered, including from the
  point of view of the citizen

• It is unclear how the project aims and public
  commitments are achieved
• Success is described in system or institutional
  terms, rather than from the perspective of the
  citizen
• Language is technical and jargon is used

Sets out priority actions to deliver the outcomes

• The key actions required are clear,
  comprehensive, have owners and key milestone
  dates
• Timescales are proportionate to complexity of
  delivery
• Actions have been well-researched and based on
  sound evidence and analysis of their impact
• A timeline is in place with intermediate points
  showing expected progress
• The most important challenges to be addressed are
  identified, prioritised and have mitigation
  actions

• There is no clear link between the key actions
  and the delivery of outcomes
• Lack of evidence to demonstrate that the right
  actions have been identified as critical
• Action owners and/or deadline dates are missing
• Timescales are unrealistic given the complexity
  of the task or resources committed
• Major challenges are not identified and important
  actions to change performance are not set out

Clarity on incentives, resources, roles and
responsibilities throughout the delivery chain

• All key players in the delivery system, and wider
  stakeholders who can influence delivery, are
  identified, and appropriate action taken to
  involve them in planning
• The resources available from contributing
  Ministries and delivery partners are set out.
• There are clear and effective cross-ministry
  working arrangements in place at all levels.
• Incentives throughout the delivery system are
  clearly identified
• Key interdependencies have been identified and
  delivery timescales take these into account

• The delivery system is not well-understood or
  clearly defined
• Little or no assessment of the delivery systems
  capacity and/or capability
• Planning and resource estimates are unsupported
  by evidence
• A lack of cross-ministry commitment to take
  difficult decisions and remove barriers
• Interdependencies have not been identified nor
  has their impact been factored into plans

9
Features of a Delivery Plan/2
Key characteristics
WHAT NEEDS TO BE IN PLACE
PITFALLS TO AVOID
Effective delivery partner and stakeholder
engagement

• Key delivery partners and stakeholders are
  identified and prioritised
• Engagement and communication plans are in place
  and based on sound analysis and understanding of
  the delivery system
• Plans demonstrate understanding of delivery
  partner and stakeholder attitudes (throughout the
  delivery system) with appropriate targeting of
  outline key messages
• Plans set out proposals for monitoring the
  effectiveness of engagement and communications
  activities

• No analysis of delivery partners and stakeholders
  and how to influence their expectations and
  attitudes
• Engagement and communication plans are incomplete
  or ill-defined
• Lack of engagement of citizens and delivery
  partners in identifying what works
• No assessment of how well the plan is understood
  by all delivery partners

Effective governance arrangements

• A clear line of accountability from the most
  senior levels through to all levels of the
  delivery system
• Conflicts between the project and other areas of
  focus by those involved in governance have been
  identified and mitigated
• Scrutiny of progress is timely, regular, informed
  and focussed on delivery of outcomes
• Mechanisms to hold to account all those who need
  to contribute to delivery in Ministries and the
  delivery system

• There are unclear lines of accountabilities
  and/or a lack of ownership of key aspects of the
  delivery plan
• Governance arrangements add complexity and
  burden.
• Governance does not support focus on the key
  actions that will deliver the outcomes

Effective performance, programme and risk
management

• An agreed monitoring system system for monitoring
  progress with timely and accurate information to
  support decision makers and keep delivery on
  track
• Robust Programme and Project Management
  arrangements are in place throughout the delivery
  system
• Clear, timely and accurate reporting mechanisms
  that provide a clear indication of whether
  delivery is on trajectory
• Delivery risks and effective strategies for
  mitigating these risks have been developed and
  implemented
• Risk management processes throughout the delivery
  system

• No common understanding amongst those responsible
  as to how progress should and can be monitored
• Monitoring mechanisms that only look back rather
  than seeking to identify problems before it is
  too late to take remedial action
• Strategies for addressing identified risks are
  non-existent or are unlikely to be effective
• Risk management is seen as a tick-box exercise
  and not a planning tool risk registers, where
  they exist, are out-of-date

10
Delivery Tools

• What is a tool?
• A sequence of methods to analyse and problem
  solve
• Why do we need them?To develop insights on how
  to get more for less and
  better outcomes more efficiently
• Boundaries?
• In the context of approaches to reform and
  delivery

11
Public Service Reform
Personalised services through empowered citizens
and professionals working together
Greater accountability and transparency enabling
citizens to hold services to account
Government enabling changes through incentives
and support without micro-managing
12
An approach to Delivery
13
The importance of systematic intervention
INTERVENTION WHEN PROGRESS IS NOT MEETING
AMBITIONS
14
Delivery Unit Patterns
Go live
Planning Phase
Delivery Phase
Activity

• Assessment of readiness to deliver
• Six monthly Delivery Reporting
• Delivery Leads events
• Update notes to PM
• Updates to Cabinet Committee
• Departmental stocktake
• PSA Board attendance
• Progress Review
• Priority Reviews
• Develop action plans to implement
• Other tools

Conducted as neccesary
15
Delivery Tools
16
Preparing for Delivery
17
Preparing Ministries for Delivering Performance
OBJECTIVE to identify, through a collaborative
approach , underlying barriers to delivery and
increase the prospects for secure outcomes of a
Government priority

• WHAT IS REQUIRED
• A set of criteria that will enable judgement of
  current performance and give direction to future
  development
• An approach to collecting evidence of
  preparedness
• An approach to synthesising the evidence
• An agreement on reporting

18
Overview

• Fieldwork
• Document Analysis
• Interviews and visits
• Workshops
• Cross Ministry
• With officials
• With stakeholders
• With providers

Analysis and Recommendations
Build a Review Team
Report and Action
Self Assessment
19
Preparing for Delivery
Vision and leadership
Working across Ministries

• Is there a clear, compelling and coherent vision
  of what success looks like and why it is
  important?
• Is there leadership at national and local level
  and is it committed to driving delivery?
• Is there a clarity of who is responsible within
  the Ministry for the delivery of the vision and
  outcomes- and do they demonstrate effective
  leadership?
• Is there regular communication of the vision and
  its implications to enthuse and inspire delivery
  partners?

• Is the goal sufficiently prioritised and linked
  to other strategies and objectives in relevant
  Ministries?
• Is there a structure (Board) and clear
  accountability for delivery among Ministries?
• s there strong co-operation and teamwork among
  officials at all levels in Ministries to drive
  delivery?
• Is there shared commitment to take difficult
  decisions and capability to actively remove
  barriers to delivery?

The delivery system
Performance and Programme Management

• Is there a shared and clear understanding of the
  delivery network including what works, levers,
  responsibilities and capabilities?
• Are delivery partners clear about and agree to
  their role in delivering the ambition?
• Are delivery partners capable and empowered to
  deliver local initiatives and share good
  practices?
• Is there active engagement with citizens to
  understand what works and what they want?

• Is there effective plan of action, programme
  and risk management across the delivery system?
• Is there high-quality and timely performance
  information, supported by analytical capability,
  which informs local and strategic decisions?
• Is there robust alignment between incentives and
  delivery, and between corrective action and
  non-performance?
• Are the right skills, resources and structures
  in place to drive delivery?

20
DISCUSS

• Using one of the policy areas you identified
  earlier, agree with your partner/group a key
  government priority for delivery.
• Consider each quadrant in turn by asking
• What information would I need to evaluate
  preparedness?
• How would I collect the information?

21
Preparing for Delivery
Vision and leadership
Working across Ministries

• Is there a clear, compelling and coherent vision
  of what success looks like and why it is
  important?
• Is there leadership at national and local level
  and is it committed to driving delivery?
• Is there a clarity of who is responsible within
  the Ministry for the delivery of the vision and
  outcomes- and do they demonstrate effective
  leadership?
• Is there regular communication of the vision and
  its implications to enthuse and inspire delivery
  partners?

• Is the goal sufficiently prioritised and linked
  to other strategies and objectives in relevant
  Ministries?
• Is there a structure (Board) and clear
  accountability for delivery among Ministries?
• s there strong co-operation and teamwork among
  officials at all levels in Ministries to drive
  delivery?
• Is there shared commitment to take difficult
  decisions and capability to actively remove
  barriers to delivery?

Joint teams and shared burdens
Clarity of vision
Leadership
Cross Ministry representation
Links to other work
Shared incentives
Clarity of delivery agreement
Ministerial commitment
Officials understand consequences of non-delivery
Specific Ministry resources
Inspiring communication
What success looks like
The delivery system
Performance and Programme Management

• Is there a shared and clear understanding of the
  delivery network including what works, levers,
  responsibilities and capabilities?
• Are delivery partners clear about and agree to
  their role in delivering the ambition?
• Are delivery partners capable and empowered to
  deliver local initiatives and share good
  practices?
• Is there active engagement with citizens to
  understand what works and what they want?

• Is there effective plan of action, programme
  and risk management across the delivery system?
• Is there high-quality and timely performance
  information, supported by analytical capability,
  which informs local and strategic decisions?
• Is there robust alignment between incentives and
  delivery, and between corrective action and
  non-performance?
• Are the right skills, resources and structures
  in place to drive delivery?

Stakeholder management
User engagement in planning
Overall allocation of resources
Role and skills of policy and delivery teams
Understanding evidence and sharing what works
How Ministries work with the system
Data, baselining, trajectories
Availability of performance interventions
Capability of the system itself
Quality of planning
Delivery through regional, local, private and
voluntary sectors
PPM and Risk management
22
Overview

• Fieldwork
• Document Analysis
• Interviews and visits
• Workshops
• Cross Ministry
• With officials
• With stakeholders
• With providers

Analysis and Recommendations
Build a Review Team
Report and Action
Self Assessment
23
Fieldwork Hypothesis Led Analysis
24
Overview

• Fieldwork
• Document Analysis
• Interviews and visits
• Workshops
• Cross Ministry
• With officials
• With stakeholders
• With providers

Analysis and Recommendations
Build a Review Team
Report and Action
Self Assessment
25
Recommendations and Actions
Recommendations and Actions
26
DISCUSS

• Consider the process.
• What relevance does it have for your country?
• Are there particular Ministries where this would
  be helpful?
• Identify how you might apply the approach.

27
Understanding a Delivery System
28
Understanding and Improving Delivery Systems
OBJECTIVE to develop an understanding of the
roles, responsibilities and motivations of a
delivery system and how Government can
intervene to enhance delivery of its priorities

• WHAT IS REQUIRED
• An approach to mapping a delivery system
• An approach to analysing its components
• An approach to gathering evidence
• An agreement on reporting
• A strategy for intervention

29
What is a delivery system?

• A delivery system is a network of organisations,
  that need to work together to achieve improved
  outcomes for local people.
• This includes central and local government,
  other public sector partners and partners from
  the third and private sectors

30
Some key questions on delivery systems
What may be preventing them from contributing to
my goal?
Which organisations have a role in delivering my
goal?
UNDERSTANDING THE DELIVERY SYSTEM
How can I improve the prospects for delivery?
Are all these organisations effectively
contributing to my goal?
31
A Three Stage Approach

• Identify who needs to do something to deliver the
  ambition
• Group actors
• Identify what each actor needs to do

• Identify gaps in your understanding of the chain
• Undertake fieldwork
• Carry out the links analysis to fill knowledge
  gaps

• Communicate and address identified weaknesses

32
A Delivery System Map

• How you will achieve the governments ambition
• How the money follows the critical path
• The nature of the relationships (accountability,
  common purpose etc) between the various
  organisations involved
• The synergies and conflicts which help or hinder
  their ability to work together
• How the system interacts with the citizen
• Opportunities for improving efficiency and
  effectiveness

33
Moving from a delivery chain.
Ministry
Regional Government
Local Government
Service provider
Citizen
34
DISCUSS
With your partner/group draw a delivery chain for
your chosen government priority.
35
to a delivery system
36
Delivery System for PSA 18 Better Health For
All DH strand
Better Health For All
Legislation (incs EU legislation) health
prevention promotion advertising
Public Health Observatories
provides support
Patients, People who need and use social
care, Citizens, Socially Excluded /
Disadvantaged Groups, Carers.
Workforce advertising, influencing recruitment
retention NSFs
Competition, pricing, vfm
Funding, LDPs
Primary Care Trusts (inc PBC)
DH
Strategic Health Authorities
Performance Management
Performance Management of delivery agreements
Influence
Acute / MH Trusts and FTs
Equitable outcomes
LAAs, LSPs
JSNA
Choice Voice influencing provision and
commissioning e.g. LINks
GPs Practices
Specialist Units / Trusts
joint commissioning
National
Regional Agencies
Local Community
Delivery Agencies
Delivery mechanisms
national levers and covers HCC, CSCI, NICE,
Audit Commission and wider agencies e.g. CQC etc
including clinicians e.g. GMC, NMC, HPC, etc
Key
Department / Agency
Footnote 1, 07/8 programme to co-locate public
health and social care presence in the regions.
Footnote 2, consider in light of wider regulatory
review.
Working jointly and in partnership
joint local working relationship
37
Delivery System for PSA 18 Better Health For
All DH/CLG strand
Better Health For All
Legislation (incs EU legislation) health
prevention promotion advertising
Public Health Observatories
provides support
Patients, People who need and use social
care, Citizens, Socially Excluded /
Disadvantaged Groups, Carers.
Workforce advertising, influencing recruitment
retention NSFs
Competition, pricing, vfm
Funding, LDPs
Primary Care Trusts (inc PBC)
DH
Strategic Health Authorities
Performance Management
Performance Management of delivery agreements
Influence
Acute / MH Trusts and FTs
DCLG
Equitable outcomes
LAAs, LSPs
JSNA
Choice Voice influencing provision and
commissioning e.g. LINks
GPs Practices
Government Offices (incs PH1)
Local Authorities inc OSCs
Specialist Units / Trusts
Performance Management, funding
Commissioning incs joint commissioning
environment, housing, roads, schools, benefits,
etc
LA Teams and Specialist LA Units
Social Care providers
Independent providers e.g. ISTCs
Voluntary providers
Regulators Inspectorates2
Inspect, Reviews Reports
National
Regional Agencies
Local Community
Delivery Agencies
Delivery mechanisms
national levers and covers HCC, CSCI, NICE,
Audit Commission and wider agencies e.g. CQC etc
including clinicians e.g. GMC, NMC, HPC, etc
Key
Department / Agency
Footnote 1, 07/8 programme to co-locate public
health and social care presence in the regions.
Footnote 2, consider in light of wider regulatory
review.
Working jointly and in partnership
joint local working relationship
38
Delivery System for PSA 18 Better Health For
All Complete system
Better Health For All
Legislation (incs EU legislation) health
prevention promotion advertising
Public Health Observatories
provides support
Patients, People who need and use social
care, Citizens, Socially Excluded /
Disadvantaged Groups, Carers, etc
Workforce advertising, influencing recruitment
retention NSFs
Competition, pricing, vfm
Funding, LDPs
Primary Care Trusts (inc PBC)
DH
Strategic Health Authorities
Performance Management
Performance Management of delivery agreements
Influence
Acute / MH Trusts and FTs
DCLG
Equitable outcomes
LAAs, LSPs
JSNA
DCSF
Choice Voice influencing provision and
commissioning e.g. LINks
GPs Practices
Government Offices (incs PH1)
DWP
Local Authorities inc OSCs
Specialist Units / Trusts
Performance Management, funding
Cross-government working
Commissioning incs joint commissioning
HO
environment, housing, roads, schools, benefits,
etc
LA Teams and Specialist LA Units
DCMS
influence
influence
Defra
Social Care providers
RDAs, Regional Assemblies, etc
OGD influence
DIUS
Independent providers e.g. ISTCs
regeneration
Business community
DfT
OGD influence
Voluntary providers
Regulators Inspectorates2
Inspect, Reviews Reports
National
Regional Agencies
Local Community
Delivery Agencies
Delivery mechanisms
national levers and covers HCC, CSCI, NICE,
Audit Commission and wider agencies e.g. CQC etc
including clinicians e.g. GMC, NMC, HPC, etc
Key
Department / Agency
Footnote 1, 07/8 programme to co-locate public
health and social care presence in the regions.
Footnote 2, consider in light of wider regulatory
review.
Working jointly and in partnership
joint local working relationship
39
Components of the best
What is a delivery chain and what is it for?

• The outcomes
• the customer groups
• The delivery partners
• The type and nature of the links between the
  customer groups and delivery partners and how
  effectively these are driving performance.
• The flow of funding

40
A complete version includes
Linkage Lines direct accountability
contract/regulatory common purpose
wider influence
41
Education, Training and Employment 16 - 18
Regional
National
Local
Ofsted
Voluntary sector organisations
Education providers
Businesses
Parents/ carers/peers
Inspect schools and colleges identify and share
best practice
Work placements expertise and innovation to EET
targeted provision
NCSL, AoC
Training and development for school and college
leaders
Education and Training
School sixth form
FE
Business Innovation and Skills
Work Based Learning
Skills Funding Agency/National Apprenticeship
Service
Apprenticeships
Apprenticeship funding contract management
16-18 year olds
Local Authorities
Young Persons Learning Agency
Commissioning funding performance management
of schools and training providers
Information Advice and Guidance
Connexions
16-19 funding EMA support for LA Commissioning
LA childrens services
Third sector
Department for Education
Businesses
Government Offices
Jobs with and without training
14-19 development
Agree targets and performance manage LAs
Qualifications and curriculum capital
Department for Work and Pensions
JCP district offices
JCP local offices
42
Crime Reduction
Local Criminal Justice Boards (42)
HM Court Service
42 CPS
Witnesses
140 Prisons
MoJ
10 DOMS
626 Courts
Directors of Offender Management
(Also includes police forces, probation areas and
Youth Offending Teams)
NOMS
Home Office
Neighbourhood Watch
5 CJS Inspectorates
Regulate and inspect CJS in England and Wales
NPIA
National Policing Improvement Agency
152 YOT
CLG
Victims
ACPO
Voluntary Sector Providers
As probation, for under 18s.
10 Govt Offices
Professional association of chief police officers
Provision of services in many CJS areas
Regional representation of Government
43 Police forces
Responsible for front-line policing
DfE
Offenders
Audit Commission
43 Police Authorities
Audit local authorities (including police)
Hold the police to account on behalf of
communities
Community Safety Partnerships (340)
152 Local Authority
152 Primary Care Trusts
Family/Peers
63 Fire Brigades
22,728 Schools
1,121 Hospitals
DoH
Health Providers
10 SHA
Landlords
Enacting directives and implementing policy
(Also includes police forces, probation areas and
Youth Offending Teams)
Manufacturers
NHS
Production of crime-proof products
BIS
43
DISCUSS

• With your partner/group convert your delivery
  chain into a delivery system map by identifying
• Other Ministry interests
• Other regional or local actors
• Potential synergies or confusion

44
Fieldwork Hypothesis Led Analysis
45
Checking the Analysis
Issue Rating
Is the outcome clearly defined and understood by all? -1 0 1 The outcome is vague The outcome is clear but partners not signed up The outcome is clear, understood and agreed
Is there sufficient capacity, including resources, to deliver? -1 0 1 No resource assessment Assessment of resource need undertaken Resources are in place
Are roles, responsibilities and accountabilities clear? -1 0 1 People are not clear on their contribution There are outlines but not agreed People know and agree where they fit in
Are staff committed to achieving the goal? -1 0 1 Motivation to achieve the goal is low There are too many goals The goal is a high priority for everyone
Does the system use robust and timely data on performance? -1 0 1 The goal is hard to measure Indicators are not entirely accurate Regular monitoring and action taken
Are levers and incentives fit for purpose -1 0 1 Few levers or incentives in place Some incentives in place but not agreed Levers are based on context and ability
46
DISCUSS

• Consider the process.
• What relevance does it have for your country?
• Are there particular priorities where this would
  be helpful?
• Identify how you might apply the approach.

47
Delivery from a Citizen Perspective
48
Customer Journeys
Customer Journey Mapping Improves Performance
By showing that different groups have different
requirements
By revealing pressure points using journey maps
By identifying key issues in current delivery
By prioritising next steps for action
49
Customer Journey Mapping
OBJECTIVE To understand what public services are
like from a citizens perspective. And to
analytically track the experience in order to
improve efficiency and delivery of outcomes.

• WHAT IS REQUIRED
• An approach to mapping contact with citizens
• An approach to gathering experience
• An approach to analysing its components
• An approach to reporting
• A strategy for intervention

50
Customer Journey Mapping

• Customer Journey Mapping is the process of
  tracking and describing the experiences that
  customers have as they interact with a service(s)

Customer experience This focuses on the emotional
insight of the actual customer who tells the
story of their journey. The system (process
mapping) Creates a graphic showing the steps,
actions, interactions and decisions points as
customers interacts with public services
Measuring the experience To determine how well
an experience is being delivered.
8
51
Customer segmentation
52
Detailed Journey Maps
53
(No Transcript)
54
Making it happen
55
End-to-End Customer View
Awareness
Engagement
Acceptance
Investigation
I know that I have financial difficulties
I recognise my home could be repossessed
I need to find out where I can get help and advice
I am receiving support and guidance
Review
Decision
Response
Progression
I understand what my options are for help
My case is under review by relevant parties
I am receiving appropriate support I am
proceeding through court proceedings
My decision is now being put into action
56
Discuss

• Think of a service
• How easy would you find it accessing your
  service?
• Where are there blockages in the provision of
  services from a citizens point of view?
• How often do you, and delivery partners get
  feedback from citizens?
• To what extent are services personalised to meet
  the needs of individuals?
• How often do you get out to see the impact of
  policy on practice?

57
Some final observations

• Outcomes Matter
• The skills and dynamics of delivery can be
  analysed and understood
• The basics of good delivery are key
• Consistency and alignment enhances outcomes
• Systematic intervention can improve systems and
  outcomes
• Collaboration pays dividends at all levels
• Citizens can co-produce services

58
Continuing the dialogue..

• Ray Shostak, CBE
• ray_at_shostak.uk.com