The Public Service Commission Monitoring and Evaluation System

1
-The Public Service Commission Monitoring and
Evaluation System

• Monitoring and Evaluation Learning Session
• 24 February 2005

2
Presentation outline

• Historical background to ME in SA
• Emphasis from above
• The PSCs ME system
• Improving performance through ME
• Challenges in implementing ME

3
Historical background to ME in SA

• ME only introduced to SA government post-1994,
  due to conditions for ME not existing previously
  (democracy, transparency and accountability).
• Initially largely Donor driven set up to answer
  questions around Donor spend

4

• Growing emphasis since 1994 on ME, largely to
  support democracy and transit the country from a
  largely secretive to more transparent one
• Civil society (NGOs, CBOs, universities),
  previously critical, now challenged to be
  constructive, many engaged directly with
  government

5

• From 1995 emergence of ME, in some departments,
  but not within a coherent and overarching
  framework.
• Monitoring was largely driven by the need to
  report on financial accountability, due to
  requirements of the Auditor-General etc.

6

• However, widespread failure to address issues of
  impact or effectiveness focus more on
  activities and outputs, rather than outcomes
• Lack of integration around monitoring efforts
  in terms of programme and project often viewed
  as a burden by project staff

7
Emphasis from above

• The Presidents State of the Nation Address
  2004 gave targets and in 2005 reported thereupon
  ME in practice
• Resulted in increased ME activity within
  Departments, as systems are aligned to meet
  information demands

8
The PSC system..

• The PSC is mandated by the Constitution of SA to
  investigate, monitor and evaluate the
  organization and administration, and the
  personnel practices of the public service.
• These values and principles are put into
  practice, and supported by various legislative
  measures, aligning the SA public service to that
  of international good governance principles and
  practices

9

• It also works collaboratively with other agencies
  in performing this function.
• It uses the Constitutional values and principles
  as the framework within which to perform its
  function, by implementing a differentiated ME
  system that addresses the needs of a diverse
  client base

10

• The PS ME system is based on the 9
  constitutional principles, and concretises these
  into indicators which are tested across
  departments
• Results both quantitative and qualitative and
  presented, and recommendations tracked

11
The principles

• High standard of professional ethics
• Efficient, economic and effective use of
  resources
• Development oriented public administration
  Provision of services in an impartial, fair,
  equitable and unbiased manner
• Accountability
• Transparency

12

• Good human resources management and career
  development practices to maximise human potential
• Representativity

13
Scoring

• Scoring system has been made more explicit,
  moving from a global scoring, to where points
  are allocated for the presence of certain
  policies and attributes
• System makes scoring easier, and reduces the
  level of discretion of researchers.
• This also creates clarity for departments on what
  is required to meet good governance standards.
  This in itself focus departments in their
  efforts.

14

• The area illustrated by the indicator is
  critically important and issues in its management
  need to be noted (what gets measured gets done)
• Performance indicated is illustrative of overall
  performance in terms of the principle
  (performance in terms of that indicator can be
  extrapolated to suggest performance in complying
  with the principle as a whole)

15
Choosing indicators

• Choice of the indicator for professional ethics,
• Chosen indicator Cases of misconduct are dealt
  with effectively and promptly
• When cases take too long to be dealt with,
  departments are lenient on wrongdoers, it can be
  assumed that it is doing little to address
  professional ethics in the workplace.
• Specific measures are systems in place for
  reporting, recording and managing cases, cases
  are responded to promptly and finalized and the
  dept has adequate capacity to handle cases

16
The resultant products

• Based on the PSCs ME System, other internal and
  external research the Annual State of the Public
  Service Reports are produced
• 2004 SOPS assessed progress over the past 10
  years
• 2005 SOPS forward looking

17
Improving performance through ME

• The system contributes to service delivery
  improvement as it determining areas for attention
• It empowers managers to make decisions about
  where to target resources and energy
• The system focuses on administration and
  management, because sound administration leads to
  service delivery improvement

18

• Findings and recommendations need to be accepted.
  
• In dealing with complex institutions it is not
  always possible to assess whether the right
  decision-maker is targeted.
• Varied experience around this project ranging
  from support and acceptance to hostility and
  defensiveness.

19

• A more effective implementation would require a
  standardized government-wide ME framework.
• Research needs to be relevant, with strategic and
  operational changes being needed over time.

20

• Post-evaluation usage remains a problem. One
  needs to track the ratio of actions to
  recommendations so as to improve the quality of
  recommendations and address issues of strategy
• A differentiated ME programme addresses
  evaluation needs of a diverse group of users

21

• Both quantitative and qualitative data is
  produced, as well as methodologies such as PRA
  and appreciative enquiry
• Specific evaluations were conducted of policy and
  programme, as well as specific departmental
  requests

22

• PSC approach
• The programme logic model

Overall Results Better governance and service
delivery in South Africa
Problems are addressed
Learning from good practice examples takes place
Departments focus on priority areas
Achievements are affirmed and promoted
FOLLOW UP
Problem areas are identified
Good practice by others is identified and promoted
Priority areas in public admin are communicated
Departments reflect on their own performance
REPORTING
Public Service Monitoring
23

• The PSC and Methodological Pluralism
• The PSC has implemented participatory,
  appreciative enquiry in partnership with
  Parliament to promote the incorporation of the
  views and perceptions of citizen in the process
  of service delivery improvement.
• These forums are both empowering and educational,
  entailing a multidirectional flow of knowledge
  through interactions between individuals, groups
  and institutions.
• It created a better acceptance of decisions made
  by authorities, and educates citizens to act
  independently.

24

• These forums provides a unique context for
  participatory evaluation where citizens and the
  state collectively contribute towards quality and
  improved service delivery.
• The methodology may be viewed as a form of
  appreciate enquiry, insofar it focuses both on
  problems and how to resolve them, as well as what
  is working well
• There was also feedback, with the forums
  reconvened to inform citizens which of their
  recommendations were implemented.

25

• The PSC and Citizens Satisfaction
• The PSC has undertaken three (3) satisfaction
  surveys over the past three years, each focusing
  on a different sector of Government
• Sectors covered thus far are the Social Services
  Cluster, the Criminal Justice Cluster, and the
  Infrastructure and Regulatory Services Cluster
• Baseline data obtained against which program
  performance can be compared at another time point

26

• Views obtained from citizens experience on the
  quality and adequacy of services. In other
  words, the service delivery gap.
• Quantitative figures on the size and distribution
  of impact.
• Able to generalise findings to population as a
  whole.

27

• The PSC and Case Studies
• Evaluation of the Housing Subsidy Scheme
• Evaluation of the National Department of
  Transport and the Evaluation of the Transport
  Agencies
• Evaluation of Land Administration Systems in the
  Eastern Cape
• Evaluation of the Multi-Purpose Community Centres
  (MPCCs).

28
Plans for next steps to strengthen ME initiatives

• Central institutions of government need to take
  the initiative in developing an impact monitoring
  system for the whole of government
• These would included institutions that are part
  of the Governance and Administration cluster, and
  which have identified rigorous monitoring and
  evaluation to improve performance in the second
  decade of freedom

29

• A national statistical system should complement
  and strengthen the ME effort by providing
  statistical data and shared technical definitions
  this should be fast tracked. The census and
  survey information is collected periodically, and
  is useful for 3-5 year assessments

30

• It is unrealistic for an all-embracing
  whole-of-government ME system a more realistic
  one should be to focus on the establishment of
  various government wide systems
• ME should become more utilisation-focused if it
  is to have the intended impact on state
  performance, meaning that managers must be
  assessed more broadly.

31

• Co-operation can take place in terms of
• - the development of government-wide systems
• - use of appreciative enquiry and participatory
  techniques for evaluation

32

• Building a culture of ME in the public service
• Appreciative evaluation techniques and when they
  should be used
• Development of specific training programmes
  drawing in skill and expertise from different
  developing countries

33

• Mentorships in which experienced specialist are
  given fellowship in partnering institutions in
  different countries
• Build a global network of evaluators in
  developing countries by forming an Evaluating
  Society for the South
• These suggestions could allow for exchange of
  ideas and skills between countries

34
Conclusions

• Given the history of SA, much progress made, ME
  plays a key role in improving government
  performance
• The Governance and Administration cluster under
  the leadership of the Presidency is poised to
  develop a framework for evaluation.
• The PSC has developed a differentiated ME
  programme which will assist in the framework for
  evaluation being effective.

35

• A key challenge lies in building the required
  implementation capacity, and that results and
  findings are utilized to improve performance
• Required is a culture that accepts that building
  a developmental state requires that evaluation
  findings be linked to performance improvement and
  excellence.