United We Stand. Evaluating, Cooperating, and other Unlikely Stories of Evaluation Capacity Building in Italy

1
United We Stand. Evaluating, Cooperating, and
other Unlikely Stories of Evaluation Capacity
Building in Italy

• Martina Bolli, Silvia Ciampi, Francesco Giordano
  Laura Tagle
• Italys National Evaluation System

2
Content

1. Cohesion policy in Italy long-term background
   contrasting directions
2. The Italian evaluation policy
3. Efforts (unlikely) achievements plus some
   drawbacks

3
COHESION POLICY IN ITALY LONG TERM
BACKGROUND CONTRASTING DIRECTIONS
4
2007-2013 - Evaluation as one of the pillars of a
(much needed) renewal of Italys regional policy

• Citizen-centered policies
• Strong focus on essential servicesrather than on
  procedures expenditure
• Emphasis on local knowledge
• Responsibility to regional governments
• Links with national policies
• Greater importance to administrative capacity

5
The regional ( cohesion) rural development
policy in Italy

• Intentionality of territorial goals
• Additionality vis-à-vis ordinary policy

• Resources come from EU National funds

6
Levels of programming a complex framework

• National umbrella strategies for regional
  rural development policies
• 7 Ministries develop their own programs
• Each of the 21 Regions develops
• its multi-sector strategy
• 5 Programs (each program is funded by one source)
• Naturally, each of the 5 programs is usually
  managed by a separate Directorate
• Programs end up being bureaucratic constructs
  rather than guides for action

7
Over time, silos have been created

• Sources of money

• Coordination organisms

• Ministry for economic development
• Ministry for agriculture
• Ministry for labour Other departments

• EU Regional Fund
• National funds
• EU Social Fund
• EU Rural Development

8
.Beyond silos

• 2007-2013 intentional efforts to bridge gaps
  introduce policy dialogue among and within
  authorities in a new perspective

9
At the same time, EU regulations ask for separate
programmes by EU source of money ( within Italy
the momentum for integration dwindles)

• The National Strategy for 2007-2013 tried to make
  the various sources of funding work together
  through common
• Programming
• Implementation
• Monitoring
• Surveillance
• Evaluation

10
21 Regions (70) 6 Ministries (30)
Centre-North 15 South 85
3bln to be given to Southern Regions on the
basis of performance in waste, water, education
and social services

• Regional development strategy
• Between 2007 2010 lower national resources
  (other than national cofinancing)
• different distribution North-South
• the instruments needed to implement
  nationally-relevant innovations (among which
  greater integration) dwindle

11
The Italian evaluation policy
12
Evaluation capacity building/1

• From EU evaluation requirements towards a
  national policy
• Institutional building
• Creation of Evaluation Units within executive
  branch of Regional authorities
• Network of Evaluation Units
• Specialized
• Regional and Rural Development policies only
• Both demand supply side
• Improve regional authorities ability to Demand
  for Use evaluations
• Improve Evaluation Units ability to Manage and
  Conduct evaluations

13
Evaluation capacity building/2

• Organized guidance support (National Evaluation
  System)
• Enabling approach
• No rigid prescriptions
• Reputational mechanisms
• Legitimize support innovators at regional level

14
Evaluation policy /1

• Nationally-funded interventions must be
  evaluated--not only those funded by EU
• Objects
• effects of a well-defined (however complex)
  intervention or group of interventions on a
  problem, a social group, or an arearather than a
  bureaucratic construct (e.g., a multisectoral
  programtipically operating on a large area,
  affecting millionsof people)
• across bureaucratic borders the evaluation
  object may have been financed by more than one
  source
• selectivity choose to answer defined questions
  about crucial controversial issues

15
Evaluation policy/2

• Each Region drafts an Evaluation Plan
• What is evaluated (and what is not)
• Who (internally or externally) performs each
  evaluation
• When
• Organization of evaluation function
• Resources (human and financial) devoted to
  evaluations

16
The National Evaluation System

• UVAL Ministry of Economic Development
  (Coordinates)
• INEA Ministry for Agricultural, Food and
  Forestry Policies
• ISFOL Ministry of Labour
• Department for Equal Opportunities
• Evaluation Units of Regional Authorities
• Its composition shows a clear intention to work
  together across silos
• involving Regions in a federal function

17
The National Evaluation System Each component
differs from the others

• Central units
• Have differing rules, interpretations, goals,
  tasks status
• Regional evaluation units
• Conduct or manage evaluations
• but also provide project selection, planning,
  knowledge management to the Region
• Have background and skills closer to regional
  fund policies (or national funds) rather than
  rural development or active labour policies

18
Efforts (UNLIKELY) achievements PLUS some
drawbacks
19
A federal function performed across silos by
central regional units

• Common guidance - include adjust for differing
  rules, traditions, goals
• Personalized support to Regions (all managing
  authorities together)
• Ensure quality evaluation of evaluations (....)
• Observe processes disseminate information
  (jointly)
• Facilitate dissemination (conference-like
  meetings, workshops, create maintain eval
  database www.dps.tesoro/valutazione.it)

20
Some results

• All Regions developed an Evaluation Plan
  http//www.dps.tesoro.it/uval_linee_valutazione.as
  p10
• Evaluations have been launched
• and even completed 50! (in addition to the
  mandatory ex post evaluations of rural
  development)
• and a lofty 60 available on the internet
• Evaluation units (established between 2000
  2002) have started conducting internal
  evaluations
• Still far from perfection but

21
Why we are happy anyway

• Activities started (and 50 concluded) even though
  
• (or just because?) there are no enforceable
  requirements (no sanctions)
• there is still strong inertia in the system
• old evaluations fulfilled needs of managing
  authorities, national coordinators, and
  Commission rapporteurs
• restricted market with marriages between
  managing authorities evaluators
• still little pressure from social partners
  general public
• Innovations go together evaluations seem to be
  used as an instrument by Regions willing to
  improve
• Innovations implemented where there has been more
  support from central state (not only on
  evaluation)

22
Thank you

• Laura.tagle_at_tesoro.it
• s.ciampi_at_isfol.it
• Valutazione.uval_at_tesoro.it
• Please send us feedback at these e-mail addresses