How to apply PSIA 10 Steps and some lessons learned

1
How to apply PSIA? 10 Steps and some lessons
learned

• --------------------------------------------
• Renate Kirsch
• Nairobi, December 2006

2
A 10 Step approach to PSIA

• 1. Selecting the Reform
• 2. Identifying stakeholders
• 3. Understanding transmission channels
• 4. Assessing institutions
• 5. Gathering data and information
• 6. Analyzing impacts
• 7. Enhancing design and compensatory schemes
• 8. Assessing risks
• 9. Establishing monitoring and evaluation systems
• 10. Fostering policy debate and feedback into
  policy choice

3
1. Selecting the reform and mapping out research
questions

• Criteria for selection of reform
• Expected size and direction of impacts
• Prominence of issue in the governments policy
  agenda
• Timing and urgency of policy or reform
• Level of national debate surrounding the reform
• Formulating the key questions
• Identify key problems/constraints that policy
  will address
• Make development objectives explicit
• Formulate causal hypotheses linking objectives to
  actions to likely short-term and long-term
  impacts
• Define the alternative (other option, status quo)

4
Operational lesson 1 identify reforms

• Need for PSIA should emerge from PRS
• Identifying reforms for PSIA should be part of
  national PRS process (no duplication)
• In practice, work in progress. Selection should
  strengthen broader process, not
  undermine/duplicate it
• Selectivity/prioritization essential
• Costly and time consuming
• PSIA most meaningful and effective when applied
  to specific reforms

5
2. Identifying stakeholders

• Stakeholders affected by policy reform positively
  and negatively
• Differentiated by ethnic, religious, age,
  spatial, livelihood, or other criteria
• Stakeholders affecting the reform
• Institutional stakeholders
• Powerful interest groups within the public
  sector, private sector, and civil society
• Focus on
• key characteristics
• interests in relation to the policy
• importance to the reform, influence on the process

6
Analytical lesson 2 different groups

• Traditionally, distributional impacts measured on
  income/consumption groups
• Useful to understand overall effectiveness and
  comparing aggregate impacts of alternatives
• But groups are artificial constructs and do not
  allow analysis of behavioral responses
• Need also focus on spatial, social, occupational
  groups that allow to understand behaviors
• ? Operational dimensions

7
3. Understanding transmission channels

• Impacts transmitted through multiple channels
• Employment (Guyana sugar)
• Prices production, consumption, wages (utility
  prices)
• Access to goods and services (credit, basic
  services)
• Assets physical, natural, financial, human,
  social (land, education, health)
• Transfers and Taxes (tariffs, subsidies, import
  tax, VAT)
• Authority (power relations, legal regulations,
  institutional capacity, political economy)

8
Analytical lesson 4 Multiple channels

• Most reforms have multiple transmission channels
• Impacts might change direction/size when
  considering them

9
4. Assessing Institutions

• Institutions mediate the effect of policy changes
  on the welfare of people
• Examine relevant social and market institutions
• Institutions may themselves be the objective of
  policy reforms
• Analyze changes in incentives and rules
• Policy changes depend on organizations for their
  implementation
• Incentives, performance and capacity are key
• Transaction costs affect reform outcomes
• Markets, legal systems, public organizations

10
5. Gathering data and information

• Map out desirable data and information
• Take stock of existing data and analysis
• Adapt PSIA to data limitations
• Adapt analytical approach
• Collect further data (multiple types)
• Postpone the reform
• Build data basis and capacity for future poverty
  and social impact analysis

11
6. Analyzing impacts

• Expected direction magnitude of impact
• Describe nature and size of principal impacts
• Income and non-income impacts
• Long-term and short-term impacts
• Direct and indirect impacts
• State underlying assumptions regarding
• Intended benefits
• Organizational capacity and institutional
  performance
• Stakeholder behavior, including behaviors of
  affected persons, investors and regulators

12
Analytical lesson 1 negative and positive impacts

• Central concern is the poor
• But, PSIA is not only about mitigation measures
• Hence, needs to focus on all impacts, both
  positive and negative on all groups
• Analysis of support and opposition to reform
• Allows for influence on design of the reform, not
  only on mitigation
• Political economy critical

13
Analytical lesson 3 short/long term

• Reforms have short and long term impacts
• Often linked to direct versus indirect impacts
• A same group could have positive net impacts in
  short term and negative ones in longer term
• Assessing short term impacts is relatively easy
• Longer term impacts require more complex and
  challenging analysis
• Assess importance of short and long term, direct
  and indirect impacts
• ?Analyze all relevant ones to define net impacts

14
7. Enhancing design compensatory schemes

• In light of analysis of impacts
• Consider an alternate design
• Alternative design
• Different pace and sequence
• Triggers to invoke additional risk management
  measures, reform modifications or an exit
  strategy
• Consider direct compensation measures
• Consider delay or suspension of the reform

15
8. Assessing risks

• Types of risk
• Institutional risks (reform complexity exceeds
  institutional capacity, vested interests in
  agency)
• Political economy risks (interest groups
  undermine reform or capture benefits)
• Exogenous risks (conflict, financial crisis,
  terms of trade shocks, natural disaster)
• Country risks (elections or political
  instability, ethnic conflict, post-conflict
  environment)
• Assess likelihood of occurrence and importance
  to the policy

16
9. Monitoring Evaluating Impacts

• ME allows to
• Validate policy analysis
• Inform policy adjustment during implementation
• Promote ownership of reforms (participatory
  monitoring)
• Promote accountability
• ME should
• Indicators defined before the reform is
  implemented
• tied to transmission channels and assumptions
• correlated with reform
• that can be measured in time to suggest
  improvements
• Build on existing systems to develop national
  monitoring system and capacity

17
Operational lesson 5 Monitoring

• PSIA often ex-ante and based on assumptions
• Assumptions, actual impacts must be monitored
  during implementation to allow corrections if
  needed
• PSIA indicators should be integrated in country
  systems to ensure continued improvement
• If PSIA elements too specific for country
  systems, ensure they will be monitored after end
  of core PSIA
• NGOs
• Development partners on the ground
• Research institutes and universities

18
10. Fostering policy debate and feedback into
policy choice

• PSIA draws on public discussions
• When identifying reform for analysis
• When analyzing stakeholders,
• When validating technical impact analysis,
• When leveraging social accountability.
• PSIA should inform policy discussions and
  consideration of alternatives
• PSIA needs an institutional home to incorporate
  results into the policy process

19
Operational lesson 2 Analysis design

• Analysis typically includes participatory
  elements (e.g. stakeholder analysis)
• Does not mean analysis is designed in
  participatory manner
• One doesnt analyze by consensus. Analysis a
  scientific process, based on professional norms
  and standards
• But methodology must be transparent, in public
  domain for informed decision-making process

20
Operational lesson 3 Analytical process

• No monopoly on who does the research Government,
  university, research institute, NGO, private
  sector, development agency
• as long as methodology transparent, rigorous
• Rigor doesnt mean ignoring stakeholders - their
  views are essential inputs
• But analysis independent, not an expression of
  the views of a particular (vocal) group
• Agencies responsible for reform must be part of
  analytical process, to be able to utilize the
  results

21
Operational lesson 4 Policy dialogue

• PSIA contributes most when closely aligned with
  ongoing policy dialogue
• Research design based on options actually
  considered
• Results relate to all stakeholders
• Needs to be anchored in government policy cycle
  (national PMS, policy research group)
• PSIA part of broader policy dialogue
• Dissemination of PSIA results is key
• Results produced early enough to influence
  dialogue
• Policy processes w/o clear beginning or end
• Discrete action part of series of inter-related
  actions
• PSIA one element to inform broader process
• PSIA needs to be absorbed by main actors in
  governments, civil society and within donor
  agencies
• Ex ante, during, ex post

22
Analytical lesson 5 Choice of methods and team

• PSIA can use various techniques and tools
• Depends on question, data, resources, time
• Complementarity, triangulation
• analytical techniques,
• quantitative/qualitative data
• Building teams Skills for different aspects
• Economists prices/quantity, equilibrium
• Social development specialists stakeholders,
  institutions, risks
• Sector specialists policy issue and reform
  design
• ? Multi-disciplinary work provides best rigor,
  but expensive and difficult

23
(No Transcript)
24
Mixed method approach

• Combining Social and Economic Analysis
• Bringing a social, economic and sectoral lens to
  the research questions
• Combining quantitative and qualitative methods
• Assess research questions with different methods
  and tools

25
Analytical focus vs type of data and analysis
Qualitative analysis
Quantitative analysis
Social
Economic
26
Complementarity of methods

• Qualitative Methods
• Identifies relationships and patterns
• can help to probe and affirm those relationships
  and explain contextual differences in the quality
  of those relationships
• inductively throws up interesting, often
  surprising and sometimes counterintuitive
  relationships and patterns,
• are applied to a specific locality, case or
  social setting are described as contextual..

• Quantitative Methods
• produce data that can be aggregated and analyzed
  to describe and predict relationships
• able to ask how much? and establish how
  confident we can be in these working
  hypotheses.
• can be applied across the entire population or a
  section of the population, e.g. a region. They
  are referred to as non-contextual.

27
Combining tools from different disciplines

• Use qualitative methods to understand context,
  relationships, patterns informs the design of a
  survey questionnaire
• Use quantitative methods to assess extent to
  which phenomena occur (generalization,
  representation)
• Use qualitative methods to unpack issues which
  are hard to explain from survey results

28
Three ways to combine methods

• In parallel
• In sequential
• Iterative

Joint conceptual framework
Basis for identifying results and developing
recommendations