Asking the Right Questions: The Design of Training Courses for Regulatory Impact Assessment in Devel

1
Asking the Right Questions The Design of
Training Courses for Regulatory Impact Assessment
in Developing Countries
www.regulatoryreform.com

• Scott Jacobs, Managing Director
• Jacobs and Associates
• Third International Conference on Pro-Poor
  Regulation and Competition Issues, Policies and
  Practices
• Center on Regulation and Competition
• University of Manchester
• 7 9 September 2004, Cape Town, SA

2
What is regulation?
www.regulatoryreform.com

• Regulation is a way to make trade-offs between
  competing interests in democratic societies.
• Regulation is inherently and inescapably about
  making choices.

3
RIA seen as a process of informed policy making
-- is a good governance tool
www.regulatoryreform.com

• An important capacity of a regulator is the
  ability to assess the economic and social impacts
  of a regulation before it is adopted.
• Regulatory impact analysis (RIA) is a tool now
  used in most developed countries to improve
  understanding of the economic and social welfare
  impacts of laws and other forms of regulation.
• RIA properly viewed as a process of
  evidence-based decision-making rather than as an
  analytical method is an institutional reform
  supporting good governance.

4
What is Regulatory Impact Analysis?
www.regulatoryreform.com

• Structurally, RIA is a method of
• asking the right questions in a structured format
  to support a wider and more transparent policy
  debate.
• systematically and consistently examining
  selected potential impacts arising from
  government action or non-action,
• communicating the information to decision-makers
  and stakeholders,

5
Good governance goals of RIA
www.regulatoryreform.com

• Faster learning, increasing benefits of
  government action, finding lowest cost solutions,
  reducing policy failures
• Transparency, building trust, and reducing
  regulatory risks for private sector, reduce
  information monopolies
• Policy coherence in a complex world break down
  vertical silos and promote horizontal thinking
• Client-oriented, credible, and responsive
  government

• Analysis Calculating the costs and benefits of
  government action
• Consultation and responsiveness to a wider range
  of interests
• Integrating multiple policy goals (social and
  economic policies)
• Accountability for actions and results (within
  ministries, to the public)

6
RIA is constrained by a lack of skills in the
public sector
www.regulatoryreform.com

• Core skills to use RIA include
• familiarity with a few micro-economic concepts
  (opportunity cost)
• presentation of complex technical and analytical
  issues to policy officials so that the
  consequences and uncertainties of options are
  clear and relevant
• processes of RIA, including the role of public
  consultation and quality control
• familiarity with innovative alternative policy
  instruments, such as market-based approaches
• Core skills to produce RIA include
• design and implementation of cost-effectiveness,
  benefit-cost analysis and other methods
• Familiarity with uncertainty analysis
• collecting data through a variety of methods
• economic analysis in competitive global economies

7
Core content of a RIA training course
www.regulatoryreform.com

• Regulatory reform principles what is smart
  regulation for the market?
• Defining the problem
• Identifying the baseline
• Choosing the decision criteria and the method
• Collecting data
• Presenting benefits and costs, treating
  uncertainty
• Consulting with stakeholders
• Institutions for RIA, and the importance of
  quality control

8
1. Principles of good market regulation
www.regulatoryreform.com

• Security (legal security, consistent respect for
  market solutions, predictable enforcement)
• Transparency (clear and simple rules, openness
  through the entire policy process, less
  corruption)
• Legitimacy (must protect safety, health,
  environment, consumers, public interests)
• Efficiency (competition, low-cost rules, orderly
  and timely decisions, move swiftly to meet market
  needs)
• Expertise (good regulatory skills and
  understanding of complex markets and
  technologies)

9
RIA should be based on government-wide principles
of good regulation
www.regulatoryreform.com

• It is essential to explicitly define regulatory
  quality, especially in a transition country,
  where understanding of market needs is low.
• Most governments have issued instructions to
  regulators about the quality of their regulatory
  decisions.
• These commonly take the form of checklists and
  decision criteria.
• Regulators must show that their proposed
  regulations will meet these quality standards
  BEFORE they adopt them.

10
A widely used example the OECD Checklist for
Regulatory Quality
www.regulatoryreform.com

• 1. Is the problem correctly defined?
• 2. Is government action justified?
• 3. Is regulation the best form of government
  action?
• 4. Is there a legal basis for regulation?
• 5. What is the appropriate level (or levels) of
  government for this action?
• 6. Do the benefits of regulation justify the
  costs?
• 7. Is the distribution of effects across society
  transparent?
• 8. Is the regulation clear, consistent,
  comprehensible and accessible to users?
• 9. Have all interested parties had the
  opportunity to present their views?
• 10. How will compliance be achieved?

11
2. Defining the problem
www.regulatoryreform.com

• Regulators must understand why regulations fail.
  Failure means that rules do not achieve
  objectives, cost more than anticipated, or even
  hurt the very people they are meant to help.
  Failure has many roots
• Failure to define the problem correctly
• Failure to understand other forces and incentives
  acting on the problem,
• Failure to predict changes in human behavior
• Failure to see the full consequences of
  government action, such as risk-risk trade-offs
• Failure to update to meet changing needs,
  technologies, and situations
• Failure to understand limits in government
  capacities

12
The importance of defining the problem
www.regulatoryreform.com

• No analysis can compensate for poor problem
  definition.
• Lessons from policy failures Understand the
  scope of the issue, its trade-offs, and
  incentives, and define the problem broadly enough
  to include changes in behavior.
• A real example
• Problem Child are being poisoned by eating pills
  from bottles they find in their homes.
• Problem as defined by regulators How can we make
  medicine bottles harder for children to open?
• Their solution Design bottle caps that are hard
  to open
• Incentives Many users of medicines (older
  people) did not close bottles because they cant
  get the tops off
• Result Child poisonings INCREASED
• Real problem How can we reduce access to
  medicines by children?
• Solution Change behavior and technology. Label
  bottles with child warnings, design caps with
  directions so that people who can read can open
  them easily.

13
To reduce regulatory failures, problem definition
requires examination of a wider range of impacts
and possible outcomes
www.regulatoryreform.com

• Define the desired outcome in terms of results
  (fewer poisonings, not safer bottles).
• Identify the range of choices that people can
  make to influence the results (put bottles on
  high shelves, close bottles tightly).
• Assess how various regulatory approaches will
  affect their choices and behaviors. Recognize the
  most regulations have positive AND negative
  effects (if bottles are harder to close, more
  people will leave them open).
• Use a RIA method that permits the relevant
  choices to be compared so that net effects can be
  identified. Usually, this will be a form of
  benefit-cost analysis.
• Identify regulatory options that use incentives
  and change behaviors to get results at lowest
  cost.

14
3. Identifying the baseline
www.regulatoryreform.com

• A RIA is an incremental analysis that compares a
  regulatory action with a baseline.
• Assessing impacts of an action requires that we
  first identify what would happen if we did
  nothing (the counterfactual).
• A RIA must be based on a clear statement of the
  baseline. What will happen in future if no action
  is taken?
• This is often not easy, because the consequences
  of the current situation are not often not clear
  (consider the problem of global warming)

15
4. Choosing the RIA decision criteria and the
method
www.regulatoryreform.com

• Basic decisions
• What kind of impacts should be assessed? (This is
  a practical and political question as much as an
  analytical question.)
• What are the decision criteria that must be
  satisfied?
• What kind of analytical method should be used?
  (This will flow from the answers to the first two
  questions.)

16
Decision criteria and analytical methods
www.regulatoryreform.com

• Decision criteria determine how analysis is used
  to reach a policy decision
• -- Do the benefits justify the costs?
• -- Is the approach the least costly option that
  achieves the objectives?
• -- Does the rule violate or prevent the
  violation of a threshold test for action?
• Analytical methods examine, order and manipulate
  different kinds of information to present
  reality in simplified and understandable ways
• -- Benefit-cost analysis (what are the benefits
  and costs?)
• -- Cost-effectiveness analysis (proportionality
  test)
• -- Threshold analysis (precautionary principle)

17
Guidelines for choice of methods
www.regulatoryreform.com

• Benefit cost analysis Should be used when there
  are many possible choices policy objective is
  uncertain, broadly defined, or contingent
  interactive effects and trade-offs are possible.
  Most policy problems fit these criteria.
• Cost-effectiveness analysis Should be used when
  policy objective is inflexibly defined there are
  low risks of trade-offs or when b-c analysis has
  defined the approach.
• Threshold analysis Should be used when society
  has agreed on a policy that should be achieved
  at any cost. Should be restricted to a few
  policies.

18
Good practices in RIA methods
www.regulatoryreform.com

• Assess all important costs and benefits.
• Use the benefit-cost principle. Regulations
  should produce benefits that justify costs.
• But form of analysis should be based on practical
  judgments about feasibility and cost (a more
  rigorous method for high-cost regulations and a
  less rigorous method for low-cost regulations).
• Supplement quantitative analysis with qualitative
  analysis where needed.
• Guidelines should be standardized (such as the
  social discount rate, use of best estimates).
• Quantitative risk assessment improves the
  capacity of a government to focus on the most
  important risks and reduce them at lowest cost
  
• Bad practices in RIA methods
• Adding more and more analysis to satisfy interest
  groups that they are represented in RIA.
• Leaving too much flexibility in choice of method
  to regulators.

19
5. Collecting data
www.regulatoryreform.com

• Good practices
• Development of strategies and guidance for
  ministries is essential.
• Identify and gain access to existing government
  databases that will be useful (business
  registries, food consumption, etc).
• Public consultation is useful but must be
  carefully structured.
• Expert groups such as academic and industry
  groups
• Business Test Panels organized for a particular
  regulation. They can be fast, but data quality
  can be a problem.
• Model enterprises that represent the affected
  population
• Opinion surveys can identify major issues
• Direct interviews can provide more detail.
• Focus groups can provide a wider range of data.
• Bad practices
• No advance planning for data collection, slowing
  down RIA when a regulation is being drafted.
• Relying too much on regulated companies for data.
  Not enough diversity in data sources.

20
6. Presenting benefits and costs, treating
uncertainty
www.regulatoryreform.com

• Sensitivity analysis can reveal whether, and to
  what extent, the results of the analysis are
  sensitive to plausible changes in the main
  assumptions and data.
• Treatment of uncertainty must be guided by
  principles of full disclosure and transparency.
• In some cases, the level of scientific
  uncertainty may be so large that you can only
  present discrete alternative scenarios without
  assessing the relative likelihood of each
  scenario quantitatively. Results should be
  presented from a range of plausible scenarios,
  with information that might help in qualitatively
  determining the most likely scenario.
• When uncertainty has significant effects on the
  final conclusion about net benefits, consider
  additional research before rulemaking. The costs
  of being wrong may outweigh the benefits of a
  faster decision.

21
Integrate RIA with the policy-making process,
beginning as early as possible
www.regulatoryreform.com

• Good practices
• Justification for new regulatory proposals should
  always be presented in RIA format
• Benefits and costs should be clearly stated in
  non-technical terms
• Bad practices
• RIA justifies decisions already taken
• RIA becomes a paperwork exercise that is not
  well understood and not used

22
7. Consulting with stakeholders
www.regulatoryreform.com

• Good practices
• Public involvement in RIA has several benefits.
• Those affected by regulations can provide the
  data necessary to complete RIA.
• Consultation provides important checks on the
  feasibility of proposals, on the alternatives
  considered, and on the degree of acceptance of
  the proposed regulation by affected parties.
• Releasing RIAs with the draft regulatory improves
  understanding of the regulation by the public

23
Consulting with stakeholders (2)
www.regulatoryreform.com

• Good practices (cont)
• An organized and predictable consultation process
  is necessary. But who should be involved? And
  how?
• Consultation should involve all major affected
  interests
• Place RIA on the Internet
• Notice and comment procedures
• Public hearings
• Advisory groups reflecting all the stakeholders
• Respond to comments in final RIA

24
Consulting with stakeholders (3)
www.regulatoryreform.com

• Good practices
• Train key interest groups in how to respond
  constructively to RIA
• Bad practices
• Public involvement too limited among special
  interest groups such as big companies
• Consultation too time intensive and costly.
• Not an opinion poll. Poor quality responses if
  consultation is not carefully structured.
• Expectations too high about changes to be made.

25
8. Institutions for RIA, and the importance of
quality control
www.regulatoryreform.com

• Good practices
• Responsibilities for RIA shared between
  ministries and a central quality control unit.
• Ministries should be the primary drafters of RIAs
  to help them ask the right questions
• Oversight and quality control exercised by bodies
  that are independent of the regulating
  ministries.
• Bad practices
• Taking too much responsibility from ministries
• Leaving too much discretion in the ministries

26
Two levels of oversight Ministerial committee
and an expert Secretariat
www.regulatoryreform.com

• A ministerial-level committee for regulatory
  reform and SME development is needed to oversee
  reforms at the political level and insist on
  ministerial action.
• A dedicated and expert group is needed at the
  working level, usually as a Secretariat to the
  ministerial committee

27
Putting a minister in charge
www.regulatoryreform.com
28
with central oversight of regulatory reform
www.regulatoryreform.com
29
Designing better RIA training
www.regulatoryreform.com

• Good practices
• Set the basic capacities to be established in
  training target the audience
• Incorporate regular RIA training into national
  training programs for the public administration
• Manuals or guidance must complement training. The
  best are simple, based on concrete examples, and
  are updated frequently.
• Help desks to offer expert advice to ministries.
  
• Bad practices
• One-time training that is superficial
• Training of analysts and not policy officials
• Guidance too complex and technical