Policy%20Analysis%20and%20Program%20Evaluation

1
Policy Analysis and Program Evaluation

• Lecture 15 Administrative Processes in
  Government

2
Stages of the Policy Process

• Problem Definition and Agenda Setting.
• Key actors policy entrepreneurs, elected
  executives, interest groups
• Secondary actors public administrators,
  legislators
• Policy Formulation.
• Key actors Public administrators, policy
  experts, legislators
• Secondary actors Elected executive and
  legislative staff.

3
Stages of the Policy Process

• Policy legitimation.
• Key Actors Legislators, elected executives,
  judges.
• Policy implementation.
• Key actors Public administrators.
• Secondary actors Elected executives,
  legislators, interest groups.
• Policy evaluation.
• Key actors Policy experts, public
  administrators.
• Secondary actors elected officials.

4
Agenda Setting

• Key questions
• What is the agenda setting process like?
• What is the role of the public administrator in
  this set of decisions?
• What can and should that role be?

5
Agenda Setting

• What is the agenda-setting process like?
• The list of issues up for public consideration at
  a particular time.
• Not that neat. Actually an array of agendas
  crosscut by separation of powers and federalism.
• Systemic versus routine agendas.
• Systemic list of larger concerns of society.
• Routine ideas and procedures of government
  itself.

6
Agenda Setting

• What roles do public administrators play?
• Administrators do not dominate the process at
  every stage or from beginning to end.
• At the level of the systemic agenda as society
  considers various problems and issues,
  administrators play a secondary role.
• At the level of the routine agenda as society
  focuses on technical matters, administrators play
  a more prominent role.

7
Agenda Setting

• What roles do public administrators play?
• Phases of the process at the stage where issues
  are placed on the agenda, administrators play a
  secondary role.
• Phases of the process at the stage where
  alternatives are developed, administrators play a
  more significant role.
• But, you cannot assume sharp distinctions.
• Policy process more like a garbage can the
  process is haphazard.

8
Agenda Setting

• Summary The agenda setting process is really a
  set of processes, and the administrative role is
  actually an amalgam of varying and complex roles.

9
Agenda Setting Guidelines for Effective Practice

• Effective administrative participation in agenda
  setting varies with context.
• Public administrator should be prepared to play
  major roles in routine agenda setting.
• While administrators are unlikely to be the main
  actors on the systemic agenda, they can fulfill
  important roles by stimulating and managing
  creativity in administrative institutions
  (diversity, external interaction, providing
  structures for voice, avoiding reactive policy,
  and acting as policy entrepreneurs.

10
Agenda Setting Guidelines for Effective Practice

• Effective administrative participation in agenda
  setting varies with context.
• Administrative roles and their effectiveness
  differ by time, jurisdiction and substantive
  issue. Administrators should be sensitive to
  these variations and adopt appropriate strategies
  and tactics.
• Timing Realignment periods.
• Jurisdiction Lower more influence.
• Issue definition. Level of political conflict
  affects administrative influence.

11
Agenda Setting Guidelines for Effective Practice

• Despite variations, public administrators
  generally can enhance their role in agenda
  setting by doing the following things.
• They can develop and refine problem-finding
  routines.
• They can develop systematic efforts to identify
  and articulate, in policy circles, the concerns
  and issues of the public.

12
Agenda Setting Guidelines for Effective Practice

• Despite variations, public administrators
  generally can enhance their role in agenda
  setting by doing the following things (contd.)
• They can create and refine contacts with groups
  of specialists external to the agency.
• They can retain in-house research and policy
  monitoring instead of contracting it out.
• They can concentrate agenda-setting efforts in
  substantive specialties close to the
  administrators (and agencys) legitimate
  jurisdiction and demonstrated achievement.

13
Agenda Setting Guidelines for Effective Practice

• Despite variations, public administrators
  generally can enhance their role in agenda
  setting by doing the following things (contd.)
• They can time agenda-setting efforts wisely.
• They can iron out differences within the policy
  or the professional community before seeking
  agenda status at some broader level.
• They can administer the preparation and
  maintenance of feasible options for possible
  adoption.
• They can perform the function of policy
  entrepreneur within their own specialties.

14
Agenda Setting Guidelines for Effective Practice

• Through all of these efforts at effective
  participation in the process, administrators
  should remain acutely aware of the normative
  issues at stake.
• Administrators have a positive duty not only to
  be obey and defend the rules, but to work on
  behalf of democratic politics and the cause of
  effective public policy.

15
Agenda Setting Guidelines for Effective Practice

• In general, administrators should become more
  active in agenda setting
• The more biased or skewed the existing
  governmental agenda is vis-à-vis the systemic
  agenda.
• The more the issues seem to be generated by the
  routine bureaucratic functions of government.

16
Agenda Setting Guidelines for Effective Practice

• In general, administrators should become more
  active in agenda setting (contd.)
• The less the administrator disguises his or her
  activities behind a veneer of ministerial
  behavior.
• The more that debate about key issues or
  alternatives is based on demonstrably false or
  dubious facts.
• The fewer the alternative channels are available
  to those who are concerned about the issue and
  are seeking access.
• The more powerless are those who seek such access.

17
Effective Policy Analysis

• Policy analysis can be defined as simply an
  estimate of what would happen if some
  governmental procedure were changed.
• Most new policy analysts do not appreciate the
  interaction of the political process with factual
  information and quantitative analysis.

18
The Politics of Policy Analysis

• For the most part, policy analysis is a
  discretionary commodity. Some political
  executives like a lot of it. Some (usually
  ideologues) dont use it at all. Policy staff
  often have to demonstrate their utility to each
  new administration.

19
The Politics of Policy Analysis

• The theoretical model of policy analysis uses
  review and evaluation of factual information
  provided by totally objective, apolitical, expert
  professionals to provide a firmer basis for
  political actions.

20
The Politics of Policy Analysis

• Once the results are available, they are
  considered in the context of a political template
  provided by elected and appointed officials and
  producing a set of alternatives with different
  physical, social, economic, and political costs
  and benefits.

21
The Politics of Policy Analysis

• After a decision is made, the time comes to close
  ranks, cease internal debate, and provide a
  united front on the wisdom of the decision, with
  the policy analysts prudently keeping their
  opinions to themselves.
• There are two judges for the suitability of a
  politicians decisions the press and the public.

22
The Politics of Policy Analysis

• Policy analysts who wish to survive their
  politicians administration must appear to be as
  factual, objective, balanced, and value-free as
  possible.
• The skill to balance an objective analysis with
  the production of politically motivated public
  documents for the policy-maker is one of the
  toughest balancing acts for an analyst.

23
The Politics of Policy Analysis

• On occasion, the policy analyst may provide
  advice on politically relevant aspects of the
  analysis.

24
The Politics of Policy Analysis

• During a crisis, the duty of the analyst is to
  place the emergent issue in a context meaningful
  to the decision level how the issue surfaced,
  who else cares, what everyone else is doing, and
  what the organization has done in the past.
• The hardest single part of the interaction
  between the analyst and the executive is the
  precise definition of the question to be
  addressed.

25
The Politics of Policy Analysis

• The political context of public sector decisions
  is constantly changing and the analyst must
  frequently reevaluate the stakeholders (the
  people and groups affected).
• Specification of the criteria for evaluation of
  policy alternative should be done jointly with
  the policy officials.
• The real output of policy analysis is not tidy
  options on paper, but increased understanding of
  and confidence in decisions.

26
Characteristics of a Successful Policy Analyst

• Technical skills quantitative techniques not
  used in most analyses, but should be understood
  by analyst.
• Multidisciplinarity knowledge from many
  different disciplines.
• Creativity Analysts must be able to frame
  issues quickly into fundamental questions,
  explore related information from all conceivable
  sources, and provide at least some useful
  insights in very little time.

27
Characteristics of a Successful Policy Analyst

• Clarity The model must get the point across
  clearly and quickly.
• Poise Policy analysis is part of an adversarial
  process. Instant analysis and intense debate is
  the rule.
• Expertise Analysts should understand the issue
  under consideration. Competence affects
  credibility.
• Political savvy Policy analysis must be
  politically sensitive.

28
Technical Sophistication

• Widespread use of computer technology is a
  reality, but the technology does not trump the
  politics.

29
Efficient Use of Technical Analysis

• Analyst must perform triage to determine when
  technical analysis is mandatory, potentially
  helpful, or irrelevant.
• Once a decision is made to do an analysis, the
  scope must be determined. This will be affected
  by resources and complexity.
• The quality and depth of the analysis will be
  affected by how high on the decision chain it
  must go.

30
Implementing Public Programs

• What happens after a decision is made?
• Key questions.
• Does the policy get put in place as desired by
  the legislators or the agency head?
• If not, what has occurred to block or modify the
  desired change.
• If successful, what factors in the institutional
  process were important in bringing about the
  desired change.

31
Implementing Public Programs

• Key steps for individual projects.
• Keep projects relatively simple, and seek
  relevant models.
• Plan ahead, but not beyond the available data,
  and involve key actors.
• Error correction and flexibility are keys to
  implementation.
• Get the right information and get it quickly.
• Never forget that serving target groups is your
  main objective.

32
Implementing Public Programs

• Key steps for cross-government programs.
• Avoid a compliance mentality (dont push rules
  and regulations).
• Stress staff competence.
• Get good information.
• In the agency management strategy for
  grants-in-aid, have as a central objective
  increasing state and local commitment to agency
  goals and capacity to provide particular services
  and make needed discretionary judgments.

33
Evaluating Public Programs

• Program evaluation is a way of bringing to public
  decision-makers the available knowledge about a
  problem, about the relative effectiveness of past
  and current strategies for addressing or reducing
  that problem, and about the observed
  effectiveness of particular programs.

34
Administrative Purposes for Evaluation

• Policy formulation to assess or justify the
  need for a new program and to design it optimally
  on the basis of past experience.
• Information on the problem addressed by the
  program how big is it? What is its frequency and
  direction? How is it changing?
• Information on the results of past programs that
  dealt with the problem were those programs
  feasible? Were they successful? What difficulties
  did they encounter?
• Information allowing the selection of one program
  over another what are the comparative costs and
  benefits? What kinds of growth records were
  experienced?

35
Administrative Purposes for Evaluation

• Policy execution to ensure that a program is
  implemented in the most cost-effective and
  technically competent way.
• Information on program implementation how
  operational is the program? How similar is it
  across sites? Does it conform to the policies and
  expectations formulated? How much does it cost?
  How do stakeholders feel about it? Are there
  delivery problems or error, fraud, and abuse?

36
Administrative Purposes for Evaluation

• Policy execution to ensure that a program is
  implemented in the most cost-effective and
  technically competent way.
• Information on program management what degree of
  control exists over expenditures? What are the
  qualifications and credentials of the personnel?
  What is the allocation of resources? How is
  program information used in decision making?
• Ongoing information on the current state of the
  problem or threat addressed in the program is
  the problem growing? Is it diminishing? Is it
  diminishing enough so that the program is no
  longer needed? Is it changing in terms of its
  significant characteristics.

37
Administrative Purposes for Evaluation

• Accountability in public decision making to
  determine the effectiveness of an operating
  program and the need for its continuation,
  modification, or termination.
• Information on program outcomes or effects what
  happened as a result of program implementation?
• Information on the degree to which the program
  made or is making a difference what change in
  the problem or threat has occurred that can be
  directly attributed to the program?
• Information on the unexpected (and expected)
  effects of the program.

38
Functions and Roles of Evaluation Sponsors

• Executive branch (federal, state, local).
• Program managers (cost-effectiveness).
• Agency heads and top policy makers (need,
  effectiveness).
• Central budget or policy authorities
  (effectiveness, need).

39
Functions and Roles of Evaluation Sponsors

• Legislative branch
• Congressional and legislative policy and
  evaluation offices (all aspects).
• Legislative authorization, appropriations, and
  budget committees (program funding and
  refunding).
• Oversight committees (all aspects).
• Regardless of sponsor, evaluators should clearly
  specify the objectives and limitations of each
  evaluation.

40
Functions and Roles of Evaluation Sponsors

• As a general rule, public administrators should
  expect their work on program effectiveness and
  feasibility to be of more general use than their
  work on implementation, which will be of most use
  to program managers and agency heads.
• Information needs will be larger for large
  programs than small, new programs over old.

41
Evaluation Approaches

• Front-end analysis evaluative work conducted
  before a decision to move ahead with a program.
• Evaluability assessment reasonableness of
  assumptions and objectives, comparison of
  objectives to program activities, feasibility of
  full-scale evaluation.

42
Evaluation Approaches

• Process evaluation describe and analyze the
  processes of implemented program activities
  management strategies, operations, costs,
  interactions, etc.
• Effectiveness or impact evaluation how well has
  a program been working? Are the changes the
  result of the program?

43
Evaluation Approaches

• Program and problem monitoring continuous
  rather than snapshot inform on problem
  characteristics or track program or problem
  progress in several areas.
• Metaevaluation or evaluation synthesis
  reanalyzes findings from several analyses to
  determine what has been learned.

44
Evaluation Approaches