Criteria and methodology for the evaluation of minority policies

1
Criteria and methodology for the evaluation of
minority policies

• Informal International Consultative Meeting in
  the Area of Minority Issues, European Centre for
  Minority Issues, Flensburg, 16-18 September 2004
• François Grin
• University of Geneva Education Research Unit

2
Structure of presentation

1. Introduction rationales for action
2. Three features of policy analysis
3. Applying policy analysis six  vignettes 
4. Conclusion implications for minority policy
   development

3
Combining different rationales for action in
minority policy

• Principle-based ? application of norms and
  standards, e.g. rights-based approach
• Goal-oriented ? selection and design of
  policies yielding the  best  results.

4
Goal-oriented types I and II

• Type I goal-oriented approaches give substance
  to principles (principles precede
  analytically-informed action)
• Type II Goal-oriented approaches shape
  principles (analysis precedes the setting of
  standards)

5
Different subcultures the diversity clover

• It is important to remember that different
  subcultures exist in the area of minority issues.
• Four main families can be identified among
  professionals as well academics, centred
  respectively on the issues of  national
  minorities ,  language minorities ,
   migrants  and  indigenous peoples .
• Interaction between these  families  and their
  respective  subcultures  remains limited, but
  needs to be developed in order to locate the most
  appropriate expertise and intellectual resources.

6
Policy v. political questions

• What (exactly) should we do (once principles are
  by and large agreed upon)?
• How should we do it?
• ? Choice and application of criteria
• ? Contribution of policy analysis in three
  principles and six  vignettes 

7
Basics of policy analysis

• Aims at creating knowledge about the consequences
  and performance of possible (ex ante) or existing
  (ex post) policies
• Stresses the cause-and-effect links between
  policy action (upstream) and results (downstream)
• Relies on the principle of comparison between
  competing options

8
Methodological implications

• Policy analysis therefore requires
• Careful investigation of the cause-and-effect
  relationships through which policy decisions are
  related to results ? model building
• Identification and measurement of policy measures
  ( policy inputs ), intervening variables,
  direct policy effects ( outputs ) and results
  in terms of the processes actually targeted
  ( outcomes ).

9
Vignette 1 outcomes v. outputs

• Outputs direct effect of a policy
• Outcome final effects of a policy, in terms of
  the variables one wishes to influence.
• ? Policy measures produce outputs, which
  influence processes that result in outcomes

10
Vignette 2 on the nature and role of
analytical models

• A model is needed to connect the policy decision
   upstream  (A) to its direct  outputs  (B)
  through to the resulting   outcomes  (C) that
  occur downstream, given certain intervening
  variables (D).
• Bf(A) and Cg(B, D), hence Cgf(A), D
• Rather than a representation of reality, a model
  is a tool to help us think about reality ? focus
  on causal links, with particular attention to
  necessary and sufficient conditions
• A model is  a metaphor whose implications have
  been spelled out .

11
Vignette 3 dearth of targeted models

• Relatively little is known about the causal
  relationships between policy measures
   upstream  and outcomes  downstream , when
  the outcome is (i) a defining feature of
  minorities, or (ii) a variable whose level is
  structurally correlated with minority identity or
  experience.
• Need for model development ? example the P-TOP
  ( policy-to-outcome path ).

12
(No Transcript)
13
Vignette 4 working backwards

• Identify the desired outcome (e.g. minority
  language use)
• Then analyse this outcome as the result of a
  process in which different variables operate
• Then select, among these variables, those that
  can be influenced through policy measures.

14
Vignette 5 The role of indicators

• Indicator measurement of a variable which is
  relevant to the policy-influenced causal
  relationship. Indicators must be
• Context-specific
• Non-circular
• Clearly located along a policy-to-outcome path
  (?? (i)  systemic  (ii)  interrelated  (iii)
  some reponsive to policy)
• Relatively easy to collect

15
Vignette 6 criteria for good/best
practice/policy
Processes Results
Resource allocation Rule of law accountability Effectiveness Cost-effectiveness
Resource distribution Democracy Fairness
16
Defining and understanding criteria

• Resource allocation (1) effectiveness ( making
  a difference )
• Resource allocation (2) cost-effectiveness
  ( least-cost effect )
• Resource distribution fairness as result
  democracy as process
• All four criteria can be measured through
  indicators

17
Implications 1we need to

• distinguish clearly between the outputs and the
  outcomes that can be associated with each option.
• develop a model linking the policy measure, its
  outputs, and the outcome identify causal links,
  necessary and sufficient conditions, etc.

18
Implications 2 and to

• explicitly state the substance of
   effective ,  cost-effective ,  fair  and
   democratic  in the context of the
  policy/programme/project under consideration
• pick or design indicators for effectiveness,
  cost-effectiveness, fairness, and democracy,
  making sure that these indicators display the
  necessary properties