International comparisons of performance in public services

1
International comparisons of performance in
public services

• Outcome based measures for education

Mary OMahony Philip Stevens
National Institute of Economic and Social Research
2
Introduction
Aims of this presentation

• Focus on outcome-based measures for education
• Volume measures
• Earnings outcomes
• Quality adjustment
• International comparative context

3
Applying the outcome method in practice

• Need to define the outcome measure
• Outcomes will depend not only on the service
  provided but also on other extraneous or
  background influences
• Need to control for these influences
• Regression methods regress outcomes on controls

4
An application to education

• Measures of output and outcomes
• Volume measure number of pupils
• Lifetime earnings wages and activity rates by
  qualification group
• Test scores - pupils reaching threshold level
• Qualification cohorts compare wages of people
  obtaining qualifications now vs. some time
  earlier

5
Volume of output
Pupil/Student Numbers, UK, 1994-2001
Share in total pupils in 1995
Annual growth rate, 1994-2001
6
Earnings outcomes

• Education impacts on two types of labour market
  outcomes
• Earnings
• Economic activity
• Earnings estimated as a function of education,
  experience and a set of control variables

7
Earnings outcomes

• Economic activity
• Employed (earns a wage, Y)
• Unemployed (earns benefit, U)
• Inactive (earns nothing)
• Total effect on lifetime earnings a product of
  earnings in each state and the probability of
  being in each state, i.e.

8
Earnings outcomes

• Econometric methods
• Returns to education
• Standard Mincer equations, correcting for
  selection bias
• Activity equations
• multinomial logit with three outcomes
  employment, unemployment and inactivity
• Screening vs. human capital explanations
• No instruments available to differentiate between
  causes of wage premia

9
Relative Lifetime earnings
10
Translating lifetime earnings to education
outcome measures

• Use increment to lifetime earnings, EO, to
  calculate weights on volume measures
• Compare lifetime earnings with persons with
  qualification i with the next higher
• These can be combined to obtain Tornqvist index

where
11
Output and outcome measures for UK education,
1994-2001
12
Output and outcome measures for US education,
1994-2001
13
Quality adjustments

• Three types of quality change
• Growth in higher quality groups
• Differential growth in lifetime earnings
• Growth in pupil effectiveness
• 1 2 are between group, 3 is within group
• Method above only deals with first two

14
Quality of output ITest scores
15
Pupil effectiveness index

• Compare growth in natural units versus growth in
  effectiveness units

16
Compare wage rates by decile persons aged under
30
17
Quality adjusted indices based on test scores for
UK education
18
Quality adjustments Problems with test scores

• How do we measure effectiveness?
• Ideally need relative wages for each
  qualification grade
• Incomplete coverage
• Changes over time in examination marking - Grade
  Inflation

19
Quality of output II Cohort Analysis

• Two cohorts one leaving school before 1994, one
  after that date
• We can consider the Raw difference between
  earnings of cohorts
• Adjusted difference removes effect of
  experience, as estimated in earnings equations

20
Quality of output II Cohort Analysis

• Impact of cohort on wages, adjusted for
  experience (annual change)
• GCSE 0.069
• A-level -0.04
• degree 0.006

21
Inputs and productivity
Inputs

• Measurement similar to outputs
• Use cost share weighted changes in inputs
  (labour, capital, intermediate purchases)
• Result is a quality-adjusted aggregate input
  measure
• Few conceptual problems specific to public
  service provision since we have market
  information on input costs

22
Labour input in education, UK
23
UK labour productivity
24
US productivity growth, 1994-2001
25
Annual differences in labour productivity growth
rates, (UK-US)Education and the aggregate economy
26
Conclusions and extensions

• First analysis suggests UK labour productivity
  growth in education higher than in US in period
  1994-2001
• Estimate for longer period 1979 to present
• Include cohort effects to quality adjust
• Estimate TFP rather than labour productivity
• requires estimates of capital intermediate
  inputs
• Estimate relative productivity levels