Title: Evaluation, Accountability and Improvement in Education: Perspectives from the OECD Review on Evaluation and Assessment Frameworks
1Evaluation, Accountability and Improvement in
Education Perspectives from the OECD Review on
Evaluation and Assessment Frameworks
- ww.oecd.org/edu/evaluationpolicy
Paulo Santiago Senior Analyst, Directorate for
Education, OECD
Improving Education through Accountability and
Evaluation Lessons from around the
World International Conference Rome, 3 October
2012
2Outline
- OECD Review on Evaluation and Assessment
Frameworks for Improving School Outcomes - Key themes
- The rise of educational measurement and
indicators development - The growing prominence of accountability as a
purpose of evaluation - Establishing links to classroom practices
- Building capacity for evaluation and assessment
- Fostering synergies within the evaluation and
assessment framework
3- 1. OECD Review on Evaluation and Assessment
Frameworks for Improving School Outcomes
4OBJECTIVES
- Purpose To explore how systems of evaluation and
assessment can be used to improve the quality,
equity and efficiency of school education. - A Review of national approaches to evaluation and
assessment in school education student
assessment, teacher appraisal, school evaluation,
school leader appraisal, education system
evaluation - The Review
- Synthesises research-based evidence on the
impact of evaluation and assessment strategies
and disseminate this knowledge among countries. - Identifies innovative and successful policy
initiatives and practices. - Facilitates exchanges of lessons and dialogue.
- Identifies policy options for governments to
consider.
Comprehensive approach Investigation of each
component individually, as well as the coherence
of the framework as a whole (including the links
between the different components).
5METHODOLOGY
- Analytical strand
- Identifying the key questions for analysis and
the background information needed from countries - Reviewing the literature and evidence on the
impact of evaluation and assessment procedures - Gathering data on countries policies and
practices (Country Background Reports) - Country Review strand
- Country Reviews provide specific advice to
individual countries. - OECD-led Review Team with external experts
- The scope and focus is determined by the country
in consultation with the Secretariat - Synthesis report
- Comparative report to analyse policy options and
highlight good practices across countries. - Twenty six systems participating
- Australia, Austria, Belgium (Flemish Community),
Belgium (French Community), Canada, Chile, Czech
Republic, Denmark, Finland, France, Hungary,
Iceland, Ireland, Italy, Korea, Luxembourg,
Mexico, Netherlands, New Zealand, Norway, Poland,
Portugal, Slovak Republic, Slovenia, Sweden,
United Kingdom (Northern Ireland).
6 7The rise of educational measurement and
indicators development
- Trends
- The growing emphasis on measuring student
outcomes the proliferation of education
indicators and the establishment of education
targets - Potential
- Student outcomes as the focal point for analysis
- Monitoring key student learning outcomes
- Formative use of standardised tests
- Hold stakeholders accountable
- Challenges
- Ensuring breadth of performance measures
- Securing fair and meaningful comparisons
- Avoiding detrimental effects of uses for
accountability - Preventing dominance of quantitative over the
qualitative - Options
- Ensure policy making is informed by high quality
measures, but not driven by their availability
the need to complement with qualitative
analysis - Ensure a broad approach to national monitoring
- Clarity of purposes for the uses of standardised
tests results
8The growing prominence of accountability as a
purpose of evaluation
- Trends
- Public reporting of school results (greater
transparency, reporting requirements) rewards
and sanctions on the basis of evaluation and
assessment results (e.g. teachers, schools). - Potential
- Creation of incentives for improved performance
opportunity to recognise and reward - Identification of underperformance
- Facilitates school choice
- Challenges
- Range of potential detrimental effects (e.g.
teaching-to-the-test, narrowing of
curriculum) - Accountability function of evaluation not to
hinder the improvement function - Conveys a control, compliance, measurement
concept of evaluation - Focus on accountability as a result of a strong
top-down national vision of evaluation might
constrain the ownership of EA by school agents - Options
- Communicate rationale for evaluation and
assessment objective is improvement - Build on a range of evaluation and assessment
components achieving a variety of functions - Publication of quantitative data to be perceived
as fair by schools set in a wider set of evidence
(of a qualitative nature) - Establish safeguards against overemphasis on
student standardised tests (more relevant for
whole-school evaluation)
9Establishing links to classroom practices
- Trends
- There is often focus on structures, procedures,
programmes and resources in a top-down approach - Potential
- National agendas for education to be strengthened
by consistency of evaluation procedures - Ensuring equity of objectives
- Challenges
- Evaluation and assessment have no value if they
do not lead to the improvement of classroom
practice and student learning - Improvement function accomplished more at a local
level need to secure adequate links to the
classroom - Risks that evaluation procedures do not place
adequate focus on what is arguably the most
important area teaching and learning - Options
- Critical to ensure that the evaluation of
teaching and learning quality is central to
evaluation procedures - Build on teacher professionalism
- Teachers to have ownership of student assessment,
build teachers ability to assess against
educational standards, teachers to be involved in
school evaluation, emphasis on teacher evaluation
for the continuous improvement of teaching
practices.
10Building capacity for evaluation and assessment
- Trends
- Evaluation and assessment policies often
introduced with no due attention to capacity
development (e.g. competencies for evaluation and
assessment) - Challenges
- Legitimacy of evaluators and accountability
procedures - The effectiveness of evaluation relies to a great
extent on ensuring that both those who design and
undertake evaluation activities as well as those
who use their results have the proper skills and
competencies. - Examples of areas for capacity development
standardised test development formative
assessment assessment against standards running
systems of externally-based student examinations
analytical capacity in education agencies to use
system level information data handling skills of
school agents formal evaluators of individual
school agents competencies for classroom
observation. - Options
- Sustaining efforts to improve the capacity for
evaluation and assessment - Strengthen school leaders capacity for school
development and instructional leadership - Ensuring support from the centre and
identification of best practice - Need for a strong capability at the national
level to steer evaluation
11Fostering synergies within the evaluation and
assessment framework
- Trends
- Most countries do not have an integrated
evaluation and assessment framework but instead a
series of components operating at different
levels that developed independently of each other
over time. - Potential
- Build synergies
- Generate complementarities
- Avoid duplication
- Prevent inconsistency of objectives
- Challenges and options
- Establishing articulations within the evaluation
and assessment framework - Within specific components of the overall EA
framework - e.g. teacher appraisal and teacher professional
development self- and external school
evaluation. - Between specific components of the overall EA
framework - e.g. teacher appraisal, school evaluation and
school development school evaluation and system
evaluation school evaluation and the appraisal
of school principals. - Processes to ensure the consistency of EA
procedures - Moderation processes for teacher appraisal and
teacher-based assessment - Clarity of responsibilities across the framework
12- Thank you for your attention!
- paulo.santiago_at_oecd.org
- www.oecd.org/edu/evaluationpolicy