National Developmental evaluation - PowerPoint PPT Presentation

Loading...

PPT – National Developmental evaluation PowerPoint presentation | free to download - id: 5534d0-YjlkM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

National Developmental evaluation

Description:

Title: National Developmental evaluation Author: Lyytinen, Heikki Kalevi Last modified by: Lyytinen, Heikki Kalevi Created Date: 8/8/2008 10:24:58 AM – PowerPoint PPT presentation

Number of Views:112
Avg rating:3.0/5.0
Slides: 51
Provided by: LyytinenH
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: National Developmental evaluation


1
National Developmental evaluation
  • Kauko Hämälainen
  • Heikki K. Lyytinen
  • The Finnish Education Evaluation Council
  • The Miracle of PISA Conference 10. -12-Sept. In
    Helsinki

2
Content
  • 1.Where did we come from and where are we going
    to in the national evaluation?
  • 2. Within what kind of administrative and
    organizational structure we act?
  • 3. What kind of demands/requirements we make for
    the national evaluation?
  • 4. What are the values and principles we set on
    the evaluation?
  • 5. How do we carry out the developmental
    evaluations in practice ?
  • 6. What have been our evaluation themes?
  • 7. Examples of evaluation practices (evaluation
    projects in practice)

3
Change of Educational Steering System in Finland
Situation in 1970s and 1980s Situation in 1990s/2000
Centralised control and decision-making Centralised control and decision-making Devolution of power
centralised curricula long-term plans budgeting based on expenditures controlling follow up external evaluation inspections conducted by state authorities etc. self-governance school-based curricula distinctive educational profile of educational institutions self-direction and -regulation learning organisation as a model of functioning self-evaluation and own control performance-based funding etc. new evaluation organisations National evaluation plan
4
Evaluation instead of inspection
  • no separate school inspectorate and inspection
    visits to schools
  • the activitis of education providers are quided
    by objectives laid down in legislation and the
    national core curricula
  • the system relies on the proficiency of teachers
    in their efforts to meet the objectives
  • strong focus on self-evaluation and external
    evaluation

5
The Forms of Steering and Improving the Quality
of Education in the Finnish School System Focus
in steering by information, monitoring and
evaluation and steering by professional expertise
  • Steering by information, monitoring and evaluation

Steering by professional expertise freeing the creative resources of the school Aim improved quality of education Market-driven steering competition between schools
Norm- and resources-based steering
6
Challenges for renewing evaluation in educational
services
  • responsibility to organise educational services
    also includes evaluating them
  • directing educational services requires goals
    and evaluating how well they have been met
    (so-called profit responsibility)
  • evaluation seen as a comprehensive process (cf.
    e.g. quality assurance)
  • evaluation as a tool for learning and
    development( developmental evaluation)

7
  • evaluation adds to the active role of
    actors(empowerment)
  • knowledge of the current state of affairs
    essential when setting goals for development
  • controlled change when steering towards future
    goals (controlled change based on facts/evidence
    based cahnge management)

8
TOWARDS NEW EVALUATION CULTURE
  • 1 Explorative/evaluative work community
    EVIDENCE BASED
  • continuing evaluation transparency
  • making use of research and evaluation knowledge
  • 2 Reflective work community - towards culture of
    EMPOWERMENT
  • independence, motivation, atmosphere,
    appreciation, security
  • 3 Network-oriented operation cultures
  • 4 Making use of parallelisms
  • peer evaluation, benchmarking
  • 5 From data management to knowledge management
    collective awareness
  • 6 Demands for involvement, multiple perspectives,
    and consensus authentic dialog
  • 7 Managing the whole of evaluation

9
Towards a new evaluation system
Change
Transition period 2-3 years
New evaluation awareness 2-3 years
Continuous development (new evaluation
culture) Change of evaluation culture Towards
developmental evaluation
External evaluation system of expert networks
Internal administrative evaluation system
Administrative dependence (internal evaluation)
Expert network based independence
(external/internal)
10
Benefitting from the network both in evaluation
and in project planning
  • Evaluation projects are organised in two phases
    to maximise
  • best expertise
  • commitment of the field, partners and interest
    groups
  • independence and
  • impact
  • 1. Planning group
  • expertise of the field and interest groups
  • draws up a plan and schedule for the project and
    its different stages
  • proposes assessors to be appointed in the
    evaluation group to the Council
  • 2. Evaluation group
  • independent experts who work in assessment and
    research
  • evaluate and draw conclusions with further
    recommendations
  • write a report and are responsible for it

11
National production of evaluation information to
serve policy-making
Parliament
The Government
The Ministry of Education
The other evaluation organisations (national
and international)
The Finnish Higher Education Evaluation Secreteria
t 12 members
The Finnish Education Evaluation Council 14
members
The Education Evaluation Secreteriat
The Evaluation Secreteriat
National evaluation and information production
network
National Board of Education Education Providers
Private Evaluation Organisations
- Regional Authorities - Provincial State
Offices - Provincial Councils - Employment and
EconomicDevelopment Centres
Universities Polytechnics Statistics Finland Res
earch Institutes in Univeristies
Schools
12
  • EducationEvaluation council in conjunction with
    the Ministry of Education since April 2003
  • planning, co-ordinating, managing and
  • developing the evaluation of basic education
  • and upper secondary education and training
  • The polytechnics and univeristies are responsible
    for the evaluation of their own operations and
    outcomes (support from the Higher Education
    Evaluation Council)

13
(No Transcript)
14
The Finnish Higher Education Evaluation Council
(FINHEEC)
  • (FINHEEC) is an independent expert body assisting
    universities, polytechnics and the Ministry of
    Education in matters relating to evaluation.
  • The evaluations conducted by FINHEEC can be
    divided into three rough categories
  • Evaluations of higher education institutions
    Institutional evaluations (universities and
    polytechnics), audit of quality work, and
    evaluations for the accreditation of the
    polytechnics
  • Programme and thematic evaluations
  • Accreditation of professional courses offered by
    higher education institutions

15
NATIONAL EVALUATION SYSTEM
Indicators -quantitative -qualitative
Learning outcomes
Situation and system- evaluations
Thematic evaluations
International cooperation PISA Copenhagen process
Methods and models
Self-assessment Quality control External and
internal audits Paperless enviroment Indicators
  • General qualifications
  • learning to learn

General knowledge/ School subjects
Professional competence
Evaluation research
16
AIMS OF EVALUATION IN THE FINNISH EDUCATION SYSTEM
The aims of evaluation are to gather and analyse
information which serves educational
decision-making at the national policy and local
levels and contributes to the development of
education in general. In addition, the aim is
to support students learning and the work of
educators, as well as institutional development.
17
Educational evaluation Focus on many levels
  • Individuals (pupils, students, teachers)
  • Classrooms/courses
  • Curriculum/programmes
  • Organizations (schools, universities)
  • Fields (all schools in a municipality, all
    programmes in a discipline)
  • The national level (national quality development
    and quality assurance systems)
  • The international level (PISA,IEA, EQUIS in the
    business school area)

18
Regional evaluation The Provincial Government
Act (22/1997) obliges the provincial governments
to evaluate the accessibility of basic services
within their borders ? The Ministry of the
Interior and other minitstries define shared
evaluation targets for a number of years (i.e.
the term of the government) ? evaluations
should be carried out co-operatively,
consistently, and drawing primary on existing
statistics and department specific databases
19
DISTRIBUTION OF WORK AND CO-OPERATION IN
EVALUATION
  • Ministry of Education and the Education
    Evaluation Council are responsible for external
    evaluation and its development. Education
    providers are primarily responsible for local
    evaluation and its development.
  • The National Board of Education
  • participates in evaluating syllabuses and
    learning
  • achievements within degree regulations
  • as a part of educational evaluation expert
    network
  • Links between external evaluation, local
    evaluation and evaluation of basic services
    should be taken into account when drawing up
    evaluation programmes and plans.
  • Educational institutions and providers who have
    participated in the evaluation must have access
    to the evaluation reports on their own
    activities.

20
EVALUATION OF LEARNING ACHIEVEMENTS
  • The National Board of Education
  • participates in evaluating syllabuses and
    learning
  • achievements within degree regulations
  • as a part of educational evaluation expert network

21
Evaluation of education in The Finnish National
Board of Education
  • quantitative and qualitative indicators will be
    defined
  • information in the form of indicators will be
    produced
  • learning achievements will be compared
  • learning achievements will be evaluated in
    different school types and levels as follows
    (sample-based approach)
  • basic education mathematics and mother
    tongue, regularly
  • basic education other subjects and
    themes (for ex. sustainable development)
  • methods of evaluation will be developed

22
REQUIREMENTS FOR EVALUATION SET BY THE
PARLIAMANENT EVALUATION POLICY)
  • strengthening the role of evaluation in
    developing education, adds the neutrality of
    national evaluation of education and
    promotesco-operation and communication between
    actors in evaluation and interest groups
  • promotes equality
  • main emphasis should be in supporting the
    development of teaching and learning at school
    and municipality level
  • education providers should be given priority
  • evaluation must concern the whole learning
    environment

23
  • Organising evaluation
  • evaluation adequately independent of the rest of
    the education administration
  • independent and impartial evaluation actors
  • increasing the impact of evaluation especially
    from the viewpoint of educational institutions
  • a system must be established to monitor the
    impact of evaluation

24
VALUE BASIS OF EVALUATION
  • Equality means promoting educational equality by
    means of evaluation.
  • Fairness means creating an ethically sustainable
    foundation for evaluation and refraining from
    comparison which may be harmful for the target of
    evaluation.
  • Truthfulness means high ethical responsibility
    and critical examination, as well as optimisation
    of the reliability of evaluation.

25
Principles for educational evaluation
  • Evaluation is based on education policy decisions
    (development aspect)
  • The legitimacy is founded on educational
    legislation
  • Education evaluation should be transparent
  • The essential outcomes need to be published
  • Methods used are to be recorded and justified, as
    reliable, valid and comparable as possible

26
  • In this regard, evaluation must provide
  • central and local administration with valid
  • data on how succesfully educational
  • objectives are being achieved. Thus,
  • evaluation should foster self-control and
  • self regulation at the level of central and
  • local administration and promote
  • accountability to the various parties
  • involved.

27
  • From the perspective of educational policy,
  • it is important to ensure that education
  • providers deliver the educational services
  • that they are intended to produce and that
  • they do this in accordance with the goals
  • set for these services and on the basis of
  • professional skills of adequately high
  • standard.

28
Evaluation process and decision making
Publishing a report
EEC
Quality assurance
Writing a report
Decision on evaluation
Evaluation visits Interpreting results
Appointing the planning group
1.
feedback
4.
EVALUATION GROUP
PLANNING GROUP
INSTITUTIONS Self-assessment, other material
Planning the evaluation
Accepting the plan Appointing the evaluation
group Accepting methods and measures and
self-assessment models
Evaluation group proposal Developing data
gathering and feedback systems Drawing up
instructions for self- assessment Evaluation
visits and panels
3.
2.
EEC
29
THE IMPLEMENTATION OF EVALUATION (PLANNING AND
EVALUATION NETWORK)
30
Implemented evaluation projects
  • Basic education
  • 1. The implementation and quality of basic
    security in education
  • 2. The implementation, availability and quality
    of special needs education, remedial teaching and
    student welfare services in basic education
  • Upper secondary education
  • The need for special needs education in upper
    secondary school education

31
Adult education
  • Liberal Adult Education Institutional Structure,
    Service Ability and Societal Effects
  • Training Opportunities and Provider Network of
    Vocational Adult Education on the Secondary Level
  • Both projects long lasting three years in two
    phases (2004-2005, and 2006-2007)
  • Based on available data statistics, research
    reports, documents, the variety of material
    produced in various development projects etc no
    primary data gathered
  • Process (1) multiperspective planning group
    comprised of education providers and relevant
    interest groups, (2) independent, scientific
    research oriented evaluation group

32
  • Basic education
  • Teaching methods in the comprehensive school from
    January 1st 2007 to December 31st 2007
  • Upper secondary education
  • Collaboration in upper secondary school education
    and vocational education from October 1st 2005 to
    October 10th 2006
  • Vocational education
  • Workplace learning from Sebtember 2004 to
    October 2006
  • The regional effectiveness of vocational
    education from January 1st 2006 to December 31st
    2007
  • Key competencies in vocational education from
    January 1st 2007 to December 31st 2007
  • Adult education
  • The competence-based examination system

33
Different evaluation frameworks
34
EFQM EXCELLENCE MODEL
Action 50
Results 50
Leadership Management 10
Processes 14
Key Performance Results 15
Personnel (HR) 9
Personnel Results 9
Customer Results 20
Policy and Strategy 8
Society Results 6
Partnership and Resources 9
Innovation and learning
35
A Common Quality Assurance Framework
Model (Common Core Criteria)
Gains Mutual Trust Trancparence Credit Transfer
Methodology (Self assement/Peer Review)
Measurement (Indicators)
Monitoring (External)
Quality assurance methods and process,
indicators, standards and norms, approach of
selfevaluation
36
QUALITY ASSURANCE MODEL EUROPEAN COMISSION
(LISBON STRATEGY)
PLANNING i.e. strategies, goals, objectives,
resources, anticipation and evaluation
mechanism
IMPLEMENTATION i.e. action plan, resources,
partnerships, leadership, process management,
training of trainers, didactical materials
METHODOLOGY quality management, self
evaluation, stakerholderss role management of
information external evaluation
REVIEW feed back and prosedures for
change continue improvement development
MONITORING AND EVALUATION i.e. self evaluation,
external inspection, internal quality control
Results of teaching, training and learning
staff-oriented results, key performance results,
sosietal results
37
Examples on evalaution projects
38
EVALUATION MODEL FOR LEARNING ON THE JOB LEARNING
  • Quantitative indices as part of self-assessment
  • 2 Self-assessment by education providers
  • multisectoral evaluation
  • consensus
  • 4 Description of best practices
  • - Portfolio

5. Stakeholder questionnaire education
committees
web-based evaluation environment
Backgound data
Initial assessment by an evaluator team
In-depth analysis of self-assessments, and
related visits to educational institutes by the
evaluator team
5. Authority interviews
Field-specific evaluation seminars
Provider feedback Final evaluation and evaluation
report
39
FROM GOALS TO INTERACTION The evaluation of
basic education pedagogy
40
What was evaluated?
  1. Key principles of pedagogy
  2. Teachers goal awareness
  3. Range of teaching methods
  4. Attendance to the pupils unique characteristics
  5. Equality of learning processes
  6. Functionality of the learning environments
    factors
  7. Teaching, studying and ???

20.10.2014
40
41
How was information produced?
  • Survey of teachers (N 2351)
  • Survey of rectors (N 410)
  • Visits to schools ( N 12)
  • Hearings ( N 12)
  • Data and research results from previous
    statistics, studies and evaluations

20.10.2014
41
42
TEACHERS IMPORTANT PEDAGOGIC PRINCIPLES
  • Fairness
  • Student oriented approach
  • Goal awareness
  • Importance of classroom piece
  • Stressing action and variation
  • The objectivity of pedagogic thinking
  • Socio-constructivism has its opponents

20.10.2014
42
43
TEACHERS GOAL AWARENESS
  • The status of curriculum is significant.
  • Goal awareness highlights many aim fields of
    education and upbringing.
  • Text book still has quite a significant role.
  • Critical approaches to curriculum
  • Wishes for time division (lesson plan)
  • Goal awareness given contemplation on aims thin

20.10.2014
43
44
TAKING STUDENTS INTO ACCOUNT
  • Students appreciated as individuals
  • Equity central / significant
  • Students differences and needs important
  • Student oriented approach has increased
  • Making use of students activity
  • No new remedies for bullying.
  • Few means of managing disruptive students.
  • Special needs students cannot always be
    supported.
  • Possibilities of differentiation limited in
    practice.

20.10.2014
44
45
RANGE OF TEACHING METHODS
  • The range and variety of teaching methods is
    highlighted.
  • Externally most often teacher oriented teaching
    with student oriented elements.
  • Instrumental use of teaching materials is
    becoming more common.
  • Information technologies have enriched working
    methods and materials.
  • Examining and experimental methods of working not
    usual.
  • Minor changes in teaching.

20.10.2014
45
46
PREREQUISITES FOR ACTION
  • A positive atmosphere for development and shared
    perception of teaching positive (rectors view)
  • The pedagogic know-how of teachers good in this
    country.
  • Technical equipment and options for teaching
    material are usually of good quality.
  • Great variation in the preconditions of action
    between schools.
  • Large and heterogeneous teaching groups is a
    problem.
  • Too much variation in the resources for
    complementary education.
  • Haste and the feeling of haste in school work
    have increased
  • Messages of problems with air-conditioning and
    schools with mold and mildew problems!

20.10.2014
46
47
THE IMMEDIATE CHALLENGES OF PEDAGOGY?
  1. Large heterogeneous groups (taking into account
    the broad questions of differentiation)
  2. The frequency of class-room disturbance
    remedies?
  3. The needs of developing self-evaluation how?
  4. The curriculum includes too much content, too
    demanding goals, and too high criteria for good
    learning outcomes, the instructions for
    evaluation call for clarifications.
  5. Teachers supplementary education has to be taken
    more seriously more time and funding, more
    equity, more long-term planning, teacher oriented
    approach.

20.10.2014
47
48
LONG-TERM NEEDS IN THE DEVELOPMENT OF PEDAGOGY
  1. Supporting pedagogic leadership at schools
  2. General education in the future launching the
    discussion
  3. What are the contents?
  4. How do we learn?
  5. What do we evaluate?
  6. Enhancing goal awareness and pedagogic thinking
  7. levels
  8. steps of advancement
  9. goals
  10. Increasing the amount of the studying, evaluating
    and developing approach in the teachers work
    in the work communities
  11. Enhancing the understanding and encountering of
    the students differences, and the pedagogy of
    developing.
  12. Enhancing learning as collective construction of
    knowledge
  13. Enhancing the appropriation of media, information
    technologies, and information flows as part of
    e.g. developing the learning to learn skills
  14. Increasing the amount of interaction with the
    surrounding community /society (e.g. childrens
    and young peoples lifestyles and cultures)
  15. Developing the evaluation skills (of the schools
    / teachers /students)

20.10.2014
48
49
DISSEMINATION OF EVALUATION INFORMATION
  • Publication / announcements
  • Report, feedback for organizers?
  • National, regional, local events
  • Articles etc.
  • Developing the overall evaluation process from
    the point of view of making use of information
    (collaboration, networking, participation, client
    oriented approach)

20.10.2014
49
50
The process in short
  • The evaluation group
  • brings all information together
  • describes the state of affairs
  • defines the criteria
  • evaluates the state of affairs
  • points out the needs of development
  • makes some general recommendations
  • takes part in the distribution of the results
About PowerShow.com