Preparation of a good self assessment report and external evaluation

1
Preparation of a good self assessment report and
external evaluation
EU oriented self-evaluation report procedures
for BIH Universities (EUSER-BIH)

• Prof dr André Govaert

2
Connection between internal, external quality
assurance and accreditation
3
Connections between internal, external quality
management and accreditation
Dynamics
Improvement
Internal quality management
Visitation
Internal
External
Accreditation
Assurance
Accountability
Stability
Uit Tijdschrift voor Onderwijsrecht
Onderwijsbeleid 2002-2003 nr. 1, 53-60 GARRE, P.
4
Bologna declaration and quality assurance

• All 45 Bologna signatories assigned themselves
  the task of co-operation in a system that
  delineates and strictly observes the level of
  bachelor and masters programmes.
• There are several ways to implement this system.
• ENQA (European Association for Quality Assurance
  in Higher Education)
• Bergen Standards and guidelines for members
• Code of Good Practice of the European Consortium
  for accreditation in Higher Education ECA

5
Quality Assurance in Flanders

• A system of external quality assurance was
  created at the beginning of the 1990s
• The higher education institutes (university
  colleges) and the universities are responsible
  for the internal and external quality management
  of the learning activities.
• They permanently monitor the quality of the
  learning activities on their own initiative
• They involve students, alumni and external
  experts from the professional field in processes
  of internal and external quality control
• Together they organise at a regular base an
  external evaluation, at least every 8 years
• VLHORA, VLIR apart of VLHORA, VLIR together they
  coordinate the external evaluation (and select
  the assessment panel)
• The management of the institute applies for the
  accreditation of a course after the external
  evaluation based on the assessment report (NVAO
  2003).

6
INTERNAL QUALITY MANAGEMENT

• Obliged by the government
• The organisation chooses the quality control
  system
• Preparation to and succession of external
  quality management
• Indirect helps determining accreditation
• Direct a subject of the framework of
  accreditation

7
QUALITY MANANGEMENT

• The institutes write a self evaluation report
  (not public)
• The assessment panel (independent, often
  international, external peers) is responsible for
  the external evaluation
• The assessment panel produces a
  subject-/discipline specific frame of reference
  (generic learning outcomes of the programme.
• During the site visit the panel examines
  additional information, holds discussions with
  the management and representatives of the
  programme (staff, students, professional
  practise). Furthermore the facilities are
  inspected and several aspects of the
  self-evaluation report are verified,
  supplemented, clarified,

8
QUALITY MANANGEMENT

• After the site visit the assessment panel writes
  a public report about the results of the
  evaluation (assessment of the quality of the
  programmes and recommendations for improvement)
• Before the assessment report is endorsed, it is
  first presented to the institution for
  verification.
• The institutes act upon the results of the
  quality evaluation in the policy of the institute
  (annual report)

9
EXTERNAL QUALITY Assurance

• Per course or group of courses
• Simultaneously at higher education
  institutes/universities with the same course or
  group of courses (cohort)
• By a visitation commission a group of external
  experts (one with didactical expertise and one
  with quality audit expertise) and one student who
  is subscribed in one of the courses that have to
  be evaluated
• Evaluation of the quality of the course
• Evaluation of the quality improvement process
  PDCA-circle
• Collection of strong elements and elements that
  the management should pay attention to
• Counterstatement report course managers and
  staff can react
• Published external evaluation (within 24 months)

10
Visitations VLHORA/VLIR

• Visitation protocol
• Guidelines for courses
• Guidelines for visitation commissions
• Guidelines to draw up the published external
  evaluation
• External quality control as leverage,
  meta-evaluation of internal quality control
• Qualitative description of processes
  (PDCA-spiral)
• Formulation of strengths, points of attention and
  strategic objectives at medium term
• Formulation of the quality level and the final
  judgement (this final judgement is either
  sufficient or insufficient)

11
Process-based approach

• 1. Qualitative description of processes
• How does the course deals with every process?
• How broad and thoroughly is that approach
  implemented in the organisation of the course?
• How is the internal evaluation of this approach
  being organised and what are the results?
• To which degree is/was the existing approach
  adjusted?

PLAN
DO
CHECK
ADJUST
12

• 2. Strength-weakness-analysis and strategic
  policy
• Concise list of strong points
• Concise list of aspects that need to be improved

13
ACCREDITATION (definition)

• The formal acknowledgement of a course based upon
  a decision of an independent organisation in
  which it is determined that the course meets
  previously fixed quality and course level
  standards
• Formal acknowledgement
• 2003 Independent organisation Netherlands -
  Flemish organisation for accreditation (NVAO)
• The management of the institute applies for the
  accreditation of a course after the external
  evaluation based on the assessment report.

14
NVAO

• The NVAOs major task is accreditation of higher
  education programmes in the Netherlands and
  Flanders.
• NVAO makes independent judgements and takes
  autonomous decisions.
• Importance of the international perspective for
  the decisions and position of NVAO (Playing an
  active role in international networks of
  accreditation and quality assurance
  organisations)

15
Accreditation

• Responsibility by decree of the organisation of
  Accreditation (NVAO)
• Condition to finance the course by the
  government
• Condition to grant recognized diplomas
• NVAO reports annually to the Flemish parliament
• Procedures for appeal

16
Accreditation framework existing courses

• Framework
• SIX SUBJECTS or Themes
• Objectives of the course / the programme
  (learning outcomes)
• Programme / Curriculum
• Effort of staff
• Means / Facilities
• Internal quality management system
• Results
• Accompanying 23 ASPECTS (FACETS) and CRITERIA

17
Decision rules for accreditation

• ASPECTS are being evaluated by means of a scale
• Excellent
• Good
• Satisfactory
• Unsactisfactory
• The judgement per SUBJECT is established on the
  basis of the importance of the different aspects
  that are involved (a way of deliberation
  (compensation))
• To obtain a POSITIVE FINAL JUDGEMENT every
  subject has to be at least sufficient
• For each main subject
• For each location

18
Characteristics of the framework

• The framework can be characterised as universal
  and accessible. The standards and criteria were
  formulated to be highly abstract rather than
  rigid and constraining.
• In leaving the frameworks open for
  interpretation, NVAO desired to accommodate the
  autonomy of institutions and programmes. In this
  way, institutions are able to elaborate their
  educational concept themselves and can position
  themselves more effectively within the higher
  education sector.
• At the same time, quality assessment agencies are
  given the liberty to make professional
  assessments.

19
Characteristics of the framework

• The themes incorporated in the frameworks broadly
  correspond with the themes that are
  internationally applied in the accreditation of
  programmes.
• The criticism from stakeholders concerning the
  fact that internationalisation and learning
  assessment are not given enough attention to is
  understandable.
• In other international accreditation frameworks,
  learning assessment is often incorporated as a
  theme. The position of learning assessment in the
  framework is an issue that should be taken up
  when new (initial) accreditation frameworks are
  drawn up. The same goes for internationalisation.

20
Introduction

• Homework
• How writing the SER and who are cooperating in
  writing the SER?
• Place in the organization and the organization of
  the course
• Historical context
• Quality framework of the course (mission, vision,
  values, strategic plans, operational aims,
  educational vision and action plans)
• Plans to improve the academisation
• Plan to improve cooperation with workfield

21
1. Educational objectives
22
1. Educational objectives
23
1. Educational objectives
24
1. Educational objectives
25
1. Educational objectives
26
Chapter 1

• Homework
• What are the educational objectives of the
  course, what will be the learning outcomes.
• 1.1 Do you think the competences are in
  accordance with the level and orientation of the
  degree
• 1.2 Which of the competences are domain specific?
  
• - Are they relevant for the workfield?
• Are they in accordance with the competences of
  the same course of your colleagues (also
  international)
• Are they tested and familiar with the workfield
  (the international academic world)
• Self evaluation
• Genesis, knowledge of the objectives by
  students, staff, ..
• Introduction of competence based learning
• International dimension
• Profiling

27
Chapter 1

• Homework
• Strength-weakness-analysis and strategic policy
• Concise list of strong points of chapter 1
• Concise list of aspects that need to be improved
  of chapter 1

28
2. Programme
29
(No Transcript)
30
2. Programme
31
2. Programme
32
2. Programme
33
2. Programme
34
2. Programme
35
2. Programme
36
2. Programme
37
2. Programme
38
2. Programme
39
2. Programme
40
2. Programme
41
2. Programme
42
3. Effort of staff
43
3. Effort of staff
44
3. Effort of staff
45
3. Effort of staff

• Homework
• Strength-weakness-analysis and strategic policy
• Concise list of strong points of chapter 3
• Concise list of aspects that need to be improved
  of chapter 3

46
4. Means / facilities
47
4. Means / facilities
48
4. Means / facilities

• Homework
• Strength-weakness-analysis and strategic policy
• Concise list of strong points of chapter 3
• Concise list of aspects that need to be improved
  of chapter 3

49
Porto

• Chapter 5
• Chapter 6

50
5. Internal quality control
51
6. Results
52

• Thanks
• Questions?