An Evaluation Process of the University Engineering Degrees: the Portuguese Case

1
An Evaluation Process of the University
Engineering Degrees the Portuguese Case

• Carlos Sá Furtado
• ICEMS Coimbra Departamento de Engenharia
  Electrotécnica e de Computadores, LTMEU,
  Universidade de Coimbra, Pólo 2, 3030-290
  Coimbra, Portugal, safurtado_at_deec.uc.pt

ICEE 2005 July 25-29, 2005,
Gliwice, Poland
2
Introduction
Portuguese university engineering courses called
licenciaturas
Reasons for the Evaluation/Assessment The
Published Legislation Principles and Objectives
of the Quality Assessment The Process of
Evaluation Criticisms at the Evaluation Process
Some General Conclusions and Recommendations
ICEE 2005 July 25-29, 2005,
Gliwice, Poland
3
Reasons for the Evaluation/ Assessment
1290 University of Coimbra 1911 Universities of
Lisbon and Oporto 1930 Technical University of
Lisbon 1970s eleven public universities, more
than twenty polytechnic institutes and dozens of
private Higher Education schools

• The Revolution of the 25th April 1974
• Centralised and autocratic regime to a
  democratic and rooted participated system
• Traditional authoritarian rules at the
  University by a modern representative democracy.

• Number of students increased exponentially
• Middle and working classes eroded the exclusive
  character of university education
• This explosion not duly prepared, organised, and
  accompanied

ICEE 2005 July 25-29, 2005, Gliwice,
Poland
4

• 1980s adequate legislation has been passed
• 1st Cycle of Evaluation Public Universitary
  Licenciaturas Assessment started in 1994.
  Foundation of the Portuguese Universities (
  Fundação das Universidades Portuguesas) FUP
• 2nd Cycle Evaluation National Council for the
  Evaluation of Higher Education ( Conselho
  Nacional de Avaliação do Ensino Superior-
  CNAVES).Public and private universities and
  public and private polytechnics.

ICEE 2005 July 25-29, 2005, Gliwice,
Poland
5
The Published Legislation
Law of the Bases of the Educational System, Law
nr. 46/86, of 14th October Law of the Autonomy
of the Universities, Law nr. 108/88, of 24th
September Law of Higher Education Evaluation,
Law nr. 38/94, of 21st November -- Protocol
signed on the 19th June 1995 between the Ministry
of Education, CRUP, and FUP Decree-Law nr.
204/98, of 11th July, regulates Law nr.38/94, and
creates CNAVES
ICEE 2005 July 25-29, 2005, Gliwice,
Poland
6
Principles and Objectives of the Quality
Assessment

• Main principles of Quality Assessment
• The Higher Education Quality Assessment System
  is unitary
• Independent and impartial character of the
  Evaluating Agency
• Mechanisms and procedures lead to impartial and
  independent judgements and conclusions
• Regular periodicity
• Permanent information through selected and
  appropriate indicators
• Encourage the Self- Evaluation

ICEE 2005 July 25-29, 2005, Gliwice,
Poland
7

• System set on the following principles
• The institutions own the evaluation system
• Based on self-assessment and peer review
• Cyclical
• Nationwide
• Teaching and research are evaluated separately
• Reports of the expert commissions are made
  public
• Faculties/Departments nothing to say about
  external evaluation committees
• No link with funding.

• Objectives of the Quality Assessment System
• Promote and improve quality of teaching and
  research
• Stimulate self-regulation based on quality
• Inform Portuguese society on the performance of
  each Institution

ICEE 2005 July 25-29, 2005,
Gliwice, Poland
8
The Process of Evaluation
Process established by CNAVES with the agreement
of all the Institutions

• Internal and external stage
• Self- evaluation of strengths, weaknesses and
  future prospects
• External stage peer review includes a site visit
  and an external report
• Contradictory hearing of the assessed institution

• CNAVES announces evaluation in the following
  year
• Faculties prepare Self-Evaluation Report (
  Relatório de Auto-Avaliação )
• Report distributed among the members of the EEC
• A 2-3 days site visit Visiting Committee- 3-5
  members
• Preliminary comments at end of visit
  afterwards, an interim report
• EEC approves final report
• Contradictory arguing

ICEE 2005 July 25-29, 2005,
Gliwice, Poland
9
The self-evaluation report contains the following
chapters a) Genesis and evolution of the
programme b) Structure and content of the
programme c) Curriculum organisation d)
Learning and teaching environment e) Intake f)
Success rates g) Quality of the graduates h)
Effectiveness of organisation and staff
qualities i) The facilities j)
Internationalisation and external contacts
and k) Internal quality assessment.
Guidebook, issued by CNAVES, helps the Visiting
Committee terms of reference and external report

• Basically, the Committee has
• Form an opinion based on supplied information
  and discussions
• Make suggestions on quality improvement

ICEE 2005 July 25-29, 2005,
Gliwice, Poland
10
Criticisms at the Evaluation Process

• Evaluation reports made more visible and known
• Some parameters clarified with assignment of
  minimum values
• Lack of indicators and parameters
• Need for an effective articulation between
  evaluation and inspection
• Need to clarify the relations between the
  evaluation conclusions and the Portuguese
  Association of Engineers (Ordem dos Engenheiros)
• Rationalization and increase of financial
  resources/ foreign experts
• Consideration of market/ Respect to the
  independence and creativity of the Institutions

ICEE 2005 July 25-29, 2005,
Gliwice, Poland
11
Some General Conclusions and Recommendations

• All the University Courses of Engineering
  evaluated
• General conclusions
• Large majority of these courses got a good
  appreciation
• Lack of experience on themes related to the
  enterprises
• Insufficient attention to ethical and
  deontological areas and integrated evaluation of
  students competence
• Great retention of students in two initial years
  
• Lecturers find difficulties to deal with great
  number of badly prepared and little motivated
  students
• Professors besides teaching and research must
  have professional practice
• Too long time to Final Work of the Course

ICEE 2005 July 25-29, 2005,
Gliwice, Poland
12

• Some general recommendations have been
  identified
• Government must be aware of the increasing
  financial constraints
• Improvement of the preparation obtained at the
  Secondary School
• Promotion of a dynamic interaction between
  University and Enterprises
• More lecturers with involvement in engineering
  projects
• Improvement by the Institutions of a
  Self-Quality System

ICEE 2005 July 25-29, 2005,
Gliwice, Poland
13
Conclusions

• Main objectives have on the whole been achieved
• Universities have improved

ICEE 2005 July 25-29, 2005,
Gliwice, Poland