Conducting programme evaluationsreviews with a view to sustainable quality promotion - PowerPoint PPT Presentation

Loading...

PPT – Conducting programme evaluationsreviews with a view to sustainable quality promotion PowerPoint presentation | free to view - id: 64cb0-MzgwO



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Conducting programme evaluationsreviews with a view to sustainable quality promotion

Description:

Conducting programme evaluations/reviews with a view to sustainable ... (Slide: Ian Bunting) Misconceptions about Quality Assurance. in Higher Education ... – PowerPoint PPT presentation

Number of Views:121
Avg rating:3.0/5.0
Slides: 76
Provided by: oac8
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Conducting programme evaluationsreviews with a view to sustainable quality promotion


1
Conducting programme evaluations/reviews with a
view to sustainable quality promotion
  • Jan Botha
  • University of Stellenbosch
  • South Africa
  • Keynote Address
  • Oman Quality Network Conference
  • Muscat, 28 October 2008

2
Key questions
  • Can we apply the 80/20 principle to programme
    evaluations?
  • What are universities learning from the
    evaluation of (formative) programmes?

3
Outline of the paper
  • Suggestions for the academic leader
  • QA in higher education context
  • Misconceptions about QA in higher education
    context
  • Programme evaluations and the 80/20 principle
  • What are we learning from programme evaluations
  • Broader perspectives Context
  • Literature
  • Experiences
  • Narrower focus on a specific issue
  • Empirical
  • Analytical

4
Suggestions for the academic leader
5
The predicament of the academic leader
  • I am a dean / HoD, my research ?
  • I have become one of them
  • In the middle of the sandwich
  • I am herding cats
  • I have to manage quality
  • From critic to custodian

6
Academic Leadership
  • Learning Leaders in times of change
  • Geoff Scott, Hamish Coates, Michelle Anderson
    (May 2008)

7
Suggestions for the Academic Leader
  • Use quality assurance as instrument to achieve
    your vision and goals for your School /
    Department / Programme
  • Quality assurance can be a powerful change
    mechanism
  • Use a research-informed approach to quality
    assurance
  • QA? Understanding / Insight ? Action

8
QA as an Academic / Scholarly Project
  • QA to be approached as a form of research
  • an intellectual enterprise
  • driven by curiosity, creativity and pragmatics
  • underpinned by methodologically sound inquiry
  • accompanied by experimentation and reflection
  • informed by theorising
  • resulting in action, application and peer
    reviewed publication

9
A Principle of Design
Specifications are continually moving upwards
Opportunity for incremental changes
Specifications
Calculate this cost
Poor design
Poor design
Good design
10
Quality Assurance in Higher Education Context
11
Living with ideology in the university
  • Ronald Barnett
  • The idea of the university as a site of
    reason
  • Universities are beset by many ideologies
  • ideology is that which is opposed to truth
  • Competition and Entrepreneurialism
  • Quality
  • Managerialism
  • Research
  • Access, inclusivity, multiculturalism
  • Turn the power of ideology onto itself, move from
    ideologies to ideaologies recall the idea of
    the university

Beyond all reason. Living with ideology in the
university. 2003. Open University Press.
12
The idea of the university?
13
What are universities for?
  • Universities state their vision, mission and
    objectives
  • A place of quality, a place to grow
  • Creating futures
  • Your Knowledge Partner
  • Innovation Generation
  • To advance Learning and perpetuate it to
    Posterity
  • Sultan Qaboos University aspires to be an
    academic institution of excellence recognized
    both nationally and internationally, a center for
    learning where individuals can develop their
    capabilities in an environment that promotes
    academic achievement and research excellence and
    a community where the highest moral and ethical
    values prevail for the purpose of both
    self-advancement and service to the community.

14
International snapshot
  • International Network for Quality Assurance
    Agencies in Higher Education (INQAAHE) a
    world-wide association of some 200 organisations
  • Members include
  • Commission for Academic Accreditation (CAA), UAE
  • Netherlands Quality Agency (NQA)
  • Malaysian Qualifications Agency (MQA)
  • Australian Universities Quality Agency (AUQA)
  • Comision Nacional de Evaluacion y Acreditacion
    Universitaria. (CONEAU), Argentina
  • National Accreditation Agency (NAA), Russian
    Federation
  • Oman Accreditation Council (OAC)

15
South Africa
  • Council on Higher Education (CHE)
  • just hang on, it will go away,
  • as has happened with the other acronymous
    bodies
  • Institutional Audits
  • Programme Accreditation
  • National Reviews of Programmes
  • Quality Promotion and Capacity Building
  • Three steering instruments in Higher Education

16
(Slide Ian Bunting)
17
Misconceptions about Quality Assurance in Higher
Education
18
Challenging the management of quality
  • Is managerialism not contradictory to the
    nature of a university ?
  • Management of quality used to cover for poor
    quality?
  • More management of quality results in less
    quality
  • CHE publication
  • Academic Freedom, Institutional Autonomy and
    Public Accountability in South African Higher
    Education (August 2008) (www.che.ac.za)

19
Misunderstandings (1)
  • Point of departure lecturers are doing poor
    quality work,
  • we need QA to catch them out
  • I am satisfied with 95 of the work of the people
    in my university/faculty/school
  • therefore QA is not necessary
  • Just make sure that you meet the minumim
    standards and keep them off your back
  • Aim for the minimum
  • Comply

20
Misunderstandings (2)
  • If good academics are appointed at universities,
    we do not need QA systems. If the wrong people
    are appointed, no document (however thick it is),
    will ensure quality.
  • If nobody steals, we do not need laws and police.
    If there are thieves, no law (however thick it
    is), will ensure that no theft takes place.
  • If nobody is ill, we do not need doctors. Since
    we know beforehand that everyone is in any case
    going to die some day, the whole medical
    profession is senseless.

21
The case for Quality Management
  • Quality management is a necessary condition for
    quality,
  • however,
  • quality management is not sufficient for quality

22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Programme evaluations and the 80/20 principle
28
Acknowledgements
  • Co-researchers
  • Professor Carools Reinecke
  • Emeritus Vice-Chancellor, Potchefstroom
    University (South Africa)
  • Mr Kamal Bhagwandas Jogibhai
  • University of the Witwatersrand, Johannesburg
    (South Africa)
  • Research Assistant
  • Mr André Müller, University of Stellenbosch

29
The Pareto Principle
  • Vilfred Pareto (1848-1923)
  • Wealth distribution in 19th Century England is
    predictably unbalanced
  • Most income and wealth go to a minority of people
  • Consistent mathematical relationship between
    proportion of people and amount of income and
    wealth
  • Principle of Imbalance
  • Zipfs Principle of Least Effort
  • 20-30 of resources accounted for 70-80
    activity
  • Jurians Rule of the Vital Few
  • the 80/20 Principle

30
The 80/20 Principle illustrated
Effort
Results
R Koch 2007. The 80/20 principle. The secret of
achieving more with less. London N Breadly
Publishing
31
What is a programme?
  • A social intervention
  • A set of planned, structured learning experiences
    leading to a qualification

32
Purposes and types of programme evaluations
  • Evaluation outcomes are used by different role
    players for different purposes
  • Mouton Babbie (The practice of social research.
    Oxford, 2001)
  • judgement-orientated evaluations
  • improvement-orientated evaluations
  • knowledge-orientated evaluations
  • Trow (Academic reviews and the culture of
    excellence. Stockholm1994)
  • internal supportive
  • internal evaluative
  • external supportive
  • external evaluative

33
Unit of evaluation/review
  • An institution as a whole
  • An academic organizational unit within an
    institution
  • department, centre, institute, bureau, school,
    faculty
  • Specific activities or processes within an
    institution
  • learning and teaching programmes
  • research programmes
  • community engagement programmes
  • capital campaigns

34
Types of programme evaluations
  • Evaluation of professional programmes as part of
    the periodic programme accreditation process
  • conducted by professional councils
  • National reviews of programmes
  • conducted by a national QA agency
  • the same programme evaluated and (re)accredited
    simultaneously at all institutions
  • Internal programme evaluations/reviews (for
    different purposes improvement/closure/consolidat
    ion etc.)
  • conducted by institutions
  • professional and formative programmes
  • undergraduate and postgraduate programmes

35
Outcomes of national reviews in SA
  • 37 MBA programmes offered by 18 public and 9
    private higher education institutions evaluated
    during 2003
  • 5 fully accredited,
  • 15 conditionally accredited
  • 19 Master of Education programmes evaluated
    during 2005
  • 7 fully accredited,
  • 2 provisional,
  • 5 conditional,
  • 2 postponed,
  • 3 de-accredited

36
Challenges in programme evaluation
  • Coherence within broader internal and external QA
    system
  • Formative programmes as unit of evaluation
  • BA, BSc, BComm, B SocSc
  • Spread over many departments in Schools and
    Faculties
  • Not to conflate process of evaluation with
    reporting the results of an evaluation process
  • Too much effort and resources for outcomes? 80/20
  • Administrative burden
  • Evidence

37
(No Transcript)
38
Expectations of Programme Evaluations
  • Sustainable quality promotion
  • An instrument for change
  • Integrated with other forms of evaluation
  • Outcomes should justify the effort
  • for academics and evaluators and HE managers
  • Standard methodology used
  • self-evaluation based on criteria/standards
  • Self-report
  • panel visit, report
  • improvement action, monitoring
  • Applicable to formative programmes

39
A Pareto Approach to Programme Evaluation
  • Consider the ideal situation
  • refer to criteria / standards
  • clustered in 11 themes
  • simplified (and translated)
  • Formulate a key question (or questions)
  • focus, contextualise (what does this mean for us)
  • Express the evaluation in a symbol (5 point
    scale) discipline to make a judgement
  • Motivate the evaluation symbol (brief narrative,
    with substantiating evidence)
  • Formulate strategies (in bullet points)

40
Template for the Self-evaluation Report
  • Programme rationale
  • Criteria
  • The programme is consonant with the facultys
    mission, planning and resource allocation. The
    design maintains an appropriate balance of
    theoretical, practical and experiential knowledge
    and skills. It has sufficient disciplinary
    content and theoretical depth at the appropriate
    level.The programme offers opportunities for
    community interaction.The design offers learning
    and career pathways to students with
    opportunities for articulation with other
    programmes within and across institutions, where
    possible. The existing formulation of the
    strategic importance of the programme was
    re-evaluated.
  • 1.2 Key Question
  • 1.3 Evaluation
  • 1.4 Motivation
  • 1.5 Strategy
  •  

Reports limited (8,9 pages). Strategies
formulated as actions
41
Criteria/standards clustered in themes
  • Programme rationale
  • Academic integrity
  • Student recruitment, admission and selection
  • Staff
  • Learning facilitation
  • Assessment
  • Infrastructure and academic information sources
  • Programme co-ordination
  • Student success and academic support for student
    success
  • Service Learning and work-based learning
  • Programme Evaluation and Development
  • Research basis (postgraduate programmes)

42
What are we learning from programme evaluations?
43
Case study
  • Faculty of Sciences, Stellenbosch University
  • Bachelor of Science in Physics Chemistry,
    Mathematical Sciences, Earth Science,
    Biodiversity, Molecular Biology, Human Life
    Sciences, Sport Science, Science Education
  • Faculty of Agri-Sciences, Stellenbosch University
  • Bachelor of Science in Agricultural Sciences in
    Animal Production Systems Agricultural
    Economics Wine Production Systems Crop
    Production Systems Forestry Food Science
  • Faculty of Humanities, University of the
    Witwatersrand (Johannesburg)

44
Agri-Sciences
45
Agri-Sciences
46
Sciences
47
Sciences
48
(No Transcript)
49
Concerns, mild concerns, commendations
  • Serious concerns
  • Recruiting the right students, adequately
    prepared widening participation (Omanisation),
    optimal number
  • (Agricultural Sciences and Natural Sciences)
  • Work-based learning
  • (Agricultural Sciences and Natural Sciences)
  • Concerns
  • Programme rationale
  • (Agricultural Sciences)
  • Infrastructure, facilities, equipment
  • (Agricultural Sciences
  • Commendable
  • Infrastructure, facilities, equipment (Natural
    Sciences)
  • Learning facilitation (Agricultural Sciences)
  • Assessment (Agricultural Sciences)

50
Strategies for Improvement(Selected and
complied from 574 strategies for improvement
listed in the self-evaluation reports)
51
1. Programme rationale (69)
  • To enhance interaction with stakeholders (subject
    specific societies, industry, extraordinary
    lecturers, alumni) to broaden academic and
    industry specific networks (through a voluntary
    advisory committee)
  •  To review/restructure the focus and content of
    the programme annually to ensure that module
    outcomes strongly reflect the programme specific
    and generic outcomes, reviewing programme
    structures annually and implementing student
    feedback and industry input, and developing a
    formal system to gather feedback from graduates
    on their appraisal of the course, and suggestions
    for improvement.
  •  To adapt and better integrate courses, share
    expertise/reduce duplication and use resources
    more cost-effectively, to reduce lecturing load
    by constructing generic courses across
    departments and including and designing modules
    to fill theoretical gaps and deepen theoretical
    knowledge or to add attractive career paths
  •  To define service/experiential/work based
    learning within the programme and incorporate
    electives/optional subjects to prepare students
    better for the work environment, extending
    practical training/internship periods and
    awarding credits for lab or field work
    experience To offer more opportunities for
    community interaction, especially for graduate
    students.

52
1. Programme rationale (continued)
  •  To communicate outcomes clearly to students to
    contextualize lectures and learning experiences
    to communicate information about administrative
    and support services to students, staff and
    stakeholders (advertising student assistantships
    effectively)
  •  To identify and support students at risk in
    small groups and give academic and non-academic
    help and to communicate the importance of class
    attendance also addressing low/increased student
    numbers
  •  To sharpen recruitment and marketing efforts,
    improving outreaches where scholars, prospective
    and current students can be informed about the
    programme and its strategic value. (Open days,
    information sessions...)

53
2. Academic integrity (65)
  • To increase reseach and benchmarking
    opportunities with international scholars, to
    ensure the programme remains at the forefront of
    new developments, and to encourage
    research/teaching interaction
  •  To review the undergraduate programmes annually,
    continuousand coherently planning of courses in
    terms of content, level of difficulty and credit
    value constantly reviewing the curriculum,
    learning materials, learning methods and
    programme outcomes moderating it and ensuring
    that the feedback from external moderators is
    taken into account
  •  To increase collaboration between lecturers to
    improve programme cohesion, filling in
    theoretical gaps through new and adapted courses,
    and to cut out duplication also working with
    timetabling administrators to free up course
    combinations to provide better articulation with
    cognate programmes, to structure the programme to
    expose students to the core themes, to balance
    practice and theory better to ensure the
    relevance of prescribed courses that are
    presented by other departments from both within,
    and external to, the school/faculty
  •  To identify and remove obstacles inhibiting the
    use of experential learning and increase
    laboratory time, re-evaluating the module
    composition and structuring of the programme in
    order to make place for a longer period of
    internship to evaluate the efficacy of the
    practical portions of the modules and investigate
    coherent year-long practical modules at second
    and third year levels
  •  To communicate the reason for the approach
    followed during the first year of study, and
    maintain a challenging learning environment for
    students despite low/increased numbers

54
2. Academic integrity (continued)
  • To develop generic skills (writing, information
    and computer) and improve the teaching of
    practical skills through the use of continuous
    assessment for all modules from second year
    upwards, and to identify and adapt modules fairly
    early in the academic programme To focus on
    improving student mathematic skills To complete
    computer literacy and scientific communication
    skills modules in the first year
  • To manage or establish new research
    institutes/units/centres such as an Institute for
    Food Technology (IFT) or Institute of Food
    Science and Technology (IFST)
  • To recruit students from historically
    disadvantaged backgrounds
  • To recruit more international students

55
3. Student recruitment, admission and selection
(93)
  • To involve industry stakeholders in recruitment
    and funding for undergraduate scholarships To
    acquire additional bursaries from all sector role
    players To expand facilities and staff To
    increase support staff, especially in terms of
    popular media and marketing To involve subject
    societies in recruitment actions To increase
    interaction with bursary providers about the
    criteria for successful students and the problems
    that are experienced by current bursary holders
    To investigate the roles professional bodies play
    in promoting the image of certain careers
  •  To review admission requirements for mathematics
    and science related to forest science programme
    To evaluate the tempo at which graduates are
    employed To investigate existing recruitment
    actions and material in terms of success and
    relevance To monitor the extended degree
    programme to ascertain whether it is increasing
    student diversity To monitor student numbers at
    the beginning of the year and consider the
    adjustment of entrance criteria if necessary To
    review policy on ranking on application form so
    as to admit students who desire to follow To
    review academic support available to struggling
    students in all years
  •  To adapt intake in terms of quantity, quality
    and equity of students. To align recruitment
    efforts of the university, faculty,
    programme-committee and departments To develop
    and implement mechanisms, including assessment
    methods, to widen access, (e.g. summer school,
    bridging programme) To increase the diversity of
    the student body in terms of South African
    cultural groups and international students To
    create a structured marketing and recruitment
    plan as top priority, buying in expertise if
    needed, consulting with SU marketing divisions,
    utilizing existing marketing mechanisms,
    negotiating for undergraduate bursaries and
    addressing the student diversity To improve the
    administrative implementation of admission
    criteria To reconsider admission requirements at
    first year level to curb the high failure rate,
    as well as at honours level (possibly) To
    re-assess the modules which have become service
    courses

56
3. Recruitment, admission and selection
(continued)
  • To make the bridging programme compulsory for
    students with a Gr 12 mark between 50-56 To
    visit underprivileged schools in our immediate
    vicinity To sponsor prizes (e.g. book prizes)
    for the best Biology (now Life Sciences?) student
    in Grade 12 at a few selected schools To
    consider an orientation period including visits
    to departments as well as the experimental farm
    before registration of second-year students to
    allow informed choices on major subjects To
    supply information on programmes at the Expo for
    Young Scientists and Olympiad candidates, as well
    as high school science teachers To encourage
    third years to attend final years product
    development presentations To focus on the
    recruitment of coloured students due to the
    demand from Agri businesses
  •  To implement extended degree programmes (and
    first year academy) to benefit students that have
    to overcome academic inadequacies due to
    historical barriers, and help students to
    overcome cultural and language difficultiesTo
    prevent over-subscription to the course Cap
    student numbers (no more laboratory space
    available). To identify the optimum number of
    students that can be accommodated in the
    programme (estimated at between 40 and 50
    students), keeping economic factors in mind To
    market a BSc(Hons) as a career route both
    commercially and in parastatal, particularly
    academic spheres To pay careful attention to
    entrance requirements in the next few years as
    the new school leaving certificate is
    implemented. Aim to raise entrance requirements
    To critically review failures and examine the
    reasons for these
  •  To initiate discussions with the government on
    the fragmentation of higher forestry education in
    South Africa To broaden the communication base
    with students at first- and second year level To
    develop a culture of uniqueness and excellence
    amongst staff members and students To raise
    awareness of language issues in the academic
    programme

57
3. Recruitment, admission and selection
(continued)
  • To make better use of recruitment opportunities
    (e.g. Open Day, visits to schools) To recruit
    black students via industry contacts To send
    promotional material (pamphlets) to all Secondary
    schools in the Western Cape To represent the
    department at open days organized by IPS To
    write an article promoting Conservation Ecology
    and Entomology in general and our department in
    particular for the popular science magazine
    Quest To ensure the web-site inspires
    students To continue with current promotions
    activities and transparent admission policies To
    have greater flexibility within the language
    policy To provide admission requirements into
    the program (especially as of 2009) on
    departmental and faculty web pages and
    brochures. To advertise the programme campus
    wide on notice board, emphasizing the financial
    support through industry bursaries, especially to
    students from disadvantaged backgrounds. To
    promote the need for a Faculty Open Day with a
    smaller, but more specific learner group (eg. top
    10 learners within a grade with mathematics as
    school subject or learners from strong feeder
    schools) so that departments can participate more
    effectively. Our perception is that the
    University Open Day fails to recruit quality
    motivated students for the program. To improve
    the distribution of marketing material To
    recruit talented learners with bursaries To
    contact CPS to disseminate the information on
    courses in the programme, job opportunities and
    available bursaries To give more students access
    to bridging programmes To try to attract the
    best Afrikaans and English students To attract
    black students with the Fundza Lushaka bursary
    scheme

58
4. Staffing (62)
  • To enhance opportunities for continuous
    professional development To make sure all newly
    appointed lecturers attend the Universitys
    induction and training courses To ensure
    on-going professional development, letting staff
    attend various courses (e.g Web-CT training) as
    the need arises. To deliver graduates who are in
    demand in the food and cognate industries To
    develop C2 and C3 staff new skills and knowledge
    To enable personnel to develop academically in
    their fields of specialization To encourage
    research activity by giving all staff members an
    off semester with no undergraduate teaching
    commitments and additional training in assessment
    methods for large classes To maintain good
    research profiles and to expose students and
    lecturers to current research and teaching issues
    and trends To quantify time usage and check
    against workload document to enforce contract
    hours To manage time more effectively by setting
    fixed consultation times
  •  To beter utilize the annual performance
    evaluation process, Permanent staff members to be
    required to compile a personal work agreement and
    performance contract. This assists in quality
    assurance of teaching and learning, research and
    service To have all teaching staff, with one
    exception, have a PhD To review teaching load of
    staff through programme evaluation To have
    positive departmental evaluations To ensure an
    equitable distribution of the teaching load by a
    yearly review To review and evaluate the
    deployment of technical staff for optimum
    technical support
  •  To ensure more realistic work loads for academic
    staff To align specialisation and research focus
    of staff needs with programme. To ensure greater
    cooperation and less duplication

59
4. Staffing (continued)
  • To utilize personnel seconded from industry to
    the departments can assist in maintaining
    research capacity To investigate mechanisms by
    which specialised technical / practical knowledge
    can be imparted to students. This may be linked
    to a specific academic staff member, specific
    tasks during the internship, and/or exposure
    during practicals To ensure that student
    assistants are well-trained
  •  To better communicate the benefits of an
    excellent diversity profile and the spin-offs for
    the student body, who are exposed to a new
    experiences, approaches and ideas. To inform the
    faculty through the departmental evaluations that
    a critical mass of academic staff and sufficient
    administrative support are not present within all
    departments in the program to effectively perform
    and excel in all 3 areas expected namely
    teaching, research and community service To
    actively encourage students and lecturing staff
    to interact about the expectations of web
    presence of modules, particularly in smaller
    senior modules
  •  To maintain standards and ensure quality despite
    the increase in student numbers To manage
    part-time staff carefully To have adequate
    facilities and personnel to allow students to do
    practical work and experiments
  •  To address the perceived insufficient levels of
    remuneration to attract and retain young
    academics, especially for young black staff. The
    current personnel evaluation has shortcomings and
    financial incentives linked to excellence are
    insufficient to motivate academic staff.

60
5. Learning facilitation (70)
  • To get more clarity on the meaning of student
    centered teaching and its implications To
    encourage participation by academic staff in the
    activities of Centre for Teaching and Learning
    To utilize WebCT to effectively communicate with
    large groups, but not replace lecturer-student
    interaction or class notes To evaluate and
    compare learning material with international
    universities To develop a policy on student
    centered teaching so that independent,
    enthusiastic and spontaneous learning
    consistently takes place To incorporate
    fundamental knowledge much more overtly
  •  To act on students feedback and evaluation To
    give a copy of each module evaluation to the
    programme coordinator (improve efficiency of the
    feedback) To request lecturers to complete
    module frameworks, literature bibliographies,
    goals, outcomes in accordance with the module
    framework requirements of Senate To review the
    relevance of all units To revisit the PBL
    process in terms of life long learning, critical
    thinking and professional reasoning To review
    what is lectured, the links between problems and
    lectures and workshops and feedback To review
    problems and evaluate their incremental demands
    over the four years To review decision to make
    curriculum books only available to students in
    e-format
  •  To align all study guides with the Universitys
    policy requirements for study guides
    (specifically the inclusion of module-level
    outcomes in study guides) Investigate ways to
    make class experience more stimulating To make
    more use of text books and journal publications,
    and less class notes Powerpoint slides always to
    be provided to students To employ a variety of
    assessment opportunities to evaluate the
    facilitation of learning To use text books and
    scientific publications to a greater extent and
    in preference to class notes To review the
    modules to ensure that they contain learning
    opportunities for the development of these
    skills, without unnecessary duplication To
    relook at the orientation course in first year
    (preparation for PBL curriculum) and add a test
    to ensure the content is mastered and
    internalized

61
5. Learning facilitation (continued)
  • To investigate the option of providing credits
    for laboratory or field work experience
  •  To adequately communicate the module outcomes to
    the students yearly by the chairperson and via
    the website To advertise student assistantships
    opportunities effectively To expose second
    year students to the layout and cohesion of the
    programme To involve students in more personal
    feedback (similar to this evaluation exercise) at
    a module and programme level To highlight the
    fact that ability to work in a team is part of
    the supposed programme outcomes and establish
    where this is in fact done
  •  To assign tutors dedicated to the support of
    disadvantaged students To inform departments of
    the need to apply computer literacy skills and
    gradually introduce sourcing and use of
    scientific literature earlier in the program
    (3rd-year level). Basically, do not leave all
    tasks, seminars and research projects for the 4th
    year To discuss with computer literacy convenors
    options to allow mathematical science students to
    do fewer but more relevant modules within
    computer literacy To investigate the possibility
    of introducing opportunities for students to
    improve and perfect their written and verbal
    communication skills at early stages in their
    studies To develop oral presentation skills for
    senior students

62
6. Assessment (58)
  • To ensure that all staff members are aware of
    departmental policies and requirements for
    assessment and the publication and storage of
    marks To use peer-reviewing within student/study
    groups as an effective supplementary method of
    assessment To encourage continues assessment
    training of staff To change fieldwork rubrics to
    be more user-friendly and precise (with student
    input) To review the number of assessment
    activities that contribute to the marks and
    activitiesTo review time needed to answer
    questions and consider the advisability of seen
    vs unseen questions To analyze all exam
    questions according to Blooms taxonomy
  •  To align assessment practices all course
    frameworks, study guides with the universitys
    requirements To ensure that all tests and exams
    are aligned with the principles of the
    Universitys Assessment Policy To investigate
    Turn-It-In for electronic submission of
    assignments To investigate ways to reduce the
    significant increase in administrative load
    associated with internal and external moderation
    To have rigorous internal moderation, and
    external moderation for third year and honours
    modules To establish, manage and oversee an
    accurate and reliable administrative system to
    record marks onto the SIS.
  •  To enhance strategies to eradicate plagiarism
    To make assessment challenging, in particular to
    test problem-solving abilities To give more
    smaller tests rather than only a few major tests
    and an exam To assess individuals fairly in a
    group assignment and CTL should be approached for
    assistance in this regard To pursue a more
    systematic manner for collecting student feedback
    on each module To improve module frameworks to
    include assessments details (dates, type of
    assessment as well as expected timeframe for
    feedback) To use a range of assessment methods
    such as a seminar, laboratory, written and oral
    examinations

63
6. Assessment (continued)
  • To clearly communicate the means by which
    problem-solving abilities will be assessed, i.e.
    the quality of the questions to be expected, the
    amount of insight that will be required To
    update the assessment dates and weights on the
    website To keep lecturing staff (and yearbooks)
    updated with regulations regarding assessment and
    moderation (internal and external) at
    departmental level To communicate the different
    assessment methods of different modules clearly
    to the students To provide reasons or
    motivations for giving a particular mark,
    especially for essay-type projects and similar
    essay-type exam questions
  •  To conduct individual interviews with students
    scoring gt30 in semester test, determine reasons,
    plan for support To devise an early warning
    system for students who are struggling (more
    difficult with larger classes) To give attention
    to entrance requirements and streaming of
    students to preserve standards To handle
    question papers with care to avoid corruption of
    the assessment process
  •  To monitor individual student progress in terms
    of the First Year Academys mechanisms.

64
7. Infrastructure and academic information
sources (55)
  • To make sure that teaching facilities remain to
    up to date To improve, maintain and replace
    laboratory equipment To challenge the
    insufficient budgets to support the running cost
    of undergraduate practical training in some
    courses (especially those with high student
    numbers) To expand laboratory space, personnel
    To develop IT support services to personnel
  •  To use our excellent library services optimally,
    faculty librarian specialists and proper student
    training and regular purchases of new DVDs and
    material To investigate the possibility of using
    the computer users area for training and tests
  •  To re-allocate facilities to departments so that
    lecture theatres, laboratories and the cellar can
    be in the same building To arrange for continued
    access to academic scientific information
    (library) once graduates are working in rural
    areas To maintain the effective academic support
    offered by the Library, SunMedia and Information
    Technology. To monitor access to and choice of
    electronic journals To identify one or more
    modules to be enriched in teaching of writing
    skills and use of library facilities and
    undergraduate students should be required to
    submit a literature review at some point in the
    academic programme to ensure that the desire to
    continue reading and learning beyond graduation
    is fostered.

65
7. Infrastructure and academic information
sources (cont)
  • To improve the equipment for undergraduate
    practicals so that each student can complete
    practical assignments individually To update the
    computer facilities and associated software To
    optimize laboratory equipment for teaching
    purposes (more student-proof research equipment)
  •  To optimally utilize the class space, modern
    equipment, laboratory facilities, and computer
    and library services
  •  To maintain laboratory equipment To monitor the
    lecture hall size amount of students
  •  To attend to the insufficient field
    laboratories To prevent overshooting capacity of
    a maximum of 40 students in the programme To
    challenge the tariffs quoted by the General
    Maintenance Section of the University for
    services are perceived to be inflated due to a
    sub-contracting system and is not seen to be
    consistently offering value and high quality
    service.

66
8. Programme coordination (61)
  • To integrate programme inputs from different
    stakeholders into curriculum development
    undergraduate, post-graduate, extraordinary
    appointments, alumni and industry
  • To align teaching practices with international
    best practrice To state programme outcomes
    (generic and specific) for all lecturers and
    students involved in the programme through
    updated module frameworks
  •  To monitor/review service levels regularly on
    all levels for relevancy within the programme To
    evaluate and implement final year feedback To
    monitor throughput of modules and support
    modules To monitor student feedback earlier in
    the modules to report back To consider student
    representation in the Programme Committee
  •  To institute ongoing actions to improve and
    evaluate the efficiency and quality of the
    teaching programme To align individual courses
    and course materials towards the main focus of
    the programme, especially for new members of
    staff To improve communication between
    departments to avoid unnecessary overlap between
    courses and course material

67
8. Programme coordination (contunued)
  • To ensure better communication between lecturers
    in different departments about the contents of
    programme To introduce programme coordinator to
    1st year students provide contact details To
    ensure regular communication between teaching
    administration at the University, and teaching
    staff and students To introduce the programme
    co-ordinator to first and second year students,
    and provide contact details in class and on the
    internet To ensure that module frameworks exist
    for all modules, also on the website To hold
    more frequent committee meetings and to create
    more regular opportunities for student input and
    participation To explain lines of communication
    and authority to students
  •  To identify and support at risk students by
    tracking their records and analyzing it to help
    the department To explore how the information in
    the Student Oracle system can be utilized To
    ensure that deadlines are met
  • To gain better understanding of the reasons why
    certain students do not study successfully

68
9. Student success and academic support (60)
  •  To provide more opportunities for individual
    face to face contact between lecturers and
    students To develop presentation skills for
    lecturers and technical staff To engages with
    CHSE to ensure academic staff have adequate
    educational background and facilitation and
    assessment skills To give both new and
    experienced staff the opportunity to attend CTLD
    assessment and teaching courses To adhere to
    University Teaching Assessment policy To
    maintain classes on/below 40 students per class
  •  To improve the monitoring system for 1st and 2nd
    year students, To monitor student success more
    systematically To review mentorship strategy
    with a view of developing a formal approach,
    close contact student / lecturer To monitor
    continuously that the pressure to improve the
    pass rate of students, especially at first year
    level, is not carried over on to the second- and
    third year level to finally compromise the
    integrity of the whole program To monitor the
    success rate in every module in the department
    (and faculty) where it resides.
  •  To adapt assessment techniques to maintain good
    academic standards To make the existing
    mentoring program more accessible to students in
    the programme. To evaluate and adjust the
    mentoring system in order to make it more
    effective and to ensure continuity from 1st year
    onwards. To look afresh at entrance requirements
    and options for students who struggle To
    investigate the possibility of continuing the
    mentor system for support of students at second
    and third year levels, in addition to the current
    first year support.

69
9. Student success and academic support
(continued)
  • Improve support to part time/sessional staff in
    their teaching role To teach students to improve
    skills rather than just to supervise and critique
    them
  •  To assist students to overcome the challenge
    posed by language. Dubble medium options for
    courses could be made more readily available To
    remind lecturing staff that all assessments,
    including small tutorial tests, should be
    bilingual To encourage language use accessible
    to all students
  •  To alert students to the available student
    support services To scrutinise studentsstudy
    records in all their modules To improve
    language, numerical and cognitive skills within
    the programme To let students participate in
    the first year Learning Academy To assist
    students with the challenges of the first-year of
    the program. The Extended degree program (also
    First Year Academy) is seen as an option to
    improve student success To make better use of
    tutor demonstrators. To extend the successful
    tutor system to other modules if financial
    support can be obtained from the Faculty To
    identify at risk students earlier and offer
    compulsory extra tutorial time To identify good
    role models for students
  •  To provide much broader access to an extended
    degree, especially within the mathematical
    sciences. Come to terms with the fact that a
    4-year degree is the norm and use this to plan
    curricula. Consider a universal 4-year programme
    with some students emerging with an honours
    others with a BSc.

70
10. Service learning and work-based learning (43)
  • To cooperate with stakeholders to create
    opportunities for WBL interaction with industry
    for all students To update lists of industry
    knowledge partners To explore collaboration
    efforts with MoUs to clarify roles To ensure
    that the commercial entity is suitable for
    receiving undergraduate students and that there
    is no family connection with the student
  •  To review and improve the WBL coordination
    responsibility guidelines and monitor the success
    and/or need for WBL through departmental
    evaluations To follow up past-graduates to see
    how feasible their recommendations are
  •  To adapt programmes to integrate WBL modules To
    use experimental farms optimally To lengthen
    period of (vacation) internships To extend
    mentoring system to alumni in the industry
  •  To ensure the commercial entity is fully aware
    of its WBL obligations to the student before
    placement (e.g. internship) To require
    literature searches during internships To
    formalize community interaction projects with
    SUSPI To consider the possibility of site visits
    and voluntary internships
  •  

71
10. Service learning and work-based learning
(cont)
  • To communicate to all parties the outcomes of the
    WBL and what the student should be achieving, To
    ask all module coordinators to discuss work
    opportunities in the context of their modules
  •  To monitor WBL risks continuously e.g. to
    provide mentoring, support and networking options
    for supervisors (employers) To overcome logistic
    difficulties of WBL in certain industries, e.g.
    the mining and petroleum industries in the
    Western Cape
  •  To encourage students to take initiative for
    finding WBL placement
  • To address any WBL misconceptions in the industry
    regarding the abilities of graduates and the
    balance of theoretical, practical and
    experiential teaching and training during the
    academic programme. To challenge lecturing staff
    to guard against modules being too theoretical
    given the context in which they are taught. To do
    an impact study to measure the effectiveness of
    the programme in preaparing community servers for
    practice

72
11. Programme evalution and coordination (44)
  • To maintain regular interaction with sector
    stakeholders and survey employers on the impact
    of the programme and levels of satisfaction To
    improve liaison with industry / employers To
    improve industrys understanding of the purposes
    and outcomes of the programme To review the
    mechanism used to give stakeholders information
    about curriculum changes based on their input
  •  To implement coordinated strategies to review
    programme content and outcomes annually To
    obtain data on student recruitment generated
    during departmental evaluations To ensure that
    all departments obtain feedback from industries
    To send evaluation forms to alumni one year after
    graduation To evaluate all quality assurance
    practices used in the department
  •  To adapt courses to eliminate redundancy and
    streamline the learning process To evaluate the
    relevance and efficacy of all modules, including
    the practical portions thereof
  •  To initiate a process of communication and
    interaction with departments at this university
    and at other institutions to ensure that we are
    offering modules of the highest standards and
    relevance also consider the employability of
    exit level students
  •  To encourage final year students to continue
    with post-graduate studies to better their
    qualifications and improve their employability
    and develop a society for past graduates

73
So what are we learning from programme
evaluations?
  • Remember Scientists, Formative, Undergraduate
  • Commitment to maintain high academic standards
  • Deep concern about students
  • Quality of students (school system) Student
    numbers
  • Access and Success
  • Not clear about student centered teaching and
    its implications
  • Need to communicate clearly to students
  • programme and course outcomes
  • Programme coherence and rationale
  • Need to cooperate and communicate with lecturers
    in other departments (Break through academic and
    organisational silos)
  • Improve liaison and cooperation with the world
    of work

74
Reflections and conclusions
  • Is the selectivity justified (due to 80/20
    approach)?
  • The 20 doctor all students must achieve all
    outcomes
  • Each programme must meet all the
    criteria/standards
  • Sampling and focus justified during evaluations?
  • How to select evidence?
  • Just right and just enough

75
Reflections and conclusions
  • Different strategies at different levels
    (Programme, Department, Faculty, Institution,
    Higher Education System, and beyond)
  • Encourage creativity, research, reflection
  • Encourage debate, discussion across silos
  • Closing the loop remains one of the biggest
    challenges
  • Good systems, procedures
  • Good people (at all levels)
About PowerShow.com