Supporting the Design of Discipline-Specific Learning Outcomes: Experiences of the Tuning Group for Physics

1
Supporting the Design of Discipline-Specific
Learning OutcomesExperiences of the Tuning
Group for Physics

• Gareth Jones
• Imperial College London

2
OUTLINE

• The Tuning Project
• What, Why, Who?
• Competences and Learning Outcomes
• Hierarchy of Learning Outcomes and link to Level
  and Standards
• Surveys and Results
• Degree Programme (Re)Design
• Main Requirements
• A fresh start or improve what exists
• Incorporating competences and content
  requirements
• Specific Examples
• IOP Accreditation requirements for Physics
  degrees
• Example of Module and Thematic Learning Outcomes

3
What is the Tuning Project?

• The universities response to the Bologna Process
  Most work done by separate but coordinated teams
  of discipline experts each with one member from
  each EU country
• To find ways to implement a three-cycle degree
  structure
• To develop learning outcomes and competences for
  each cycle (reference points) on basis of
  consensus after much discussion
• To survey views of students, graduates, academics
  and employers on importance of both generic and
  subject specific competences
• To survey and compare programme content and
  structure
• Development of ECTS as a credit accumulation
  system
• Best Practice in teaching learning and quality
  enhancement
• Tuning Coordinators/Leaders Julia Gonzalez
  Robert Wagenaar
• Tuning Physics Group Leader Lupo Dona dalle
  Rose

4
From the Tuning Final Report
5
Two of the key driving ideas of the Tuning Project

• One of the main objectives of the Bologna process
  is to make study programmes and periods of
  learning more comparable and compatible. This
  objective is strongly promoted by making use of
  the concept of levels, learning outcomes,
  competences and ECTS credits.
• The Tuning emphasis on competences and learning
  outcomes is intrinsic to the paradigm shift from
  a professor-centred to a student-centred approach
  which is seen as a key way of improving the
  effectiveness of European HE.

6
Competences

• Ability to do something.
• Competences range from
• specific and small, e.g. competence to use an
  oscilloscope, to
• general and large, e.g. competence to solve
  problems
• Acquired by students and assessed either in a
  specific part of a course or throughout programme
  in an integrative, holistic way
• Learning Outcomes often expressed in terms of
  competences (but not all)
• Generic Competences, e.g. general cognitive
  abilities, interpersonal skills
• Subject Specific Competences
• Competences required and/or valued by
  profession/discipline
• Different universities may emphasise particular
  competences and de-emphasise others ? Profile of
  degree

7
Examples of Generic Competences from Tuning

• Ability to apply knowledge in practical
  situations
• Capacity for analysis and synthesis
• Capacity to learn
• Creativity
• Adaptability
• Critical and self-critical abilities
• Concern for quality
• To act in accordance with a basic knowledge of
  the profession

8
Tuning Survey 2008 Employers ResponseMost
important generic competences
9
Physics Specific Competences/Learning Outcomes

• Able to enter new fields through independent
  study
• Familiar with the work of genius, i.e. with the
  variety and delight of physical discoveries and
  theories, thus developing awareness of the
  highest standards
• Have a good understanding of the most important
  physical theories including a deep knowledge of
  the foundations of modern physics
• Able to evaluate orders of magnitude in
  situations which are physically different but
  show analogies
• Able to understand and master the most commonly
  used mathematical and numerical methods
• Able to perform calculations, including the use
  of numerical methods and computing, to solve
  problems
• Able to construct mathematical models of a
  process/situation by identifying the essentials
  of a process/situation and making justified
  approximations
• Have a good knowledge of at least one frontier
  physics specialty

10
Physics Specific Competences/Learning Outcomes
(Practical/Experimental/Research)

• Able to perform experiments independently, as
  well as to describe, analyze and critically
  evaluate experimental data and to be familiar
  with the most important experimental methods
• Understanding of the nature and methods of
  physics research and how it can be applied in
  other fields e.g. engineering
• Familiar with the culture of physics research,
  including the relation between experiment and
  theory and ability to span many areas
• Able to find physical and technical information
  relevant to research work and technical project
  development using literature search methods
• Able to work with a high degree of autonomy,
  accepting responsibility in planning and managing
  projects
• Able to carry out professional activities in the
  area of applied technologies and industry

11
Physics Specific Competences (Human Dimension)

• Able to present ones own results (research or
  literature search) to professional and lay
  audiences orally and in written form using
  appropriate language
• Able to work in interdisciplinary teams
• Prepared to compete for school teaching positions
  in physics
• To show a personal sense of responsibility, e.g.
  meeting deadlines, and to show professional
  flexibility
• To behave with professional integrity and an
  awareness of the ethical aspects of physics
  research and its impact on society

12
Tuning Survey on Competences 2008
13
(No Transcript)
14
Learning Outcomes What and Why?

• Statements of what students should know,
  understand or be able to do as a result of
  following a course
• Knowledge and understanding
• Problem solving
• Skills experimental, mathematical, design,
• Ability at communication, teamwork etc.
• Use in defining levels 1st and 2nd cycle level
  descriptors
• Part of Bologna Process and Qualification
  Frameworks
• Use in Programme Design QA methodology
• What education is all about
• Must be assessed

15
Hierarchy of Learning Outcomes

• Module Level Learning Outcomes
• Specified by Module Teacher and Programme
  Director
• Should/must be assessed ? mark or grade
• Desired and threshold Learning Outcomes ?
  criteria
• Need to be specific but not too detailed
• Thematic Learning Outcomes, e.g. Quantum
  Mechanics
• Refer mainly to overall or final abilities.
  Forest not the trees
• Year Learning Outcomes useful for progression
  criteria
• Programme Learning Outcomes, e.g. BSc (Hons) in
  Physics
• General and summative statements
• Holistic
• Dublin Descriptor type statements but applied
  to discipline ? Refer to Academic Level

16
Academic Level and Learning Outcomes

• Intended Learning Outcomes give a good indication
  of competence for performing particular tasks,
  but
• Need to be fairly specific, e.g. able to use time
  dependent perturbation theory to solve problems
  in atomic and nuclear physics. But
• What kind of problems?
• How difficult?
• Need to refer to how assessed, e.g. exam
  questions.
• Learning Outcome statements for programmes are
  not enough to compare standards. How do you add
  up Learning Outcomes? Need to specify
  content/volume.

17
Are Learning Outcomes Helpful?

• Can be very helpful for programme design
• Focus mind on What are the students getting out
  of it?
• Can improve teaching and the output competences
  of graduates
• How to assess whether or not they are achieved?
• Exams OK for academic problem solving but not so
  good for realistic problem solving
• Difficulty of questions is crucial for standards
  but is hard to control and interpret
• Mark Scale Raw data for testing hypothesis Has
  this LO been achieved? but what is threshold
  mark?
• Practical competences easier to test

18
Traditional Programme Design

• (Professor) i ? (Course) i
• I will teach them what I know
• Programme S (Course) i
• Leads to content and professor dominated
  curriculum
• Danger of
• Content overload and excessive derivations
• Obscurity of purpose Why are we doing this?
• Little increase in competence
• Advantages (if have good professors!)
• Produces deep understanding for best students
• Good for producing future professors!!!

19
The Programme Design Problem

• An existing module synopsis can be basis for a
  list of Learning Outcomes for that module
• The general characteristics of a degree programme
  can be defined by Qualifications Framework
  statements
• But what goes in the middle?
• Subject specific qualification and level
  descriptors (Benchmark)
• Thematic Learning Outcomes
• Structuring of content to ensure linkage and
  progression
• Development of teaching, learning and assessment
  methods to enable learning outcomes to be
  achieved and assessed holistically
• Construction of a matrix of competences vs.
  modules is very helpful
• Helps to ensure competences appear explicitly in
  the design

20
Matrix of Competence vs. Content
Knowledge Understand Apprec.work of Genius Problem Solving Maths Skills Experimental Skills Communication Skills
Mechanics Relativity 50 10 30 10
Maths 1 20 30 50
1st Yr Lab 10 10 50 30
Quantum Physics 60 20 15 5
Professional Skills 20 10 30
21
Steps in Physics BSc Programme Design

• INPUTS
• (a) IOP Accreditation Requirements and QAA
  Benchmark statement
• (b) National Framework of Qualifications (NQAI)
• (c) Desired Qualification Profile (e.g. Applied,
  Pure,)
• (d) Desired/expected student intake and potential
  employers
• (e) Resources and existing degree programme
  modules
• (f) Tuning results on Competences, Learning
  Outcomes, Content,
• PROCESS
• Internal Discussion where we are ? where we want
  to be, SWOT
• Construct Matrix of Competences vs. Modules,
  using (a), (b), (f)
• Check (c), (d), (e)
• Develop Learning Outcomes for whole programme,
  themes and modules
• Check academic level
• Develop Teaching and Learning Methods and
  Assessments
• ITERATE! Will it work? Does it meet
  requirements? Is it realistic?
• Seek wide support and administrative approval

22
Use of Learning Outcomes in Practice(Reverse
Engineering)

• Start from where we are now
• LOs for each module Improve them, check how
  assessed
• Examine content remove redundancies, add missing
  items
• Check accreditation, benchmark, Tuning
  competences are met
• Construct matrix of competences vs. modules
• Iterate! It is likely there are gaps or
  deficiencies
• Construct more generalised LOs for themes,
  years, programme
• Ensure logical progression, e.g. C depends on A
  and B
• Check requirements of NQAI. Check academic
  level.
• Iterate, again! Pay particular attention to
  assessment and recent student results (marks,
  drop-out rates, employment, )
• Present new programme for approval

23
Example of Approaches to Teaching
LearningTuning Physics Group

• Modelling (second cycle)
• Modelling in a narrow sense means finding a
  simplified mathematical description of a complex
  phenomenon. It often means also applying tools of
  theoretical physics to non-physics situations.
• There is no course unit named Modelling.
  Students learn the modelling description of
  nature throughout their whole degree-course.
  Possible examples are the modelling neglect of
  friction in the description of free fall, the
  abundant use of harmonic oscillator for phenomena
  in the neighbourhood of stable equilibria, the
  shell model average field for nucleons in nuclei,
  the modelling of two-nucleon and three-nucleon
  forces, and so on.
• The whole teaching offer is then important in
  lectures, exercise classes, in lab classes, in
  student seminars and during research training
  students learn about how theories were developed,
  how to select and then apply theoretical tools
  (e.g. models) to a particular physical problem
  and how to model the building blocks of a theory,
  by adapting these latter to the experimental data
  description.

24
Example of Approaches to Teaching
LearningTuning Physics Group

• Problem solving skills (first cycle)
• Active Learning in all classes (theory, lab or
  problem solving)
• Several questions are posed to the theory class
  and a certain amount of time is allowed for
  discussion in the same class.
• Several question-problems are set to the class
  and assigned to groups of students. They should
  find an answer (either exact or approximate) in a
  certain amount of time. They are also requested
  to explain their reasoning to other students (Did
  they divide the problem in simpler problems? did
  they use analogies with problems, for which they
  already knew the answer? why are they confident
  about their own answer?)
• In the exercise classes the students are
  requested to correct and comment other students
  ways of solving the exercises.
• In the lab classes students are frequently asked
  to solve experimentally or propose ways for
  solving other more complex problems that may be
  considered extensions of the material proposed in
  the class. (ex after studying an LC circuit they
  are encouraged to solve the problem of coupled LC
  circuits and think about the problem of impedance
  adaptation in a transmission line).

25
IOP Accreditation Requirements

• The degree programme should foster intellectual
  curiosity in the minds of students
• Graduates should have acquired
• A secure knowledge of an agreed core of physics
  a few extra frontier topics
• Competences represented by graduate skills base
• The degree programme must incorporate project
  work
• BSc level project work may be a dissertation
• MSc/MSci level project work must involve research
  skills
• The degree programme must be consistent with QAA
  Benchmark

26
IOP Graduate Skills Base(Part of Programme
Learning Outcomes)

• Physics Skills Physics students should be able
  to
• Tackle problems in physics
• Use mathematics to describe the physical world
• Plan, execute, analyse and report experiments
• Compare results critically with predictions from
  theory
• Transferable Skills A Physics degree should
  enhance
• Problem solving skills (well defined and
  open-ended)
• Investigative skills
• Communication skills
• Analytical skills
• IT skills
• Personal skills (group work, use of initiative,
  meet deadlines)

27
Graduates should have a secure knowledge of the
IOP Core of Physics

• Mathematics for Physicists
• Mechanics and Relativity
• Quantum Physics
• including atomic, nuclear and particle physics
• Condensed Matter Physics
• Oscillations and Waves
• Electromagnetism
• Optics
• Thermodynamics and Statistical Physics

28
Example of Module and Thematic LOs

• 1st Year Mechanics Module LOs (selection)
• Understand the concept of conservative force and
  its relation to the potential function (in 3
  dimensions)
• Be able to solve single particle motion from a
  given potential function in two dimensions
• Be able to use angular momentum and energy
  conservation in central force problems
• Can be tested by answers to exam questions but
  how to interpret exam marks
• Not just Yes or No but partial Yes
• Index of cleverness or speed of working
• Thematic Learning Outcome for Mechanics
• Able to use Newtons Laws in a wide range of
  areas of physics
• Aware of the power of conservation laws
• Aware of more advanced methods of Lagrangians
  etc.

29
Conclusions

• The traditional approach to programme design
  stresses content too much and does not pay
  sufficient attention to the change we are trying
  to produce in students in terms of their
  competences.
• A Learning Outcomes approach requires a
  re-thinking of why, what and how we teach and of
  how we assess students achievements.
• It will require more effort initially from
  teachers but will probably enable reductions to
  be made in the amount of content taught.
• Students must be given more scope for activities
  like problem solving, team-work and
  communications but also must accept more
  responsibility for their own learning.
• The Learning Outcomes approach is firmly embedded
  in the Bologna Process. Tuning has shown how it
  can be used in a Pan-European way