Evaluating the student learning experience

1
Evaluating the student learning experience
Higher Education Consultant Learning and Teaching
Ivan Moore

• Ivan Moore
• HE consultant
• ivan.moore_at_manchester.ac.uk

Centre for Excellence in Enquiry Based Learning
2
Objectives
Higher Education Consultant Learning and Teaching
Ivan Moore

• By the end of the session, you should
• be able to describe a range of approaches to
  designing an evaluation form, based on
  internationally recognised methodologies
• have experienced an activity through which you
  will have begun to develop a customised
  evaluation methodology
• have experienced an approach to evaluating the
  student learning experience

Centre for Excellence in Enquiry Based Learning
3
What is evaluation?

• Its something we talk about in Educational
  Development. It is important for Learning and
  Teaching but we often dont do it very wellor
  in time to do much about it. Somehow, we often
  seem to not be able to find the time to evaluate
  what we are doing or the effects of our
  interventions. It seems just a little too
  difficult to do, or we have other priorities.

4
The purposes of evaluation

• Evaluation for accountability (measuring results
  or efficiency)
• Evaluation for development (providing information
  to help to improve practice)
• Evaluation for knowledge (to obtain a deeper
  understanding of some particular area of practice
  such as student learning or change management)
• Chelimsky (1997)

• Summative
• Formative
• Research

5
Why should WE evaluate?

• Because it is part of our professional practice

6
Principles of evaluation

• An integral part of our teaching practice
• An ongoing process, so that we learn from
  systematic reflection
• Should be participatory
• Should enable us to make appropriate
  modifications along the way
• Should enable us to make judgments on specific
  sessions, but also to draw out wider implications

7
Purposes of evaluation

• Mike Prosser
• Quality Assurance
• student satisfaction
• the mean score is important as a measure of
  quality
• Quality Enhancement
• Student conceptions/how they experience the
  course
• The deviation is important more focused view

8
Purposes of evaluation

• Mike Prosser
• Is the learning environment/teaching approach
  having any influence on student
  conceptions/approaches?
• A student experience survey is more important
  than a student satisfaction survey

9
Approaches

• Off the peg questionnaires
• Learning Styles (Honey and Mumford)
• Motivated Strategies for Learning
• Learning and Study Strategies Inventory (LASSI)
• Approaches to Study Inventory
• Course Experience Questionnaire (Ramsden)

10
The Course experience questionnaire(P. Ramsden)

• Designed as a performance indicator
• 24 statements relating to 5 aspects
• 1 overall satisfaction statement
• Research-based
• Drawn from statements made by students in
  interviews
• Students with positive responses take a deep
  approach

11
The five sub-scales

• Good teaching
• Clear goals and standards
• Appropriate assessment
• Appropriate workload
• Generic skills

12
Enquiry-Based Learning

• Learning is driven by a process of enquiry
• Students determine the lines of enquiry
• They identify what they need to know, how they
  will learn it, how they will know that they have
  learned it
• They will share this learning with others
• They have a range of support resources
• Online paper-based equipment room other
  students the tutor

13
Enquiry-Based Learning

• Students
• Academic development- information skills,
  analysis, synthesis, evaluation, hypothesis
• Professional development- teamworking,
  leadership, shared goals, communication
• Personal development- taking responsibility,
  planning, problem solving, confidence
• Change of approach
• Locus of control, motivation

14
From Ramsden CEQ

• EBL has helped me develop my ability to work as
  part of a group
• I have usually had a clear idea of where I was
  going and what was expected of me
• I have found EBL motivating
• The EBL tutors motivated me to do my best work
• EBL has helped sharpen my analytical skills
• There has been more assessment of what I have
  memorised than of what I have understood
• As a result of EBL, I believe that it is more
  important to have found new ways of thinking than
  to have gained specific knowledge about the
  subject areas

15
Designing an evaluation questionnaire

• In groups 30 minutes
• Outcomes based design
• Agree up to 4 goals/objectives for your group
• For each, design 4 statements that will help to
  determine if the goal is being achieved
• Record the goals and statements on a flip chart

16
Approaches

• Scriven
• Goal free evaluation
• EBL has helped me develop my ability to work as
  part of a group
• Presupposes and leads the student
• What parts of the module motivated you most?
• (ok so it presupposes motivation, but not EBL)
• Why?
• Illuminative evaluation

17
Goal-free evaluation

• In pairs
• Choose another pair on which to focus
• Devise up to 5 questions that you might ask
  students to answer that might provide information
  on what they are experiencing
• Record your questions
• Compare them with the other pair
• Discuss what information you might receive from
  these questions
• Draw up your conclusions for feedback

18
Context-free evaluation

• Since the beginning of the year
• What skills, if any, have you developed?
• What helped you to develop these skills?
• How have you changed the way you study?
• What prompted you to make these changes?

19
Other methods

• Focus groups
• Structured interviews
• Continuous feedback
• What did you find most difficult/confusing today?
• What things did you find helped you learn last
  week?
• What should I
• Start/stop/continue?

20
The three minute paper

• What was the most useful of meaningful thing you
  learned during this session?
• What question(s) remain uppermost in your mind as
  we end this session?
• What was the muddiest point in this session?
• What would you like me to stop doing?
• What would you like me to start doing?
• What would you like me to continue doing?

21
Other instruments

• Kolbs Learning Style Inventory (LSI)
• Honey and Mumfords LSQ
• Sternbergs thinking styles
• Felders Index of Learning Styles
• Weinsteins Learning and Study Strategies
  Inventory (LASSI)
• Entwistles Approaches to Study Inventory

22
Evaluating the student learning experience
Higher Education Consultant Learning and Teaching
Ivan Moore

• Ivan Moore
• HE consultant
• ivan.moore_at_manchester.ac.uk

Centre for Excellence in Enquiry Based Learning