USING THE COURSE EXPERIENCE QUESTIONAIRE CEQ TO IMPROVE TEACHING or at least the DEST view of it

1
USING THE COURSE EXPERIENCE QUESTIONAIRE (CEQ) TO
IMPROVE TEACHING(or at least the DEST view of it)

• Professor Alan Farley
• Director
• Institute of Teaching and Learning

2
WHY THE CEQ?

• FOR
• It is one of the main tools being used by DEST to
  assess teaching within universities
• It gives a national comparison across all
  universities on 3 common scales
• It is being used by various bodies as part of a
  process to rank universities.
• Good tools available for its analysis
• Allows university to select four scales of most
  interest
• AGAINST
• Questions are limited in scope
• When released it relates to students who
  completed study more then a year prior and are
  commenting on studies up to 4 or 5 years prior

3
How can we improve our CEQ Performance?

• Long Term Improve the learning experience for
  Deakin students
• Medium Term Make students more aware of what we
  do for them and what are reasonable expectations
• Short Term Make students who have recently
  graduated, or are about to graduate, more aware
  of the use being made of the CEQ (and Graduate
  Destination Survey)

4
ANALYSING CEQ

• QUANTITATIVE RESPONSES
• Allows assessment of which areas need
  investigation
• Use breakdowns by different criteria to detect
  problem areas, eg by faculty
• Does not indicate the importance to the student
  of the issue addressed by the question
• QUALITATIVE RESPONSES
• New tool for analysis CEQuery
• Gives a possible measure of importance to
  students of domains/sub-domains
• Allows investigation of what the problem is

5
CEQuery

• Codes Best Aspects and Needs Improvement
  comments on CEQ
• Uses a dictionary of words and phrases linked by
  proximity
• Codes into 5 domains and 26 sub-domains
• Coded 80 of student comments across 14
  universities (10 were meaningless)
• Validated as 90 accurate in coding

6
CEQuery Coding Domains

• Assessment
• Course Design
• Outcomes
• Staff
• Support

7
CEQuery Subdomains

• Assessment
• Expectations
• Feedback
• Marking
• Relevance
• Standards

• Course Design
• Flexibility
• Learning methods
• Practice-theory links
• Relevance
• Structure

8
CEQuery Subdomains contd

• Outcomes
• Further learning
• Intellectual
• Interpersonal
• Personal
• Knowledge/skills
• Work application

• Staff
• Accessibility
• Practical experience
• Quality attitude
• Teaching skills

9
CEQuery Subdomains contd

• Support
• Infrastructure
• Learning resources
• Library
• Social affinity
• Student administration

• Student Services
• Course administration

10
Hit Rates Domain Level
11
(No Transcript)
12
Odds of a comment being BA rather than NI
13
Management CommerceNumber of Hits Importance
(This Field of Education attracted 78000 hits
out of 285906)

• 1st Course Design Learning Methods
• 2nd Staff Quality
• 3rd Course Design Flexibility
• 4th Staff Accessibility
• 5th Course Design Structure
• 6th Course Design Relevance
• 7th Staff Teaching Skills
• 8th Course Design Theory-
• Practice links
• 9th Support Social Affinity

• 10th Outcomes Knowledge
• Skills
• 11th Support Learning
• resources
• 12th Outcomes work
• application
• 13th Assessment standards
• 14th Outcomes intellectual
• 15th Support infrastructure
• 16th Support student administration

14
Management CommerceOdds of a BA Quality

• Assessment - Feedback 1 to 10
• Assessment - Marking 2 to 10
• Assessment - Expectations 2 to 10
• Course - Structure 3 to 10
• Support - Student Admin 3 to 10
• Assessment - Standards 4 to 10
• Support - Library 4 to 10
• Staff - Teaching Skills 5 to 10
• Support - Student Services 5 to 10
• Support - Infrastructure 5 to 10
• Support - Learning Resources 7 to 10
• Course - Design Relevance 8 to 10
• Staff - Accessibility evens

• Outcomes - Work application evens
• Staff - quality
  evens
• Course Design - methods 1.1 to 1
• Course Practical-theory links 1.2 to 1
• Course Design - flexibility 1.3 to 1
• Staff practical experience 1.4 to 1
• Assessment - relevance 2 to 1
• Support social affinity 2.1 to 1
• Outcomes knowledge/skills 3.9 to 1
• Outcomes interpersonal 4.5 to 1
• Outcomes intellectual 11.5 to 1
• Outcomes further learning 13.8 to 1
• Outcomes personal 30.2 to 1

15
Applying CEQuery to Deakin

• Preliminary analysis of 2004 data completed
• Further analysis will be done, eg by faculty
• Can be applied to internal evaluations if format
  amended