Excellent Measures Precede Measures Of Excellence

1
Excellent Measures Precede Measures Of Excellence

• Dr Hamish Coates
• coatesh_at_acer.edu.au

2
Focus of the presentation

• Review of key considerations
• Stocktake of current alternatives
• Highlight possible developments
• Conversation about possibilities

3
Designing quality indicators

• Data-driven evaluations require access to
  quality data
• The quality of indicators underpins the authority
  and validity of the evaluation
• Current national data resources are limited
• What quantifiable indicators would enhance
  national evaluations of teaching and learning in
  Australian higher education

4
An evaluative perspective

• Australia is a leader in the large-scale
  evaluation of teaching and learning
• Large-scale evaluations are part of contemporary
  higher education
• We need to place greater significance on
  evaluation methodology
• Methods for evaluating educational quality have
  been developed over many years
• It is imperative that appropriate and
  contemporary analytical methods are used

5
Qualities of robust indicators

• Interpretability/transparency/simplicity
• Parsimony/sufficiency
• Sensitivity/generalisability
• Validity/stability/reliability
• Efficiency/practicality/integrity
• Relativity/context/framework

6
A general indicator framework
7
Review of current possibilities

• Trends in national conversations about teaching
  and learning
• The current national alternatives
• CEQ GTS GSS OSI
• GDS graduate work further study
• DEST attrition progress
• Limitations of current alternatives

8
An enhanced indicator framework

• Many developments outside the framework
• Current framework is dated due to major sector
  changes
• Authority of a framework and associated
  indicators relies on their synergistic
  relationship
• Need to reconceptualise the framework for
  teaching and learning indicators

9
A measure of student engagement

• The idea of student engagement
• The value-added by student engagement data
• Engagement data helps to enhance higher education
• An Australian student engagement survey

10
Measures of academic learning

• Problems with current state of affairs
• Develop staff capacity for manage assessment
• Develop calibrated assessment instruments
• Develop grade-point averages

11
Graduate skills measures

• Routine administration of a graduate skills
  assessment to provide data on the net effect of
  university study
• Develop authentic assessments of graduate
  skills
• Gather insights on graduate skills from graduate
  employers

12
A review of the perspective

• Quality considerations are influenced by the
  information on which they are based
• Data-driven policies must be underpinned by valid
  and accessible indicators
• Current data available at the national level is
  limited
• A revised indicator framework and new data
  collections would enhance determinations of
  quality in Australian higher education