Australian Universities Quality Agency and Quality Audit

1
Australian Universities Quality Agencyand
Quality Audit
Professor Deryck Schreuder Chair, AUQA October
2004
A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
2
Briefing Outline

• Overview of AUQA
• Quality Audit
• Other Aspects of AUQAs work
• Auditing Transnational Education.

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
3
1. Overview of AUQA

• The Australian Universities Quality Agency (AUQA)
  is an independent national agency that will
  promote, audit, and report on quality assurance
  in Australian higher education.
• It is a not for-profit company limited by
  guarantee.
• The shareholders are the Commonwealth, State and
  Territory Ministers for Education.

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
4
AUQA Board of Directors

• Three directors nominated by the Commonwealth
  Minister responsible for higher education
• Three directors nominated by the State
  Territory Ministers responsible for higher
  education
• Four directors elected by the self-accrediting
  institutions
• One director elected by the non-self accrediting
  institutions and
• AUQAs Executive Director

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
5
AUQA Staff

• AUQA has a full-time staff of nine

• Executive Director
• Audit Directors (4)
• Information Manager
• Personal Assistant to Executive Director
• Audit Support Administrator
• Project Officer

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
6
Australian Universities Quality Agency
In Australia, universities are created by
legislation after surmounting significant hurdle
requirements. Therefore, they are not
subsequently required to go through an
accreditation process. They do however undergo
lustral audit by the national Agency, AUQA.
A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
7
AUQAs Objectives

• Arrange and manage a system of periodic audits
  of QA arrangements relating to the activities of
  Australian universities, other self-accrediting
  institutions (SAIs) and state and territory HE
  accreditation bodies.

1
A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
8
AUQAs Objectives

• Monitor, review, analyse and provide public
  reports on QA arrangements in SAIs, and on
  processes and procedures of state and territory
  accreditation authorities, and on the impact of
  those processes on quality of programs.

2
A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
9
AUQAs Objectives

• Report on the criteria for the
  accreditation of new universities and
  non-university HE courses as a result of
  information obtained during the audit of
  institutions and state and territory
  accreditation processes.

3
A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
10
AUQAs Objectives

• Reporting on the relative standards of the
  Australian HE system and its QA processes,
  including their international standing, as a
  result of information obtained during the audit
  process.

4
A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
11
2. Quality Audit

• A systematic, independent determination whether

• An organisations planned arrangements are
  suitable to achieve its goals (i.e. check the
  overall approach).
• Actual practice conforms to the planned
  arrangements (i.e. the deployment).
• The arrangements achieve the desired results.
• The organisation is learning and improving from
  its self-evaluation of its approach, deployment
  and results.

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
12
AUQA Audit Scope
Indicative scope includes

• Organisational Leadership, Planning etc.
• Teaching Learning (all modes), Courses etc.
• Research, Consultancies, Community Service etc.
• Internationalisation
• Understanding Students and Stakeholders
• Staff and Staff Support
• Student Support
• Academic Support
• Admin Support, Facilities, Resources etc.

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
13
Main Audit Stages

• 1. Auditee conducts self-evaluation
• Consultation on panel membership
• 3. Performance Portfolio submitted
• 4. Portfolio Meeting
• 5. Preparatory Visit preparing for the Audit
  Visit
• 6. Provision of supporting documentation

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
14
Main Audit Stages (cont.)

• Off-shore Audit Visit (if any)
• 8. On-shore Audit Visit(s)
• 9. Draft Audit Report to auditee
• 10. Audit Report published
• 11. Follow-up Action Plan ( 3-4 months)
• 12. Follow-up Progress Report ( 2½ yrs)

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
15
AUQA Auditors

• AUQA has a Register of 100 honorary auditors from
  Australia and overseas
• The AUQA Board approves all appointments to the
  Register, following checks of resumes and
  referees reports
• All auditors undertake some form of training
• AUQA holds annual Auditor Meetings

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
16
Typical Panel Composition

• two members from within Australian universities
  or other SAIs
• an Australian member from outside the university
  sector
• an overseas member (usually from within a
  university) and
• an AUQA professional staff member.

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
17
Questions andComments?
A u s t r a l i a n U n i v e r s i t i e s Q
u a l i t y A g e n c y
18
Other Aspects of AUQAs Work
A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
19
1. National Activities
AUQA has a significant national presence in
Australia in addition to its audit work

• Australian Universities Quality Forum

• Good Practice Database

• workshops

• publications

• policy advice

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
20
2. International Activities
AUQA has a significant international presence in
addition to its audits of the overseas activities
of Australian institutions.

• International Network for Quality Assurance
  Agencies
• in Higher Education (INQAAHE)

- Development of Principles of Good Practice
for Quality Assurance Agencies

• Asia-Pacific Quality Network (APQN)
• - AUQA Secretariat

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
21

• Mutual Recognition

- quality assurance agencies to minimise load
on institutions and facilitate credit transfer
and recognition of qualifications eg. MoU
with South Africa

• South East Asian Ministers of Education
• Organisation (SEAMEO)

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
22

• ASEAN Universities Network (AUN)

• AUQAs staff are frequently invited to
• speak at a number of international
• conferences and to assist other quality
• agencies with their work.

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
23

• Over recent years, staff have conducted
• reviews of institutions in Hong Kong, Mexico,
• South Africa and the United Arab Emirates,
• and provided advice to agencies in Japan
• and elsewhere.

• Staff have trained reviewers for other
• quality agencies.

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
24

• AUQA frequently receives government
• and agency delegations, from all parts
• of the world.

• Staff from overseas quality assurance
• agencies have also observed AUQA audits.

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y
25
Auditing Transnational Education
A u s t r a l i a n U n i v e r s i t i e s Q
u a l i t y A g e n c y
26
Offshore Audit Visit Stages

• Request further information
• Decide an Offshore AV is required
• Notify Auditee request further information
• Construct visit program in consultation with, and
  via, Auditee
• Delegation undertake visit, accompanied by
  Auditee
• Delegation reports back to full Panel
• Follow-up issues identified

A u s t r a l i a n U n i v e r s i t i e s Q
u a l i t y A g e n c y
27
Tests for Transnational Audit Visits

• Materiality
• Significance
• Risks (to the provider)
• Risks (to the students)
• Host Country Accreditation
• Necessity
• Practicality

28
1. Materiality

• What are the numbers of staff and students per
  transnational venue (perhaps as a ratio to total
  student numbers)?
• If numbers are modest and stable, then Audit
  Panel may choose to put its efforts elsewhere.
• If numbers are big/variable enough to have a
  discernible impact on the SAIs financial
  position, then Audit Panel may consider
  thissufficiently material to warrant attention.

29
2. Significance

• What is the relationship of the transnational
  activities to the SAIs organisational
  strategies?
• If ad hoc, or not scheduled for growth in
  comparison with other organisational goals, then
  Audit Panel may conclude it not sufficiently
  significant to apply limited resources in
  auditing it.
• If transnational activities are subject of
  aggressive growth strategy then Audit Panel may
  deem them sufficiently significant to warrant
  particular attention, even if currently quite
  small.

30
3. Risks (to the provider)

• What is the likelihood and consequences of things
  going wrong with the transnational activities?
  Factors to consider include
• How long the activity has been established.
• How stable has been the operation.
• What risk management systems are in place.
• Whether there are any particular external
  sensitivities (e.g. media, political).

31
4. Risks (to the students)

• The institution may have thorough systems, tested
  refined through experience, for large bodies of
  transnational students, but not for the smaller
  operations.
• Use of transnational staff may increase risks for
  students, particularly regarding HDR supervision.
• Students may be denied access to University
  faculty.
• Support via a language other than Englishmay
  pose risks.

32
5. Host Country Accreditation

• Many transnational locations in which Australian
  SAIs operate have their own external quality
  mechanisms.
• The scope and rigour of host country
  accreditation processes varies considerably.
• What is the effect of these arrangements on the
  QA for Australian transnational activities?
• Can the Audit Panel rely on mutualrecognition?

33
6. Necessity

• Are there other, more cost-effective means
  available by which the Audit Panel can acquire
  the necessary knowledge?
• Examples include
• Teleconferencing
• Videoconferencing
• Documented materials
• Bringing people to the Panel

34
7. Practicality

• Can a visit be properly planned and achieved in
  the time available?
• What is the number and location of transnational
  venues? If the transnational activities are
  widely dispersed around the world, then there may
  be practical limitations of time and cost.
• Sampling is a technique used for auditing
  domestic activities and is no less applicable
  totransnational activities.

35
Implications for Quality Assurance

• Admission
• Teaching
• Assessment
• Support Services
• Evaluation and Review

• Philosophy
• Contract
• Approvals
• Curriculum
• Promotion

36
1. Philosophy

• What is the purpose of the transnational
  activity?
• What values underpin it?
• Who are the stakeholders/beneficiaries, what are
  their interests, and how are their interests
  being represented?

37
2. Contract

• Contract authorisation
• Was a thorough due diligence undertaken first?
• Signed by who? Was there appropriate
  consultation?
• Contract scope
• Academic QA issues covered in a clear manner?
• Are the risk management provisions adequate?
• Contract management
• Good communication processes?
• Is the contract subject to review?

38
3. Approvals

• Internal Approval
• Was it a planned or emergent event?
• Was it the contract explicitly approved within
  the SAI? How? Were the right people involved?
• What info contributed to the approval process?
• What values underpinned the approval process?
• External Approval
• Have the necessary host countryapprovals been
  obtained?

39
4. Curriculum

• Is the curriculum identical, equivalent,
  significantly tailored or unique?
• Is it appropriately contextualised (e.g. case
  studies, cultural sensitivity)
• In what language are curriculum materials?
• Have appropriate approvals been sought re
• credit recognition/exchange agreements?
• amendments to the program?
• changes to the delivery mode?

40
5. Promotion

• In what language is the program promoted?
• Who is doing the promotion, and to what
  guidelines? Is it consistent with domestic
  promotion regarding curriculum and policies?
• How accurate and fair is the promotion, e.g. re
  availability of electives?

41
6. Admission

• Entry Standards
• Identical, equivalent or different?
• Are credit transfers properly quality assured?
• Decision Makers
• Who is managing the student admission approvals?
• Where are approval decisions being made?
• Who is checking the validity of the evidentiary
  requirements (incl. prior credit)?

42
7. Teaching

• The Teachers
• Are there appropriate selection, induction,
  briefing and support of teachers?
• What accountability systems are in place?
• The Teaching
• Does the schedule provide adequate time for
  student reflection and assignments?
• Is the pedagogy appropriate?
• In what language is teaching provided?

43
8. Assessment

• Is the assessment schedule identical, equivalent
  or different to the base program?
• What is the language of all assessment?
• Who does the assessing? Is assessment moderated?
  Precisely how?
• Are results compared against the base program
  results?
• What academic security provisionsare in place
  (e.g. invigilation)?

44
9. Academic and Pastoral Support

• What (and how are) Library, IT and pastoral
  resources and services made available?
• How do these support services compare with the
  Australian version? What form of evaluation is
  in place?
• Are the decisions made in respect of support
  services deliberate and based on an assessment of
  student learning support requirements?

45
10. Evaluation and Review

• Are the learning outcomes demonstrably equivalent
  to those of the home program?
• Are teaching evaluations, course evaluations and
  course reviews being conducted? Using whose
  models? How are results used?
• How and how often is the contract reviewed?
• What information contributes to the review, and
  who is involved?

46

• Professor Deryck Schreuder
• Chair, AUQA

A u s t r a l i a n U n i v e r s i t i e s Q u
a l i t y A g e n c y