The RQF at UWA

1
The RQF at UWA

• April 2007

2
RQF _at_ UWA

• Announced by Minister Bishop December 2006
• New system to assess research quality
• Distribute research block grant funding
• Based on the UK RAE process

3
RQF _at_ UWA

• No promise of new money
• Full details and rules unknown
• Key Document
• The Recommended RQF
• (October 2006) http//www.dest.gov.au/research/rqf
  

4
RQF _at_ UWA

• Eligible Staff
• Teaching and Research and Research only
• Employed 40 or more
• Level B and above
• Competitive grant funded research fellows (ARC
  Postdocs Level A may be assessed)

5
RQF _at_ UWA

• Possibly included (not yet decided)
• Adjuncts, Honorary, plus
• less than 0.4 appointments
• May be assessed (included on complete list of
  publications? context statements?)
• May be assessed for quality BUT not for funding

6
RQF _at_ UWA

• Eligible staff
• Must have 4 research outputs to assess
• BUT may be less for Early Career Researchers
  (between 2 and 4)

7
RQF _at_ UWA

• Key Dates
• Assessment Period 2001 to 2006
• Staff Census Date 31 March 2007
• (plus service period rules)
• Submission date 30 April 2008

8
RQF _at_ UWA

• Submissions
• Work will be submitted, assessed and reported via
  RFCD codes (research fields, courses and
  disciplines)
• Individuals will not be assessed
• Results will not be published for individuals
• Scores published for Universities

9
RQF _at_ UWA

• Submissions
• Not by individuals but by research groups
• University determined groups
• Based on RFCD codes and directed to (13)
  assessment panels
• Submit Evidence Portfolios

10
RQF _at_ UWA

• Evidence Portfolios
• 4 best research outputs per person
• Full list of publications
• Context statements (evidence of research quality)
• 10 page impact statement per group
• Plus institution list of submitted vs eligible
  staff

11
RQF _at_ UWA

• Quality Assessment
• Expert review not quite peer review
• Holistic view of a groups work
• May have some expert peer advice

12
RQF _at_ UWA

• 1. Biological sciences
• RFCD 2701-2799
• Biochemistry and cell biology, Genetics,
  Microbiology, Botany, Zoology, Physiology,
  Ecology evolution, Biotechnology, Other
  biological sciences

13
RQF _at_ UWA

• 6. Clinical sciences and clinical physiology
• RFCD 3201-3206, 3208-3210(less 321021),321204
• Medicine general, Immunology, Medical
  biochemistry clinical chemistry, Medical
  microbiology, Pharmacology pharmaceutical
  sciences, Medical physiology, Dentistry,
  Optometry, Clinical sciences (exc. Psychiatry),
  Mental health

14
RQF_at_UWA

• TABLE 1 THE RATING SCALE FOR RESEARCH QUALITY
• Rating Descriptor
• 5 Research that is world leading in its field or
  makes an equally exceptional contribution in an
  area of particular significance to Australia.
• 4 Research that meets world standards of
  excellence in its field or makes an equally
  excellent contribution in an area of particular
  significance to Australia.
• 3 Research that is recognised internationally as
  excellent in terms of originality, significance
  and rigour but which nonetheless falls short of
  the highest standards of excellence.
• 2 Research that is recognised as methodologically
  sound in its field and of high originality,
  significance and rigour.
• 1 Research that is deemed to fall below the
  standard of recognised quality work.

15
RQF_at_UWA

• Impact Assessment
• Research impact is defined as the social,
  economic, environmental and/or cultural benefits
  of research
• 10 page submission (verifiable evidence)
• 4 case studies
• End users to contact for references
• Must have a quality rating of 2 to be assessed

16
RQF _at_ UWA

• TABLE 2 THE RATING SCALE FOR RESEARCH IMPACT
• Rating Description
• A Adoption of the research has produced an
  outstanding social, economic, environmental
  and/or cultural benefit for the wider community,
  regionally within Australia, nationally or
  internationally.
• B Adoption of the research has produced a
  significant social, economic environmental and/or
  cultural benefit for the wider community,
  regionally within Australia, nationally or
  internationally.
• C Research has been adopted to produce new
  policies, products, attitudes, behaviours and/or
  outlooks in the end user community.
• D Research has engaged with the end user
  community to address a social, economic,
  environmental and/or cultural issue regionally
  within Australia, nationally or internationally.
• E Research has had limited or no identifiable
  social, economic, environmental and/or cultural
  outcome, regionally within Australia, nationally
  or internationally.

17
RQF _at_ UWA

• The UWA Approach
• Internal Research Quality Assessment process
• Helped us learn about the issues for managing and
  interpreting ratings for research quality

18
RQF _at_ UWA

• The UWA Approach
• Socrates as an RQF tool
• https//socrates.uwa.edu.au
• Log-on staff ID plus password
• Collects data from many university systems and
  presents it in a set of different views on
  research performance

19
RQF _at_ UWA

• The UWA approach
• Socrates
• Grants
• Publications
• Note CV function
• Socratic Index

20
RQF _at_ UWA

• Socrates
• Citation data (incomplete)
• journal impact factor,
• number of citations
• Expected versus actual citations
• Can help with selecting 4 best publications

21
RQF _at_ UWA

• Top down vs bottom up
• Minimise unproductive work
• Identify research groups (university structure of
  disciplines)
• Mostly school based but some centres
• Identify whose work falls in to which group
• Who is assessed and who is not

22
RQF _at_ UWA

• Identify a key person per group
• Draft up group statements
• Context and impact statements
• Internal review
• Write, revise, review and rewrite

23
RQF _at_ UWA

• Timeline
• February - April 2007
• talk to schools
• Identify groups (and rfcd codes)
• Identify key person

24
RQF _at_ UWA

• April May 2007
• Draft context and impact statements
• May - November 2007
• Review and redraft impact statements
• November- December 2007
• Finalise selection of research outputs to
  submit

25
RQF _at_ UWA

• Basic Assumptions
• More assessed rather than less
• UWA budget model unchanged
• UWA Mission and strategic goal more important
  than transient DEST funding formulas

26

• Assessment Panels
• with RFCD codes

27
RQF _at_ UWA

• 1. Biological sciences
• RFCD 2701-2799
• Biochemistry and cell biology, Genetics,
  Microbiology, Botany, Zoology, Physiology,
  Ecology evolution, Biotechnology, Other
  biological sciences

28
RQF _at_ UWA

• 2. Physical, chemical and earth sciences
• RFCD 2401-2699
• Astronomical sciences, Theoretical condensed
  matter physics, Atomic molecular physics
  Nuclear particle physics Plasma physics,
  Optical physics, Classical physics, Other
  physical sciences, Physical chemistry, Inorganic
  chemistry, Organic chemistry, Analytical
  chemistry, Macromolecular chemistry, Theoretical
  computational chemistry, Other chemical
  sciences, Geology, Geophysics, Geochemistry,
  Oceanography, Hydrology, Atmospheric sciences,
  Other earth sciences

29
RQF _at_ UWA

• 3. Engineering and technology
• RFCD 2902-2999
• Aerospace engineering, Manufacturing engineering,
  Automotive engineering, Mechanical industrial
  engineering, Chemical engineering, Resources
  engineering, Civil engineering, Electrical
  electronic engineering, Geomantic engineering,
  Environmental engineering, Maritime engineering,
  Metallurgy, Materials engineering, Biomedical
  engineering, Computer hardware, Communications
  technologies, Interdisciplinary engineering,
  Other engineering technology

30
RQF _at_ UWA

• 4. Mathematical and information sciences and
  technology
• RFCD 2301-2399, 2801-2899
• Mathematics, Statistics, Other mathematical
  sciences, Information systems, Artificial
  intelligence signal image processing,
  Computer software, Computation theory
  mathematics, Data format, Other information,
  computing communication sciences

31
RQF _at_ UWA

• 5. Agricultural, veterinary, food and
  environmental sciences
• RFCD 2901, 3001-3099
• Industrial biotechnology food sciences, Soil
  water sciences, Crop pasture production,
  Horticulture, Animal production, Veterinary
  sciences, Forestry sciences, Fisheries sciences,
  Environmental sciences, Land, parks
  agricultural management, Other agricultural,
  veterinary environmental sciences

32
RQF _at_ UWA

• 7. Public health and health services
• RFCD 3211-3299 (less 321204)
• Nursing, Public health health services (exc.
  Mental health), Complementary/ alternative
  medicine, Human movement sports science, Other
  medical health sciences

33
RQF _at_ UWA

• 8. Psychology, psychiatry, neurological,
  behavioural and cognitive sciences
• RFCD 3207, 321021,380, 3802,3803-3899
• Neurosciences, Psychology, Psychiatry, Cognitive
  science, Other behavioural cognitive sciences
  Linguistics.

34
RQF _at_ UWA

• 9. Social sciences and politics
• RFCD 3601-3705 (less 3702)
• Political science, Policy administration, Other
  policy political science, Sociology,
  Anthropology, Human geography, Demography

35
RQF _at_ UWA

• 10. Economics, commerce and management
• RFCD 3401-3599
• Economic theory, Applied economics, Economic
  history history of economic thought,
  Econometrics, Other economics, Accounting,
  auditing accountability, Business and
  management, Banking, finance and investment,
  Transportation, Tourism, Services, Other
  commerce, management, tourism and services.

36
RQF _at_ UWA

• 11. Law, Education and Professional Practices
• RFCD 3301-3399, 3901-4099, 3702
• Education studies, Curriculum studies,
  Professional development of teachers, Other
  education, Journalism, communication and media,
  Librarianship, Curatorial studies, Social work,
  Other journalism, librarianship curatorial
  studies, Law, Professional development of
  practitioners, Justice legal studies, Law
  enforcement, Other law, justice, law enforcement

37
RQF _at_ UWA

• 12. Humanities
• RFCD 3706-3799, 419901, 4201-4499
• History philosophy of science medicine, Other
  studies in human society, Art History and
  appreciation, Language studies, Literature
  studies, Cultural studies, Other language
  culture, Historical studies, Archaeology
  prehistory, Other history archaeology,
  Philosophy, Religion religious traditions,
  Other philosophy religion

38
RQF _at_ UWA

• 13. Creative arts, design and built environment
• RFCD 3101-3199, 4101-4199 (less 419901)
• Architecture and urban environment, building,
  Other architecture, urban environment and
  building, Performing arts, Visual arts crafts,
  Cinema, electronic arts multimedia, Design
  studies, Other arts (exc. Art history
  appreciation)