Professor Jeffrey Braithwaite Director, Institute of Health Innovation Director, Centre for Clinical

1
Professor Jeffrey BraithwaiteDirector, Institute
of Health InnovationDirector, Centre for
Clinical Governance ResearchUniversity of New
South Wales, AustraliaEPSO Meeting
Copenhagen, 23 October 2008
Accreditation research
2
Research partnership
3
The ARC Linkage Team

• Professor Jeffrey Braithwaite
• Dr David Greenfield
• Dr Marjorie Pawsey
• Professor Johanna Westbrook
• Professor Bill Runciman
• Professor Sally Redman
• Professor Robert Gibberd
• Conjoint A/Professor Mary Westbrook
• Dr Justine Naylor
• Ms Sally Nathan
• Ms Maureen Robinson
• Ms Judie Lancaster

• Mr Brian Johnston
• Dr Desmond Yen
• Ms Lena Low
• Ms Heather McDonald
• Ms Darlene Hennessey
• Mrs Margaret Jackson
• ACHS staff
• ACHS surveyors
• ACHS member organisations
• Consumer Reference Group

4
Systematic review
5
Systematic review
6
Systematic review
7
The research four studies

• A prospective, multi-method, multi-disciplinary
  , multi-level, collaborative project for
  researching health sector accreditation

8
The research four studies

• Research aim A
• To explore the relationships
• between accreditation,
• clinical performance,
• organisational culture, and
• consumer participation

9
The research four studies

• Research aim B
• To examine the influence of surveyors on both
  the accreditation process and outcome and their
  own health organisations

10
The research four studies

11
Study 1

• To determine whether there are relationships
  between EQuIP performance and
• organisational culture
• organisational climate
• consumer participation
• leadership
• clinical performance

12
Study 1

• This is a three year ARC grant which conducted
  four major studies of accreditation
• One key study, study 1, examined 19 randomly
  sampled health care organisations looking at
  accreditation performance, organisational
  climate, organisational culture, consumer
  involvement, leadership, and clinical indicator
  performance

13
Study 1

• We took six variables
• And measured them in the 19 randomly sampled
  health care organisations, each of which had
  participated in accreditation in Australia
  through ACHS EQuIP
• Each of the variables was measured and the data
  interpreted by a research team blinded from the
  other research teams

14
Study 1
Characteristics of the sampled organisations

• Small 7
• Medium 6
• Large 6
• Public 13
• Private 6

• Metropolitan 8
• Regional 3
• Rural 7
• Remote 1
• Each Australian state and a territory represented

15
Study 1

• Data and procedures
• Participant organisations were ranked 1 19 on
  the basis of performance
• Accreditation statistical ranking of performance
  based on ACHS EQuIP surveyor reports
• Organisational culture 1,000 semi structured
  interviews with organisational members

16
Study 1

• Data and procedures
• Organisational climate ethnographic,
  non-participant observations and informal
  interviews, one week at each site
• Consumer participation semi-structured
  interviews
• Leadership semi-structured interviews

17
Study 1

• Data and procedures
• Clinical indicators proportion of clinical
  indicators for that organisation that were better
  than the national average
• In summary independent measures of the six
  variables were taken, and on each variable the
  data were subject to a rank order correlation

18
Study 1

• A note on rank order correlation
• Correlation coefficients can range from -1 to 1
  they measure the degree of relationship between
  two variables
• A perfect linear relationship between two
  variables gives a correlation coefficient of 1

19
Study 1

• Summary
• Random sample n 19 organisations
• ACHS EQuIP outcome survey research tool
• UNSW research team cultural, climate, consumer
  participation, leadership assessment clinical
  data analysis
• Research outcome sought understanding
  organisational relationships

20
Study 1
21
Study 1 Outcomes

• Braithwaite et al. Health service accreditation
  as a predictor of clinical and organisational
  performance a blinded, random, stratified study.
  Submitted to International Journal for Quality in
  Health Care, 13/10/08.
• Braithwaite, J., Greenfield, D., Johnston, B.,
  Scrivens, E. and Shaw, C. (2008) Recent results
  analysed from large accreditation studies with
  new research results on accreditation, standards
  and surveying. Abstract in ISQua 2008.
  Twenty-fifth International Safety and Quality
  Conference Healthcare quality and safety
  meeting the next challenges, Copenhagen, Denmark
  International Society for Quality in Health Care,
  October 19-22

22
Study 1 Outcomes

• Nathan, S., McLure, K., Greenfield, D., Pawsey,
  M. and Braithwaite, J. (2008) Taking our first
  steps together the experiences of a research
  team and a consumer advisory panel in a national
  study of health service accreditation. Involving
  People in Research Symposium, Perth 5-6 March.
• Braithwaite, et al. (2007) Accreditation
  assessing the evidence. Transforming patient
  safety, accreditation and risk in health care in
  international context. Abstract in the ISQua
  2007. Twenty-fourth International Safety and
  Quality Conference Transforming health care in
  the electronic age, Boston, United States of
  America, International Society for Quality in
  Health Care, October 2.

23
Study 2

• A comparison between health services
  participating and not participating in an
  accreditation program

24
Study 2

• Details
• Purposive sample N 3 organisations
• UNSW research team cultural assessment
  clinical data
• Research outcome sought understanding
  accreditation effects

25
Study 2
26
Study 2 Outcomes

• Of the twelve indicators, in 9/12, or 75, the
  health services participating in accreditation
  outranked those health services that did not
  participate in accreditation

27
Study 2

• Other research findings
• One study failed to find any differences between
  accredited and non-accredited (rehabilitation)
  programs (Mazmanian et al. 1993)
• Another study found improved outcomes when a
  (trauma) service accredited (Simons et al. 2002)

28
Study 2

• Outcome
• Paper being drafted

29
Study 3

• To assess the intra- and inter-rater reliability
  of ACHS-EQuIP surveyors and survey teams

30
Study 3

• Details
• Five parts
• Scenario exercise intra-reliability
• Scenario exercise inter-reliability (team)
• Focus groups ACHS staff surveyors
• Focus group member organisations,
• An examination of two teams in practice
• Research outcome sought reliability of survey
  teams

31
Study 3 Outcomes

• What promotes reliability of surveyors and survey
  teams?
• A defined accreditation program where members
  participate in developing standards
• Trained, experienced surveyors
• Trained survey team leaders
• Shared expectations of accreditation, standards,
  surveying processes
• A system of self-governance with checks and
  balances along the way
• reliability

32
Study 3 Outcomes

• Greenfield, D., Pawsey, M., Naylor, J. and
  Braithwaite, J. (2008) Are healthcare
  accreditation surveys reliable?, International
  Journal of Health Care Quality Assurance, in
  press (Accepted 14/01/08).
• Greenfield, D., Braithwaite, J. and Pawsey, M.P.
  (2008) Health care accreditation surveyor style
  typology, International Journal of Health Care
  Quality Assurance, 21 (5), 435-443.
• Greenfield, D., Pawsey, M., Naylor, J. and
  Braithwaite, J. (2008) Improving the reliability
  of an accreditation program using research to
  educate and to align practice. Poster in ISQua
  2008. Twenty-fifth International Safety and
  Quality Conference Healthcare quality and
  safety meeting the next challenges, Copenhagen,
  Denmark International Society for Quality in
  Health Care, October 19-22

33
Study 4

• To examine the influence of ACHS-EQuIP surveyors
  on their own health services

34
Study 4

• Details
• Two PhD studies in progress
• Retrospective study of accreditation outcomes
  with surveyors
• Prospective in-depth case studies of health
  services with surveyors
• Research outcome sought understanding surveyor
  influence

35
Study 4 Outcomes

• Details
• Becoming a surveyor has benefits for both the
  surveyor and the surveyors host organisation
• Surveyors get exposed to other organisations
  processes, how they do things, what they do well
  and not so well
• This allows them to take back that learning to
  their home organisation
• They do this formally and informally

36
Study 4 Outcomes

• PhD expected to be completed 2010 and 2011.
• Lancaster, J., Braithwaite, J. and Greenfield,
  D. Benefits of participating in accreditation
  surveying. Submitted to International Journal of
  Health Care Quality Assurance, 9/9/08.

37
Other research outcomes

• Selected papers
• Greenfield, D., Braithwaite, J., Pawsey, M.P.,
  Johnston, B. and Robinson, M. (2009) Distributed
  leadership to mobilise capacity for accreditation
  research, Journal of Health Organisation and
  Management, in press (Accepted 20/6/08).
• Greenfield, D. and Braithwaite, J. (2008) Health
  care accreditation research a systematic review,
  International Journal of Quality in Health Care,
  20 172-183.

38
Other research outcomes

• Selected papers
• Greenfield, D. and Braithwaite, J. (2007)
  Researching accreditation, E-Hospital, Journal of
  the Association of European Hospital Managers, 9
  (5) 18-19.
• Braithwaite, J., Westbrook, J.I., Pawsey, M.,
  Greenfield, D., Naylor, J., Iedema, R.A.,
  Runciman, B., Redman, S., Jorm, C., Robinson, M.,
  Nathan, S. and Gibberd, R. (2006) A prospective,
  multi-method, multi-disciplinary, multi-level,
  collaborative, social-organisational design for
  researching health sector accreditation
  LP0560737, BMC Health Services Research, 6,
  113-123.

39
Other research outcomes

• Selected presentations
• Braithwaite, J., Greenfield, D., Johnston, B.,
  Scrivens, E. and Shaw, C. (2008) Recent results
  analysed from large accreditation studies with
  new research results on accreditation, standards
  and surveying. Abstract in ISQua 2008.
  Twenty-fifth International Safety and Quality
  Conference Healthcare quality and safety
  meeting the next challenges, Copenhagen, Denmark
  International Society for Quality in Health Care,
  October 19-22.
• Braithwaite, J., Greenfield, D. and Westbrook,
  M. (2008) Contrasting and converging perspectives
  on organisational culture and climate. In
  Conference Proceedings of Culture and Climate
  Cracking the Code. The Sixth International
  Conference on Organisational Behaviour in Health
  Care obhc2008, Sydney Society for the Study of
  Organising in Health Care, March 26-28.

40
Other research outcomes

• Selected presentations
• Greenfield, D., Travaglia, J., Pawsey, M.,
  Lloyd, J. and Braithwaite, J. (2008) Who is
  accountable for quality and safety? Poster in
  Bold aims, bold outcomes The Sixth Australasian
  Conference on Safety and Quality in Health Care,
  Christchurch, New Zealand Australian Association
  for Quality in Health Care, September 1-3.
• Greenfield, D., Braithwaite, J. and Pawsey, M.
  (2007) Mobilising academic, industry and
  government stakeholders in collaborative research
  partnerships for improved patient safety through
  accreditation research. Abstract in the Patient
  Safety Research Conference Shaping the European
  Agenda, Porto, Portugal European Commission
  Sixth Framework Programme for Research and the
  Portuguese Ministry of Health, September 24-26.

41
Summary and conclusion

• 12 papers published or in press
• 7 papers in production
• 10 presentations to international conferences
• 4 further presentations planned
• Australian Accreditation Research Network
  established
• Research project, comprising four studies,
  successfully completed
• International interest stimulated

42
Questions?