Structured Assessment Format for Evaluating Operative Reports (SAFE-OR) in General Surgery - PowerPoint PPT Presentation

1 / 27
About This Presentation
Title:

Structured Assessment Format for Evaluating Operative Reports (SAFE-OR) in General Surgery

Description:

Title: Structured Assessment of Operative Reports Author: Ashley Vergis Last modified by: Susan Kepner Created Date: 4/30/2006 6:54:03 PM Document presentation format – PowerPoint PPT presentation

Number of Views:300
Avg rating:3.0/5.0
Slides: 28
Provided by: AshleyV150
Category:

less

Transcript and Presenter's Notes

Title: Structured Assessment Format for Evaluating Operative Reports (SAFE-OR) in General Surgery


1
Structured Assessment Format for Evaluating
Operative Reports (SAFE-OR) in General Surgery
  • Ashley Vergis1 MD, Lawrence Gillman1 MD, Jason
    Park1 MD, Samuel Minor MD FRCSC2, Mark Taylor1 MD
    FRCSC
  • 1University of Manitoba, Winnipeg, Canada
  • 2Dalhousie University, Halifax, Nova Scotia,
    Canada

2
  • Medicine is an information intensive and reliant
    profession whose practice is dominated by how
    well information is processed or reprocessed,
    retrieved and communicated

Murphy et al. (1992). Good European Health
Record. Centre for Health Informatics and
Multi-professional Education University College
London 1-62.
3
Background
  • Operative dictations essential for
  • Caregiver communication
  • Quality assurance
  • Research
  • Medico-legal

? Must be comprehensive and concise!
Eichholz et al. (2004). Obstetrics and
Gynecology 103(2) 342-6.
4
Background
  • Multiple previous attempts to judge the quality
    of OR notes
  • Third party billing criteria
  • Comparison to consensus criteria
  • Overall quality appears poor
  • Critical details often omitted
  • Superfluous aspects detailed

5
Background
  • However
  • No validated tools that examine quality of
    operative dictations
  • Consequences
  • Cannot currently measure the adequacy of an
    operative report
  • Unclear if operative reports meet their stated
    objectives

6
Objective
  • Evaluate the Structured Assessment Format for
    Evaluating Operative Reports (SAFE-OR) in
    General Surgery
  • Construct validity
  • Inter-rater agreement
  • Internal consistency

7
Methods
  • Subjects
  • University of Manitoba general surgery residents
  • Novices PGY 1-3
  • Experienced PGY 4-6

8
Methods
  • Design
  • View and dictate a videotaped laparoscopic
    sigmoid colectomy
  • Dictation transcribed

9
Methods
  • Design
  • Evaluated by 2 blinded, independent faculty
    general surgeons
  • SAFE-OR instrument
  • Structured Assessment Format
  • Adapted from CAGS clinical practice committee
    guidelines
  • Global Quality Rating scale

10
Methods
  • Structured Assessment Format
  • 19 Categorical items (yes/no)
  • E.g., Date of Procedure
  • 7 Ordinal items (5 point anchored Likert Scale)
  • E.g., Findings on Exploration

11
Structured Assessment Form
12
Methods
  • Global Quality Rating Scale
  • 5 point anchored Likert scale
  • 9 categories
  • 1. Readability 6. Clarity
  • 2. Operative Indications 7. Knowledge
  • 3. Operative Steps 8. Reproducibility
  • 4. Operative Findings 9. Overall Quality
  • 5. Succinctness

13
Global Quality Ratings Scale
14
Analysis
  • Construct Validity
  • Novice and experienced groups
  • compared (Independent samples T test)
  • Inter-rater Agreement
  • Intra-class correlation coefficient (ICC)
  • gt0.8 desirable
  • Internal Consistency
  • Cronbachs alpha

Wanzel, et al. (2002). Current Problems in
Surgery 39(6) 573-659.
15
Results Construct Validity
Mean Score Mean Score
Novice (n13) Experienced (n8) p
Structured Assessment (out of 44) 23.3 5.2 34.1 6.0 0.001
Global Quality (out of 45) 25.6 4.7 35.9 7.0 0.006
16
Results Inter Rater Reliability Internal
Consistency
Overall ICC (95 CI) Range Cronbachs Alpha
Structured Assessment 0.98 (0.96 - 0.99) 0.73 - 0.90 0.85
Global Quality 0.93 (0.83 - 0.97) 0.65 - 0.91 0.96
Ordinal items, Kappa 0.73 - 1.00 for
categorical items
17
Conclusion
  • SAFE-OR
  • Satisfies construct validity
  • Excellent inter-rater reliability
  • Excellent internal consistency

18
Project Significance
  • SAFE-OR is the first tool that assesses the
    quality of an operative report in a valid and
    reliable manner

19
Project Significance
  • Means to more objectively measure operative
    report characteristics
  • Assess degree to which reports meet stated
    functions (quality assurance)
  • If unsatisfactory, may employ methods to improve
    reporting

20
Project Significance
  • Potential to develop methods of improving
    dictations in surgical education
  • Implement SAFE-OR for assessment in training
    programs
  • Formative and Summative
  • Feedback (self and faculty)
  • Global attributes
  • Specific details

21
Project Significance
  • Develop, monitor and refine curriculum for
    operative dictation
  • Improve communication
  • ? Implications for patient care

22
Future Investigation
  • Analyze characteristics of real world operative
    dictations
  • National survey (Cdn) of resident and program
    directors training perceptions
  • Develop and assess specific templates for
    operative dictation
  • Assess computerized dictation templates (WebSMR)
  • Edhemovic et al., Annals of Surgical
    Oncology, 11(10)941947.

23
Acknowledgements
  • Royal College of Physicians and Surgeons of
    Canada Medical Education Research Grant

24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
Structured Assessment Format for Evaluating
Operative Reports (SAFE-OR) in General Surgery
  • Ashley Vergis1 MD, Lawrence Gillman1 MD, Jason
    Park1 MD, Samuel Minor MD FRCSC2, Mark Taylor1 MD
    FRCSC
  • 1University of Manitoba, Winnipeg, Canada
  • 2Dalhousie University, Halifax, Nova Scotia,
    Canada
Write a Comment
User Comments (0)
About PowerShow.com