PrelimCheck: Software to Facilitate Follow-Up of Resident Preliminary Reports

1
PrelimCheck Software to Facilitate Follow-Up of
Resident Preliminary Reports

• Phillip Cheng, MD MS
• Assistant Clinical Professor of Radiology

2
Objective

• The purpose of this exhibit is to illustrate
  an open source desktop tool for follow-up of
  resident preliminary notes, allowing residents to
  rapidly evaluate their notes for discrepancies
  compared to final dictated reports.

Patient and study identifiers shown in this
presentation have been obscured, altered or
synthesized.
3
Introduction

• At our teaching institution, radiology residents
  begin taking independent call at night in their
  second year of training.
• During the night, their preliminary
  interpretations are recorded as notes in the PACS
  (Fuji Synapse). Typically 50-100 notes are
  placed by a resident per call night.
• The attending dictates the final reports in the
  morning and reviews preliminary notes by the
  resident.

4
Resident Notes in PACS
5
Resident Notes in PACS
6
A Feedback Problem

• Independent call experience is considered by our
  residents and faculty as one of the most
  important educational experiences of their
  residency.
• However, feedback is an essential part of this
  education, and is not immediately available since
  the resident leaves in the morning before the
  attendings dictate final reads for the overnight
  studies.
• In particular, there is no mechanism in the PACS
  for discrepancies between preliminary and final
  reads to be flagged for later review.

7
Reviewing Discrepancies

• We have reviewed several unsatisfactory options
  for reviewing overnight discrepancies with
  residents
• Morning readout with attending post-call
  resident is typically tired and not receptive to
  feedback
• Attending e-mail to residents requires extra
  effort by attending especially for minor
  discrepancies, so such feedback is sporadic
• Residents reviewing own reads from a previous
  call night
• Attractive since the residents are usually
  motivated to review their call performance
  however, identification and sorting of the
  preliminary and final reads from a call night is
  tedious and prone to incompleteness.

8
PrelimCheck Facilitating Feedback
PrelimCheck is a tool that completes the
communication loop between attendings and
residents.
?
Resident
Notes in PACS, Dictated reports
Notes in PACS
Attending
9
PrelimCheck

• Open source, portable Windows desktop application
• Written in C, requires .NET Framework 2.0
• Communicates directly with Fuji Synapse PACS

10
PrelimCheck - Usage

• Easy-to-use tool for residents to compare their
  preliminary notes with attending follow-up notes
  and final reports
• A resident specifies the location and time period
  of the call shift and clicks the Download button

Location
Time Period
11
PrelimCheck - Usage

• Download
• Performs NTLM authentication to Synapse PACS
  server
• SQL query over network for notes performed in
  requested time period
• Retrieve text of each note, and any dictated
  report for the corresponding study
• Typical download time for 12 hr shift approx 1
  minute

12
PrelimCheck - Usage
Sortable list of studies with notes placed in the
specified time frame
Search list by keywords or save to comma
separated value (CSV) file
All preliminary notes for highlighted patient
study
Dictated report, automatically scrolled to
impression section
13
PrelimCheck - Usage

• Requires approximately 5-10 minutes for review of
  overnight call cases
• No built-in intelligence to detect and flag
  discrepancies between preliminary and final
  report
• Ease of rapid review partially obviates this
  problem

14
Survey

• Use of the program is encouraged but optional in
  our residency
• One year after release, residents of all training
  levels were anonymously surveyed on usage, survey
  response rate 83

15
Survey - Usage

• Most respondents used the program. The main
  stated reason for non-usage was lack of
  familiarity with program, or lack of call
  experience.

16
Survey Usability

• Most respondents rated the program high in
  usability / ease of use.

17
Survey - Effectiveness

• Despite lack of automated flagging of
  discrepancies, most respondents indicated that
  the program improved discrepancy detection.

18
Sample of Anonymous Comments
19
Limitations and Future Work

• Lack of automated discrepancy detection
• Attendings can use standardized code words in
  notes to accurately flag and rate discrepancies
• Slow network client-based implementation
• Requires approximately 1 minute to download notes
  for a call shift
• Faster response possible with a server-side
  implementation
• Inability to filter results during network
  retrieval
• Since notes are not indexed on PACS server, all
  notes within time period are retrieved and then
  filtered on the client side
• Solvable with server-side implementation

20
Source Code Availability

• PrelimCheck source code is available under the
  Apache 2.0 open source license.
• All source available at Github
• http//github.com/pmcheng/prelimcheck

21
Summary

• PrelimCheck, an open source desktop
  application, facilitates rapid resident
  self-evaluation of on-call preliminary radiologic
  interpretations.

http//hsc.usc.edu/phillimc/prelimcheck phillip.
cheng_at_usc.edu