A Practical Approach to Evaluating Systems Based Practice Nelson Perret, MD LSUEarl K. Long Medical

1
A Practical Approach to Evaluating Systems Based
PracticeNelson Perret, MDLSU/Earl K. Long
Medical CenterBaton Rouge, LA
2
Why LSU Baton Rouge?

• Had to answer this question before moving on to
  SBP

3
Positive thinking

• If you can understand the LSUHSC/NO LSU/Earl K.
  Long Medical CenterBaton Rouge General Health
  SystemCM Medical Services arrangement then you
  clearly know systems!

4
The Real Answer

• Systems-Based Practice Core CompetencyThe
  Problematic One?
• ---Michael S. Beeson, MD, SAEM Newsletter,
  January/February 2003, Vol. XV, Number 1
• No one else would do it!

5
SBP The Ugly Stepdaughter

• Patient Care
• Medical Knowledge
• Practice Base Learning and Improvement

• Interpersonal and Communication Skills
• Professionalism
• Systems-Based Practice

6

• SBP is that part of the clinical competencies
  which causes the most frustration in medical
  practice

7
Number Six

• Patient Care
• Medical Knowledge
• Practice Base Learning and Improvement

• Interpersonal and Communication Skills
• Professionalism
• Systems-Based Practice

8
Patient Care

• Cognitive skills
• Medical knowledge
• Practice Base Learning and Improvement

• Relational Skills
• Interpersonal and communication skills
• Professionalism

9
Where does systems based practice fit in?

• Entity unto itself?
• Integrates all the other competencies?

10
Systems-based practiceSBP

• Awareness of and responsiveness to the larger
  context and system of health care
• The ability to effectively call on system
  resources to provide care that is of optimal
  value
• www.ACGME Outcome Project.org

11
CORD-EM Consensus GroupEM Goals for SBP

• Understand, access, appropriately utilize, and
  evaluate the effectiveness of resources,
  providers, and systems necessary to provide
  optimal emergency care
• Understand different medical practice models and
  delivery systems and how to best utilize them to
  care for the individual patient

12
CORD-EM Consensus GroupEM Goals for SBP

• 3. Practice cost-effective health care and
  resource allocation that does not compromise
  quality of care
• 4. Advocate for and facilitate patients
  advancement through the health care system

13
Understand, access, appropriately utilize, and
evaluate the effectiveness of resources,
providers, and systems necessary to provide
optimal emergency care

• Understand
• Effectiveness of resources
• Providers
• Systems
• Access
• Resources
• Providers
• Systems

14
Understand, access, appropriately utilize, and
evaluate the effectiveness of resources,
providers, and systems necessary to provide
optimal emergency care

• Appropriately utilize
• Resources
• Providers
• Systems
• Evaluate
• Resources
• Providers
• Systems

15
Summaryof SBP

• Know what is out there
• Know what is available to you
• Do the best with what you have
• Work with all the people, personalities and
  resources within the system
• Remember that cost is important
• Work in the system to make it better

16
SBP in a Nutshell

• Give each individual patient the best care
  possible with the hand dealt to you and the
  patient and work to make sure the next hand dealt
  is a better hand

17
Our Process

• Listed the areas that we thought were important
  in SBP
• We determined how we were going to measure the
  outcomes of training in those areas

18

• Tiger Woods
• vs.
• Bill Glasson

19
The Tiger Woods WayDrs. Dyne, Strauss, Rinnert
Way

• Take the physician tasks from the Model of
  Clinical Practice of Emergency Medicine and
  assign a relevance number to each one in relation
  to SBP
• Dyne PD, Strauss RW, Rinnert S. Systems-Based
  Practice The Sixth Core Competency. Academic
  Emergency Medicine 200291270-1277.

20
Table 1. Physician Tasks from the Model of the
Clinical Practice of Emergency Medicine with
Relevance to Systems-Based Practice (SBP)
21
The Tiger Woods WayDrs. Dyne, Strauss, Rinnert
Way

• Take the physician tasks with the highest
  relevancy to SBP and determine what
  subcompetencies fall under that task

22
Table 2. The Physician Tasks from the Model with
a Systems-Based Practice (SBP) Value of 1 with
the Corresponding Subcompetencies.
23
The Tiger Woods WayDrs. Dyne, Strauss, Rinnert
Way

• Determine what tools you will use to measure the
  outcomes of the subcompetencies

24
Assessment Methodologies

• Primary Assessment Methodologies for SBP
• Direct observation
• Global rating
• 360-degree evaluation
• Portfolios
• Standardized oral examination
• Written Examination

25
The Bill Glasson WayThe LSU/Earl K. Long Way

• Take the 4 goals of the CORD-EM Consensus Group
  for SBP and list important components of each
  goal
• Understand interaction of practice within the
  larger system
• Knowledge of practice and delivery systems
• Practice cost-effective care
• Advocate for patients within the health care
  system

26
The Larger Context and System of Health Care

• Other Health Care Workers (Non EP)
• Hospital System
• Preventative Medicine

27
The Larger Context and System of Health Care

• Interaction with other HCWs
• Consultants
• Nurses in ED
• Other hospital departments (x-ray, lab)
• Emergency medical services
• Public health and Office of Emergency Preparedness

28
The Larger Context and System of Health Care

• Within the hospital system
• Medical records
• Hospital committees
• Public relations for hospital
• Safety net for a strained system
• Performance improvement (PI)

29
The Larger Context and System of Health Care

• Preventative Care
• Teaching in clinical setting
• Community service

30
Knowledge of Practice and Delivery Systems

• EMTALA
• HMOs
• Charity Hospital System
• Medicare
• Medicaid
• Formularies
• Prescription Writing Regulations

31
Practice Cost-Effective Care

• Patient work-up
• Labs
• Imaging
• Ability to explain no workup necessary
• Patient treatment
• Medicines in the ED
• Outpt prescriptions
• Medications not indicated
• Therapeutic modalities (esp. ortho)

32
Advocate for Patients Within the Health Care
System

• Getting those that need admitting, admitted
• Getting proper testing done in a strained system
• Getting proper outpatient follow-up
• EP follow-up of ED patients
• Not just a MD but a MSW
• Community Service

33
Evaluating Competency

• Direct observation
• Standardized oral examinations
• Written examinations
• 360-degree evaluation
• Portfolios
• Global rating

34
Direct observation

• Most useful tool
• To be accurate
• All evaluators need to be familiar with the
  elements being evaluated
• Evaluations need to be done in real-time and not
  past recall
• Some agreement on what the numbers on the ordinal
  scale mean (Can 75 of class be excellent?)
• Grade as EM resident do not grade level of
  training

35
Checklist

• List the SBP elements that can be evaluated by
  direct observation
• It is to be completed at the end of the
  residents shift and hopefully discussed with
  resident
• Not every resident is evaluated on every
  competency on every shift

36
Consistency in evaluating on an ordinal scale

• Mini lectures or discussions during faculty
  meetings
• Identify outliers

37
360-degree evaluation

• Very good for SBP because the system is
  evaluating the doctor
• Hard to get the patient aspect of the evaluation
• May come in from backdoor (pt. complaints)
• Include consultants as evaluators

38
Written examinations

• Quarterly written examinations
• Will code each question for which of 6 Core
  Competencies it covers
• Resident will get a global score on test and also
  scores for each of the competencies

39
An 18 month old child presents with suppurative
OM. Which of the below is the most appropriate
treatment?

• Amoxicillin
• Amoxicillin/clavulanate
• Ceftriaxone
• Cefprozil
• SMX/TMP

40
Clinical competencies evaluated by previous test
question

• Patient care
• Medical knowledge
• Practice based learning and improvement
• Systems-based practice

41
Standardized oral examinations

• Do we give an SBP grade for a global oral
  examination or do we make an oral examination
  just to evaluate SBP?

42
Portfolios

• Great concept
• Needs more emphasis
• Not for everyone
• May be one good method for deciding who is at the
  head of the class in SBP

43
Global rating

• The REC (Resident Evaluation Committee) takes all
  of the assessment tools used to evaluate SBP and
  assigns a global rating for SBP for each resident
• Essentially done in the past but very subjective
  because done primarily from recall without
  objective evidence in front of evaluators

44
Caveat in evaluating the elements of
systems-based practice

• All elements should not be weighted equally

45
SBP elementsThe Heavy Weights

• Interaction with other health care workers
• Interaction with administration
• Being sure the sick (insured and uninsured) get
  proper medical care
• EMTALA

46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
These competencies to be evaluated every 6 months.
51
Last but not Least!

• Patient Care
• Medical Knowledge
• Practice Base Learning and Improvement

• Interpersonal and Communication Skills
• Professionalism
• Systems-Based Practice

52
Systems-based practice

• Probably more important in the discipline of EM
  than any other specialty
• The only way to effect change in the larger
  health care system is to be involved in
  systems-based practice

53
Which resident is practicing good SBP?

• The Perret Test
• If you are working a night shift and you look at
  the resident schedule and you are happy with the
  resident who is working then that resident most
  likely practices good systems-based medicine.

54
In Conclusion

• Dont make this harder than it is
• It is always good having Tiger Woods on your team
• But, Bill Glassons technique is also pretty good