Competency-based Evaluation (and Competency in Evaluation!)

1
Competency-based Evaluation(and Competency in
Evaluation!)

• Kathy OKane Kreutzer, M.Ed.
• VCU School of Medicine
• Richmond, VA

2
We are confronted with insurmountable
opportunities.

• Pogo

3
Program Evaluation and Planning
Look at the Big Picture Map out your teaching and
evaluation activities against the
competencies Start with what you do best Choose
new teaching or evaluation activities to fill
gaps
4
Program Evaluation and Planning
and evaluation and planning and evaluation and
planning and evaluation and planning and
evaluation and planning and evaluation and
planning and evaluation and planning and
evaluation and planning and evaluation and
planning and evaluation and planning and
evaluation and planning and evaluation and
planning and evaluation and planning and planning
and evaluation and planning and evaluation and
planning This is an ongoing process! Excellent
Programs are Never Done
5
'It doesn't happen all at once you become. It
takes a long time.

• The Skin Horse
• The Velveteen Rabbit
• Margery Williams

6
Program Evaluation and Planning
Have clear learning objectives and criteria Make
trainees and faculty aware of the
objectives Evaluate what you are
teaching Document what you evaluate Communicate
feedback to trainees in a timely manner Have a
process for addressing individual needs of
trainees
7
Trainee Evaluation
Use different methods to evaluate different
competencies and multiple methods for each
competency Direct observation is the Gold
Standard Be sure that the evaluation method
matches the competency ACGME website go to
Outcome Project and then Assessment for
excellent examples and descriptions of assessment
tools and how and when they are most effectively
used
8
Trainee Evaluationadapted from ACGME/ABMS
Toolbox v 1.1
Patient Care Skill Maybe Better Best
Interviewing 360 Standardized Patient (SP) OSCE Checklist
Teamwork Checklist Global rating - - - - 360
Patient Ed Counseling Checklist 360 SP OSCE Pt survey
9
Ill do it because I want to, but not because you
tell me to!

• Mary OToole
• The Answer-Backer Cure
• Mrs. Piggle Wiggle
• Betty MacDonald

10
Own your evaluation methods and results
If you dont believe it dont do it If you dont
use it dont do it If you dont know why youre
doing it dont do it
11
Feedback is the breakfast of champions

• Ken Blanchard

12
Ongoing Feedback and Addressing Needs are Key
Challenge yourself to provide more frequent
feedback Weekly is not too often 5 minute
feedback can be effective Ask about and listen
to learner needs and concerns Develop a plan to
address needs Document
13
The FED Mnemonic
Feedback Encouragement Direction Hersh
ey S. Bell, MD
14
Familiarity breeds content

• Anna Quindlen

15
Challenges for Small Programs
A trainee in a small program deserves as much
planning and forethought in support of learning
and development as a trainee in a large program
16
Challenges for Small Programs
? Be sure your trainees have a peer group ?
Fellows are not exactly residents, not exactly
faculty identify opportunities for them to spend
time with other fellows ? Rounds, journal clubs,
interdisciplinary case presentations, research
projects, writing
17
Challenges for Small Programs

• Communication
• A close group can make some communication less
  effective
• Your team may be a family, but it is a family
  with a purpose
• Be explicit about when you are giving feedback
• Addressing any concerns promptly is especially
  important
• See Medios Top Ten Reasons We Avoid Talking to
  Someone About a Problem

18
Challenges for Small Programs

• Lack of anonymity is a fact in small programs
• ? Respect
• ? Privacy
• ? Focus on the behavior, not the person
• Confidentiality where possible
• Model receptivity to feedback

19
Education by design, not by chance

• Kreutzers Accreditation Axim

20
Unless someone like you cares a whole awful
lot, nothing is going to get better. Its not.

• Dr. Seuss
• The Lorax

21
Today is your day! Your mountain is waiting.
Soget on your way.

• Oh, the Places You'll Go!
• Dr. Seuss