Pros and Cons of an InService Cardiovascular Examination

1
Pros and Cons of an In-Service Cardiovascular
Examination

• George W. Vetrovec, MD.
• VCU Pauley Heart Center
• MCV Campus
• Virginia Commonwealth University
• Richmond, Virginia

2
Disclosures

• Member of the ABIM Cardiology Examination Board.

3
A Cardiology In Service ExamScope of the Issue

• What this is Not A group of training directors
  writing a few questions a piece to make an
  exam.

4
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

5
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

6
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

7
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures.

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

8
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

9
Writing a Good Question

• Testing point.
• Stem.
• Legitimate distracters.
• No negative or EXCEPT questions.
• No Multiple True False questions.

10
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

11
Testing a Question

• Discrimination.
• Compare responses of top and low scorers.
• Difficulty.
• Relevance.

12
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

13
Exam Reproducibility

• Exam reproducibility.
• Comparative multiple exams.
• Predictability for trainees future success.
• ABIM Certification.
• Practice competency.

14
Examination Security

• Where given.
• Proctors.
• Certainty of Test Taker identity.

15
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

16
Pros and Cons

• Pros
• Measurement of Fellow progress.
• Identify trainee weaknesses early.
• Comparative training program quality.
• Satisfy ACGME Competency Outcomes Measures

• Cons
• Quality Questions.
• Psychometric Management.
• Comparative Exams
• Risk of teaching to an Exam.
• Cost.

17
Surmounting the ConsExamples from Other
Programs
18
Internal Medicine In Service Exam

• Owned by the American College of Physicians.
• Contract with the National Board of Medical
  Examiners for psychometric support.

19
IM In Service Exam History of Exam

• Started in 1988.
• Evolved from the MKSAP Exam.

20
IM In Service Exam Magnitude of Exam

• 420 Programs 20,000 trainees.
• Two 3.5 hour exams.
• 4 Administrations per year.
• Requires extensive proctors/escorts to locally
  administer the exams.
• Cost - 90 per test taker.
• Financially self sustaining.

21
IM In Service Exam Structure

• Steering Committee.
• Writing Group.
• 8 Writers.
• Train Writers
• Contract with National Board of Medical Examiners
  (USMLE)

22
IM In Service Exam Future Plans

• Make On line exam.
• 2-3 Years.

23
Other Models

• Pediatric Cardiology.
• American Board of Pediatrics.
• Connective Tissue.
• National Board of Medical Examiners

24
Cardiology In Service Exam Start up

• Organizing Group.
• Clinician test writers.
• Consulting Agency.
• Psychometrics.
• Test writing skills.
• Start up funding

25
A Cardiology In Service Exam Special
Considerations

• Images.
• Technical skill testing

26
A Cardiology In Service Exam Summary

• Feasible.
• Useful in evaluating trainee progress.
• Must be done well to achieve the benefits of such
  an exam.