Is a Mock Best Option MCQ a Valid Measurement of Student Performance in Final Clinical Medicine

1
Is a Mock Best Option MCQ a Valid Measurement of
Student Performance in Final Clinical
Medicine A. Patterson, M. Shuhaibar, C.
OMorain, M. Hennessy, S.R. McCann University of
Dublin, Trinity College, Ireland AMEE CONFERENCE
2009
BACKGROUND Validity is one of the core
principles on which assessment is based.
Validity is the extent to which a test actually
measures what it claims to measure1. This study
builds on previous work2 examines both predictive
and face validity of an examination. Predictive
validity examines if the test can predict future
performance in a particular area. Face validity
is the least scientific measure of validity but
it pertains to the students perception of the
test and if they believe it is a measure of their
overall knowledge. If a test has predictive
validity it allows students see how they are
progressing in the course and highlights if they
need to amend their study strategies to perform
better.
OBJECTIVES to compare student performance in a
mock best option multiple choice question
examination with their performance in a Final
Clinical Medicine MCQ examination and their
clinical examinations for 2008 and 2009. In
addition, the analysis investigated if the mock
examination completed by students three months
prior to the real examination could predict how
students perform in the real examination. Also,
to ascertain students perceptions of the
examination.
2008 N 103 Mock MCQ Real MCQ Minor cases Major cases Final Mark
Mock MCQ 0.626 0.433 0.423 0.655
Real MCQ 0.626 0.563 0.566 0.790
Minor cases 0.433 0.563 0.576 0.819
Major cases 0.423 0.566 0.576 0.803
Final Mark 0.655 0.790 0.819 0.803
RESULTS The results for 2008 and 2009 were
compared for students who completed both tests
using a Pearson correlation coefficient (N 103,
2008 and N 116, 2009). There was a strong,
positive correlation between both tests, 2008 and
2009 r0.63, 2009 r 0.387 respectively. Both
correlations were significant. The mock
examination results were also compared with
students clinical performance and their overall
mark. There were significant correlations with
all these measurements. A paired t-test was
performed to see if there was any significant
difference in student performance between the two
tests. Students completed an evaluation of the
examination. There was no significant difference
between student performance on both tests with
tlt0.001. The means for the mock and the real
examinations were 60.6 10.5 and 68.7 12.5,
52.3 8.8 and 67.9 9.5 for 2008 and 2009
respectively.
2009 N 103 Mock MCQ Real MCQ Minor cases Major cases Final Mark
Mock MCQ 0.387 0.415 0.224 0.526
Real MCQ 0.387 0.354 0.218 0.587
Minor cases 0.415 0.354 0.298 0.789
Major cases 0.224 0.218 0.298 0.639
Final Mark 0.526 0.587 0.789 0.639
DISCUSSION Students performed better in the real
examination versus the mock examination, there
was a period of three months between the tests
when an intensive course in Clinical Medicine was
provided for students. The predictive validity
of the mock examination was tested and
proved. Students valued the results of the mock
examination as a valuable indicator of how they
were proceeding in the course and commented that
it reduced the anxiety of sitting the real
examination.
Correlation is significant at the 0.01 level
Correlation is significant at the 0.05 level
It was an insight. Thank you. It has given me
the motivation to study  
CONCLUSIONS The mock examination is a valuable
examination that provides students with an
opportunity to see how they are progressing in
the course. It is under consideration to convert
the mock into a real examination. However, as
students perform better in the real examination
and the tests are examining equivalent knowledge,
this study suggest it should be continued as a
purely formative examination.

• REFERENCES
• Brown, G., Bull, J. and Pendlebury, M., 1997.
  Assessing student learning in higher education. ,
  Routledge, London
• A. Patterson, M. Shuhaibar, Prof. C. OMorain,
  The Predictive Validity of a Mock MCQ
  Examination of Student Performance in a Final
  Clinical Medicine MCQ examination, INMED
  Conference, Ireland, 2009