You can do medical education research!

1
You can do medical education research!

• Rachel Ellaway Ph.D.
• Assistant Dean Curriculum and Planning

Lisa Graves, MD, CCFP, FCFP Associate Dean
Undergraduate Medical Education
2
Conflict of Interest

• We have no financial interests, arrangements, or
  affiliations that constitute a direct or indirect
  conflict of interest in the context or content of
  the subject of this workshop.

3
Conflict of Interest

• Intros 5
• What is MER? 10
• A1 Opportunities 15
• Getting started 10
• A2 Ask a good question 15
• Designing a study 10
• A3 sketch a study 15
• Close 10

4
Teaching objectives

• By the end of the session participants will be
  able to
• Differentiate between the many forms of
  contemporary medical education research.
• Identify needs and opportunities for undertaking
  medical education research in their own practice.
• Appraise the support and resources needed to
  conduct a medical education research project and
  how to obtain them.
• Prepare a simple medical education research plan,
  implement it and report on it.

5
Learning objectives

• Up to you but

6
What is medical education research?
7
Medical Education Research

• any investigation related to the education of
  medical professionals, including research related
  to undergraduate (medical school), graduate
  (residency), and continuing medical education

Collins J (2008) Medical Education Research
Challenges and Opportunities. Radiology, 240,
pp639-647
8
What is the focus?

• medical education research can focus on any
  number of topics, including curriculum
  development, teaching methods, student
  evaluation, teacher evaluation, course
  evaluation, faculty development, admission and
  preparation of candidates for medical training,
  factors influencing career choice, research
  methodology, and use of technology in education

Collins J (2008) Medical Education Research
Challenges and Opportunities. Radiology, 240,
pp639-647
9
What is researched?

• curriculum and teaching issues
• skills and attitudes relevant to the structure of
  the profession
• individual characteristics of medical students
• the evaluation of students and residents
• and more
• Acad Med. 2004 Oct79(10)939-47. Trends in
  medical education research. Regehr G.

10
Medical Education Research

• There are many forms of contemporary medical
  education research, many schools of thought
• Still a relatively underdeveloped field
• Critical to medical education

11
Why do it?

• Evaluation
• Quality assurance
• Quality improvement
• Increased effectiveness
• Increased efficiency
• Professional responsibility
• Curiosity
• Ultimately improving patient care
• Accountability

12
Accountability

• medical education is accountable through its
  research arm, where, in principle, research
  evidence provides the rationale for practices

Bleakley, A, Bligh, J and Browne, J (2011).
Medical Education for the Future Identity, Power
and Location. Dordrecht, Germany, Springer.
13
Evidence Based Medical Education

• Best Evidence Medical Education (BEME) is defined
  as "the implementation by teachers and
  educational bodies in their practice, of methods
  and approaches to education based on the best
  evidence available
• BEME collaboration generating systematic reviews
• CEMESTR

http//www.bemecollaboration.org
14
?Boyer's model of scholarship

1. The scholarship of discovery that includes
   original research that advances knowledge.
2. The scholarship of integration -synthesis of
   information across disciplines, across topics
   within a discipline, or across time.
3. The scholarship of application that goes beyond
   the service duties of a faculty
4. The scholarship of teaching and learning - the
   systematic study of teaching and learning
   processes

Boyer E. (1990). Scholarship Reconsidered
priorities of the professoriate. The Carnegie
Foundation for the Advancement of Teaching, New
York, NY.
15
Needs and opportunities
16
Activity 1 needs and opportunitiesIdentify at
least 1 need you have for medical education
researchIdentify at least 1 opportunity you
have to conduct medical education research
17
Getting started
18
What is done?

• Interventions efficacy and effectiveness
• Assessment validation, reliability,
  psychometrics
• Psychosocial dimensions
• Social dimensions
• Cultural dimensions
• Education systems
• Systematic reviews BEME, CEMESTR

19
How is it done?

• Intervention vs no intervention -
• Intervention 1 vs intervention 2
• Pre-post
• Psychometrics
• Performance
• Kirkpatrick

20
Kirkpatrick

• 1 Did the learners like the learning process?
• 2 What did they learn? How much did they learn?
• 3 What performance changes resulted from the
  learning process?
• 4 How has the learning process reduced cost,
  improved quality, increased efficiency etc?

After Kirkpatrick DL (1994). Evaluating Training
Programs. Berrett-Koehler Publishers, Inc.
21
Kirkpatrick in MER

• Care-focused impact
• K1 Participation
• K2a Modification of attitudes / perceptions
• K2b Modification of knowledge / skills
• K3 Behavioural change
• K4a Change in organizational practice
• K4b Benefits to patient / clients

Tochel C, Haig A, Hesketh A, Cadzow A, Beggs K,
Colthart I and Peacock H. (1999) The
Effectiveness of Portfolios for Post-Graduate
Assessment and Education BEME Guide No 12.
Medical Teacher, 31(4)299-318.
22
Kirkpatrick

• Education-focused impact
• K1 Participants views
• K2a Change in attitudes
• K2b Modification of knowledge or skills
• K3 Transfer of learning to the workplace
• K4a wider changes in the organization,
  attributable to the educational program
• K4b improvement in student or resident
  learning/performance

Steinert Y, Mann K, Centeno A, Dolmans D, Spencer
J, Gelula M and Prideaux D. (2006) A systematic
review of faculty development initiatives
designed to improve teaching effectiveness in
medical education BEME Guide No 8. Medical
Teacher, 28(6)497-526.
23
Question development

• FINER criteria for a good research question
• F Feasible adequate number of subjects,
  adequate technical expertise, affordable in time
  and money, manageable in scope
• I Interesting getting the answer intrigues
  investigator, peers and community
• N Novel confirms, refutes or extends previous
  findings
• E Ethical amenable to a study that institutional
  research ethics board will approve
• R Relevant to scientific knowledge, to clinical
  and health policy, to future research

Farrugia P, Petrisor BA, Farrokhyar Fand Bhandari
M. Research questions, hypotheses and objectives.
Can J Surg. 2010 August 53(4) 278281
24
Question format

• PICOT criteria
• P Population what specific learner/teacher
  population are you interested in?
• I Intervention (for intervention studies
  only) What is your investigational intervention?
• C Comparison group what is the main alternative
  to compare with the intervention?
• O Outcome of interest what do you intend to
  accomplish, measure, improve or affect?
• T Time what is the appropriate follow-up time to
  assess outcome

25
Activity 2 ask a good questionTake your need
or opportunity from the first activity and create
a good research question to address it
26
Designing a study
27
The process of empiricism

• Experience
• Classification
• Quantification
• Discovery of relationships
• Approximation to the truth
• the ultimate arbiter is not faith or utility or
  logic, or even truth, but the empirical world
  itself Gorham, p52

Gorham, G (2009) The Philosophy of Science A
Beginner's Guide. Oneworld.
28
Progression of enquiry

• Observe the world
• Identify possible phenomena
• Describe in increasing breadth and depth
• Identify relationships and differences within the
  phenomena and with other phenomena
• Build models and develop theories and test them
• Refine models and theories to become predictive
  and to account for the phenomena
• doesnt necessarily start with numbers

29
Methods

• Quantitative proofs, measurements
• Experimental pre-post, trials, parallel
• Epidemiologic RCTs, cohorts, randomization
• Psychometric measuring individual knowledge,
  abilities, study reliability, validity
• Correlational questionnaires, surveys
• Individuals vs groups
• Efficacy vs effectiveness

Norman and Eva (2010) Quantitative Rsearch
Methods in medical education. In Swanwick (Ed)
Understanding Medical Education Evidence, Theory
and Practice. Wiley.
30
Methods

• Qualitative
• Themes
• Motives
• Attitudes
• Behaviours
• Beliefs
• Cultures
• Ethics
• Phenomenology

31
Methods

• Mixed methods
• QuanQuan, QualQual, QuanQual
• Different methods different questions/answers
• Holism and triangulation
• Sequences
• from what to why survey gt interview
• from why to what interview gt expt.
• Lenses, theories and more

32
Study design

• Design the study
• Create a protocol
• Specific questions and rationale
• Methods and participants follow questions (and
  reality)
• Ethical limitations to educational research
• REB forms can help you design a protocol

33
Ethical review

• Research Ethics Boards (REBs)
• Not just about getting published
• TCPS2 research involving human subjects
• including autoethnography and secondary data
• Lakehead Laurentian others
• Using the REB process to your advantage

34
Activity 3 sketch a studyTake your research
question and sketch out a study to address it
35
Where next?
36
Resources?

• Monthly NOSM medical education journal club
  contact jdepatie_at_nosm.ca
• Conferences AMEE, AAMC (Generalists, RIME), CCME
• Calls for funding
• Research Support Group
• Collaborations lets talk