Though this be madness, there is a method in it: Using methodological exemplars to improve pedagogical research - PowerPoint PPT Presentation

Loading...

PPT – Though this be madness, there is a method in it: Using methodological exemplars to improve pedagogical research PowerPoint presentation | free to download - id: 69bd81-ZDZjY



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Though this be madness, there is a method in it: Using methodological exemplars to improve pedagogical research

Description:

Though this be madness, there is a method in it: Using methodological exemplars to improve pedagogical research Andrew N. Christopher Albion College – PowerPoint PPT presentation

Number of Views:1
Avg rating:3.0/5.0
Date added: 13 February 2020
Slides: 60
Provided by: Jordan122
Learn more at: http://teachpsych.org
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Though this be madness, there is a method in it: Using methodological exemplars to improve pedagogical research


1
Though this be madness, there is a method in
it Using methodological exemplars to improve
pedagogical research
  • Andrew N. Christopher
  • Albion College

Jordan D. Troisi Widener University
Presented at the 12th Annual Society for the
Teaching of Psychology Best Practices
Conference October 11, 2013
2
Though this be madness, there is a method in it
  • A brief summary of the beginning of Hamlet
  • Hamlet is visited by his fathers ghost (The
    King)
  • The ghost says that his own brother (Claudius)
    has killed him
  • 1-2 months later, Claudius has married Gertrude,
    Hamlets mother
  • The ghost says kill Claudius
  • Hamlet says sure, and that hes going to act
    mad for awhile, presumably to cover his tracks
  • Perhaps there is a method in it?

3
Though this be madness, there is a method in it
  • Hamlet is acting(?) mad, and a bunch of
    characters develop hypotheses as to why
  • Gertrude hypothesizes hes mad because his father
    has died and she married his brother in 1-2
    months
  • Claudius hypothesizes hes acting mad because he
    wants to steal the throne he has recently assumed
  • Polonius hypothesizes hes mad in love with his
    daughter Ophelia and that he just wants to sleep
    with her
  • All valid hypotheses, which they each test
    (poorly)
  • But they ignore a crucial confound. . .
  • A freaking ghost is telling Hamlet what to do!

? FAIR
? FAIR
? FAIR
4
Guidance in pedagogical research from Hamlet
  • Theres a lot of madness out there, and its
    often hard to understand
  • Luckily, psychologists are trained to determine
    the reasons why behavior occurs
  • Also luckily, such behavior is rarely guided by
    ghosts
  • What happens in the classroom may seem like
    madness, but theres a method in it
  • We need the methodological tools to make sense of
    the madness

5
Overview
  • Scholarly teaching, the scholarship of teaching
    and learning (SoTL), and the learning sciences
  • Issues of control in pedagogical research
  • Effective methodology exemplars from SoTL and the
    learning sciences
  • Take-home messages for effective pedagogical
    research

6
SoTL Learning Sciences Daniel Chew (in press)
SoTL
Learning Sciences
  • Extension of higher education
  • Evaluates specific pedagogical methods
  • Application-driven
  • Constructs may lack theoretical precision
  • Extension of basic research in psychology
  • Findings may not translate across teaching
    contexts
  • Theory-driven
  • Construct operationalizations tend to be
    widely-accepted

7
SoTL Learning Sciences Daniel Chew (in press)
SoTL
Learning Sciences
  • Predicting the weather
  • Knowing the laws of thermodynamics

8
Research on Note-Taking
  • Williams and Eggert (2002) examined the notes of
    125 students in an undergraduate human
    development course
  • Students took notes in a 125-page study guide on
    both their
  • Class notes
  • Textbook readings

9
Research on Note-Taking
  • Three blind raters coded these notes for
  • Completeness
  • Length
  • Accuracy

10
Research on Note-Taking
  • Completeness, length, and accuracy used to
    predict
  • Brief essay quizzes linked to the notes
  • Exam questions from text only
  • Exam questions from lecture only
  • Exam questions from both text and lecture

11
Research on Note-Taking
  • Peverly et al. (2013) randomly assigned 200
    undergraduates in an educational psychology
    course to either
  • Outline provided
  • No outline provided
  • while taking notes on a 23 min video-taped
    lecture

12
Research on Note-Taking
  • Predictors
  • Handwriting speed
  • Wrote letters of alphabet horizontally in orders
    as many times as possible in 45 sec
  • Language comprehension
  • 7 reading passages with questions about each
  • Verbal working memory
  • Listening span test
  • Attention

13
Research on Note-Taking
  • Dependent Variables
  • Quality of notes
  • Written summary of the lecture

14
Research on Note-Taking
  • Used structural equation modeling to assess
    predictors relationships with quality of notes
    and written summary of the lecture

15
Overview
  • Scholarly teaching, the scholarship of teaching
    and learning (SoTL), and the learning sciences
  • Issues of control in pedagogical research
  • Effective methodology exemplars from SoTL and the
    learning sciences
  • Take-home messages for effective pedagogical
    research

16
SoTL The Early Days
  • Focused on student attitudes (i.e., Did students
    like it?)
  • e.g., Christopher and colleagues (2004)
  • Wesp and Meile (2008, p. 362)
  • It appears that student opinions about the
    effectiveness of teaching techniques are
    inaccurate.
  • Researchers should prefer direct measures (of
    student learning) because they provide a more
    accurate assessment of pedagogical
    effectiveness.

17
Learning Might not Equal Liking
  • Attitudes about courses and topics may not equate
    to learning, and vice versa
  • After a course on research and statistics,
    students reported increases in knowledge but no
    changes in favorable attitudes toward the subject
    (Sizemore Lewandowski, 2009)

18
SoTL The Early Days A Vicious Cycle
  • Psychologists are control freaks
  • Correlation does not equal causation
  • equates to
  • Correlational research is not as valuable as
    experimental research

19
SoTL Why is it so hard to do?
  • Special challenges in doing SoTL
  • How often you teach a particular class
  • Number of students in the class
  • Time of day a class (or sections of a class) is
    offered
  • Lack of control relative to lab-based research

20
Overcoming the Control Issue in SoTL
  • As psychologists, we love control
  • As teachers, we want to do whatever we can to
    maximize student learning and sometimes produce
    attitude change
  • But, classrooms are very noisy places

21
Finding Appropriate Comparisons
  • Smith (2008)
  • Use a previous class
  • Compare and (ideally) statistically control for
    ACT/SAT scores
  • Use an in-tact class
  • Divide the class into 2 equal groups
  • Group 1 attends the first half of class Group 2
    attends the second half of class
  • Use your intervention on one of the 2 groups
  • Measure outcome variable on the same day

22
More Methods to Use Within 1 Class
  • Bartsch et al. (2008)
  • Typical pre/post contains same questions
  • Using different tests at pre and post
    instrumentation confound
  • Focus on using a 1-group pre/post with alternate
    forms

23
1-Group Pre/Post with Alternate Forms
  • Prepare 2 versions of an assessment instrument
    (i.e., Version A Version B)
  • Give half the students Version A at pretest and
    Version B at posttest
  • Switch for the other half of the students

24
1-Group Pre/Post with Alternate Forms
  • 2 (Version A first or Version B first) x 2 (pre
    and post) mixed ANOVA
  • Allows us to learn if
  • Groups were equivalent to start with
  • Version A was equivalent to Version B
  • The intervention does or does not facilitate
    student learning

25
1-Group Pre/Post with Alternate Forms
  • Use random assignment to
  • Form your 2 groups of students
  • Put questions on the 2 versions of the assessment
    instrument

26
Overview
  • Scholarly teaching, the scholarship of teaching
    and learning (SoTL), and the learning sciences
  • Issues of control in pedagogical research
  • Effective methodology exemplars from SoTL and the
    learning sciences
  • Take-home messages for effective pedagogical
    research

27
Rapport, Student Motivation, and Course Attitudes
  • Wed all like good rapport, motivated students,
    and favorable attitudes toward our courses
  • Legg and Wilson (2009) examined how sending a
    welcome email to students before a course began
    could create these outcomes
  • Building rapport before the first day of class!

28
Rapport, Student Motivation, and Course Attitudes
  • Half of an Introduction to Psychology courses
    students (n 66) received a welcome email one
    week prior to the beginning of the semester
  • Randomly assigned, instructor blind to
    participant condition
  • Surveyed rapport, student motivation, and course
    attitudes at
  • End of 1st class period
  • At mid-term
  • After the final exam

29
Rapport, Student Motivation, and Course Attitudes
  • Most favorable ratings for email group during the
    first class period
  • Fewer effects for subsequent measurement periods
  • Some interactions with gender of student
  • Students in the email group were less likely to
    drop the course

30
Kudos to Legg and Wilson (2009)!
  • A welcome email is a low-tech, time-efficient way
    to contact students and start building rapport
    and motivation
  • Authors randomly assigned students within one
    class and were blind to which students received
    the email
  • Measured multiple, relevant DVs, at 3 time periods

31
Student Attitude Change in a Prejudice Course
  • Reducing prejudice is often a goal of psychology
    courses
  • Reducing prejudice should be assessed over time,
    as done superbly by Kernahan and Davis (2010)
  • The utility of the classroom for ideas in basic
    research

32
Student Attitude Change in a Prejudice Course
  • Compared a Psychology of Prejudice and Racism
    course a Statistics course
  • Utilized a pretest-posttest design with a control
    group, plus a one-year follow-up
  • Design allows for examination and comparison of
    attitude changes from beginning to end of
    semester, and beyond

33
Student Attitude Change in a Prejudice Course
  • Results showed that by the end of the semester,
    those in the Prejudice and Racism course showed
    greater
  • Awareness of white privilege, white guilt,
    noticing of racism, and responsibility for taking
    action
  • At one year follow-up, some effects
  • Plateaued (e.g., awareness of white privilege)
  • Waned (e.g., responsibility for taking action)
  • Increased (e.g., comfortability in mixed-race
    interactions)

34
Kudos to Kernahan and Davis (2010)!
  • Comparison between a group with an anticipated
    effect and a control group
  • Effects examined across 3 time periods, including
    one year later
  • High external validity, and the target outcome of
    interest for basic and pedagogical research!

35
Learning or Living-Learning Communities
  • Designed to create a coherent educational
    experience for a group of students centered on a
    particular topic
  • Can improve student performance and retention
    while fostering relationships
  • Buch and Spaulding (2008) spectacularly examined
    these important outcomes utilizing a longitudinal
    design with a matched comparison group

36
Learning or Living-Learning Communities
  • Compared GPA, retention, co-curricular
    involvement in psychology courses/activities over
    7 semesters of college
  • Utilized a longitudinal design with a control
    course with students matched (n total 40) on
    important variables (SAT score, gender,
    ethnicity, generation status)

37
Learning or Living-Learning Communities
  • Cumulative GPA better in the Learning Community
    group during the 1st, 2nd, and 4th semester of
    college
  • Retention at the school and progress within the
    major were better among Learning Community group
    (some significant findings, some non-significant
    but in the expected direction)
  • Co-curricular involvement higher in the Learning
    Community group
  • Involvement in psychology club, internship,
    research

38
Kudos to Buch and Spaulding (2008)!
  • Longitudinal design (see also Buch Spaulding,
    2011), with a comparison group matched on
    important variables
  • On a hot topic in higher education
  • 7
  • Semesters
  • Of
  • Data
  • !

39
Laptop Multitasking and Performance
  • Multitasking divides attention and can lead to
    poor memory of course material and poor
    performance
  • Laptops in the classroom can provide distractions
    for the users and others in view of laptop
    screens
  • Sana and colleagues (2013) put distractions from
    laptop use to the test (literally!)

40
Laptop Multitasking and Performance
  • Two studies on how well students retained
    information during lectures for a multiple choice
    quiz
  • One study examined student performance when they
    were randomly assigned to multitask during the
    lecture (given some online tasks to perform)
  • A second study examined student performance when
    they were in view of confederates laptop screens
    on which they were multitasking

41
Laptop Multitasking and Performance
  • In the first study, students assigned to
    multitask performed worse on a quiz of the
    information of the lecture (with no difference
    between simple fact-based items or application
    items)
  • In the second study, students who were in view of
    a research confederate who was multitasking
    performed worse on the quiz

42
Kudos to Sana and Colleagues (2013)!
  • Examining a topic extremely relevant to teaching
    and learning
  • Including two studies, one on distraction from
    the self and one on distraction from others
  • Using an experimental design that simulated
    classroom experiences
  • High external validity and mundane realism

43
Time-of-Day Preference and Grade-Point-Average
(GPA)
  • Preckel et al. (2013) examined whether high
    school students (N 272) time-of-day preference
    was predictive of their GPAs in a variety of
    subjects
  • At the college level, we discuss the best times
    to offer classes from a students perspective

44
Time-of-Day Preference and Grade-Point-Average
(GPA)
  • Preckel et al. (2013) statistically controlled
    for variables suggested to be predictive of
    academic performance (e.g., conscientiousness,
    cognitive ability, achievement motivation,
    gender)
  • Included measures of other-report as well as
    self-report

45
Time-of-Day Preference and Grade-Point-Average
(GPA)
  • Entered time-of-day preference on the last step
    of three hierarchical linear regressions
  • Used three criterion
  • overall GPA
  • math-science GPA
  • language GPA

46
Time-of-Day Preference and Grade-Point-Average
(GPA)
  • Found that students with an evening preference
    had a lower overall GPA, lower math-science GPA,
    and language GPA than students with a morning
    preference

47
Time-of-Day Preference and Grade-Point-Average
(GPA)
  • Use of statistical control, a commonly-used
    technique in personality research
  • Use of multiple predictors, including self- and
    other-report data

48
Academic Dishonesty and Narcissism
  • Much has been made about the rise of narcissism
    in Western society during the past 2-3 decades
  • Teachers have lamented (likely since the
    beginning of time) academic dishonesty among
    students

49
Academic Dishonesty and Narcissism
  • Brunnell et al. (2011) randomly assigned 199
    Introductory Psychology students to one of two
    questionnaire conditions
  • Questions referred to the Self
  • Questions referred to the Typical Student on
    Campus

50
Academic Dishonesty and Narcissism
  • Questions referred to
  • Guilt experienced for cheating if
  • Exam was overly difficult
  • Classmates did not help them study
  • Friends pressured them to cheat
  • Prevalence of Academic Dishonesty
  • The number of times they (others) cheated on a
    test in the past 12 months
  • Predicted how many times the typical student on
    campus would cheat on a test

51
Academic Dishonesty and Narcissism
  • 40-item Narcissism Personality Inventory (Raskin
    Terry, 1988)
  • Contains 3 subscales
  • Power
  • Exhibitionism
  • Special Person

52
Academic Dishonesty and Narcissism
  • Low exhibitionists displayed less guilt when they
    cheated than did high exhibitionists no
    differences were found when judging others who
    cheated
  • High exhibitionists reported more academic
    dishonesty when judging themselves than others
    no differences were found when judging others

53
Academic Dishonesty and Narcissism
  • Within the self condition, mediational analyses
    revealed that the relationship between
    exhibitionism and reported academic dishonesty
    was fully mediated by guilt

54
Academic Dishonesty and Narcissism
  • Use of both experimental and nonexperimental
    measures
  • Analysis of narcissism measure at both the factor
    and subscale levels
  • Examined interactive effects of experimental
    condition and narcissism
  • Mediational analyses to hint at the underlying
    mechanism

55
Overview
  • Scholarly teaching, the scholarship of teaching
    and learning (SoTL), and the learning sciences
  • Issues of control in pedagogical research
  • Effective methodology exemplars from SoTL and the
    learning sciences
  • Take-home messages for effective pedagogical
    research

56
A happy medium?
  • SoTL and the learning sciences have somewhat
    different approaches to studying similar outcomes
  • Effective pedagogical research utilizes the
    strengths of both areas

57
Highlighted strengths of my research (as a
gracious author)
  • Strengths highlighted in reviews of my work
  • Novelty of topic and research design
  • Building in theory to support findings in an
    intuitive way
  • Meaningful, stringent control/comparison groups
  • Using established materials from basic and
    pedagogical research
  • Large-scale studies
  • Pressing, relevant, generalizable, and current
    topics
  • Clear, logical writing

58
Though this be madness, there is a method in it
  • Instead of ghosts of kings past, we have ghosts
    in the form of confounds and noise
  • But fear not! Though this may be madness, there
    is a method in it
  • We just need to use the right tools to separate
    the method from the madness
    (or the
    meaningful effects
    from the confounds and
    noise)

59
  • Thank you for your attention! Questions please!
About PowerShow.com