An Interactionist Approach to Student Success: When Conscientiousness Matters

1
An Interactionist Approach to Student Success
When Conscientiousness Matters Joan R. Poulsen,
Deborah A. Kashy, Gerd Kortemeyer Department of
Psychology - Michigan State University
www.lon-capa.org
Abstract One lesson learned from the
person-situation debate is that accurately
predicting outcomes involves the interaction
between personality characteristics and aspects
of the situation. This study examines how the
personality trait conscientiousness predicts
academic outcomes across a range of course
content domains, and estimates the degree to
which situational differences in the course
structure moderate the conscientiousness-course
success relationship. The courses varied in their
content domains (e.g., psychology, physics,
chemistry), but they all shared a common
internet-based homework and course management
system LON-CAPA. We examine how
conscientiousness interacts with a variety of
course requirements in predicting student
performance. Course requirements examined include
frequency of homework assignments, percentage of
grade that homework contributes, the number of
resubmissions (i.e., additional attempts after an
incorrect answer is submitted) students were
given for typical homework problems.
Figure 1. The frequency of homework assignments
moderates the effect of conscientiousness on
course grade.

• Many studies have examined the role of
  conscientiousness in academic domains. To
  summarize, people who are higher on
  conscientiousness, tend to
• Be more likely to complete more years of
  schooling (Barrick Mount, 1991 McKenzie Gow,
  2004)
• Have higher motivation to perform well (Judge
  Ilies, 2002)
• Have a positive attitude about school (Heaven et
  al., 2002)
• Have higher self-reported and actual performance
  at school (Heaven et al., 2002 Paunonen
  Ashton, 2001)
• Have the tendency to perform more behaviors
  (e.g., turn in assignments on time) leading to
  academic success (De Raad Schouwenburg, 1996).
• However, personality is more predictive in some
  situations than others. Thus, there may be some
  academic situations in which conscientiousness
  helps a student more than others. The situation
  we examine here is how instructors implemented
  the LON-CAPA course management and assignment
  system in their classes.

• Materials
• Characteristics of the Students
• School Conscientiousness Level of
  conscientiousness in behaviors in the school
  domain was measured using a modified version of
  the IPIP scale (alpha .84).
• Grade in course The final grade in the course
  was obtained from the registrars records on a
  0.0 4.0 scale.
• Percent homework solved This information was
  downloaded from the LON-CAPA system and
  calculated as the number of problems the student
  solved correctly divided by the number of
  problems assigned.
• GPA Students cumulative grade point average was
  obtained from the registrars records at the end
  of the term.

Figure 2. The amount homework contributes to the
course grade moderates the relationship between
conscientiousness and course grade.

• Characteristics of the Courses
• Percent grade that homework counted We examined
  the syllabi from each course to determine the
  percent of the total grade earned from LON-CAPA
  homework.
• Frequency of assignments Using course syllabi,
  we assessed how frequently homework assignments
  were given (e.g., weekly, bi-weekly, etc.).
• Number of students in the class The number of
  students enrolled in the class at the end of the
  semester according to the registrars records.
• Maximum tries The average number of submissions
  allowed per problem by the instructor over a
  course. This information was downloaded from the
  LON-CAPA system, except in cases of unusual
  outliers, in which case information from the
  syllabi were used to determine this value.

• What is LON-CAPA?
• The LearningOnline Network with a Computer
  Assisted Personalized Approach
• LON-CAPA is an integrated system for online
  learning and assessment.
• It consists of
• a learning content authoring and management
  system allowing new and existing content to be
  used flexibly
• a course management system
• an individualized homework and automatic grading
  system
• data collection and data mining system
• a content delivery system that will provide
  gateways to and from NSF's National STEM Digital
  Library

• One reason conscientiousness is interesting to
  examine in terms of internet-based academic
  behaviors is the concern that distance education
  and online components of classes are too
  unstructured to promote success for many students
  (Wilkinson Sherman, 1990 Hiltz, 1994 Guernsey
  Young, 1998). With the growing popularity of
  the use of internet technology in education, it
  is useful to understand how the structure of a
  course can impact student performance, and what
  role conscientiousness plays in this.
• Given this previous work, students who are higher
  on conscientiousness should generally have better
  grades than students lower in conscientiousness.
  Also, when courses are structured more loosely
  (e.g., students are less accountable for their
  progress), students higher in conscientiousness
  should perform better than those who are lower on
  conscientiousness.
• In this study, we investigated the following
  questions
• 1. Does conscientiousness predict student
  success?
• 2. Does the structure of the online component of
  a class influence student success?
• 3. Do students who are more conscientious tend to
  do better (or worse) if courses are structured in
  certain ways?
• 4. How do student-level conscientiousness ratings
  and course-level differences in class structure
  interact to predict student performance?
• Method
• Participants
• 3394 undergraduate students (51 women 85
  Caucasian, 5 African-American, 7 Asian
  American) enrolled in one of 34 classes at
  Michigan State University that were using the
  LON-CAPA system in either Fall semester of 2003
  or Spring semester of 2004.
• Procedure
• These data were collected as part of a larger
  study assessing the educational effectiveness of
  the LON-CAPA system and internet technology.
  Students in the classes that used LON-CAPA were
  surveyed at the start and end of the semester. We
  assessed their level of conscientiousness,
  attitudes about the LON-CAPA course management
  system. We downloaded information from the
  LON-CAPA system about students online behaviors.
  We also obtained information from the registrar
  and instructors about students grades and GPA,
  as well as information from syllabi about class
  policies, grading procedures, and how the
  instructor used the software in his or her class.

• Similarly, the relationship between
  conscientiousness and the percent of homework
  students solved was moderated by three factors
• When homework counted less towards the course
  grade, students higher on conscientiousness did
  better on the homework.
• When the class size was larger,
  conscientiousness was more strongly related to
  the amount of homework students completed.
• When students were given more tries to get
  problems correct, conscientiousness was stronger
  predictor of how much homework students got
  correct.
• The overall pattern suggests that when a class is
  structured with fewer external motivational
  factors, conscientiousness plays a larger role in
  predicting student success.

• The LON-CAPA software provides instructors with a
  common, scalable platform to assist in all
  aspects of teaching a course, from lecture
  preparation to administration of homework
  assignments and exams.
• It provides a sophisticated assignment engine
  that can create unique homework assignments and
  exams for each student in a class. Its formative
  and summative assessment tools grade a broad
  variety of objective problems and assist in
  evaluation of essays.
• It provides prompt feedback for students and
  instructors, as well as statistical information
  on performance and effectiveness of materials.
  Discussion pages attached to every homework
  assignment encourage communication among students
  and faculty.
• The LON-CAPA software is freely available and
  free (GNU General Public License), and may be
  modified and adapted.

• Results
• Sample characteristics
• Students had a mean conscientiousness rating of
  5.13 (SD 0.88) on a seven-point scale, and an
  average cumulative GPA of 3.13 (four-point
  system) (SD .57).
• Conscientiousness predicts student success
• Regression analyses were used to analyze the
  degree to which school conscientiousness
  predicted outcomes for students. We controlled
  for students overall GPA and found that higher
  levels of school conscientiousness predicted
• Higher course grades (b .06 )
• Higher percent of homework solved (b
  2.039)
• Fewer guesses on the homework (b -.884)
• Moderators of the relationship between
  Conscientiousness and success
• Multi-level regression was used to analyze how
  different course-level characteristics (Level 2)
  moderated the effects of conscientiousness (Level
  1) on student outcomes.
• The association between conscientiousness and
  course grade is moderated by two main factors of
  interest
• Frequency homework was due Conscientiousness
  had a stronger effect on course grade when
  homework was less frequently assigned. (Figure 1)
• Percent of grade homework counted
  Conscientiousness had a stronger effect on course
  grade when homework counted less towards the
  course grade. (Figure 2)

• Conclusions
• The present research suggests that
• Students higher on conscientiousness tend to
  perform better in university courses.
• Students high on conscientiousness perform
  behaviors that improve their grade (they complete
  more homework, guess less).
• But, conscientiousness may be more predictive
  of student success in courses that are less
  rigidly structured.
• When students have less external motivation for
  doing homework, conscientiousness is more
  important for success.
• Also, when students are less accountable (e.g.,
  higher class size, homework is optional), those
  higher on conscientiousness may have the
  advantage.
• Although conscientiousness predicts student
  success in college-level courses, the way in
  which courses are structured influences the
  strength of this association.

• LON-CAPA was developed by an enthusiastic group
  of faculty and professionals who recognize that
  Information Technology can play an important role
  in improving students learning and
  understanding.
• Current members of the project include faculty
  from universities, colleges, and K-12 schools in
  the US, Canada, Asia, Africa, and Europe. The
  core development group is located at Michigan
  State University.
• Today LON-CAPA is serving over 16,000 course
  enrollments per semester at MSU alone, and well
  over 23,000 course enrollments system-wide,
  ranging from middle school to graduate level
  courses.
• Disciplines include astronomy, biology, business,
  chemistry, civil engineering, computer science,
  family and child ecology, geology, human food and
  nutrition, human medicine ,mathematics, medical
  technology, physics, and psychology.
• The Effects of Conscientiousness in Educational
  Settings
• Previous work on conscientiousness suggests that
  it positively predicts many performance-related
  outcomes, such as job performance, and academic
  behavior.