Title: Predicting Student Success in Online Courses: A New Measure

1
Title Predicting Student Success in Online
Courses A New Measure

• Dr. Marcel S. Kerr Dr. Kimberly Rynearson
• Tarleton State University Central Texas
• Mr. Marcus C. Kerr
• University of Texas - Brownsville

2
Introduction

• Currently there exist several self-report
  measures of learning styles (Honey Mumford,
  1992 Kolb, 1985 Felder Silverman, 1988
  Rundle Dunn, 1999 Schmeck, 1983).
• Scores on these measures have shown to be related
  to a variety of student outcomes.
• Preliminary research indicates that the learning
  styles measured with these instruments may not
  translate into success for the online student.

3
Introduction

• Perhaps a different set of learning styles and/or
  student characteristics are more important for
  online learning.
• The researchers propose three studies to answer
  this question.
• This session focuses on the results of Study I.
• The goal of Study I was to develop the initial
  measure and determine its construct validity.

4
Internet Search

• Fifty institutions were sampled randomly to
  determine whether they offered distance learning
  courses/programs.
• Thirty (60) of those sampled had distance
  learning courses/programs and an online
  self-report student assessment.
• The Institutions consisted of private/public
  four-year universities, community colleges,
  technical schools, public education districts,
  and online-only degree granting entities.

5
Internet Search

• The majority of the institutions (43) used Is
  Online Learning for Me? by Prentice Hall
  Publishing Company (2000).
• Across the surveys, a number of issues were
  assessed.
• Issues included computer literacy, technology
  usage, communication skills, readiness,
  persistence, self-efficacy, learning styles,
  lifestyle and other student characteristics.

6
Qualitative Analysis

• The 30 measures were comprised of 428 individual
  items.
• Sixty-eight (16) items were unique.
• Therefore, the remaining 360 (84) items appeared
  on two or more assessments.
• This outcome suggests that the majority of
  institutions offering distance education options
  overlap in what they consider important
  characteristics for online success.

7
Test Development

• The 50 most frequent items were retained to
  create the initial version of the TOOLS.
• Items were worded to reflect a behavior as
  opposed to an attitude or perception.
• This approach allows the researchers to identify
  students behavioral strengths and weaknesses
  regarding online performance.

8
Initial TOOLS

• The initial measure consisted of fifty items.
• Items were grouped into Six categories
• Computer Skills
• Time Management
• Motivation
• Academic Skills
• Need for Online Delivery
• Learning Skills

9
Literature Search

• A literature review identified predictors of
  college student success.
• Five variables appeared in the research
  literature and the online assessment analysis.
• These variables included learning style,
  self-esteem, reading comprehension, intrinsic
  motivation, and locus of control.
• These variables were selected to help determine
  the construct validity of the TOOLS .

10
Hypotheses

• Students learning styles would be related to
  learning success.
• Self-esteem would be positively related to
  learning success.
• Comprehension strategy use would be positively
  related to learning success.
• Intrinsic motivation would be directly related to
  learning success.
• Internal locus of control would be directly
  related to learning success.

11
Hypotheses

• Gender was examined in relation to these
  variables as well.
• It was expected that more females than males
  would emerge as verbal learners and more males
  than females would emerge as visual learners.
• Due to the developmental nature of intrinsic
  motivation, it was predicted that age would be
  directly related to intrinsic motivation.

12
TOOLS Revision Criteria

• Items that did not meet a factor loading of .35
  were removed,
• the number of factors retained was based upon an
  Eigenvalue criterion of one, and
• a natural break in the curvilinear function on
  the scree plot (Bell, 1996 Tabachnick Fidell,
  2001).

13
Method

• Participants
• Procedures
• Materials

14
Participants

• 188 volunteer students (126 undergraduate 62
  graduate) from TSU
• The sample (38 males and 150 females) mean age
  was 31 years.
• The sample was ethnically diverse.
• 93 reported having Internet access at home.

15
Procedures

• Graduate and undergraduate students were
  solicited for participation by their instructors.
• 62 participants completed the battery of six
  self-report surveys online.
• 122 participants completed the surveys in a
  traditional paper and pencil fashion.
• Survey instructions and the order in which the
  items were presented were the same for both
  delivery formats.

16
Materials

• Test of Online Learning Success (2002)
• Rosenberg Self-Esteem Scale (1965)
• Index of Learning Styles (1999)
• Soloman Felder
• Reading Comprehension Strategies Questionnaire
  (2000)
• Taraban, Rynearson, Kerr
• Academic Intrinsic Motivation Questionnaire
  (1998)
• Shia
• Trice Academic Locus of Control Scale (1985)

17
Results

• TOOLS Scale Reduction
• TOOLS Internal Consistency
• TOOLS and Predictors of Success
• Gender Differences in Learning

18
Results TOOLS Scale Reduction

• Principle Component Analyses (PCA) were computed
  on the initial 50-item TOOLS measure
• Regardless of rotation and extraction methods
  used, similar factors and factor loadings
  emerged.
• The same items loaded on the same factors across
  all analyses. The consistency of these outcomes
  suggests a stable factor structure (Cody Smith,
  1997 Tabachnick and Fidell, 2001).

19
Results TOOLS Scale Reduction

• Using Eigenvalue of one, 13 factors emerged.
• Scree plot revealed an inflection point at the
  fifth factor.
• Five factors were retained.
• Computer Skills
• Independent Learning
• Need for Online Instruction
• Academic Skills (Reading Writing)
• Dependent Learning
• Five items did not meet the .35 factor loading
  criterion and were removed.

20
Results TOOLS Internal Consistency

• The initial measure yielded a coefficient alpha
  of .87 (n 183).
• The revised measures coefficient alpha did not
  decrease (alpha .87, n 183).
• The revisions to TOOLS subscales resulted in
  increased internal consistency.
• Initial 6 subscales coefficient alphas ranged
  from .55 to .88 (n 188).
• Revised 5 subscales coefficient alphas ranged
  from .72 to .89 (n 188).

21
Results TOOLS and Predictors of Success

• Pearson product-moment correlations were
  computed.
• The five specific hypotheses were partially
  supported.
• Self-esteem was positively related to learning
  success, r(188) .35, p .01
• Comprehension strategy use was positively related
  to learning success, r(188) .28, p .01.
• Intrinsic motivation and locus of control were
  not significantly related to learning success,
  but their relationships were in the right
  direction.
• Learning styles was not systematically related to
  learning success.

22
Results Additional Hypotheses

• The hypothesis regarding age and intrinsic
  motivation was also supported, as a small
  significant correlation was found, r(188) -.17,
  p .01.
• To determine gender differences in learning
  styles, t-tests were computed.
• Significantly more females (M 4.52, SD 2.35)
  than males (M 3.42, SD 2.26) reported being a
  verbal learner, t(178) 2.54, p .02.
• Significantly more males (M 7.58, SD 2.50)
  than females (M 6.63, SD 2.46) reported being
  a visual learner, t(186) 2.11, p .04.

23
Discussion

• The goal of establishing construct validity of
  the new measure was accomplished
• Five-factor stable structure
• High internal consistency
• The majority of the relationships with the
  convergent validity variables were supported

24
Discussion

• Limitations
• Self-report data
• Used poor measure of internal motivation
• Participants attended same university
• Directions for Future Research
• Field will benefit from studies using
  experimental controls
• Revised TOOLS should be used with different
  student populations

25
Studies 2 and 3

• The goals of Study 2 is to establish the TOOLS
  criterion validity and further revise the items.
• The goals of Study 3 is to establish the TOOLS
  predictive ability and determine its reliability
  over time.

26
Contact Information

• To learn more about TOOLS as it develops, or if
  you desire to obtain a copy of the current
  measure, contact one of the following
• Dr. Marcel S. Kerr
• kerr_at_tarleton.edu
• Dr. Kimberly Rynearson
• rynearson_at_tarleton.edu