Technology-assisted learning: a longitudinal field study of knowledge category, learning effectiveness and satisfaction in language learning - PowerPoint PPT Presentation

Loading...

PPT – Technology-assisted learning: a longitudinal field study of knowledge category, learning effectiveness and satisfaction in language learning PowerPoint presentation | free to download - id: 7cdcdd-NzJlM



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

Technology-assisted learning: a longitudinal field study of knowledge category, learning effectiveness and satisfaction in language learning

Description:

Technology-assisted learning: a longitudinal field study of knowledge category, learning effectiveness and satisfaction in language learning W. Hui,* P.J.-H. Hu, T ... – PowerPoint PPT presentation

Number of Views:76
Avg rating:3.0/5.0

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Technology-assisted learning: a longitudinal field study of knowledge category, learning effectiveness and satisfaction in language learning


1
Technology-assisted learning a
longitudinalfield study of knowledge category,
learningeffectiveness and satisfaction
inlanguage learning
  • W. Hui, P.J.-H. Hu, T.H.K. Clark, K.Y. Tam
    J. Milton
  • College of Information Technology, Zayed
    University, Abu Dhabi, United Arab Emirates
  • Accounting and Information Systems, David Eccles
    School of Business, University of Utah, Salt Lake
    City, Utah, USA
  • Information and Systems Management, School of
    Business and Management, Hong Kong University of
    Science and Technology, ClearWater Bay, Hong
    Kong, China
  • Language Center, School of Humanities, Hong Kong
    University of Science and Technology, ClearWater
    Bay, Hong Kong, China
  • Journal of Computer Assisted Learning, Vol. 24,
    245-259, 2007

2
Introduction
  • Some researchers, including Zhang et al. (2004),
    suggest technology-assisted learning can
    substitute for some conventional, face-to-face,
    classroom-based learning.
  • In this context, an instructor can deliver course
    materials through a designated Web site, from
    which students access materials and interact with
    the instructor (and perhaps peers) remotely.
  • A competing view holds that technology-assisted
    learning should be used only to complement
    face-to-face learning, which demands a hybrid or
    blended approach to leverage the respective
    strengths of each type of learning, such as using
    technology-assisted learning in some areas rather
    than replacing face-to-face, classroom-based
    learning altogether.

3
Introduction
  • According to Masie (2002) and Frederickson et al.
    (2004), this hybrid approach represents a
    preferable and arguably more advantageous means
    of using technology-assisted learning.
  • However, with either approach, a fundamental
    question Does the use of technology-assisted
    learning improve students learning effectiveness
    and satisfaction?
  • With this background, we propose a factor model
    that explains students learning satisfaction and
    empirically test the model using evaluative
    responses collected from a longitudinal field
    experiment.
  • We focus on students learning of English as a
    foreign language, which typically spans different
    aspects of language learning, including
    vocabulary, grammar, listening, speaking and
    reading.

4
Introduction
  • Therefore, we examine the following research
    questions (1) Is learning effectiveness
    associated with technology- assisted
    learning contingent on target knowledge? (2)
    What are the essential antecedents of learning
    satisfaction in technology-assisted
    learning?
  • Our longitudinal field experiment investigates
    the effects of technology-assisted learning by
    comparing students learning effectiveness across
    different important aspects of English learning
    with technology-assisted versus face-to-face
    learning.
  • Our first study group contains students who use
    face-to-face learning exclusively, whereas the
    second uses both technology-assisted and
    face-to-face learning (i.e. hybrid approach).

5
Introduction
  • Our study design thus supports an analysis of
    between-group differences with respect to the
    target knowledge category and the combined effect
    of target knowledge and learning medium.
  • We also use the collected data to test the
    proposed learning satisfaction model, which
    consists of essential satisfaction antecedents
    (i.e. perceived learning community support,
    course learnability, learning effectiveness) in
    technology-assisted learning.

6
Literature review
  • Previous research into learning effectiveness in
    technology-assisted learning
  • Several studies report positive effects of
    technology-assisted learning, including Johnson
    et al. (2000), who compare learning methods in
    human resource developments and show that
    students in the technology-assisted group
    perceive the instructor more positively and rate
    the overall course quality higher than their
    counterparts in the face-to-face group.
  • Abraham (2002) designs a virtual classroom for
    student learning about information systems and
    finds that technology-assisted learning improves
    learning feedback to students but the higher
    resulting learning effectiveness is not
    significantly better than that observed in
    face-to-face, classroom-based learning.

7
Literature review
  • Previous research into learning effectiveness in
    technology-assisted learning
  • A meta-analysis by Bernard et al. (2004) suggests
    that the impact of technology-assisted learning
    is not significant, consistent with Clarks
    (1983) contention that the delivery medium has
    only marginal effects on the outcomes of planned
    instruction, measured according to learning
    effectiveness or satisfaction.

8
Literature review
  • Previous research into learning satisfaction in
    technology-assisted learning
  • A review of extant literature on the critical
    topic of learning satisfaction (Allen et al.
    2002 Wang 2003) suggests limited investigations
    of the essential factors that affect learning
    satisfaction, even though such investigations are
    particularly important when considering the
    relatively high dropout rate associated with
    technology-assisted learning (Hiltz Wellman
    1997 Kumar et al. 2001).
  • Consistent with Keller (1983), we define learning
    satisfaction as the perception of being able to
    achieve success and positive feelings about
    achieved outcomes.

9
Literature review
  • Previous research into learning satisfaction in
    technology-assisted learning
  • Furthermore, on the basis of an extensive
    literature review, we identify three essential
    satisfaction determinants (1) Learning
    effectiveness (Keller 1983 Wang 2003), (2)
    Perceived course learnability (Roca et al. 2006)
    and(3) Perceived learning community support
    (Wang 2003 Liaw 2004 Chou Liu 2005).
  • According to Martin-Michiellot and Mendelsohn
    (2000), materials delivered in an easy-to-learn
    fashion can enhance students learning
    effectiveness and satisfaction.

10
Literature review
  • Previous research into learning satisfaction in
    technology-assisted learning
  • In this study, perceived course learnability
    refers to the degree to which a student considers
    the course materials delivered through
    technology-assisted or face-to-face learning easy
    to learn.
  • Consistent with Wang (2003), we define perceived
    learning community support as the extent to which
    a learning environment creates an active,
    strongly bonded community that encourages and
    facilitates knowledge exchanges among peers and
    their instructors.

11
Experiential learning model and implications for
language learning
  • The experiential learning model assumes an
    iterative nature of learning through experience,
    from reflection and conceptualization to action
    and then enhanced experience (Osland et al.
    2001).
  • According to this model, technology-assisted
    learning may be less effective for some aspects
    of language skills. For example, by engaging in
    live speaking drills or role plays, students can
    recognize their speaking problems directly and
    concretely (i.e. concrete experience).
  • Such iterative processing reinforces student
    learning, but technology-assisted learning
    provides only limited support in this sense.

12
Experiential learning model and implications for
language learning
  • However, technology-assisted learning may better
    support other aspects of language learning
    because of the convenient access it offers to
    learning materials pertinent to vocabulary,
    reading, or grammar, which students may study
    repetitively at their preferred time and pace.

13
Hypotheses and research model
  • We objectively measure students learning
    effectiveness using test scores on listening,
    vocabulary and grammar exercises.
  • An online learning environment can provide
    listening exercises, but the effectiveness may
    not be comparable to classroom-based learning
    because all students in the classroom acquire
    listening comprehension when one student engages
    in a speaking exercise with the instructor.
  • As a result, technology-assisted learning should
    offer less learning support through concrete
    experience, which diminishes the effectiveness of
    the learning cycle conceptualized by Kolb (1976).

14
  • However, in the acquisition of vocabulary and
    grammar skills, concrete experience plays a
    lesser role, so the electronic channel can
    provide effective lessons.
  • Because the learning materials are available to
    the students anytime and anywhere, they can
    absorb materials better at their own place and
    take the time to reflect on the proper use of
    words and grammar.
  • H1 Students in the face-to-face group show
    greater improvement in listening comprehension
    than their counterparts in the technology-assisted
    learning group.
  • H2 Students in the technology-assisted learning
    group show greater improvement in vocabulary than
    their counterparts in the face-to-face group.
  • H3 Students in the technology-assisted learning
    group show greater improvement in grammar than
    their counterparts in the face-to-face group.

15
Hypotheses and research model
  • In addition, we examine students satisfaction
    with technology-assisted learning using a factor
    model that contains key satisfaction
    determinants, such as perceived learning
    community support, learnability and
    effectiveness.
  • Existing pedagogical theories emphasize the
    socially constructed nature of learning, which
    indicates it essentially involves sharing and
    negotiation (Gulz 2005).
  • Neo (2003) empirically supports collaborative
    learning for enhancing students problem-solving
    and critical thinking skills.
  • Accordingly, we posit a positive correlation
    between perceived learning community support and
    learning effectiveness.

16
Hypotheses and research model
  • H4 In technology-assisted learning, perceived
    learning community support is positively
    correlated with perceived learning effectiveness.
  • H5 In technology-assisted learning, perceived
    learnability is positively correlated with
    perceived learning effectiveness.
  • As suggested by Keller (1983), learning
    satisfaction relates directly to perceptions and
    feelings about learning effectiveness or
    outcomes.
  • Therefore, we expect a positive correlation
    between perceived learning effectiveness and
    learning satisfaction.
  • Both Liaw (2004) and Chou and Liu (2005) reveal
    that information and experience sharing among
    peers and group members increases students
    learning satisfaction.

17
Hypotheses and research model
  • A relatively learnable course gives students a
    sense of satisfaction because they overcome
    challenges they encounter during the learning
    process.
  • H6 In technology-assisted learning, perceived
    effectiveness is positively correlated with
    learning satisfaction.
  • H7 In technology-assisted learning, perceived
    learning community support is positively
    correlated with learning satisfaction.
  • H8 In technology-assisted learning, perceived
    course learnability is positively correlated with
    learning satisfaction.

18
Fig 1. Hypotheses and research model for
explaining learning satisfaction
19
Study Design
  • Experimental design
  • A computer program assigned subjects to either
    the technology-assisted learning or face-to-face
    learning scenario, which creates to a
    between-groups design.
  • Our control group uses face-to-face learning
    exclusively, whereas the treatment group receives
    a combination of face-to-face and
    technology-assisted learning. (Hybrid approach)
  • Subjects
  • The participants are first-year students at a
    major university in Hong Kong who enrolled in the
    freshman English class mandated by the
    university.

20
Study Design
  • Dependent variables and measurements
  • We measure learning effectiveness objectively by
    comparing the difference between the pre- and
    post-study test scores, conducted at the
    beginning and end of the semester.
  • We examine subjects learning satisfaction and
    assessments of perceived course learnability and
    learning community support (Piccoli et al. 2001
    Aragon et al. 2002Wang 2003) by adapting
    previously validated question items to
    operationalize each investigated construct, with
    some minor wording changes appropriate to the
    targeted learning context.

21
Study Design
  • Dependent variables and measurements
  • All question items are based on a seven-point
    Likert scale, with 1 as strongly disagree and 7
    as strongly agree.

22
(LS 6 items)
(PLE 6 items)
(CL 3 items)
(CLS 3 items)
23
Study Design
  • Data collections
  • Our data are longitudinal, collected in the fall
    semester (September December) of 2004.
  • At the beginning of the semester, each subject
    took an English test online (pre-test), and this
    score serves as a baseline against which we
    evaluate the subjects learning effectiveness at
    the end of the semester.

24
Data analysis and results
  • A total of 507 subjects, 29.4 of the first-year
    student population, voluntarily took part in the
    study.
  • As a result, our effective sample includes 438
    subjects who averaged 19.1 years of age and were
    fairly balanced in their gender distribution.
  • Noticeably, more male than female subjects appear
    in the technology-assisted learning group, but
    more female than male subjects were in the
    face-to-face group in effective samples.

25
(four kinds of learning style)
It appears that more abstract thinkers joined the
face-to-face group than the technology-assisted
learning group (i.e. 63 v.s 53), whereas more
reflective observers were in the face-to-face
group than in the technology-assisted learning
group (i.e. 38 v.s 28).
26
  • Table 2, the Cronbachs alpha value of each
    investigated construct exceeds or is close to
    0.7, the commonly suggested threshold for
    reliability (Nunnally Bernstein 1994).
  • We also assess the instruments convergent and
    discriminant validity by performing a principal
    components analysis using the Varimax method with
    Kaiser normalization rotation.

Table 2. Summary of descriptive statistics and
construct reliability analysis.
27
  • The eigenvalue ofeach extracted factor exceeds
    1.0, the common threshold value.
  • Overall, our analysis shows that the instrument
    exhibits adequate convergent and discriminant
    validity.

Table 3. Reliability and discriminant validity of
the study instrument.
28
Data analysis and results
  • Technology-assisted versus face-to-face learning
  • For each of the learning aspects we study, we
    perform the following regression
  • where technology-assisted learning is the
    dummy variable and has a value of 1 if the
    subjects are in the technology-assisted learning
    group and 0 otherwise.
  • If the coefficient of technology-assisted
    learning is significant, we conclude there is a
    significant difference between technology-assisted
    and face-to-face learning.

o, 1
29
  • As we show in Table 4, technology-assisted
    learning has a significant impact on students
    performance with regard to listening
    comprehension and vocabulary.
  • As hypothesized, the face-to-face group achieves
    better performance in listening than the
    technology-assisted group, but the latter reveals
    enhanced vocabulary skills.
  • Likewise, for grammar skills, the
    technology-assisted learning group performs
    better than the face-to-face group, though the
    difference is not significant.

30
  • Thus, our results suggest that technology-assisted
    learning can enhance certain aspects of language
    learning, particularly those that emphasize
    reflective observation and do not require
    extensive human interaction (i.e. explicit
    knowledge).
  • However, for aspects of language learning that
    rely more on concrete experience through human
    interactions (i.e. tacit knowledge),
    technology-assisted learning may be less
    effective.

31
Data analysis and results
  • We use the CALIS procedure in SAS to test the
    hypothesized structural equation model.
  • The estimated structural model appears in Fig 2,
    together with the measurement model.
  • Overall, our model shows satisfactory explanatory
    power, accounting for 59 of the variance in
    perceived learning effectiveness and79 of the
    variance in learning satisfaction.
  • All factor loadings and path coefficients are
    statistically significant.

H8 0.31
H5 0.47
H6 0.76
H4 0.50
H7 0.38
32
  • Bentlers comparative fit index (CFI) is greater
    than 0.90, a common cut-off for a good fit.
  • The root mean square error of approximation is
    less than 0.08, indicating an adequate model fit.
  • Furthermore, the goodness-of-fit index (GFI), GFI
    adjusted for degrees of freedom, and
    Bentler-Bonett normed fit index are all close to
    the commonly suggested 0.90 benchmark.
  • Collectively, these fit index values suggest a
    satisfactory fit of our model and the data.

33
  • Technology-assisted learning can better support
    the acquisition of vocabulary (H2), whereas
    face-to-face learning seems to facilitate
    students listening skills more effectively (H1).
  • Students in the technology-assisted learning
    group perform slightly better than those in the
    face-to-face group, as hypothesized, but the
    difference is not significant (H3).

34
  • Our data further show that perceived course
    learnability and learning community support
    represent important predictors of perceived
    effectiveness, in support of H4 and H5.
  • In addition, perceived course learnability,
    perceived effectiveness and perceived learning
    community support are significant predictors of
    learning satisfaction, in support of H6H8.

Perceived Course Learnability
H8
H5
Perceived Effectiveness
Learning Satisfaction
H6
H4
Perceived Learning Community Support
H7
35
Summary and future research directions
  • This study contributes to technology-assisted
    learning literature by empirically examining
    target knowledge category (type) as a key
    moderating factor of online learning
    effectiveness.
  • We also contribute to technology-assisted
    learning research by proposing and empirically
    testing a factor model that explains and predicts
    students learning satisfaction in such settings,
    which directly affects their dropout decisions.
  • We also contribute to research in
    technology-assisted language training by showing
    that the use of technology-assisted learning can
    improve students vocabulary skills but may
    undermine their listening comprehension, with
    probable explanations.

36
Summary and future research directions
  • In addition, this study contains several
    limitations that suggest future research
    directions.
  • 1. our sampling process might introduce bias.
  • - For example, our study might have appealed
    more to students who value monetary rewards or
    are more eager to share their learning
    experiences.
  • - Therefore, readers should take caution when
    generalizing our findings.
  • 2. We do not directly measure each subjects
    effort or workload, such as the average number of
    hours each student spends on English learning per
    week.
  • - Therefore, we cannot comfortably rule out
    the possibility that the differences we observe
    between the two groups result from the
    differential workloads of their students.

37
Summary and future research directions
  • 3. The course Web site represents another
    potential limitation, in that though it resembles
    typical technology-assisted learning platforms,
    it is designed primarily to support asynchronous
    learning.
  • - Continued research should examine advanced
    technology-assisted learning platforms that
    support synchronous communications and live human
    interactions via rich media, such as video
    conferencing.
  • 4. We study technology-assisted learning
    effectiveness and satisfaction only in the
    context of language training.
  • - Further research should expand to other
    areas (e.g. information systems, management) to
    generate empirical evidence with greater validity
    and generalizability.
About PowerShow.com