Electronic resources are they used the way they were intended

1
Electronic resources are they used the way they
were intended?

• Dr Ria Hanewald
• Monash University
• Faculty of Information Technology
• Email ria.hanewald_at_infotech.monash.edu.au

2
Abstract

• Evaluating educational software and online
  learning materials to establish that intended
  learning outcomes have been achieved is a
  challenging task. Intelligent agents can help
  with this endeavour. This paper reports on recent
  research which programmed, tested and further
  developed such agents. Specifically, log files
  tracking students actions were compared to
  students observed behaviour to establish a
  correlation. It illuminated the discrepancy
  between underlying pedagogical assumptions of
  electronically delivered materials and the
  students appropriation of those learning tools.
  Presentation of these findings will help teachers
  understand their students learning processes in
  using educational resources.

3
Multimedia teaching software

• A
• Algorithm animation package contains pseudo code,
  animation and textual explanation
• B
• Web Industrial Experience Resource
• Integrated learning environment contains calender
  of events, time tracker, document management,
  news groups, discussion forum, repository of
  resources
• C
• Interactive parametric animations
• Simulated Modules for Learning Engineering Design

4
Purpose

• A
• Additional resource in lab
• with/without tutor in attendance
• for clarification, revision, exam preparation,
  consolidation, extension of algorithms covered
  in class
• B
• Additional resource in lab
• without tutor in attendance
• hands-one experience using interactive computer
  simulations
• C
• Essential component of the course/degree
• usage is integral part of the assessment

5
Data Collection

• Data collected during September and October 2004
• 36 students observed (12 volunteers for each
  application)
• Regular computer lab session/ under real life
  conditions
• 30 - 45 minute session length (one off)
• One to one basis/ student researcher (observer
  effect)
• Set of questions at the end of the computer
  session

6
Methodology

• Case studies using
• 1.) Observations of each students behaviour and
  actions
• 2.) Field notes to capture situational
  circumstances and prior experience
• 3.) Interview to clarify students intentions
• 4.) Audit trail / computer log to track
  students actions
• Comparison between audit trail and the students
  actions to establish one-to-one correspondence
• Cross referencing with interview answers and
  field notes
• Enables examination of the behavioural and
  cognitive content of learning sessions by
  comparing activities and actions

7
Traditional Instructional Design

• ADDIE formula
• Analyze-Design-Develop-Evaluate
• Based on behaviorist theories
• Uses Top-Down approach
• Established knowledge
• Packaged in software
• Transferred to student via computer
• Student has learned content

8
Contemporary Instructional Design

• Based on constructivist theory
• Enables solving of real life, practical problems
• Allows exploring and manipulating of objects
• Students have re-requisite knowledge
• Students undergo some structured experience

9
Learning approach

• Information Transmission Learning Technologies
• Passive learning
• Based on drill
• A - database material on a server
• Interactive Learning Technologies
• Engaged and sustained learning
• Characterized by challenging tasks
• B - professional engineering simulator
• C - web based resource
• Jones, B.J. Valdez, Nowakowski, J. and
  Rasmussen, C. 1994. Designing learning and
  technology for educational reform, North Central
  Regional Educational Laboratory, Oak, Brook, IL

10
Learning Theories

• Prescriptive model (based on behaviorist
  theory)
• A
• students follow set list of tasks (traditional/
  instructional)
• Phenomenological model
• B
• students own exploration (constructivist)
• C
• students tackle tasks (problem based learning)

11
Audit trails

• Computer log files record control operations and
  requests
• Track learners actions
• Enables identification and evaluation of patterns
• Gaining better understanding learners cognitive
  processes while interacting with a given piece of
  software
• Enables investigation of the relationship between
  
• the students intention and action and
• the software designers or teachers intention

12
Research issues

• Need for devices that monitor learners using
  educational technology
• To develop audit trails that track a variety of
  approaches
• Traditional and contemporary
• Using various learning theories

13
Research issues

• Focus is on the extent to which the user has
  attained certain learning objectives
• Focus on process evaluation gives a more
  complete picture of the learning process, can
  inform design of subsequent software
• Focus on effectiveness and efficiency of
  instructional design

14
Findings

• Students develop own learning strategies for use
  of educational software, vary from intended use
  of the software designer
• Discrepancy between intended and actual outcomes
• Correlation between student usage, knowledge
  acquisition and instructional design of the
  technology
• Need to develop new frameworks for instructional
  design of technology in educational contexts
• Need to develop non-linear performance models
• Need to develop mechanism to incorporate new
  ideas during learning process

15
References

• Axelrod, R. 1997. The complexity of cooperation.
  Agent based models of competition and
  collaboration. Princeton Princeton University
  Press.
• Callan, R. 2003. Artificial Intelligence.
  Palgrave Macmillian
• Epstein, J.M. 1999. Agent-based Computational
  models and generative Social Science. Complexity,
  4 (5)41-60
• Flagg, B.N. 1990. Formative Evaluation of
  Educational Technologies. Lawrence Erlbaum
  Associates, Inc. New Jersey, USA.
• Judd, T. and Kennedy, G. 2004. Making sense of
  audit trail data. Australian Journal of
  Educational Technology, 2018-32.

16
References

• Kearsley, G. 1998. Online Education. New
  Paradigms for Learning Teaching.
  http//www.horizon.uni.edu/
• Knapik, M. and Johnson, J. 1998. Developing
  Intelligent Agents for distributed systems.
  McGraw Hill Inc.
• Laurillard, D. 2002. Rethinking University
  Teaching a conversational framework for the
  effective use of learning technologies (2nd ed.)
  Routledge Falmer, London.
• Moundridou, M. Virvou, M. 2002. Evaluating the
  persona effect of an interface agent in a
  tutoring system. Journal of Computer Assisted
  Learning, 18 (3)
• Reeves, T.C. and Hedberg, J.G. 2003. Interactive
  Learning Systems Evaluation. Educational
  Technology Publications., Englewood Cliffs, New
  Jersey.