Multimedia Instruction: Using visuals and audio for learning

1
Multimedia Instruction Using visuals and audio
for learning

• David A. Mappin, PhD.
• Michele M. Kelly , March, 1999

2

• Almost everyone agrees that using visuals in
  instruction is a good idea. However, beyond the
  simplicity of direct illustration, or visual
  enhancement for purposes of motivation most
  multimedia developers do not have a good idea of
  the range of purposes which visuals may serve in
  learning and the reasons why a line drawing might
  be a better choice than a photograph in certain
  instances.

3

• Beyond research with basic visual materials,
  there are
• concessions which have emerged with the rise in
  the number of
• forms of new media available, concession with
  regard to
• maximizing image quality while keeping the file
  sizes
• manageable and dealing with other aspects of the
  selected
• medium, for example, the available transmission
  rates for any
• multimedia data needed as a part of the system.

4
Program Format

• This program will deal with seven forms of
  visuals, from verbal and visual images through
  artistic renderings and photographic images, in
  three basic states still or static partial
  motion and full motion.

5

• We might consider this as mapping a three part
  structure for this nexus of the technical demands
  of the system with the
• quality of the image to be displayed
• image quality
• file size
• transmission dimensions, and
• image handling constraints within the computing
  systemdisplaying the image(s).

6
Program Format

• Clear presentations of lists of what is known and
  what is recommended in using visuals for learning
  would be features of the final product. At the
  same time the program would enable discussion of
  those things which are not known or which are
  subject to judgements by the developer. At this
  stage new ideas may be incorporated and the basic
  structure may be altered through the process of
  discussion.
• As a background to this discussion the program
  will also address some more fundamental issues of
  what research has shown about our confidence in
  using visuals for learning and how we make
  meaning from pictures, and how sound relates to
  the process.

7
Program Format

• Why use Visuals?
• Dynamic (motion) visuals
• Animation
• Why Use Sounds?
• When to Use Visuals, Sound or a combination of
  the two.
• How to Use New Media Effectively.
• Research supporting the Use of Visuals and Sound
  for Learning.
• The proposed collaborators.
• References

8
Why Use Visuals?

• In a review of visual learning research Anglin,
  Towers, and Levie (1996) maintain there is
  significant evidence that people remember
  pictures better than words, generally speaking.
  There are at least three theories commonly
  presented to explain this claim, often referred
  to as the picture superiority effect.
• These three are
• Paivios dual-code model (Paivio, 1978 1990),
• The single code model (Anderson, 1978 Kosslyn,
  1981 and Pylyshyn, 1978)
• The sensory-semantic model (Nelson, 1979 Smith
  Magee, 1980)
• While the discussion of the three will
  undoubtedly continue, each has some implications
  for the use of pictures.

9

• Duchastel (1978) described three roles for
  illustrations in text based on their functions
• an attentional role
• a retentional role
• and an explicative role.

10

• As Anglin, Towers, and Levie describe them,
• the attentional role relies on the fact that
  pictures naturally attract attention. (p. 765).

11

• The retention role aids the learner in
  recalling information seen in an illustration.
  
• (Anglin p. 765)

12

• the explicative role explains, in visual
  terms information that would be hard to convey in
  verbal or written terms. (Anglin p. 765).

13
Why Use Visuals?
Duchstel and Walter (1979) have identified seven
ways that pictures may provide an explicative
role

• Descriptive
• Expressive
• Constructional
• Functional
• Logical-mathematical
• Algorithmic
• Data display

14
Descriptive

• The role of the descriptive function is to show
  what an object looks like physically.
• (Duchstel and Walter 1979)

15
Expressive

• The expressive role is to make an impact on the
  reader beyond a simple description.

16
Constructional

• The intent of the constructional role is to show
  how the parts of a system form the whole.

17
Functional

• The functional role allows a learner to follow
  visually the unfolding of a process or the
  organization of a system.

18
Logical-mathematical

• The purpose of this role is to show mathematical
  concepts through curves, graphs, etc.

19
Algorithmic

• The algorithmic role is used to show action
  possibilities. Data display

20
Data display

• The functional role of data display is to allow
  quick visual comparison and easy access to date
  such as pie charts, histograms, dot maps, or bar
  graphs.

21
Alessandrini

• In addition, it will be tempered with the
  observations by Alessandrini (1984) that some
  functional frameworks deal to much with
  representational images.

22
A Seven Step Continuum
Wileman (1995) has presented a seven step
continuum for dealing with images from
representational photo realism, through various
levels of graphic abstraction, to verbal symbols.

23
Wilemans seven step continuum combined with
still, partial motion, and full motion visuals
can be seen as a matrix to explore how various
choices may look and the effect they may have on
learning.
24
Dynamic (motion) visuals

• Much of the work on the use of dynamic visuals,
  or what we would usually call motion media, comes
  from work that has been done on learning from
  film and television. There is little reason to
  suspect that most of these principles will not
  continue to be valid.

25
Dynamic (motion) visuals

• As early as 1924 after a series of research
  studies freeman concluded that motion media could
  contribute to learning in cases where
• the motion was critical to what was being
  learned and
• the motion was used to cue or draw the learners
  attention to the material being presented.

26
Dynamic (motion) visuals

• What we know about the use of motion visuals
  today may be thought of as extensions and
  refinements of freemans principles. In the
  section on when to use visuals we provide some
  examples of where motion might be critical to
  learning.
• In addition to these examples the use of video as
  a social modeling tool in learning various
  interpersonal skills has been shown to be very
  valuable.

27
Dynamic (motion) visuals

• Perhaps the newest addition to our ideas of
  learning is in using dynamic visuals to create a
  context for that learning. One example of this
  may be found in the work on anchored
  instruction by the Cognition and Technology
  Group at Vanderbilt University. (Cognition and
  Technology Group, 1992).
• Through extensive use of a story line presented
  with video to provide a context for the learning
  activity, thus making that learning more
  authentic, CGTV have developed programs such as
  those in the Jasper Woodbury series which provide
  an excellent example of this use of video.

28
Dynamic (motion) visuals

• Also within the dimension of dynamic visuals is
  the impact which camera angles, shot lengths, and
  various editing techniques may have on learning
  from and on the impact of educational video.
  Authors such as Devaney (1991) have mapped some
  of these dimensions, as has some work by Baggaley
  (1974, 1975).

29
Animation

• Research in the use of animation in learning has
  generated mixed results in linking the use of
  animation and learning (park Hopkins, 1993
  Reiber, 1990).
• Reiber however, (1990) has made three
  recommendations regarding the role of animation
  in designing learning
• It should be used only when its attributes are
  congruent with the learning task
• Novice learners in any given content area my find
  difficulty in knowing how to attend to relevant
  cues or details And
• Its greatest contribution to computer-based
  learning may lie in its use in interactive
  graphic applications.

30
When to Use Visuals, Sounds or a combination of
the two.

• Many aspects of the role of visuals may be
  represented by the Cone of Experience developed
  by Edgar Dale in 1946.
• Dales cone represents individuals experiences
  as shading from real experience through contrived
  (vicarious) experiences to the increasingly
  abstract realm of verbal symbols.
• As seen in the drawing on the next slide,
  Bruners ideas may be shown to parallel Dales.

31
(No Transcript)
32
When to Use Visuals, Sounds or a combination of
the two.

• What this suggests is that the use of visual and
  other multimedia materials to replace direct
  human experiences provides a very powerful
  experience, closely analogous to having the real
  experience.
• By implication, and this has been borne out in a
  variety of research, such experiences can be very
  powerful in developing learning. The further one
  gets from direct experience toward the symbolic
  and abstract aspects of human experience the more
  the learner has to have developed skills in
  understanding and manipulating symbols in order
  for learning to take place.

33
When to Use Visuals, Sounds or a combination of
the two.

• This is not to suggest that learning from
  abstract symbols, for example reading, is not a
  powerful way of learning, that would be nonsense.
  
• Rather, because of the abstraction learners can
  always make clear, direct connections to the uses
  of their learning without further, less abstract
  experiences.
• We should also remember that visuals themselves
  can be highly abstract and very symbolic.

34
Visuals Can

• At a practical level, visuals of differing kinds
  can show us what the eye can see including
• Things too far away or
• Too dangerous to view in person

35
Visuals Can

• At a practical level, visuals of differing kinds
  can show us what the eye might see

Historical events Historical re-creations Socia
l Modeling
36
Visuals Can

• Show us what the eye cannot see by extending
  human capabilities including

Things which are too fast or too slow for us to
perceive. Things which are too large or too
small for us to perceive.
37
Visuals Can
Show us things that do not (cannot) exist
Fantasy people, places, things Science
projections Science fantasy
Presentations of literature
38
Visuals Can
Show us things that don't (can't) exist.
Fantasy people, places, things Science
projections Science fantasy
Presentations of literature
39
Why Use Sounds?

• Tripp and Roby (1996) have reviewed the ways in
  which audio has been used for learning in the
  past century.
• They conclude that there is no reason to doubt
  that students learn from audio, but, if print is
  available, research has clearly shown that
  students learn material significantly better from
  print that from audio if the content is held
  constant.

40
Why Use Sounds?

• In the parallel case of the use of audio in
  combination with visual materials, research has
  clearly shown that redundant audio information
  can aid learning.
• Mann (1992) has described how sound may function
  within a video program in a variety of roles and
  Millerson (1985) has described a series of ways
  that sound can be used to represent ideas
  symbolically and he describes a series of nine
  functions sound might have in this regard
  factual, environmental, interpretive, symbolic,
  imitative, identifying, recapitulative, coupling,
  and montage.

41
Why Use Sounds?

• In the proposed program the possible uses of
  sound on its own and in conjunction with images
  will be explored more extensively.
• Given the role that sound can and does play in
  multimedia it would seem import to make students
  more conscious of the possibilities.

42
How to Use New Media Effectively.

• Working with any of the new media to provide
  images and sound which are useful in learning and
  also of the quality necessary for the purpose
  require us to balance four factors

• Image quality.
• File size.
• Transmission dimensions, and.
• Image handling constraints within the computing
  the system displaying the image(s).

43
How to Use New Media Effectively.

• Some of the obvious issues which might be
  considered are
• Ensure students are not being distracted (or
  turned off) because the image is jerky or blurry.
  
• Ensure students are not distracted from careful
  observation because there are too many other
  things to do at the same time.
• Are students able to see the actual colour if it
  is important because of colour coding .
• What are the download times for the web ojects
  which are being used? Video animation both
  require careful attention to frame rate and
  display window size.

44
Research Supporting the Use of Visuals and Sound
for Learning

• Four primary lines of research into visuals used
  as illustrations are described by Levie (1987)

• Picture perception
• Memory for pictures
• Learning and cognition
• Affective responses to pictures.

45
Research Supporting the Use of Visuals and Sound
for Learning

• In the same work Levie makes the point that
• It is clear that research on pictures is not a
  coherent field of inquiry. An aerial view of the
  picture research literature would look like a
  group of small tropical islands with only a few
  connecting bridges in between. Most researchers
  refer to a narrow range of this literature in
  devising their hypotheses and in discussing their
  results. Similarly, authors of picture memory
  models, for example, take little notice of the
  theories of picture perception. (P. 26)

46
Proposed Collaborators

• The following individuals and institutions have
  been discussing the development and use of this
  material
• Dr. David Mappin
• University of Alberta, lead institution
• Dr. Jon Baggaley
• Athabasca university
• Dr. Bill hunter
• University of Calgary
• Eric rice
• Access/LTA

47
References

• Alesandrini, K. L. (1984). Pictures and adult
  learning. Instructional science 13, 63 - 77.
• Anderson, J. R. (1978). Arguments concerning
  representations for mental imagery.
  Psychological review 85, 249-77.
• Anglin, G., Towers, R., Levie, H. (1996).
  Visual message design and learning the role of
  static and dynamic illustrations. In D. Jonasen
  (ed.), Handbook of research for educational
  communication and technology, 755-794. New York,
  NY Macmillan.
• Baggaley, J. (1974). Research notes experiments
  in ET effects of adding backgrounds.
  Educational broadcasting international 7,
  208-209.
• Baggaley, J. (1975). Research notes experiments
  in ET effects of edited cutaways. Educational
  broadcasting international 8, 36-37.

48
References

• Cognition and technology group at Vanderbilt.
  (1992). The jasper experiment an exploration of
  issues in learning and instructional design.
  Educational technology research and development
  40 (1), 65-80.
• Dale, E. (1946). Audio-visual methods in
  teaching. New York, NY holt, Rinehart, and
  Winston.
• Devaney, A. (1991). A grammar of educational
  television. In D. Hlynka and J. Belland, eds.,
  Paradigms regained the uses of illuminative,
  semiotic, and post-modern criticism as modes of
  inquiry in educational technology. Englewood
  cliffs, NJ educational technology.
• Duchastel, P. C. (1978). Illustrating
  instructional texts. Educational technology 11,
  36-39.

49
References

• Duchastel, P. C. Walter, R. (1979, Nov.).
  Pictorial illustration in educational texts.
  Educational technology, 20 - 25.
• Dwyer, F. (1978). Strategies for improving
  visual learning. State college, PA learning
  services.
• Freeman, F. N., Ed. (1924). Visual education a
  comparative study of motion pictures and other
  methods of instruction. Chicago, IL university
  of Chicago process.
• Kosslyn, S. M. (1981). The medium and the
  message in mental imagery a theory.
  Psychological review 88, 46-66.
• Levie, H. (1987). Research on pictures a guide
  to the literature. In D. M. Willows H. A.
  Houghton, eds. The psychology of illustration
  volume 1 basic research, 1-50. New York
  Springer.

50
References

• Mann, B. (1992). The SSF model structuring the
  functions of the sound attribute. Canadian
  journal of educational communication 21 (1),
  45-65.
• Millerson, G. (1985). The technique of
  television production, 9th ed. New York, NY
  focal press.
• Nelson, D. L. (1979). Remembering pictures and
  words appearance, significance, and name. In L.
  S. Cermak F. I. M. Craik, eds. Levels of
  processing in human memory, 45-76. Hillsdale,
  NJ Erlbaum.
• Paivio, A. (1978). A dual-coding approach to
  perception and cognition. In J. H. Pick E.
  Saltzman, eds. Modes of perceiving and
  processing information, 39-51. Hillsdale, NJ
  Erlbaum.
• Paivio, A. (1990). Mental representations a
  dual coding approach, 2nd edition. New York, NY
  oxford Univesity press.

51
References

• Park, O., Hopkins, R. (1993). Instructional
  conditions for using dynamic visual displays a
  review. Instructional science 22, 1-24.
• Pylyshyn, Z. W. (1981). The imagery debate
  analogue media versus tacit knowledge.
  Psychological review, 88, 16-45.
• Reiber, L. (1989). A review of animation
  research in computer-based instruction. In
  proceedings of selected research papers presented
  at the annual meeting of the association for
  educational communications and technology,
  Dallas, Tx, Feb. 1-5 (ERIC document reproduction
  service no. 308-832).
• Reiber, L. (1990). Animation in computer-based
  instruction. Educational technology research and
  development 38 (1), 77-86.

52
References

• Reiber, L. (1994). Computers, graphics, and
  learning. Madison, WI WCB brown benchmark.
• Salomon, G. (1981). The use of visual media in
  the service of enriching mental thought
  processes. Instructional science, 9 (4),
  327-329.
• Salomon, G. (1979). Interaction of media,
  cognition, and learning. San Francisco, CA
  Jossey-bass.
• Smith, M. C., Magee, L. E. (1980). Tracing the
  time course of picture-word processing.
• The Vietnam War
• http//dspace.dial.pipex.com/leuhusen/nam/intro.ht
  m