The Asynchronous Discussion Board as an Assessment Tool: A Critical Appraisal

1
The Asynchronous Discussion Board as an
Assessment Tool A Critical Appraisal
0
Dr Jeremy Williams Universitas 21 Global

• 8th CAA Conference
• Loughborough University
• 6-7 July 2004

2
Overview
0

• The context
• Learning through discussion
• Criteria for assessing discussion
• The hazards of assessing discussion
• Authentic discussion board assignments
• Preliminary findings

3
0
1. The context
4
Universitas 21 Global
0
5
The global nature of U21G

• Online graduate business school, headquartered in
  Singapore
• Around 300 students enrolled from more than 20
  countries, in many different time zones
• Adjunct faculty from Europe, Australasia, Asia
  and North America

6
Pedagogy

• Instructor-led, student centred
• Problem-based learning
• Extensive use of Harvard Business school cases
• A commitment to authentic assessment
• Modus operandi asynchronous discussion gt how
  discussion boards are used ? critically important

7
0
2. Learning through discussion
8
Communities of practice

• With the expansion of online learning
  tremendous opportunities are becoming available
  to teachers and learners to foster peer
  relationships, team skills, collaboration, group
  problem solving and debate, as well as less
  formal or structured communications between
  students and their wider professional
  communities
• Morgan OReilly (1999, p. 86)

9
Peer learning

• In the online classroom, it is the relationships
  and interactions among people through which
  knowledge is primarily generated
• Palloff and Pratt (1999, p. 15)
• Collaboration between students from far-flung
  institutions around the globe can considerably
  enrich the experience and broaden the contextual
  perspectives of each participant
• Day (1998)

10
Dialogue and greater learner autonomy

• Enhancing dialogue is a way of promoting
  independence and autonomy in learners, and
  challenges power relationships in teaching and
  learning
• Evans Nation (1989)
• The digital learning environment will probably
  be the most efficacious enabler of independent
  and self-determined learning
• Peters (2000, p. 16)

11
Laurillards Conversational Framework

• studies show that a collaborative discussion
  environment is highly valued by students
• Students have access to an expert whom they can
  question to clarify the experts description
• Students can articulate and re-articulate their
  descriptions of the topic in response to others
  ideas and comments
• Students can reflect on the discussion to clarify
  their own understanding
• Laurillard (2002, p. 148)

12
Encouraging online discussion

• Grades the currency that students deal in
• Swan et al (2000)
• Needs to be well-integrated into the subject, and
  will also be of greater purpose to students if it
  is assessed
• Day (1998) cited in Morgan OReilly (1999, p.
  86)

13
0
3. Criteria for assessing discussion
14
MacKinnon (2000)

• Four categories of interaction
• Challenging a point of view
• Forwarding a new perspective
• Relating the theory to ones experience
• Offering support for a position based on the
  literature.

15
Sabin et al (2000)

• P is for participation
• A is for additional commentary
• C is for constructive criticism
• E is for encouraging

16
Meyer (2004)

• Proffers 4 different analytical frameworks
• Two were developmental models
• King and Kitcheners Reflective Judgment Model
• Perrys model of intellectual and ethical
  development
• Two captured levels of thinking
• Garrisons four-stage critical-thinking model
• Blooms taxonomy of educational objectives

17
0
4. The hazards of assessing discussion
18
You can take a horse to water, but

• Collaborative learning is undeniably important,
  and the communicative media are powerful enablers
  that match what is needed for discussion and
  collaboration, but to what extent do they succeed
  in enabling learning?
• the properties of a medium do not determine
  the quality of learning that takes place

Diana Laurillard (2002) Rethinking University
Teaching, p. 148
19
Assessing discussion Counter-intuitive?

• Harlen and Deakin Crick (2003) the motivation
  for learning can be discouraged unwittingly by
  assessment and testing practices
• Early experience at U21G suggests that when
  discussion contributions are assessed, discussion
  becomes a little contrived
• A case of Goodharts Law applied to assessment?

20
0
5. Authentic discussion board assignments
21
Discussion boards for discussion

• Discrete small-scale problems abandoned in favour
  of discussion board assignments integrated with
  case study assignments.
• Small assessment weighting for participation
  remains
• The main incentive to contribute the grounding
  it will provide for the submission of a case
  study assignment with a larger assessment
  weighting.

22
Assessment in disguise

• Trials now taking place
• All discussion board activity is evaluated with
  an overall mark awarded
• Students nominate selected discussion board
  contributions for assessment (e.g. their 5 best)
• Discussion boards will be considered by the
  instructor in validating peer assessments of
  student performance

23
0
6. Preliminary findings
24
Student feedback

• This prompted actual threads of discussion rather
  than what I've found in other modules where one
  makes rather artificial comments on other
  student's postings as the class attempts to get
  marks for making exactly the same points in
  slightly different words.

25
Student feedback

• I thought the discussion boards were useful in
  preparing for the assignments because they
  focused on the content of those assignments so
  you were able to get tips from your fellow
  students and the tutor. In other modules,
  discussion boards were separate to the written
  assignment and so represented an additional
  rather than complementary task.

26
Summary

• Appropriately structured, discussion boards can
  provide a useful vehicle for learner assessment
• The act of assessing need not have a stultifying
  effect on the quality of debate and the depth of
  learning

27
Future work

• A comparative analysis of the qualitative
  substance of discussion board postings before and
  after the change in discussion board format

28
Questions?