ICT and formative assessment in the learning society Bertil Roos, PhD

1
ICT and formative assessment in the learning
society?Bertil Roos, PhD
2
From elite to mass education

• Increasing number of students
• During the last decade the number of students
  have increased by more than 50 percent (from 200
  000 to more than 300 000 at the end of the
  millenium).

3
Decreasing unit of resources

• In the 1990s student/staff ratios increased from
  101 to 151

4
Stress and fatigue

• Consequences
• Students are taught in large classes and big
  lecture halls
• Possibility for students to ask questions is
  limited
• Difficult for teachers to return individual
  feedback to the students
• Difficulty to promote student learning
• Stress
• Fatique

5
Crisis

• How can teaching and learning be fostered in
  these raw circumstances?
• How can quality be maintained?
• How can resources be reallocated to enhance
  learning?

6
Solution?

• Most of the Swedish people have access to the
  internet
• Sweden tops the IT-league (Source IDCs
  Information Society Index)
• Can new technology promote student learning?

7
Research and development

• Can on-line assessment be used to support
  teaching and learning in the current context of
  European higher education?

8
Paradigm shift in assessment

• From the current paradigm
• testing culture
• psychometrics
• To the problem-solving paradigm
• assessment culture
• assessment of learning

9
Formative assessment

• Assessment carried out for the purpose of
  improving learning or teaching while it is still
  going on assessment for improvement, not grading.

10
Summative assessment

• Assessment carried out at the end of instruction
  to determine student learning and assign grades.

11
Formative versus summative assessment

• Robert Stake has defined the difference in
  culinary terms
• when the cook tastes the soup, thats formative
  evaluation when the guest tastes it, thats
  summative evaluation.
• Black Wiliam define formative assessment as
• all those activities undertaken by teachers,
  and/or by their students, which provide
  information to be used as feedback to modify the
  teaching and learning activities in which they
  are engaged.

12
High stakes assessment

• a close association between assessment outcomes
  and social outcomes
• summative assessment is high stakes if it has
  important consequences

13
Low stakes assessment

• its social consequences may be positive, when it
  is used to guide teaching and learning
• moving assessment closer to students and their
  learning

14
Low versus high-stakes assessment

• Linn (1998)
• It is important to use multiple indicators when
  judging the students which will increases the
  validity.
• A key, for long term success, is to create a
  system for evaluating both the intended positive
  effects and the more likely unintended negative
  effects of the assessment system.

15
Testing, assessment and examinations as social
practices

• Messick (1989)
• the value of the testing must depend on the
  total set of effects it achieves, whether
  intended or not (p. 85).

16
Formative low-stakes assessment

• Methods of assessment are determined by our
  beliefs about learning
• Can assessment support learning as well as
  measure learning through on-line processes?

17
Feedback

• Ramaprasad (1983) defines feedback as
• information about the gap between the actual
  level and the reference level of a system
  parameter which is used to alter the gap in some
  way (p. 4).
• Black Wiliam (1998)
• For feedback to exist, the information about the
  gap must be used to alter the gap. If the
  information is not actually used in altering the
  gap, then there is no feedback (p.39).

18
Question to consider

• Are you assessing product or process?
• Are you assessing knowledge or use of that
  knowledge?
• Is the assessment formative or summative,
  high-stakes or low-stakes?

19
Teachers view of on-line formative assessment

• Worth repeating
• Time-saving
• Item analysis possible
• Performance data
• More sophisticated feedback

20
Students view of on-line formative assessment

• Speedy feedback
• Good support
• Time friendly
• Easy access
• Preferable over pencil and paper

21
Reflections

• How can we use information technology to
  transform learning?

22
Lessons learned

• On-line assessment is incremental - an activity
  built around cultural, hands-on change rather
  than top-down, technological innovation-by-substit
  ution
• On-line assessment requires a institutional
  horizon based on long-term pedagogic change.

23
Pedagogical issues

• Is on-line assessment appropriate?
• Integration with existing assessment methods and
  strategies
• Integration within course

24
Implementation of on-line assessment

• Identify aims
• Identify appropriate technology
• Staff development
• Student training
• Operational issues
• Risk analysis
• Evaluation

25
Reflections
www.onlineassessment.nu