How to make sure your students learn what you want them to

1
How to make sure yourstudents learnwhat you
want them to
Claus Brabrand ((( brabrand_at_itu.dk ))) Associate
Professor ((( Programming, Logic, and Semantics
))) IT University of Copenhagen
Claus Brabrand ((( brabrand_at_itu.dk ))) VISITING
Professor ((( Software Productivity Group (SPG)
))) Univ. Fed. de Pernambuco
2
An Introduction to the Theory Framework of
Constructive Alignment
Claus Brabrand ((( brabrand_at_itu.dk ))) VISITING
Professor ((( Software Productivity Group (SPG)
))) Univ. Fed. de Pernambuco
3
Exercise
T
"What is good teaching?"
4
Outline (Aug 03, 2010)

• 1) Introduction
• Constructive Alignment
• The SOLO Taxonomy
• 2) From Content to Competence
• Advocate a shift in perspective
• Elaborate on The SOLO Taxonomy
• 3) In Practice
• Concrete recommendations
• Alignment implementation process

5
Introduction to

• Constructive Alignment SOLO Taxonomy

John Biggs popular and heavily cited book
Teaching for Quality Learning at
University - What the student does
Note 3rd Edition now available J.Biggs
C.Tang, 2007
Teaching Teaching
Understanding Understanding
19 min award-winning short-film on Constructive
Alignment (available on DVD in 7 languages,
epilogue by John Biggs)
6
Neighbour Discussion
T
"What are the film'smain messages (in your
opinion)"?
7
Outline (Aug 03, 2010)

• 1) Introduction
• Constructive Alignment
• The SOLO Taxonomy
• 2) From Content to Competence
• Advocate a shift in perspective
• Elaborate on The SOLO Taxonomy
• 3) In Practice
• Concrete recommendations
• Alignment implementation process

8
From Content to Competence

• My old course descriptions (Concurrency 2004)
• Given in terms of a 'content description'.
• Essentially

• Goal is
• To understand
• deadlock
• interference
• synchronization
• ...

This is a bad idea for two reasons...!
9
Problem 1 !

• Problem with 'content' as goals !

analyze ... theorize ...
analyze systems explain causes
define deadlock describe solutions
agreement
Stud. C

• Goal is
• To understand
• deadlock
• interference
• synchronization
• ...

tacit knowledge from a research-based tradition
not known by student
Teacher
name solutions recite conditons
analyze systems explain causes
Stud. B
?
Stud. A
Censor
10
Problem 2 !

• Problem with 'understanding' as goals !

• Goal is
• To understand
• deadlock
• interference
• synchronization
• ...

'concept of deadlock' ?!
?
The answer is simple
It cannot be measured !
11
Competence !

• 'Competence' as goals !

Competence knowledge capacity to
act upon it
Have the student do something and then observe
(evaluate) the product and/or process

• Objective !
• To learn how to
• analyze systems for...
• explain cause/effects...
• prove properties of...
• compare methods of...
• ...

Note 'understanding' is of course
pre-requisitional !
?
Note' inherently operational ( verbs)
'SOLO' Structure of the Observed Learning
Outcome
12
Neighbour Discussion
T
"How does this content vs. competence relate
to YOUR courses?"
13
SOLO (elaborated)
Note the list is non-exhaustive
QUANTITATIVE
QUALITATIVE
SOLO 2 uni-structural
SOLO 3 multi-structural
SOLO 4 relational
SOLO 5 extended abstract

• theorize
• generalize
• hypothesize
• predict
• judge
• reflect
• transfer theory (to new domain)

• analyze
• compare
• contrast
• integrate
• relate
• explain causes
• apply theory (to its domain)

• combine
• structure
• describe
• classify
• enumerate
• list
• do algorithm
• apply method

• define
• identify
• count
• name
• recite
• paraphrase
• follow (simple) instructions

14
SOLO verbs

• Mapped by
• B. Dahl C. Brabrand
• (Natural science context!)
• With help from
• 3 Educational research colleagues (medicine)
• J. Biggs C. Tang

15
SOLO Advantages

• Advantages of The SOLO Taxonomy
• Linear hierarchical structure (good for
  progression)
• Aimed at evaluating student learning
• Converges on research (at SOLO 5)

ResearchProduction ofnew knowledge
16
Exercise
T
"Key competences in YOUR course?"
Concurrency analyze systems compare models
17
10' Break

• Please put the Post-Its on the wall

"What is good teaching?"
Key competences (in your course)
18
Outline (Aug 03, 2010)

• 1) Introduction
• Constructive Alignment
• The SOLO Taxonomy
• 2) From Content to Competence
• Advocate a shift in perspective
• Elaborate on The SOLO Taxonomy
• 3) In Practice
• Concrete recommendations
• Alignment implementation process

19
Concrete Recommendations
1) Use 'standard formulation' a) puts
learning focus on the student b) competence
formulation "to be able to"

• Intended Learning Outcomes Genetics
  101
• After the course, the students are expected to be
  able to
• locate genes on chromosomes
• do simple calculations (e.g., recombination
  frequencies, in-breeding
  coefficients, Hardy-Weinberg,
  evolutionary equilibria).
• describe and perform connexion-analysis
• describe fundamental genetic concepts (e.g.,
  mutation variation,
  in-breeding, natural selection).
• describe and analyze simple inheritancies
• analyze inheritance of multiple genes
  simultaneously

4) Avoid 'understanding-goals' "To
understand X", "Be familiar with Y",
"Have a notion of Z" !
V
N
N
V
N
V
V
V
N
V
V
N
V
3) Use 'Verb Noun' formulation What the
student is expected to do with a given
matter .
2) List sub-goals as 'bullets' Clearer than
text
N
V
20
Implementation Process
1) Think carefully about overall goal of
course (what students learn to do?)
2) Operationalize these goals and formulate
them as SOLO intended learning outcomes
alignment
learning incentive
learning support
3) Choose carefully the form(s) of assessment
( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
Think of teaching activities as training for
exam
21
My Starting Point

• Content description (Concurrency '04'05)

What is the overall goal of the course...? (i.e.,
what are the students to learn to do?)
22
Overall Course Philosophy

• Model-Based Design for Concurrency

23
Implementation Process
1) Think carefully about overall goal of
course (what students learn to do?)
2) Operationalize these goals and formulate
them as SOLO intended learning outcomes
alignment
learning incentive
learning support
3) Choose carefully the form(s) of assessment
( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
Think of teaching activities as training for
exam
24
Intended Learning Outcomes

• Intended Learning Outcomes(based on The SOLO
  Taxonomy)

Noteexplicitly included as a non-goal
?
25
Implementation Process
1) Think carefully about overall goal of
course (what students learn to do?)
2) Operationalize these goals and formulate
them as SOLO intended learning outcomes
alignment
learning incentive
learning support
3) Choose carefully the form(s) of assessment
( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
Think of teaching activities as training for
exam
26
MC-test (pre- vs. post-alignment)

• 2004 Multiple-Choice assessment
• with a bunch of seemingly reasonable questions
• 2006 Assessment ( ILO's)
• Project (carefully assessing 'synthesis')
• construct, apply, relate, implement, ...
• Multiple-Choice test (carefully assessing
  'analysis')
• analyze and compare

BadAlignment
Good Alignment
27
Implementation Process
1) Think carefully about overall goal of
course (what students learn to do?)
2) Operationalize these goals and formulate
them as SOLO intended learning outcomes
alignment
learning incentive
learning support
3) Choose carefully the form(s) of assessment
( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
Think of teaching activities as training for
exam
28
Teaching/Learning Activities
functional knowledge (problem oriented)
Project work
Case teaching
teacher centric
student centric
Exercise class
Lecture
declarative knowledge (discipline oriented)
29
Lecture (about Application)

• Teacher activity
• Introduce
• Explain
• Elaborate
• Discuss application
• Give examples
• Show PPT slides
• Questions on slides
• Winding up

• Student activity
• Listen
• Listen (maybe take notes)
• Understand? (correctly? deeply?)
• Listen (maybe take notes)
• Listen (maybe take notes)
• Watch (maybe note points)
• Write answers to questions
• Possibly ask a question

vs.
active teacher
vs.
Biggs Tang 2007, p.137
passive student
30
Learn about vs. Learn to do

• Learning (about)
• about application
• about cooking
• about programming
• about designing
• about analysis
• about construction
• about relating
• ...

• Learning (to do)
• to apply
• to cook
• to program
• to design
• to analyse
• to construct
• to relate
• ...

studentlistening
(to something about something)
vs.
student doing !!
31
Student Activation
"The (in-famous) Learning Pyramid"
Average retention rate
5
Lecture
Listening Learning about
passive student
10
Reading
20
Audiovisual
student activation
30
Demonstration
active student
50
Discussion group
75
Practice by doing
Doing Learning to do
80
Teaching others
NTL Institute for Applied Behavioral Science,
Bethel, Maine
32
Constructive Alignment

• Make explicit ILO's ( )
• (and tell this to students)

Intended Learning Outcomes
ILO's
Teaching LearningActivities
Assessment
vs.
SUSAN intrinsically motivated
ROBERT extrinsically motivated
33
Questions...
Cognitive processes
Course descriptions
My research and teaching
"understanding" content ? competence
Association new old
The SOLO Taxonomy
'TLA' Teaching / Learning Activities
Teacher models levels 1 - 2 - 3
The Film
The Book
Susan Robert
?
Student activation
Tips'n'Tricks
CS v. NAT v. MAT
recite generalize
15 programming
Students at University
"What is good teaching?"
Constructive Alignment
John Biggs
Top Competences
34
Now, please "2-minute recap"
T

• Please spend 2' on thinking about and writing
  down the most important points from the talk
  now!

Immediately
After 1 day
After 1 week
After 2 weeks
After 3 weeks
35
Key References

• Teaching for Quality Learning at
  UniversityJohn Biggs Catherine TangSociety
  for Research into Higher Education, 2007.
  McGraw-Hill.
• Evaluating the Quality of Learning The SOLO
  TaxonomyJohn Biggs Kevin F. CollisLondon
  Academic Press, 1982
• Teaching Teaching Understanding
  UnderstandingClaus Brabrand Jacob Andersen19
  minute award-winning short-film (DVD)Aarhus
  University Press, Aarhus University, 2006
• "Constructive Alignment The SOLO Taxonomy a
  Comparative Study of University Competencies in
  Computer Science vs. Mathematics"Claus Brabrand
  Bettina DahlCRPIT, Vol. 88, ACS 3-17, R.
  Lister Simon, Eds., 2007

36
Obrigado !
Film's homepage
((( http//www.daimi.au.dk/brabrand/short-film/
)))
37
Tips'n'Tricks (activation)

• Neighbour discussions

• Post-It exercise

• focus zoom in
• anonymous (!)
• swap'able
• everyone will engage
• empathetic control
• shared knowledge pool

• more questions (students dare ask them)
• better questions (students had a chance
  to discuss)

Phil Race
1-2 min timeout

• Frequent breaks

• Form variation

pulse reader measurements
lecturing blended with in-class activation
exercises
38
Tips'n'Tricks (cont'd)

• Use many examples(build on student
  pre-knowledge)

• Explicit structure

1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
?

• self evident to you teacher
• not to a learner student
• (esp. during learning process)

• "Less-is-more"

• Student 'recap' at end

• analyze
• compare
• relate

common deadlock, uncommon deadlock,
A-synchronization, B-synchronization, hand-shake,
multi-party synchronization, multi-party
hand-shake, binary semaphores, generalized
semaphores, blocking semaphores, recursive locks,
...
vs.
now
after 1 day
after 1 week
after 2 weeks
after 3 weeks
Emphasize depth over breadth (coverage)
39
Problematic Courses

• E.g. course Databases (at RUC/Roskilde)
• Note almost entirely non-operational(!)
• i.e. measure how?!

• obtain knowledge about the structure of
  database systems
• be familiar with design of databases by use of
  special notations like E/R and analysis
  through normalization
• get an overview of the most important database
  models and a detailed knowledge about the
  most important model - the relational model
  as well as the language SQL
• get an overview of database indexing and query
  processing
• obtain knowledge about application programming
  for DB systems.

Familiar with ?!
40
UNALIGNED COURSE
?
Teachers intention
Students activity

• e.g.
• explain
• relate
• prove
• apply

"Dealing with the test"
Exams assessment

• e.g.
• memorize
• describe

• e.g.
• memorize
• describe

41
ALIGNED COURSE
?
Teachers intention
Students activity

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

Exams assessment

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

42
Student Motivation

• Susan (intrinsic motivation)
• - wants to learn !
• Robert (extrinsic motivation)
• - to pass exams !

43
Constructivism

• Transmission is Dead
• (lectures )
• Knowledge is Actively Constructed !

!
active teacher passive students
risk
44
SOLO Taxonomy

• Hierarchy for Competences
• Deep learning (not surface) !

5 generalize, theorize, predict, 4 explain,
analyze, compare, 3 describe, combine,
classify, 2 recite, identify, calculate,
45
Stud Learning Focus

• Focus on Student Learning !
• (instead of what teacher does
• labelling students good/bad)
• Student activitation ? learning

46
Alignment

• Make explicit ILOs
• (Intended Learning Outcomes)
• (and tell this to students)

Exam ILOs Teaching
47
Teaching/Learning Activities
problem-oriented
Project work
Case teaching
teacher-centric
student-centric
Lecture
Exercise class
discipline-oriented
kilde Torben K. Jensen
48
Acquisition of Competence

• Acquisition of competence progresses according to
  the following stages of learning
• 1) Unconscious incompetence
• 2) Conscious incompetence
• 3) Conscious competence
• 4) Unconscious competence
• 5) Capacity for moving consciously between
  stages 3) and 4) (which is required by a
  teacher)

49
On the Role of Examination

• Alignment
• A systemic theory (a teaching system w/
  cause/effects)
• A theory of planning (over the course of a
  course)
• A theory of motivation (and incentive)
• From the exam as a......to

"The exam does not come after, but before the
course!"
"Necessary evil"
application of alignment
Motivational and learning-guiding pedagogical
tool for the teacher(!)
50
Definition Good Teaching

• Definition
• Good news
• We now know how to do this
• Alignment!!!
• Explicitly defined course objectives (as verbs)!
• Discourage surface-learning!
• Encourage depth-learning!
• Less-is-more depth rather than breadth of
  coverage!

Good teaching is getting most students to use
the higher cognitive level processes that the
more academic students use spontaneously --
Teaching for Quality Learning at University,
John Biggs, 2003
51
The BLOOM Taxonomy (1956)

• The BLOOM Taxonomy

Analysis
Evaluation
Synthesis
SOLO 45
Qualitative
Application
Comprehension
Quantitative
SOLO 23
Knowledge

really intended to guide the selection of
items for a test rather than to evaluate the
quality of a students response to a particular
item -- (Biggs Collis, 1982)