Inside the black box: Raising standards through classroom assessment

1
(No Transcript)
2
How can we make productive contributions to
policy and practice?

• IEE Interim Strategy Board awayday
• 13 May 2008
• Dylan Wiliam

3
Pasteurs quadrant
4
Institute for Effective Engineering

• Building a bridge
• Requires a knowledge of the fundamental
  properties of materials
• But the physics doesnt tell you what the bridge
  should look like
• Designing effective learning environments
• Requires a knowledge of the fundamental
  principles of psychology
• But the psychology doesnt tell you what the
  learning environment should look like

5
Looking in the wrong place

• 3 generations of effectiveness research
• Raw results approaches
• Different schools get different results
• Conclusion Schools make a difference
• Demographic-based approaches
• Demographic factors account for most of the
  variation
• Conclusion Schools dont make a difference
• Value-added approaches
• School-level differences in value-added are
  relatively small
• Classroom-level differences in value-added are
  large
• Conclusion An effective school is little more
  than a school full of effective classrooms

6
Teacher quality matters
Barber Mourshed, 2007
7
but more for some than others
Impact of teacher quality on student outcomes
(Hamre Pianta, 2005))
8
The dark matter of teacher quality

• Teachers make a difference
• But what makes the difference in teachers?

9
Why research hasnt changed teaching (much)

• Aristotles main intellectual virtues
• Episteme knowledge of universal truths
• Techne ability to make things
• Phronesis practical wisdom
• In education, what works is often the wrong
  question
• Almost everything works somewhere
• Almost nothing works everywhere
• So the smart question is usually under what
  conditions will this work?
• And therefore much expertise in education is
  phronesis, not episteme
• Dissemination might be the right metaphor for
  episteme, but not phronesis
• How do we decide whether we need episteme or
  phronesis
• In medicine?
• In education?

10
After Nonaka Tageuchi, 1995
11
Designing for scale

• In-principle scalability
• A single model for the whole school
• But which honours subject-specificities
• Understanding what it means to scale (Coburn,
  2003)
• Depth
• Sustainability
• Spread
• Shift in reform ownership
• Consideration of the diversity of contexts of
  application
• Clarity about components, and the theory of action

12
Two opposing factors in any school reform

• Need for flexibility to adapt to local
  conditions, resources, etc
• Implies there is appropriate flexibility built
  into the reform
• Need to maintain fidelity to core principles, or
  theory of action of the reform, if it is to
  achieve desired outcomes
• Implies you have a well-thought-out theory of
  action

13
Tight but loose
Some reforms are too loose (e.g., the Effective
schools movement) Others are too tight (e.g.,
Montessori Schools) The tight but loose
formulation
combines an obsessive adherence to central
design principles (the tight part) with
accommodations to the needs, resources,
constraints, and particularities that occur in
any school or district (the loose part), but
only where these do not conflict with the theory
of action of the intervention.
14
Why the why?

• In many reforms, the why is non-existent,
  under-conceptualized, or not communicated well
• The Tight but Loose framework says
• It is imperative to explicitly weave the why (the
  theory of action and research base) into the what
  and the how, so that end users understand it
• Without that knowledge, under inevitable local
  pressures and constraints, users will make
  implementation decisions that undercut the
  effectiveness of the reform

15
Logic model for KLT
(Leahy, Leusner Lyon, 2005)
16
The theory of action matters to everyoneor it
should

• Teachers need it so they can make good decisions
  about the selection and adaptation of AfL
  techniques
• Administrators need it so they can make good
  decisions about supporting teacher change
• Parents need it because otherwise theyre nervous
  about new practices
• Politicians need it so they can make good
  decisions (fund programs that work and set
  policies that support them)

17
The effects of context

• Beliefs
• about what constitutes learning
• in the value of competition between students
• in the value of competition between schools
• that test results measure school effectiveness
• about the trustworthiness in numerical data, with
  bias towards a single number
• that the key to schools effectiveness is strong
  top-down management
• that teachers need to be told what to do, or
  conversely that they have all the answers
• In the English context, which beliefs
• are most significant for education reform?
• can we change?