Moderation of schoolbased teacher judgments of student achievement: experience and practice in Queen

1

• Moderation of school-based teacher judgments of
  student achievement experience and practice in
  Queensland
• Graham Maxwell
• Queensland Studies Authority

2
Australia overview

• Strong curriculum base affecting assessment
• Incorporation of school-based assessment in all
  certification
• Preference for standards-based assessment
• Respect for teacher judgment
• Vocational education within schooling

3
Australia overview (cont)

• Multiple pathways to future study and careers
  (without explicit tracking)
• School-based assessment in the compulsory years
• Outcomes-based frameworks
• Issues concerning national benchmarks
• Equity issues

4
Australia update

• Curriculum consistency?
• Essential learnings?
• Reporting consistency?
• Literacy and numeracy testing (aspects of?)
• Lit. Num. benchmarks and reporting?
• Other testing (Sci, ICT, Civ Cit) (sample)
• Generic and employability skills?

5
Queensland structures

• Traditional pattern
• pre-school, primary (1-7), secondary (8-12)
• Current syllabus structures
• Years 1-10 KLA syllabuses
• Years 11-12 Senior certificate
• New pattern
• early (P-4), middle (5-9), senior (10-12)

6
Queensland Studies Authorityltwww.qsa.qld.edu.augt

• Amalgamation in 2002 of previous
• Queensland School Curriculum Council (Yrs 1-10)
• Board of Senior Secondary School Studies (Yrs
  11-12)
• Tertiary Entrance Procedures Authority
• Main functions
• Syllabuses (P-12)
• Assessment
• Moderation
• Certification (Senior Certificate)

7
Queensland assessment

• Formal statewide requirements
• Year 2 Diagnostic Net
• Years 3, 5 7 literacy and numeracy tests
• Year 9 literacy and numeracy tests?
• Senior Certificate (end of Year 12)

8
Years 1-10 KLA syllabuses

• Specified core learning outcomes
• strand, sub-strand, cross-curriculum
• Developmental levels (62)
• Assessing and reporting a school responsibility
• Current statewide consultation on assessing and
  reporting
• Distinction between recording and reporting

9
Current Senior Certificate

• End-of-schooling record of achievement
• Reports
• Levels of Achievement in approved subjects
• Accredited VET certificates and competencies
• Awarded if
• student completes Year 12
• student has one recordable result

10
Senior Certificate subjects

• A subjects (count towards university selection
  index) moderated
• AR subjects (count towards alternative ranking
  index) unmoderated

11
Proposed Senior Certificate

• Will require minimum amount and quality of
  learning
• Will broaden recognised types of learning
• Other recognised awards and certificates
• Work-based, community-based and self-directed
  learning
• All students will have record of learning
• (from Year 10)

12
Fundamental principles for Senior Certification

• Partnership between QSA and schools
• Syllabuses as
• Curriculum frameworks
• Exit performance standards (five levels of
  achievement)
• School implementation plans (work programs)
• Teaching and assessment program
• Assessment continuous and formative
• Match to syllabus learning expectations
• Building a portfolio of student work
• Assessment as judgment against standards

13
Reporting final level reached

• Final report is judgment of where the student
  got to in their learning journey, not where they
  started or where they were on the average across
  the whole course
• Earlier assessments may be superseded by later
  assessments covering the same underlying
  dimensions of learning (selective updating)
• Final result is not mathematical aggregation
• Final result is an interpretation of the body of
  evidence (portfolio)

14
Stages of external moderation for A subjects

• Approval (of work programs)
• Monitoring (end of Year 11)
• Verification (end of Year 12 - Term 3)
• Confirmation (end of Year 12)
• Random Sampling (post hoc)

15
Moderation for A subjects

• School-based assessment
• Varied assessment techniques
• Building a portfolio of student work
• Teacher judgments against standards
• Internal (within-school) moderation
• Moderation by expert teacher panels
• Feedback advice to schools
• Negotiation of final results

16
What do panels do?

• District panel reviews school submissions
• School submits sample of portfolios
• School submits other material
• Consider whether they agree with school
• Focus on confirming or challenging
• Not a blind or independent judgment
• Emphasis on looking for evidence
• Aim is to reconcile any differences of opinion
• State panel reviews samples from districts
• State panel arbitrates difficult cases
• QSA Officers provide support and advice

17
Some features in 1972

• University orientation (with one university)
• 30 retention of age group
• Strong subject-matter demands
• Norm-referencing and over-testing
• Universal use of marks and cut-offs
• Emphasised end-of-semester examinations
• Reported result for each semester
• Consensus moderation meetings

18
Some features in 2004

• Broader orientation (not just university)
• 7580 retention of age group
• Focus on subject-related processes
• Standards-referencing and balance
• On-balance defensible judgments
• Whole of subject completion
• Report exit level of achievement
• Focus on where the student gets to
• Expert panel moderation

19
Other updates

• Credentialling of panellists
• On-line submissions of work programs
• Special consideration principles
• Late submission and non-submission
• Revisiting (assessments/semesters)

20
Staffing review panels

• Advertise vacant positions
• Selection and appointment by QSA Office
• Balance of representation sought
• Rollover every six years (life of syllabus)
• Chairs can have two terms
• One-third of members changed each time
• Training and credentialling
• Training tied to cycle of syllabus implementation

21
Roles

• School principal, heads-of-department, teachers
  (within-school moderation)
• Review panels chairs (state and district)
  (guardians of standards over time know teachers
  and schools)
• Standards and Assessment Officers (support
  syllabus development, provide assessment
  workshops, advise and assist schools, accredit
  panel members, support panel chairs, check
  certification data)

22
Some key characteristics

• Partnership
• Trust (in teachers)
• Involvement (of teachers)
• Parent and public acceptance
• Transparency
• Adaptation (to local contexts)
• Accountability
• Improvement

23
Some lessons of continuous assessment

• Synthesises formative and summative (continual
  feedback on progress against outcome targets)
• Builds and updates an evidential record of
  progress (dispersed rather than peak pressure no
  surprises at end)
• Fits new understandings of learning (context,
  engagement, authenticity)
• Extends range of assessable learning (broadening
  the curriculum)
• Develops student self-assessment and knowledge

24
Some lessons of moderation

• Responsibility encourages professionalism
• Empowers teachers (and students)
• Can produce acceptable comparability
• Creates powerful professional development
• Beginning with lower stakes assessment
• Building public perceptions and confidence
• Perseverance having a long term view