Introduction to the CAESL Assessment System

1
Introduction to the CAESL Assessment System
2
Overview

• Accountability the focus of NCLB.
• What do we mean by assessment?
• What is an ideal assessment system?

3
NCLB Accountability

• By 2007-08, annual assessment of science at
  least once during each of the following grade
  spans 3-5, 6-9, and 10-12.
• Must report three levels Advanced, Proficient,
  Basic.
• Must include individual student scores as well
  as disaggregation by race, ses, and other
  categories.

4
What do we mean by assessment?

• Deliberate acts to reveal student understanding
• May be formal (tests, quizzes, portfolios,
  etc.)
• Or informal (listening to students,
  questioning strategies, etc.)

5
CAESL Principles of Quality Assessment

• Quality assessment is
• Supportive of student learning
• Research-based
• Aligned to important science content
• Fair
• Accurate and efficient

6
How can assessment impact student learning?

• Paul Black and Dylan Wiliam, in a comprehensive
  review of literature, found that formative
  assessment is closely connected to instruction
  and with information about how to improve
  performance produced large positive effects on
  student performance.
• Black and Wiliam noted that this kind of
  feedback rarely occurred in classrooms.
• The importance of formative assessment was
  recognized in the National Science Education
  Standards that linked content, teaching and
  assessment standards.

7
(No Transcript)
8

• Base of the system
• Targeted goals for student learning
• These are derived from research into student
  cognition, curriculum objectives, and standards
  at the district, state or national level.

9

• Base of the system
• Quality Assessment Tools
• Tests and strategies used are purposefully
  designed to measure the targeted goals for
  student learning
• Multiple measures are used

10

• Base of the system
• Quality Use
• Teachers use quality assessment s to
• Communicate expectations and provide feedback
• Take action to improve learning
• Promote student accountability
• Assure equity

11

• Levels of the system
• Classroom level is where most assessment occurs.
• Currently, most classroom assessment is loosely
  coupled, if at all, to district and state level
  assessment
• CAESL model advocates coordinated and integrated
  system across levels

12

• Progress guides
• These are the detailed expressions of how
  students knowledge and skills develop in the
  relevant science topics.
• They are based on research about how students
  learn in science
• They can be used to evaluate performance across
  the levels of assessment

13

• Information flow
• Uses the Progress Guides as the language of
  information
• Information from state and district assessments
  can be interpreted in direct relationship to
  student learning and performance in the classroom
  and vice versa

14
Closer Look at the CAESL Assessment System
Quality Assessment Tools Foundations of
Reflective Lessons Erin Furtak, Stanford
University Using Progress Variables and
Embedded Assessment to Improve Teaching and
Learning Karen Draney, UC Berkeley Teachers
Use of Assessment to Support Learning Joan
Herman, UCLA/CRESST
15
Quality Assessment Tools Foundations of
Reflective Lessons
Erin Marie Furtak
Stanford Education Assessment Laboratory
16
Stanford Education Assessment Laboratory

• Quality Assessment Tools (QATs)
• Designed to
• Measure the targeted goals for student learning
• Provide evidence bearing on student learning
  using multiple measures
• Guide instructional improvement

QUALITY ASSESSMENT TOOLS
17
The foundation for QATs SEAL framework for
knowledge types
Declarative Procedural Schematic Knowledge Knowled
ge Knowledge
(Knowing that) (Knowing how to)
(Knowing why)
Proficiency
Low
High
Extent (How much?) Structure (How is it
organized?)
Domain-specific content facts concepts
principles
Procedures Lab Techniques Designs Analysis Inferen
ce
Problem schemata Mental models
18
QATs within a domain Buoyancy

• Foundational Approaches in Science Teaching
  (FAST) Curriculum
• Developed by the Curriculum Research and
  Development Group (CRDG) at the University of
  Hawaii
• Interdisciplinary middle school science
  curriculum
• First 12 investigations focus on density and
  relative buoyancy, i.e

Why do things sink and float?
19
What do YOU think is going to happen
FLOAT
or
SINK
20
Buoyancy A rich cognitive domain
Type of Knowledge Focus
Example Prompts Declarative Knowing
that Concepts facts What is
density? Procedural Knowing how Actions,
steps, Find the density of a liquid
procedures Schematic Knowing why Principles
Why do things sink or float? mental
models
21
Reflective Lessons What are they?

• Formative assessments embedded in a unit of study
• Provide teachers and students with opportunities
  for reflection
• Composed of multiple types of prompts designed to
  elicit students thinking to make explicit their
  developing conceptual understanding
• Type I
• Graph Interpretation and Explanation
• Predict-Observe-Explain (POE)
• Open-Ended Question Why Do Things Sink and
  Float?
• Predict-Observe (PO)
• Type II
• Concept Maps

22
FAST 1-12 Instructional Sequence
Joints
23
Example of a Reflective Lesson RL _at_ 4 - POE
Predict
24
Example of a Reflective Lesson RL _at_ 4 - POE
Observe
Explain
25
Example of a Reflective Lesson RL _at_ 4 - PO
Predict - Observe
26

• Quality Assessment Tools (QATs)
• The foundation of any assessment system
• Reflective lessons lie at the base of the CAESL
  study

QUALITY ASSESSMENT TOOLS
27
Using Progress Variables and Embedded Assessment
to Improve Teaching andLearning

• Karen Draney, Cathleen Kennedy, Mark Wilson
• UC Berkeley

28
The BEAR Assessment System

• Progress variables
• Developmental perspective
• These hold the levels of the CAESL model together
• Assessment tasks
• Match between instruction and assessment
• Progress guides
• Teacher management and responsibility
• Progress maps
• Quality evidence
• Assessment moderation
• Bringing it all together
• The system forms bottom level of CAESL model

29
The reflective lesson structure

• Pretest
• Reflective lessons
• Posttest

30
Interpreting a progress map
31
CAESL maps
32
Individuals w/ progress maps
33
Change by class using a progress map
34
Overall change using a progress map
35
Teachers Use of Assessment to Support Learning
Building Knowledge and Capacity

• Joan L. Herman
• UCLA/CRESST

36
Teachers Use of Assessment for Student Learning

• Teachers use quality assessments to

• Elicit student understanding
• Communicate/provide feedback
• Interpret/take action
• Increase student responsibility
• Assure equity

QUALITY USE
37
Two Collaborative Ventures Examining Teacher
Practices and Use

• CAESL Framework Study, in collaboration with
  Stanford and UC Berkeley
• CAESL Research-Professional Development
  Collaborative

38
Research Questions How Does the Bottom Floor
Work?

• What constitutes quality classroom assessment
  practice?
• How does quality practice develop?
• What are its effects on student learning?

39
Varied Information Sources

• Surveys
• Classroom observations
• Teacher reflections
• Assessment portfolios
• Analyses of student work
• Student performance data

40
Illustrative Practice Issues

• Whether and how teachers
• Purposively elicit students understanding
• Interpret student responses
• Provide feedback
• Orchestrate formal and informal assessments
• Take action based on results

41
Preliminary Hunches An Iterative Process

• Goals matter
• Alignment is a first aha
• Specific goals give way to progress on big ideas
• Alignment focus moves from task only to task and
  performance criteria.
• Assessment criteria are internalized and guide
  all instruction and assessment
• Feedback linked to coherent and consistent
  assessment frame

42
Some Major Challenges

• Practical issues -- classroom management, class
  size, press to cover content, space and
  environment, available time
• Teachers content and pedagogical knowledge
• Quality of available instructional materials and
  assessment tools

43
Graphic Slide
44
Break Out Sessions

• Jigsaw to better understand what CAESL Framework
  means
• Each table number off to assure representation at
  each break out session
• Your job to bring your sessions perspective
  back to your home table

45
Questions to Think About

• How does this session help me understand the
  CAESL Framework?
• Does it make sense to my work? How?
• What questions do I have about this
  interpretation?
• What are the implications from this session for
  us as a community and our future work?