Do You Know What Your Students Know Examining Student Work to Inform Instruction

1
Do You Know What Your Students Know?Examining
Student Work to Inform Instruction

• Lani Seikaly, partner
• Hillcrest and Main, Inc.
• and project director of MDK12
• http//mdk12.org
• lani_at_mdk12.org

2
What do principals need to put in place to
improve their student achievement?How do
teachers and schools identify what students know
and still need to learn in relation to state
standards?
3
State assessments only inform schools where their
students are performing at the time of testing.

• Teachers must know where their students are at
  any point in the school year.

4
AYP provides us a clear target.

• However, to meet AYP, we need to monitor
  student level data at the classroom level on an
  ongoing basis.

5

• How do state standards and NCLB change
  expectations for what happens in our schools?

6
Standards-based education changes everything.

• Defines what we expect all students to know and
  be able to do.
• Expects educators to take all students to
  proficiency on those standards.

7

• Before Standards
• Teachers taught what they thought was important.

• After Standards
• Teachers are expected to teach the content
  standards.

8

• Before Standards
• Teachers had different expectations for
  different groups of students.

• After
• Standards
• Teachers are expected to take all students to
  proficiency.

9

• Before Standards
• The focus was on how well teachers taught.

• After Standards
• The focus is on how well students learn.

10

• Before Standards
• Only selected students had access to higher
  level instruction.

• After Standards
• All students are expected to have equity of
  opportunity for higher level instruction.

11
What activities do teachers need to engage in
today that they might not have needed to do 15
years ago?
12

• Align instruction and assessment with state
  content standard indicators
• Know where their students are performing on the
  indicators
• Work toward a common understanding of content
  standards and proficient work
• Diagnose what students know and still need to
  learn in relation to those indicators and the
  criteria for proficient work
• Engage in grade level team examinations of
  student work

13
How do we get answers to these questions

• Where are each of our students in relation to the
  content standards they must attain?
• What do they know and are able to do?
• What do they still need to learn?

14
We have a good deal of research to draw on in
identifying what effective schools and leaders do
to improve student achieve-ment.

15
Black and Wiliam in their 1998 Phi Delta Kappan
article, In the Black Box Raising Standards
through Classroom Assessment, assert,

• There is a body of firm evidence that formative
  assessment is an essential component of classroom
  work and that its development can raise standards
  of achievement. We know of no other way of
  raising standards for which such a strong prima
  facie case can be made.

16
If formative assessments are so critical, then
how do we create good assessments?
17
We examined the National Research Council book,
Knowing What Students Know, which identified the
key concepts on which good assessments are
built.From Knowing What Students Know by James
W. Pellegrino, Naomi Chudowsky, and Robert
Glasser/
18
Every assessment, regardless of purpose, rests
on three pillars

• Cognition, a model of how students represent
  knowledge and develop competence in the subject
  domain
• Observation, tasks or situations that allow one
  to observe students performance
• Interpretation, an interpretation method for
  drawing inferences from the performance evidence
  thus obtained.
• James W. Pellegrino, Naomi Chudowsky, and Robert
  Glasser

19
To understand what students know, staff need to
understand

• the knowledge and cognitive domains of the
  content standard indicators
• how students learn
• how to provide students opportunity to
  demonstrate what they know
• how to interpret student responses

20
Knowing What Students Know also suggests that
Student work should focus adult-student and
adult-adult conversations to discuss standards.

21
We have also read about the importance of
collaborative examinations of student work.
22
Joan Richardson, editor of the National Staff
Development Council newsletter, believes
thatThe practice of having teachers work
together to study student work is one of the most
promising professional development strategies in
recent years. Examining student work helps
teachers intimately understand how state and
local standards apply to their teaching practice
and to student work.
23
As they see what students produce in response to
their assignments, they can see the successes as
well as the situations where there are gaps. In
exploring those gaps, they can improve their
practice in order to reach all students.
24
The Aspen Workshop on High Schools recommended in
its summary report for the Transforming High
Schools Task Force that the continuous and
collaborative examination of student work along
with the personalization of schooling are the two
critical strategies for transforming high schools
at the local level.
25
At the risk of sounding overly simplistic, the
use of student work as the unrelenting focus of
adult conversations can be the catalyst of
fundamental changes in the educational experience
of adolescents, and the transformation of
teaching and learning at the high school level.
26
Kate Nolan, Director of Re-Thinking
Accountability for the Annenberg Institute of
School Reform, believes

• Rich, complex work samples show us how students
  are thinking, the fullness of their factual
  knowledge, the connections they are making.
  Talking about them together in an accountable way
  helps us to learn how to adjust instruction to
  meet the needs of our students.

27
So we understand

• that to know where each of our students is in
  relation to content standards, we need to
  regularly examine student performance to inform
  our instruction and to monitor student progress.
  What would that look like? How would we
  operationalize that?

28
What does a team examination of student work
look like?

• How do teachers define proficiency?
• How do teachers diagnose strengths and needs?
• How do they record the diagnostic information?
• Who leads the discussion?
• What questions need to be asked?

29
Examining Student Work

• Though teachers have always examined student
  work as part of their grading process, the new
  focus on accountability and standards has driven
  a more structured and collaborative examination
  of student work.

30
Examining Student Work

• The focus of the examination has shifted from a
  summative evaluation of student performance to a
  diagnostic evaluation of student performance,
  teacher assignment, and implications for
  instruction.

31
The Examining Student Work Protocol asks teachers
to

• Reach consensus about what makes a student
  response proficient on an assignment or
  assessment
• Diagnose student strengths and needs on the
  performance
• Determine next instructional steps based on the
  diagnosis

32
In the first part of the protocol,

• a team of teachers work through the process of
  reaching consensus on what the team believes
  constitutes a proficient response on a selected
  text and question.

33
Stiggins argues that we really cant assess
accurately if we dont understand the target

• To assess student achievement accurately,
  teachers and administrators must understand the
  achievement targets their students are to master.
  They cannot assess (let alone teach) achievement
  that has not been defined. Stiggins, Richard
  J. 2001. The Principals Leadership Role in
  Assessment. NASSP Bulletin (January 2001) 1326.

34
Part 1 Reaching Consensus about Proficiency

• What did you ask the students to do?
• Which content standard indicator were you
  assessing?
• What did you consider proficient performance on
  this assignment?
• Exactly what did students need to say or write
  for you to consider their work proficient?

35
It is not enough that an individual teacher
defines proficiency.

• It is critical that at least a grade level
  team has reached consensus on the definition of
  proficiency to ensure that all students are held
  to the same performance expectations.

36
Teachers were surprised at how off the page
their teammates were.

• Think of all the mixed messages our students
  are receiving when we havent defined proficiency
  on a standard / indicator in the same way.

37
Lets take a look at a team of 3rd grade teachers
trying to get on the same page about what a
proficient answer would need to include for a
reading comprehension assessment.
38
Only after the team has agreed on what
constitutes a proficient response are they able
to diagnose student strengths and needs.
39
In the second part of the protocol,

• the team examines three student papers to
  determine if the response is proficient and to
  identify strengths, needs and instructional next
  steps.

40
Each teacher is asked to bring three samples of
student work from the same assignment or
assessment a response at the top of the class, a
response at the bottom of the class and a
response in the middle of the class.
41
Part 2 Diagnosing Student Strengths and Needs

• What did the student demonstrate that he/she
  knew?
• What misconceptions or wrong information did the
  student have?
• What did the student not demonstrate?
• How would you find out if he/she knew it?

42
Teachers must shift their mindset from scoring (a
summative examination) to diagnosing (a formative
examination) student performance.
43
In many cases teachers have spent a great deal of
time sorting student responses (either by letter
grades or by rubric scores) and virtually no time
diagnosing what students know and still need to
learn.
44
Lets take a look at our team of 3rd grade
teachers diagnosing their student responses.
45
Part 3 Identifying Instructional Next Steps

• Based on the team's diagnosis of the student
  performance, what would you do next with that
  student?
• What questions might you ask the student?
• What feedback would you give?
• Do you need to re-teach anything to the class or
  a subset of the class?

46
How have teams captured the data?
47
(No Transcript)
48
What can you learn from examining student work?

• What do teachers learn about students?
• What do teachers learn about their instruction?
• What do school teams learn about their teams
  understanding of content standards?

49
Teachers came out of this examining student work
process with

• Grade level consensus of what constitutes
  proficient work on the assignment
• Formative assessment data
• Specific information to inform their instruction
• Strategies for re-teaching
• Deeper understanding of the intent of the
  standard / indicator they were assessing.
• Probing questions to ask students to better
  understand where they were

50
Andrew Nelson, teacher at Harmony Hills E.S.

• I wish I had caught onto this earlier but at
  one of the meetings, it became apparent that kids
  werent reading the question. That was a big
  awakening to me because I was so focused on how
  to write the answer, we hadnt spent time
  unpacking the question.

51
Glenn Messier, teacher at Harmony Hills E.S.

• My teammates proficient responses were a
  little bit more advanced than what I was
  expecting and looking for. They were looking for
  a lot more in-depth answers. To get on the same
  page, I needed to raise the bar for my students.

52
Ilise Wolf, teacher at Harmony Hills E.S.

• When youre working on a team, I really find
  I get a lot of ideas from my teammates and
  support and feedback. What I might consider a
  good assignment for a child, another 2nd grade
  teacher might have a few extra words to add that
  would really make a difference to my students.

53
Lets hear what our 3rd grade team learned from
their experience in participating in this
examining student work protocol.
54
Principals and facilitators learned

• When teachers collaborate with other teachers
  on examining their own students performance on
  classroom assignments and assessments, teachers
  are more engaged in the process and take more
  ownership for making changes to their practice.

55
Principals and facilitators learned that

• Teachers began to have the discussion and reach
  consensus on what they were looking for on an
  assignment or assessment before they gave it to
  the students. Therefore, their teaching was also
  more aligned with the proficient criteria they
  had identified.

56
Principals and facilitators learned that

• Principals needed to set clear expectations for
  their teams, monitor whether they were met,
  identify useful end products, and engage in an
  ongoing discussion of the data and what students
  knew and still needed to learn.

57
How do we build teacher capacity to do this?

• Shouldnt we do that before we start the
  process?

58
This is on the job training.

• The focus on interpreting student performance
  and determining what teachers need to do to
  support student performance allows teachers to
  examine their own practice through the lens of
  student needs rather than the lens of good versus
  mediocre teaching.

59
The fact that the process is ongoing allows
teachers to build capacity over time rather than
try to absorb everything in an upfront training.

60
Bottom line ..

• It is only when teachers and schools start to
  collect the data and diagnose the performance
  that provides them the necessary information
  about where a student is in relationship to the
  indicators they must master that effective
  data-driven decision making to improve student
  achievement can happen.

61
You can find more information on the School
Improvement in Maryland Web

• Examining Student Work
• http//mdk12.org/data/examining/index.html
• Monitoring Student Progress
• http//mdk12.org/data/progress/index.html
• An online course, Using Data to Improve Student
  Achievement
• http//mdk12.org/process/course

62
Lani Hall Seikaly Contact Information

• Email lani_at_mdk12.org
• Phone [phone]
• Web site http//hillcrestandmain.com
• Web site http//mdk12.org