Understanding by Design

1
From Data Overload to Increased
Student Learning
Dr. Ronald S. Thomas Center for Leadership in
Education at Towson University rathomas_at_towson.e
du
2
Show Us the Data!
Data are observations, facts, or numbers that,
when collected, organized, and analyzed, become
information and, when used productively in
context, become knowledge.

3
Show Us the Data!
Data are observations, facts, or numbers that,
when collected, organized, and analyzed, become
information and, when used productively in
context, become knowledge. Data
Information Knowledge A
LEARNING ORGANIZATION

4
Show Us the Data!
Data Information Knowledge
A LEARNING ORGANIZATION With a colleague or
two, elaborate on the definition of data using
the questions on p. 2. How far is your school
along the continuum?

5
The DRIP Syndrome
DATA RICH
INFORMATION POOR
6
The Situation in Too Many
Schools
DATA OVERLOAD
7
We need a new process.

• Real time
• Specific to each grade and subject
• Addresses individual students needs
• Results in instructional improvements that will
  actually
• occur
• Can be re-directed frequently
• Has meaning for teachers (seen by teachers as a
• worthwhile use of their time)
• Use the scales on page 4 to rate the data
  conversations in your team, department, or school.

8
What should the new process look like?
School improvement is most surely and
thoroughly achieved when teachers engage in
frequent, continuous, and increasingly concrete
and precise talk about teaching practice . . .
adequate to the complexities of teaching, capable
of distinguishing one practice and its virtue
from another. --Judith Warren
Little
9
In other words . . .
A Classroom-Focused Improvement Process (CFIP)
10
Goal of CFIP

• Frequent, continuous, and increasingly concrete
  and precise dialogue by school teams, informed by
  data

11
CFIP is built on

• 1. Dialogue

12
What Is True Dialogue?
In dialogue, a group accesses a larger pool of
common meaning, which cannot be accessed
individually. People are no longer primarily in
opposition, rather they are participating in
generating this pool of common meaning We are
not trying to win in a dialogue. We all win if
we are doing it right.
- Senge, The Fifth
Discipline (1990)
13
SILENCE, PLEASE!
Read the description of true dialogue on page
6. Theres a lot of reading and thinking going on
here!
14
CFIP is built on

• 1. Dialogue
• 2. Protocols

15
What Is a Protocol?
A protocol consists of agreed-upon guidelines
for dialogue which everyone understands and has
agreed to that permit a certain kind of
conversation to occur, often a kind of
conversation which people are not in the habit of
having. Protocols build the skills and culture
necessary for collaborative work. Protocols
often allow groups to build trust by doing
substantive work together.
16
Why Use a Protocol?
In some educational organizations, protocols
may at first seem foolish an unwarranted
interference in ordinary business. The more
dysfunctional the organization, the stronger the
negative reaction may be.One could argue
thatcommunication precision, faithful
replication, and scripts would prove
counterproductive here. Dont we learn best by
just talking with each other?
McDonald, et al. The Power of Protocols
(2003), pp. 1,4.
17
NO!
Among educators especially, just talking may not
be enough. The kind of talking needed to educate
ourselves cannot rise spontaneously and unaided
from just talking. It needs to be carefully
planned and scaffolded. McDonald, et al. The
Power of Protocols (2003), pp. 4,5.
18
CFIP is built on

• 1. Dialogue
• 2. Protocols
• 3. Triangulation of
• Data

19
Triangulation of Data

• Use a variety of sources of data to identify the
  patterns.
• By combining multiple sources, we can overcome
  the weaknesses of individual data points and
  generate powerful insights not available from one
  source.

20
Triangulate Data

• Three types of data are triangulated
• External Assessment Data
• Coursewide Benchmark
• Assessment Data
• Classroom Assessment Data

21
THE GPS ANALOGY
22
Why Triangulate Data? One of the real dangers
of having state tests determine adequate yearly
progress is our tendency to use only those
results to guide instruction. The problem is that
most state tests are instructionally
insensitive (Popham, 2007). They are not
designed to inform daily lesson planning, but to
generate an accountability score as quickly and
efficiently as possible.
-Thomas,
Principal, NAESP (2006)
23
Why Triangulate Data? State test scores are
important, but they are only one source of
student achievement data. If we are to make
accurate and helpful decisions to guide daily
instruction, we must use at least three different
sources of data. It is a method that has been
practiced for centuries by architects, engineers,
and surveyors (White, 2004).
-Thomas, Principal,
NAESP (2006)
24
Alignment Issue When Collecting
Classroom Assessment Data
Be sure that classroom assessments mirror the
most valued external assessments in content,
format, and rigor.
25
CFIP is built on

• 1. Dialogue
• 2. Protocols
• 3. Triangulation of
• Data

26
TEAM DATA DIALOGUE PROTOCOL MOVING FROM DATA
TO INCREASED STUDENT LEARNING DATA SOURCE(S)
__________________________________________________
_________ Step 1 Identify the questions to
answer in the data dialogue. Step 2 Build
assessment literacy. Define terms (if
needed). Step 3 Identify the big picture
conclusions from the data. Step 4 Identify the
patterns of class strengths and weaknesses (using
more than one data source, if possible).
Step 5 Drill down in the data to individual
students. Identify and implement needed
enrichments and interventions.
Step 6 Reflect on the reasons for student
performance. Identify and implement needed
instructional changes for the next unit.
27
CFIP Data Dialogue Protocol Formats

• One-page overview of the model, p. 9
• CFIP model with protocol questions, pp. 11-12
• CFIP protocol worksheets, pp. 13-17
• Reflection Guide to Instructional Changes,
• pp. 19-20
• Steps 4-6 of CFIP protocol, in another format,
  pp. 21-23
• Example of CFIP protocol, as completed by a
  school team, pp. 24-30
• Take a few minutes to preview
• pages 9-30.

28
TEAM DATA DIALOGUE PROTOCOL MOVING FROM DATA TO
INCREASED STUDENT LEARNING DATA SOURCE(S)
__________________________________________________
_________ Step 1 Identify the questions to
answer in the data dialogue. Step 2 Build
assessment literacy. Define terms (if
needed). Step 3 Identify the big picture
conclusions from the data. Step 4 Identify the
patterns of class strengths and weaknesses (using
more than one data source, if possible).
Step 5 Drill down in the data to individual
students. Identify and implement needed
enrichments and interventions.
Step 6 Reflect on the reasons for student
performance. Identify and implement needed
instructional changes for the next unit.
29
STEP 1 When Analyzing Data, Begin with a
Question.
All data analyses should be designed to answer a
question. Unless there is an important question
to answer, there is no need for a data
analysis.
30
STEP 1 When Analyzing Data, Begin with a
Question.
On page 32, brainstorm several examples of the
types of questions your school should be
answering through ongoing data analyses.
31
Data Dialogues at Middleborough Elementary
School STEP 1 When Analyzing Data, Begin with
a Question.

• What can we learn about our students by looking
  at data, given our beliefs about the value of MSA
  and MSA scores?
• From Doug Elmendorf, Former Assistant
  Principal

32
STEP 2 Understand the Data Source.

• Build ASSESSMENT LITERACY with questions like
  these
• What assessment is being described in this data
  report? What were the characteristics of the
  assessment?
• Who participated in the assessment? Who did not?
  Why?
• Why was the assessment given? When?
• What do the terms in the data report mean?
• Be sure everyone on the team has clear and
  complete answers to these questions before
  proceeding any further.

33
Data Dialogues at Middleborough ES STEP 2 Build
Assessment Literacy
MSA

• Data from standards-based tests compared
• to data from norm-referenced tests

Classroom Assessment Literacy

• On-line Math Benchmarks
• Math Summatives
• HM Theme tests
• Daily Formatives

34
STEP 3 Look for the Big Picture Views in
the Data.

• Identify
• What do we see in the
• data?
• What pops out at us
• from the data?

35
Data Dialogues at Middleborough ES STEP 3
Getting the Big Picture

• Focused on special education population in grades
  3 and 4.
• Depersonalized data with anonymous data walk.
  Sample questions on page 34.
• Case managers compared assessment triangulation
  results with IEP goals.
• Less desirable data displays were thrown out for
  future meetings. Others were modified to make
  them more user-friendly.

36
STEP 4A Look for Patterns in a Single Data
Source.

• What patterns do you see over and over again in
  the data?
• What are the students strengths? What
  knowledge and skills do students have?
• What are their weaknesses? What knowledge and
  skills
• do students lack?

37
STEP 4B Look for Patterns in the Data from
Multiple Data Sources.

• What patterns do you see over and over again from
  multiple sources of data?
• What are the students strengths? What
  knowledge and skills do students have?
• What are their weaknesses? What
• knowledge and skills do
• students lack?

38
Data Dialogues at Middleborough ES STEP 4 Look
for Patterns

• Are there strengths and areas for growth that
  exist through most or all data sources?

39
STEP 5 Drill Down to Individual Students.
Identify and Implement Needed Enrichments.

• What are the implications for enrichments from
  what you learn from the data?
• Which students need
• enrichment?What should enrichments
• focus on?

40
STEP 5 Drill Down to Individual Students.
Identify and Implement Needed Interventions.

• What are the implications for interventions from
  what you learn from the data?
• Which students need
• interventions?What should interventions
• focus on?

41
STEP 5 Drill Down to Individual Students.
Identify and Implement Needed Enrichments and
Interventions.
Brainstorm on page 36 as many interventions as
you can of each type available in your school.
42
Data Dialogues at Middleborough ES STEP 5
Implement Interventions

• Structural Adjustments
• Personnel (Instructional Assistants)
• Technology

• Specific Intervention Examples
• After school tutorial for specific students.
  (Skill deficits identified in data analysis were
  the targets of this intervention.)
• Teacher websites to address pervasive homework
  concern related to math review and reinforcement.
• Lunch Bunch to address number sense concerns.
• Actual change of groupings of students to better
  meet individual student needs.

43
STEP 6 Reflect on the reasons for student
performance -- What in our teaching might be
preventing all students from being successful?

• To what extent did we implement research-based
  instructional practices as we
• Planned instruction?
• Introduced instruction?
• Taught the unit?
• Brought closure to instruction?
• Assessed formatively?

44
STEP 6 Reflect on the reasons for student
performance. Identify and Implement
Instructional Changes in the Next Unit

• How will we change instruction in our next unit?
• Content focus
• Pacing
• Teaching methods
• Assignments

45
STEP 6 Reflect on the reasons for student
performance. Identify and Implement
Instructional Changes in the Next Unit

• When will we review the data again to determine
  the success of the enrichments, interventions,
  and instructional changes?
• What do the data not tell us? What questions
  about student achievement do we still need to
  answer? How
• will we attempt to answer these questions?

46
STEP 6 Reflect on the reasons for student
performance. Identify and Implement
Instructional Changes in the Next Unit

• Study the reflection sheet on pp. 39-40
• carefully. Then, on page 38, identify the items
  that you feel that, by and large, teachers in
  your school
• Are good at doing.
• Are OK at doing, but need more practice.
• Really need to work on.

47
The Big Six of Data Analysis 1. Begin with a
question. 2. Understand the data source. 3. Look
for the big picture. 4. Look for patterns in the
data. 5. Identify and act on the
implications of the patterns for your
students. 6. Identify and act on the implications
of the patterns for your instruction.
48
Your Next Steps

• Unless you emerge from
• the data analysis process with a clear plan of
  action for your classroom, you have wasted your
  time.
• Implement your plan of interventions,
  enrichments, and changes in instruction.
• Collect your next set of data.

49
Is It Worth the Effort?

• Take a look at the following results.
• Then you tell me.

50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
Caveats about CFIP

• It is a paradigm shift from the traditional
  lesson planning format.
• It is not easy, especially at first.
• Follow the steps faithfully until they become
  second nature.
• The CFIP is a guide until you make the process
  your own.
• Expect mistakes and imprecision in the data.
• The results are worth the effort.

55
Where does a school go from here in becoming more
data-driven?
DISCUSSION What drivers and barriers would you
see schools facing in implementing data
dialogues, such as the CFIP model (pp. 43)?
56
Where does a school go from here in becoming more
data-driven?
DISCUSSION What role could you play in helping
schools overcome the barriers and move forward in
their data dialogue process (p. 44)?
57
Questions and Answers